UI automation

What is UI automation?

UI automation lets software operate an application through its user interface. The automation opens windows, fills fields, picks menu items, presses buttons, and reads what is on screen, doing by machine what a person would do by hand.

It is common in RPA and desktop flows when a legacy application has no usable API. It can run attended, next to an employee, or unattended in a managed session. A good UI flow leans on structural UI elements rather than fixed screen coordinates, because window size, resolution, and layout all change over time.

UI automation is more precise than screen scraping, which reads pixels or raw text off the screen. Where scraping guesses at content, UI automation targets a named control in the application's own hierarchy.

UI elements and selectors

A UI element represents a single control, such as a text box or a button. A selector describes how the runtime finds that control in the application's tree of windows and components. On Windows this usually means one of three accessibility layers.

UI Automation (UIA) is the modern Microsoft framework and the default choice. It gives richer element detail and works well with newer frameworks such as WPF, WinForms, and UWP apps.

Microsoft Active Accessibility (MSAA) is the older layer that UIA replaced. It carries less detail but remains useful for legacy applications, for example classic Win32 or VB6 software, that do not expose UIA elements.

A raw UIA mode also exists for awkward cases, such as custom-rendered or Electron-based apps, where it exposes intermediate layers the standard view hides. Whichever layer you use, a good selector relies on stable attributes and matches exactly one element. Dynamic process IDs, changing window titles, and position indexes make a selector fragile, so most tools let you add fallback selectors that are tried in order.

UI automation versus API automation

An API gives an explicit machine contract with requests, responses, and error codes. It is usually faster, easier to test, and far less sensitive to visual changes, so it should be your first choice.

UI automation drives the layer built for people. It stays useful when no supported integration exists, when the API does not cover every action you need, or when you need a temporary bridge while a proper integration is built. Check first for an API, a file export, a database integration, or a ready-made custom connector. Choose UI automation on purpose and write down why.

Worked example: order entry in legacy software

A supplier sends orders through a portal. A desktop flow opens the old ERP screen, searches for the customer, fills the order lines, and reads back the generated order number.

Before each important step the flow waits for the right window and checks a recognisable state instead of pausing for a fixed number of seconds. After saving, it writes the order number back to the source. When validation fails, the flow does not create a second order. It records the error and sends the item to manual handling.

The pattern that makes this reliable is state-based waiting: wait until an element exists, a window closes, or a status changes, and give every wait a timeout that produces a controlled error rather than a hang.

What to watch out for with UI automation

UI automation shares a changing environment with the operating system and the applications on it, so it needs active care as long as those interfaces keep changing.

• Test in the mode you run in. A flow that works interactively can fail unattended because of session state, resolution, an unexpected dialog, or missing privileges. To drive an application that runs with elevated rights, the runtime itself usually has to run with matching rights.

• Handle the unexpected. Pop-ups, session locks, and slow loads will happen. Plan for them instead of assuming a clean screen.

• Do not bypass authorisation. Automating a screen does not grant new permissions. The runtime identity should hold only the rights the process needs, and credentials belong in supported secret storage, not in flow variables or scripts.

• Version and re-test. Keep flows, selectors, and dependencies under source control and re-test after application, browser, resolution, or language changes. A green start status does not prove the right data was saved.