TLDR
Automating a Windows application sounds straightforward. Install a tool, record some clicks, run the script. In practice, the environments where automation matters most are exactly where most tools break down: enterprise desktops, legacy ERP systems, industrial HMI panels, locked-down production builds.
2026 has changed one thing: there are now more agents claiming to solve this. It hasn't changed the underlying problem.
Why Windows Automation Is Still Hard
Most automation tools share the same assumption: the application exposes something they can hook into. A code hook, an accessibility tree, an object ID, a stable selector. That assumption holds for modern web apps. It breaks in three places that show up constantly in enterprise Windows environments.
Legacy applications. Enterprise Windows desktops often run applications built on Win32, WPF, or WinForms, sometimes decades old. These applications may have no accessibility hooks, no stable selectors, and no APIs. Traditional tools simply cannot reach the elements they need to interact with.
Locked-down production builds. Test builds often include instrumentation hooks that make automation possible. Production builds strip those hooks out. A script that works perfectly in a test environment stops working the moment it's pointed at a production build. That's usually the environment where validation actually matters.
HMI and embedded displays running on Windows. Many HMI applications, including automotive digital cluster simulators, industrial control interfaces, and medical device UIs, can run on Windows machines. These applications don't expose accessibility hooks or structured selectors. The only interface is the screen itself. This is where the gap between "automation works in demos" and "automation works in production" is widest.
What Changes With Agentic Automation
The shift from script-based tools to agentic automation addresses the root cause of these failures. Script-based tools fail because they rely on code structure that isn't always there. Agentic automation adapts, using structured signals when available, and falling back to screen-based execution when they are not.
AskUI uses a hybrid execution model. When structured signals like selectors are available, it uses them for speed. When they are not, such as on locked-down builds, legacy applications, or HMI interfaces, it falls back to screen-based execution. The agent perceives the screen the same way a human engineer would and acts on what it sees.
This means the same automation logic works across the full range of Windows environments:
from askui import ComputerAgent
with ComputerAgent() as agent:
agent.act("Open the application and verify the status display shows Ready.")No instrumentation required. When structured signals are available, the agent uses them. When they are not, it perceives the screen directly and acts on what it sees.
Where This Matters Most: Enterprise and Industrial Windows
For general Windows desktop apps, the difference between agentic and script-based tools is a matter of maintenance overhead. For enterprise and industrial environments, it's the difference between automation being possible or not.
SIL environments on Windows. HMI simulation software running on Windows is one of the key environments where traditional tools fail completely. The display is rendered by a proprietary engine with no accessibility layer. AskUI operates at the OS level and interacts with what is rendered on screen, regardless of what's underneath.
VDI and Citrix sessions. Remote desktop environments present the same problem. The application runs inside a virtualized session with no direct element access. AskUI's Agent OS runs locally on the target device and operates at the system input layer, making it compatible with VDI and Citrix without additional configuration.
Cross-variant testing. Enterprise Windows deployments often involve the same application running in multiple configurations: different languages, different feature sets, different hardware. Script-based tools require separate scripts for each variant. Because AskUI identifies elements by appearance rather than code structure, the same test logic runs across variants without rebuilding.
Getting Started on Windows
pip install askui[all]Requires Python 3.10 or higher. You'll also need AskUI Agent OS installed on the target device. See the AskUI GitHub for setup instructions.
A basic Windows automation workflow:
from askui import ComputerAgent
from askui.tools.store.computer import ComputerSaveScreenshotTool
from askui.tools.store.universal import WriteToFileTool
with ComputerAgent(
act_tools=[
ComputerSaveScreenshotTool(base_dir="./screenshots"),
WriteToFileTool(base_dir="./reports"),
],
) as agent:
agent.act(
"Open the application, navigate to the status screen, "
"and write a summary of what is displayed."
)For a full walkthrough of how to connect Claude as the reasoning model and extend with caching, see How to Build an Agentic AI with Claude & AskUI.
FAQ
Does AskUI work with applications that have no DOM or accessibility hooks?
Yes. AskUI uses a hybrid execution model. When structured signals like selectors are available, it uses them for speed and precision. When they are not, it falls back to screen-based execution. This means it works on any application with a visible interface, including those without programmatic access.
What about locked-down production builds?
AskUI does not require instrumentation hooks or code-level access to the application under test. It works on production builds the same way it works on test builds.
How is this different from tools like Selenium or Playwright?
Selenium and Playwright are designed for web applications and rely on browser DOM access. AskUI uses a hybrid approach: for web environments it can leverage Playwright for fast, structured execution. For desktop applications, legacy software, VDI sessions, and environments without a DOM, it falls back to screen-based execution. The same test logic can span both.
For detailed comparisons with specific tools, see AskUI vs UiPath and AskUI vs Ranorex.
Does it work in VDI or Citrix environments?
Yes. AskUI's Agent OS runs locally on the target device and operates at the system input layer, making it compatible with virtualized environments.
What Windows applications can AskUI automate?
Any application with a visible screen interface: desktop apps, legacy enterprise software, HMI simulators, VDI sessions, and embedded displays running on Windows. For more on how this applies specifically to HMI and hardware validation environments, see AskUI: Eyes and Hands of AI Agents Explained.