TLDR
Infotainment systems are not web apps. They run on custom hardware, handle voice, touch, and physical inputs simultaneously, and update over the air. Script-based automation was not built for this environment. Agentic testing is.
Why Infotainment Testing Is Different
A web app has a DOM. A selector points to a button, a test clicks it, and the result is verified. The path is predictable.
An automotive infotainment system does not work this way. The interface runs on embedded hardware with no DOM and no accessibility tree. The same screen may look different depending on the vehicle variant, firmware version, or regional configuration. A voice command, a steering wheel button, and a touchscreen tap can all trigger the same function. All three need to be tested.
Script-based tools have no entry point into this environment. They cannot read what is on the screen, cannot handle multi-modal inputs, and cannot adapt when an OTA update shifts a UI element. Teams using selector-based tools find themselves stuck below 40 to 60 percent automation coverage. Not because they stop trying, but because every hour spent writing new tests is offset by hours spent fixing existing ones due to random app behavior, selector changes, and config drifts. The problem is covered in detail in our guide to HMI testing automation.
What Agentic Testing Does Differently
An agentic testing agent does not read selectors. It receives an instruction, reasons about what needs to happen, selects the right tool for that step, executes it, and observes the result.
For infotainment systems, this means the agent interacts with the interface the way a human tester would: by looking at what is on the screen and deciding what to do next. When the interface changes after an OTA update, the instruction remains valid. The agent finds the new path on its own.
The execution model handles inputs the same way regardless of modality. A test that validates a navigation command triggered by voice uses the same underlying loop as one triggered by a touchscreen tap. The instruction describes what needs to be verified. The agent handles how.
The OTA Testing Problem
Over-the-air updates are one of the hardest problems in infotainment testing. An update ships, UI elements shift, and every test that encoded a specific position or selector breaks overnight.
With script-based automation, this means a maintenance sprint before the next build can be validated. With agentic testing, the instruction stays the same. The agent reasons from intent, not from coordinates. An element that moves is still the same element with the same function. The test does not break.
This changes the economics of infotainment regression testing. Teams can run the same test suite against every firmware version without rewriting tests for each one.
Multi-Modal Input Coverage
| Input Type | What Gets Tested |
|---|---|
| Touchscreen | Element position, tap response, gesture recognition |
| Voice command | Trigger recognition, UI state after command |
| Physical button | Correct function activation, state change |
| Steering wheel control | Navigation, selection, confirmation |
Each input type triggers the same validation loop. The agent observes the resulting UI state and verifies it matches the expected outcome, regardless of which input triggered it.
Where This Fits in the HIL/SIL Stack
For HIL and SIL environments, the agent operates at the display layer. It reads the UI state after signals are processed by the system under test. The test infrastructure, such as CANoe or dSPACE, handles the signal injection. AskUI verifies that the display reflects the expected state.
Agentic testing integrates into existing automotive test pipelines without replacing them. The hardware-in-the-loop setup stays the same. The agent adds a validation layer that does not require selectors or DOM access. For a deeper look at the full SIL, HIL, and MIL testing hierarchy, see MIL SIL HIL Testing.
How Tests Are Structured
Agentic infotainment testing does not require code. Everything the agent needs lives in plain text files organized in a folder hierarchy. Test engineers own these files directly.
The structure looks like this:
main.py
prompts/
system_capabilities.md
device_information.md
ui_information.md
report_format.md
procedures/
start_can_trace.md
plans/
smoke.md
tests/
infotainment/
setup.md
rules.md
teardown.md
bluetooth_pairing.md
Each layer serves a specific role. System prompts define how the agent thinks and behaves. Procedures are reusable step sequences, such as returning to the home screen or opening a specific app. Test definitions describe what to verify, step by step, in plain language. Setup and teardown files handle bench preparation and cleanup.
A test case for switching media sources looks like this in practice:
| Step | Description | Expected Result |
|---|---|---|
| 1 | Open the Media app by running the OpenMedia procedure | Media app is in the foreground |
| 2 | If the source picker is not already open, tap the source-picker control | Source picker is visible |
| 3 | Tap the entry labeled "Bluetooth Audio" in the source list | Bluetooth Audio source is activated |
| 4 | Take a screenshot of the resulting Media screen | Screenshot is captured and embedded in the report |
Once tests are defined, running them requires a single command. The agent handles reasoning, execution, and reporting from there.
Each run produces a structured report showing the test case ID, status, agent interpretation of each step, expected and actual results, and a screenshot at each step. The report is generated deterministically, which means it can be used for audit trails and compliance documentation.
To see this running against a real infotainment system, AskUI hosts live webinars with working demos. You can find upcoming sessions at askui.com/webinars.
FAQ
Can agentic testing work on infotainment systems with no DOM or accessibility tree?
Yes. When selectors are not available, the agent uses screen-based execution to interact with the interface. This is how embedded infotainment systems without a DOM or accessibility tree are handled.
How does agentic testing handle OTA updates?
Instructions are written in natural language and encode intent, not specific coordinates or selectors. When an OTA update shifts a UI element, the instruction remains valid and the agent finds the updated element on its own.
Does this replace HIL or SIL testing infrastructure?
No. Agentic testing adds a UI validation layer on top of existing test infrastructure. The agent reads display state after signals are processed. It does not replace the tooling that sends those signals.
How are multi-modal inputs handled?
The same test intent can be validated across different input types. The agent verifies the resulting UI state after each input, whether triggered by voice, touch, or a physical control.
How does this integrate with existing automotive test pipelines?
AskUI runs as part of existing CI/CD pipelines. For automotive environments, it operates at the display layer alongside tools like CANoe or dSPACE. For a broader look at available tooling, see Top Agentic AI Tools for Infotainment Testing.