AskUI Automation Example with Flutter (2025 Guide)

Updated: June 2025

Introduction

This tutorial shows how to use AskUI to automate an Android app built with Flutter. We provide the source code for the Flutter demo app used in this tutorial. You can find the source code on our GitHub repository.

This tutorial assumes that you already have your Android device or emulator prepared. If not, follow this post to set up your device.

Live Demo in Action (Playback Speed x3)

Setup

The source code for the Flutter demo app used in this tutorial is provided in this repository.

‍

1. Build and Run Flutter Demo App

1) Install Flutter

First, ensure Flutter is installed.

2) Clone repository and create Flutter demo app:

git clone https://github.com/askui/flutter-example-automation
cd flutter-example-automation
flutter create demo_app
cd demo_app

3) Install dependencies for the Flutter demo app:

flutter pub add camera intl

4) Update minSdkVersion in android/app/build.gradle:

defaultConfig{
  ...
  minSdkVersion 21
  ...
}

5) (Optional) Clear deprecation warnings (see this issue):

Modify pubspec.yaml:

dependencies:
  camera:
    git:
      url: https://github.com/flutter/plugins
      path: packages/camera/camera
      ref: 9e46048ad2e1f085c1e8f6c77391fa52025e681f

6) Run the Android Emulator.

7) Run the demo app:

flutter run

Now you should see the demo app running on your Android device.

‍

2. Setup ADBKeyboard

In this example, AskUI automates typing on the Android device using ADBKeyboard  a virtual keyboard that handles input via ADB:

‍ADBKeyboard.apk

1) Download the ADBKeyboard package (Version 2.0):

👉 ADBKeyboard GitHub Releases

2) Unzip it.

3) Find your device:

adb devices

4) Install ADBKeyboard on the device:

adb -s <your device id> install ADBKeyboard.apk

5) Configure ADBKeyboard:

adb -s <your device id> shell settings put secure default_input_method com.android.adbkeyboard/.AdbIME

6) Enable ADBKeyboard:

adb -s <your device id> shell ime enable com.android.adbkeyboard/.AdbIME

7) Verify:

When clicking any text field, you should see ADB Keyboard {ON} notification at the bottom of the screen.

‍

3. Setup AskUI

1) Install AskUI by following the guide for your OS (Windows, Linux, macOS):AskUI Installation Guide

AskUI Installation Guide

2) Run the UiController manually with Android runtime mode:

cd <YOUR_PROJECT_DIRECTORY>/node_modules/askui/dist/release/latest/<YOUR_PLATFORM>
./askui-ui-controller -r android

On macOS:

cd node_modules/askui/dist/release/latest/darwin/askui-ui-controller.app/Contents/MacOS/
./askui-ui-controller -r android

If you can’t find the binary, run:

npx jest test/my-first-askui-test-suite.test.ts --config ./test/jest.config.ts

3) Disable UiController code inside test/helper/jest.setup.ts (since it's already running manually):

import { UiControlClient } from 'askui';

let aui: UiControlClient;

jest.setTimeout(60 * 1000 * 60);

beforeAll(async () => {
    aui = await UiControlClient.build({
        credentials:{
            workspaceId: 'YOUR_WORKSPACEID_FROM_USER_PORTAL',
            token: 'YOUR_TOKEN_FROM_USER_PORTAL',
        }
    });
    await aui.connect();
});

afterAll(async () => {
    aui.close();
});

export { aui };

Breaking Down the AskUI Test Code

The full test automates all three app tabs:

• Outline (form fill)
• Datepicker (calendar interaction)
• Camera (take picture)

General Tips for Using AskUI:

1. Use annotate() or annotateInteractively() to visualize how AskUI sees UI elements.
2. Know your screen size — swipe coordinates may vary depending on device size.
3. Use .withText() targeting whenever possible.

‍

1. Click and Type (Outline Tab)

We start by filling out the form fields on the first screen.

App Screen:

‍

Test Code:

await aui.click().text().withText('Enter your username').exec();
await aui.type('askui').exec();

await aui.click().text().withText('Enter your email').exec();
await aui.type('askui@askui.com').exec();

await aui.click().text().withText('Enter your address').exec();
await aui.type('Haid-und-Neu-Straße 18').exec();

await aui.pressAndroidKey('enter').exec();

await aui.click().text().withText('Submit').exec();
await aui.click().text().withText('Refuse').exec();

await aui.click().text().withText('Banana').exec();
await aui.click().text().withText('Sunny').exec();
await aui.execOnShell('input swipe 1000 1000 100 1000').exec();
await aui.execOnShell('input swipe 1000 1000 100 1000').exec();

‍

2. Datepicker Tab

We continue by filling out text fields and selecting dates using the date pickers.

‍

App Screen:

Test Code:

Fill Title & Description:

await aui.click().text().withText('Title').exec();
await aui.type('My vacation plan').exec();

await aui.click().text().withText('Description').exec();
await aui.type('0. Drink a lot of water').exec();

await aui.pressAndroidKey('tab').exec();

Select Departure Date:

await aui.click().text().withText('edit').nearestTo().text().withText('Departure').exec();
await aui.click().icon().withText('chevron right').exec();
await aui.click().icon().withText('chevron right').exec();
await aui.click().text().withText('7').exec();
await aui.click().text().withText('ok').exec();

Select Return Date:

await aui.click().text().withText('edit').nearestTo().text().withText('Return').exec();
await aui.click().icon().withText('chevron right').exec();
await aui.click().icon().withText('chevron right').exec();
await aui.click().icon().withText('chevron right').exec();
await aui.click().text().withText('5').exec();
await aui.click().text().withText('ok').exec();

Interact with Checkbox and Switch:

await aui.click().checkboxUnchecked().nearestTo().text().withText('Brushed Teeth').exec();
await aui.click().switchDisabled().nearestTo().text().withText('Enable feature').exec();
await aui.execOnShell('input swipe 1000 1000 100 1000').exec();

‍

3. Camera Tab

Finally, we switch to the Camera tab and trigger the camera interaction.

App Screen:

Test Code:

await aui.click().button().contains().text().withText('Take a Picture').exec();
await aui.click().icon().containsText('circle').exec();

‍

4. Complete Test Code

This full code block integrates all above steps for a complete end-to-end automation.

import { aui } from './helper/jest.setup';

describe('jest with askui', () => {

    it('should fill up the textfields and push buttons', async () => {
        await aui.click().text().withText('Enter your username').exec();
        await aui.type('askui').exec();
        await aui.click().text().withText('Enter your email').exec();
        await aui.type('askui@askui.com').exec();
        await aui.click().text().withText('Enter your address').exec();
        await aui.type('Haid-und-Neu-Straße 18').exec();
        await aui.pressAndroidKey('enter').exec();
        await aui.click().text().withText('Submit').exec();
        await aui.click().text().withText('Refuse').exec();
        await aui.click().text().withText('Banana').exec();
        await aui.click().text().withText('Mango').exec();
        await aui.click().text().withText('Sunny').exec();
        await aui.click().text().withText('Rainy').exec();
        await aui.click().text().withText('Windy').exec();
        await aui.execOnShell('input swipe 1000 1000 100 1000').exec();
        await aui.execOnShell('input swipe 1000 1000 100 1000').exec();
    });

    it('should pick the dates', async () => {
        await aui.click().text().withText('Title').exec();
        await aui.type('My vacation plan').exec();
        await aui.click().text().withText('Description').exec();
        await aui.type('0. Drink a lot of water').exec();
        await aui.pressAndroidKey('tab').exec();
        await aui.click().text().withText('edit').nearestTo().text().withText('Departure').exec();
        await aui.click().icon().withText('chevron right').exec();
        await aui.click().icon().withText('chevron right').exec();
        await aui.click().text().withText('7').exec();
        await aui.click().text().withText('ok').exec();
        await aui.click().text().withText('edit').nearestTo().text().withText('Return').exec();
        await aui.click().icon().withText('chevron right').exec();
        await aui.click().icon().withText('chevron right').exec();
        await aui.click().icon().withText('chevron right').exec();
        await aui.click().text().withText('5').exec();
        await aui.click().text().withText('ok').exec();
        await aui.click().checkboxUnchecked().nearestTo().text().withText('Brushed Teeth').exec();
        await aui.click().switchDisabled().nearestTo().text().withText('Enable feature').exec();
        await aui.execOnShell('input swipe 1000 1000 100 1000').exec();
    });

    it('should take a picture', async () => {
        await aui.click().button().contains().text().withText('Take a Picture').exec();
        await aui.click().icon().containsText('circle').exec();
    });

});

Conclusion

After following this tutorial, you can fully automate Flutter mobile apps with AskUI. While this demo uses Flutter, the approach works for any Android app. AskUI’s visual automation removes fragile selectors, making mobile UI automation stable, adaptive, and AI-powered.

👉 Join AskUI Community to share your cases or get help.

‍

‍