Automate Multiple Devices

This tutorial shows how to automate multiple devices on the same network by using the askui library. After following this tutorial, you will be able to automate more than one device at the same time across different platforms, whether Linux, macOS, Windows, or Android, with a single setup with the askui library.

In fact, there are many automation tools in the wild which you can use to automate different devices, although most of them require different configurations and different test code for different platforms. By using the askui library, an automation tool that operates on the OS level, you can simply use the same test code for any application running on different devices even without so much change. And this makes the askui a powerful Cross-platform automation tool.

Let's have a look at the askui library and see how we can accomplish a Cross-platform/-device automation test🔥

‍

📌 The following tutorial assumes that you have already installed and set up the askui library on your local device. The code for the configuration is based on the generated test code: npx askui init. See Getting Started for more details.

‍

1. Download and Prepare the askui-ui-controller Binary for Each Device

If you already used the askui library once, then the binary for your platform already exists in the node_modules/ directory, as the binary gets automatically downloaded if an instance of UiController gets initialized. See here for more details about the askui UI Controller.
Follow this path and confirm that the binary exists:

-- CODE language-bash line-numbers -- # Windows node_modules/askui/dist/release/latest/win32/askui-ui-controller.exe # macOS node_modules/askui/dist/release/latest/darwin/askui-ui-controller.app/Contents/MacOS/askui-ui-controller # Linux node_modules/askui/dist/release/latest/linux/askui-ui-controller

‍

If the remote device runs the same platform as the local device, simply copy the binary from the local device to the remote device.
If the remote device runs a different platform, then download the binary for the respective platform: Windows | macOS(intel) | macOS(silicon) | Linux

Android Only:

There is no need to save the binary to Android devices. They are controlled by the askui UI Controller running on the local device (desktop).
Be sure that your Android device is discoverable by the ADB from your local device:

-- CODE language-bash line-numbers -- # Run this command to confirm that your Android device is discoverable adb devices

If you don't have the ADB installed on your local device, set it up by following this tutorial.
Use the commands below, if you want to connect your Android device via ADB wirelessly:

-- CODE language-bash line-numbers -- # Make sure that the `USB Debugging Mode` is enabled in the Android device. # Connect the Android device with a USB cable, and run this command: adb devices # will print the "device-id" adb -s "device-id" tcpip 9000 # replace "device-id" with your device-id adb -s "device-id" connect "local-ip-address":9000 # replace the "local-ip-address" # Run this command, if you want to check the local ip address of the android device adb -s "device-id" shell ip -f inet addr show wlan0 # Now you can disconnect the USB cable from the Android device. # Confirm that the Android device is recognised by adb wirelessly. adb devices

2. Configure the jest.setup.ts

Figure out the local ip address of the remote device with the following command:

-- CODE language-bash line-numbers -- # Windows only ipconfig /all # macOS only ipconfig getifaddr en0 # Linux only hostname -I

Change the IP address <local-ip-address> in the test/helper/jest.setup.ts:

-- CODE language-ts line-numbers -- import { UiControlClient, UiController } from 'askui'; let localDevice: UiControlClient; let remoteDevice: UiControlClient; jest.setTimeout(60 * 1000 * 60); beforeAll(async () => { // Get your askui credentials from https://app.v2.askui.com/workspaces const credentials = { workspaceId: '', token: '', } // This client will communicate with // the controller running on this local device. localDevice = await UiControlClient.build({ uiControllerUrl: "ws://127.0.0.1:6769", credentials: credentials, }); await localDevice.connect(); // This client will communicate with // the controller running on the remote device. // Replace the // In case of Android device, replace it with 127.0.0.1 remoteDevice = await UiControlClient.build({ uiControllerUrl: "ws://:7001", credentials: credentials, }); await remoteDevice.connect(); }); afterAll(async () => { localDevice.close(); remoteDevice.close(); }); export { localDevice, remoteDevice };

3. Run the Controller on Each Device

Run the binary askui UI Controller on the local and remote devices with the following command:

-- CODE language-bash line-numbers -- ## Note the port number given for the Remote device here ## must correspond to the one given in the above config ## ------- On the Local device ------- # Windows only askui-ui-controller.exe --host 0.0.0.0 -d 0 -m # macOS / Linux only askui-ui-controller --host 0.0.0.0 -d 0 -m ## ------- On the Remote device ------- # Windows only askui-ui-controller.exe --host 0.0.0.0 -d 0 -m -p 7001 # macOS / Linux only askui-ui-controller --host 0.0.0.0 -d 0 -m -p 7001

If running successfully, you should see the logs printed on the terminal, e.g:
‍

[2023-01-02 17:31:19.634 +0100] DEBUG (AskuiUiController): Window is minimized.
[2023-01-02 17:31:19.639 +0100] INFO (AskuiUiController): Selecting display number 0.
[2023-01-02 17:31:19.641 +0100] INFO (AskuiUiController): Successfully started.

Android Only:

If your remote device is an Android device, run the askui UI Controller on the local device (desktop) with an extra option as below:

-- CODE language-bash line-numbers -- # Windows only askui-ui-controller.exe --host 0.0.0.0 -d 0 -m -p 7001 -r android # macOS / Linux only askui-ui-controller --host 0.0.0.0 -d 0 -m -p 7001 -r android

Make sure that your local device (desktop) is running TWO DIFFERENT askui-ui-controller, if you want to control the local device and the Android device at the same time.

(option) Running Multiple Android Devices

The askui UI Controller considers all the Android devices as a single device with multiple displays. To see the GUI of the display selection, set the minimize option(-m) to false from the commands below. The screenshot below shows what it looks like.
Note that you have to create another instance of UiControlClient in the test/helper/jest.setup.ts with the respective ip:port that is given to the android device.
Set the display number (-d option) of the binary differently for each Android device.
Set the port number (-p option) of the binary differently for each Android device.

-- CODE language-bash line-numbers -- # Windows only # For the first Android device askui-ui-controller.exe --host 0.0.0.0 -d 0 -m -p 7001 -r android # For the second Android device askui-ui-controller.exe --host 0.0.0.0 -d 1 -m -p 7002 -r android # macOS / Linux only # For the first Android device askui-ui-controller --host 0.0.0.0 -d 0 -m -p 7001 -r android # For the second Android device askui-ui-controller --host 0.0.0.0 -d 1 -m -p 7002 -r android

4. Write the Automation Code

Write the test code in test/my-first-askui-test-suite.test.ts:

-- CODE language-ts line-numbers -- import { localDevice, remoteDevice } from './helper/jest.setup'; describe('jest with askui', () => { it('should work with multiple devices', async () => { const everyElement_remote = await remoteDevice.getAll().exec(); console.log(everyElement_remote); await localDevice.moveMouse(500,500).exec(); const everyElement_local = await localDevice.getAll().exec(); console.log(everyElement_local); }); });

5. Run the Automation Code

Run the command below to run the askui test code:

-- CODE language-bash line-numbers -- npx jest test/my-first-askui-test-suite.test.ts --config ./test/jest.config.ts

6. Conclusion

Now you should be able to automate multiple devices in the network. If you got any issues while following this tutorial, don't hesitate to ask our Discord Community!

‍

More to explore

tutorials

Try askui in Your Browser With Gitpod

askui controls your operating system (OS) through mouse movement, mouseclicks, keypresses and screencapture.Thus it can be a pain to install it on your local machine. Not everything might work out of the box or there are simply permissions missing (looking at you macOS 🫣).But we have a solution we will show you in this blog post!Our GitHub repository has a ready-to-go Gitpod configuration inside where you can try out askui for free in your browser without any installation issues.All you need is a GitHub account and a Gitpod account 🦄

tutorials

How to Use Custom Elements in askui

Custom Element Selection is a feature in askui that enables you to create custom selectors for elements on the screen, instead of relying on the standard selectors provided such as Button, Textfield, etc. .Here we will demonstrate how to use a custom element to explore Google Street View.

tutorials

Automate Github 2-Factor Authentication with Android Device

Today, we're building a two-factor authentication GitHub workflow automation with a browser we have on our Personal Computer (PC) and a physical Android device.

Get in touch

For media queries, drop us a message at info@askui.com

Download Press Kit