Integrating Video Recording and Custom Reporters into AskUI Runs

Debugging UI-Automations can be a real pain especially when you run them inside a pipeline with no chance to observe what is actually happening during execution.

Wouldn't it be nice to get a report of failed automation steps that include screenshot before and after an execution and also a video?

We've recently added video recording support to our UI controller, enabling you to record a video for each step you execute and add it to a report of your AskUI runs.

We also provide an interface you can implement so you can integrate it into your favorite reporting tool like for example Allure.

Implementing A Reporter

To help you get started, we've created an example repository for Allure and also a repository where we collect reporter implementations.

Implementing your own reporter involves two main steps:

Implementing the reporter interface
Configuring the reporter within the jest.setup.ts.

Depending on your chosen reporter framework, you may not need to implement every function in the interface.

-- CODE language-ts line-numbers -- import { Step } from './step'; import { ReporterConfig } from './reporter-config'; /** * Can be used to report on a step run. * * The reporter is an interface (instead of a class) * so that the it may be implemented by a class * that already extends another class, e.g., the * reporter of a test framework. */ export interface Reporter { config?: ReporterConfig; onStepBegin?(step: Step): Promise; onStepRetry?(step: Step): Promise; onStepEnd?(step: Step): Promise; }

For instance, our Allure reporter only includes the config and onStepEnd() functions.

-- CODE language-ts line-numbers -- import { Reporter, Step, StepStatus } from "askui"; import { Status } from "jest-allure-circus"; function convertPngDataUrlToBuffer(pngDataUrl: string): Buffer { const base64Data = pngDataUrl.replace(/^data:image\/png;base64,/, ""); const buffer = Buffer.from(base64Data, "base64"); return buffer; } function mapAskuiToAllureStepStatus(status: StepStatus): Status { switch (status) { case "passed": return Status.PASSED; case "failed": return Status.FAILED; case "erroneous": return Status.BROKEN; default: return Status.SKIPPED; } } export class askuiAllureStepReporter implements Reporter { config = { withScreenshots: 'always' as const, }; async onStepEnd(step: Step): Promise { const status = mapAskuiToAllureStepStatus(step.status); const attachments = []; if (step.lastRun?.begin?.screenshot !== undefined) { attachments.push({ name: "Before Screenshot", type: "image/png", content: convertPngDataUrlToBuffer(step.lastRun?.begin?.screenshot), }); } if (step.lastRun?.end?.screenshot !== undefined) { attachments.push({ name: "After Screenshot", type: "image/png", content: convertPngDataUrlToBuffer(step.lastRun?.end?.screenshot), }); } allure.logStep( step.instruction.valueHumanReadable, status, attachments, ); } }

Optionally, you can provide mappings for the steps of __Jest__ to your reporter framework, tailoring the integration to your preferences.

To integrate video recording into your reporting tool, configure your reporter in the jest.setup.ts file within the UIControlClient. Additionally, extract the recorded video from the UIController and add it to your reporter.

Here's an example of how that is done:

-- CODE language-ts line-numbers -- ... aui = await UiControlClient.build({ inferenceServerUrl: process.env["ASKUI_INFERENCE_SERVER_URL"] ?? "https://inference-dev.askui.com", reporter: new askuiAllureStepReporter(), }); await aui.connect(); }); beforeEach(async () => { await aui.startVideoRecording(); }); afterEach(async () => { await aui.stopVideoRecording(); const video = await aui.readVideoRecording(); allure.createAttachment( "Video", convertBase64StringToBuffer(video), ContentType.WEBM ); });

Allure Example

Our Allure example repository also publishes its report to a website: Check it out here!

‍

Links for Already Implemented Reporters

We've created a community repository on our GitHub space where you can find and contribute implemented reporters. If you need inspiration or want to share your custom reporter with the community this is the right place to start!

All reporters and Instructions to implement your own

Conclusion

Incorporating screenshots and videos into your executed steps reports can significantly enhance your reports, as demonstrated in the provided example.

If a specific reporter is not implemented already, you can implement it on your own and integrate it with the instructions and examples given in our repositories.

If you need help, please join our Discord where the AskUI staff and the community are ready to help you out!

More to explore

tutorials

Automate Highlighting in WYSIWYG Editors with AskUI

Working in What-You-See-Is-What-You-Get editors (WYSIWYG) can be very tiring and also taxing on your wrists. Things like highlighting the same thing over and over again can often not be done with search-and-replace, because there is just no button for it 😤 In this tutorial I will show you how I automated this task with AskUI and also how to work around some kinks of WYSIWIG-Editors like microanimations and strange behaviour.

tutorials

Automate Your WebGL Application in 7 Easy Steps

Embrace Immersive and Interactive User Experiences: A Guide to Automating WebGL Websites with AskUI As web developers increasingly seek to captivate users with immersive experiences, WebGL-based websites have gained popularity. By harnessing the power of canvas to draw graphics and animations directly, they offer unparalleled creative control and dynamism in crafting user interfaces. The result? A more flexible and engaging platform for stunning graphical compositions, delivering real-time interactions that captivate visitors.

tutorials

Visual Regression Testing with AskUI and Jest

Visual Regression Testing (VRT) is a useful method to guard yourself against unwanted layout and visual changes that can occur with new version of a Graphical User Interface (GUI).AskUI is not a testing tool in the original sense but an automation tool for all GUIs that is platform-independent. Wouldn't it be nice to combine VRT with AskUI, so you do not have to maintain two different projects.Luckily you can do this because Jest - AskUIs favourite runner - can be extended to provide VRT, while AskUI delivers the screenshots to test on.

Get in touch

For media queries, drop us a message at info@askui.com

Download Press Kit