Create XRAY JSON Report And Upload it via API

When you use Jira as a testing team it is highly likely that you also use XRAY for testmanagement. So when you run UI-tests with AskUI you want to integrate the test run reports into xray automatically on each run.

With the latest version of askui-reporters we added the AskUIXRayStepReporter that can create XRAY-proprietory-json reports, which offers step-level-reporting with screenshots!

In this post we will show you how to setup this reporter and get the reports into your Jira/XRAY via REST-API.

Prerequisites

AskUI installed and configured on your system (Windows, Linux, macOS)

Installation and Configuration of the AskUIXRayStepReporter

The AskUIXRayStepReporter already comes bundled with the latest AskUI version. But by default our Allure Reporter is active.

In the following steps we will show you how to activate and configure the AskUIXRayStepReporter instead.

Enable and Configure the AskUIXRayStepReporter in `helper/askui-helper.ts`

First you have to modify your helper/askui-helper.ts file to use the AskUIXRayStepReporter.

Afterwards it should look like this.

‍**Do not forget to change <your workspace id> and <your access token>!**

-- CODE language-ts line-numbers -- import { UiControlClient, UiController } from 'askui'; import { AskUIXRayStepReporter } from '@askui/askui-reporters'; // Client is necessary to use the askui API let aui: UiControlClient; jest.setTimeout(60 * 1000 * 60); let xRayReporter: AskUIXRayStepReporter; beforeAll(async () => { xRayReporter = new AskUIXRayStepReporter({ // Uncomment the next line if you want screenshots also when steps pass // withScreenshots: 'always', }); aui = await UiControlClient.build({ credentials: { workspaceId: '', token: '', }, reporter: xRayReporter, }); await aui.connect(); }); beforeEach(async () => { xRayReporter.createNewTestEntry(global.testName); }); afterEach(async () => { xRayReporter.finishTestEntry(global.testStatus); }); afterAll(async () => { await xRayReporter.writeReport(); aui.disconnect(); }); export { aui };

Enable the `jest-xray-environment` in `jest.config.ts`

Our Reporter has to use a special jest-environment, so it can include the name of each test from the it-block in the report.json. You configure the jest-environment in jest.config.ts like this.

-- CODE language-ts line-numbers -- import type { Config } from '@jest/types'; const config: Config.InitialOptions = { preset: 'ts-jest', testEnvironment: '@askui/askui-jest-xray-environment', setupFilesAfterEnv: ['./helpers/askui-helper.ts'], sandboxInjectedGlobals: [ 'Math', ], reporters: [ "default" ] }; // eslint-disable-next-line import/no-default-export export default config;

An Example Report

Here is an example report.json from our example repository. As you can see the testKey property is the description from the it-block. Also in this case we configured the xRayReporter in helper/askui-helper.ts to alway take screenshots instead of only when a step failed.

-- CODE language-ts line-numbers -- xRayReporter = new AskUIXRayStepReporter({ withScreenshots: 'always', });

Full report:

-- CODE language-ts line-numbers -- { "tests": [ { "testKey": "TEST1", "start": "2024-01-10T15:17:05.998Z", "steps": [ { "status": "PASS", "evidences": [ { "data": "iVBORw0KGgoAAAANSUhEUgAAC ...", "filename": "before.png", "contentType": "image/png" }, { "data": "AAAAesCAYAAACEMRk8ABJYYklEQVR ...", "filename": "after.png", "contentType": "image/png" } ] } ], "finish": "2024-01-10T15:17:10.852Z", "status": "FAIL" } ] }

Upload over XRAY REST-API

Getting a XRAY-compatible JSON is nice, but useless without pushing it to XRAY 😉. The recommended way is to use XRAYs-REST-API.

In this Gitlab-CI-example the first job run-askui-job runs your tests inside a Gitlab-Pipeline and makes the report availaible under xray-report/report.json.

The next job transfer-report-to-xray-job requests a token from the Jira/XRAY-Server and then uploads the report via API.

Do not forget to set the Gitlab-Variables for your Jira/XRAY-Instance: CLIENT_ID, CLIENT_SECRET, JIRA_SERVER_URL, PROJECT_KEY

-- CODE language-yaml line-numbers -- # Example how to make xray-report/report.json available run-askui-job: script: - npm run askui artifacts: paths: - xray-report/report.json expire_in: 1 week # Expects the file xray-report/report.json to be # made available through another step transfer-report-to-xray-job: script: - echo "Transfer report to XRAY" - token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "${CLIENT_ID}","client_secret": "${CLIENT_SECRET}" }' $JIRA_SERVER_URL/api/v2/authenticate) - # XRAY-JSON - curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" --data @"xray-report/report.json" "${JIRA_SERVER_URL}/api/v2/import/execution" - echo "done" variables: CLIENT_ID: $CLIENT_ID CLIENT_SECRET: $CLIENT_SECRET JIRA_SERVER_URL: $JIRA_SERVER_URL PROJECT_KEY: $PROJECT_KEY needs: - run-askui-job

Conclusion

With our AskUIXRayStepReporter you can create XRAY-compatible JSON with stepreporting and screenshots on fail-only/always. For a complete example including all the setup files, please check our XRAY reporter example repository on Github.

More to explore

academy

Use AskUI and Cucumber Together

By defining the behavior of a system in a structured format like Gherkin, Behavior-Driven Development (BDD) enables teams to bridge the gap between stakeholders, testers, and developers, avoiding misunderstandings and reducing rework. In this blog post, we'll show you how to set up Cucumber in conjunction with AskUI to define AskUI workflows using BDD principles.

academy

Better AskUI Workflows with Convenience Methods in AskUI SDK (ADK)

AskUI interacts with your operating system like a real human user. This directly leads to code that specifies WHAT you have to do in a specific sequence to achieve a workflow. Unfortunately this also means a lot of repetitive code. This is why we implemented some convenience methods to help you ease these pain points.

academy

Use Playwright and AskUI Together

Playwright is the de facto standard when it comes to reliably testing browser applications across browsers and operating systems. But what if your workflow under test leaves your browser and requires you to automate on a desktop application? AskUI is not confined to a specific application because it uses visual selectors and real human-user-like input with keypresses and natural mouse movement. This makes it an ideal tool to accompany Playwright.

Get in touch

For media queries, drop us a message at info@askui.com

Download Press Kit