Create XRAY JSON Report And Upload it via API

April 22, 2024
Tablet with diagram left. Clipboard with XRAY logo on the right. Apple keyboard top right. Title in the bottom reading JSON Report.
linkedin icontwitter iconfacebook iconmail icon

When you use Jira as a testing team it is highly likely that you also use XRAY for testmanagement. So when you run UI-tests with AskUI you want to integrate the test run reports into xray automatically on each run.

With the latest version of askui-reporters we added the AskUIXRayStepReporter that can create XRAY-proprietory-json reports, which offers step-level-reporting with screenshots!

In this post we will show you how to setup this reporter and get the reports into your Jira/XRAY via REST-API.


Installation and Configuration of the AskUIXRayStepReporter

The AskUIXRayStepReporter already comes bundled with the latest AskUI version. But by default our Allure Reporter is active.

In the following steps we will show you how to activate and configure the AskUIXRayStepReporter instead.

Enable and Configure the AskUIXRayStepReporter in `helper/askui-helper.ts`

First you have to modify your helper/askui-helper.ts file to use the AskUIXRayStepReporter.

Afterwards it should look like this.

**Do not forget to change <your workspace id> and <your access token>!**

-- CODE language-ts line-numbers -- import { UiControlClient, UiController } from 'askui'; import { AskUIXRayStepReporter } from '@askui/askui-reporters'; // Client is necessary to use the askui API let aui: UiControlClient; jest.setTimeout(60 * 1000 * 60); let xRayReporter: AskUIXRayStepReporter; beforeAll(async () => { xRayReporter = new AskUIXRayStepReporter({ // Uncomment the next line if you want screenshots also when steps pass // withScreenshots: 'always', }); aui = await{ credentials: { workspaceId: '', token: '', }, reporter: xRayReporter, }); await aui.connect(); }); beforeEach(async () => { xRayReporter.createNewTestEntry(global.testName); }); afterEach(async () => { xRayReporter.finishTestEntry(global.testStatus); }); afterAll(async () => { await xRayReporter.writeReport(); aui.disconnect(); }); export { aui };

Enable the `jest-xray-environment` in `jest.config.ts`

Our Reporter has to use a special jest-environment, so it can include the name of each test from the it-block in the report.json. You configure the jest-environment in jest.config.ts like this.

-- CODE language-ts line-numbers -- import type { Config } from '@jest/types'; const config: Config.InitialOptions = { preset: 'ts-jest', testEnvironment: '@askui/askui-jest-xray-environment', setupFilesAfterEnv: ['./helpers/askui-helper.ts'], sandboxInjectedGlobals: [ 'Math', ], reporters: [ "default" ] }; // eslint-disable-next-line import/no-default-export export default config;

An Example Report

Here is an example report.json from our example repository. As you can see the testKey property is the description from the it-block. Also in this case we configured the xRayReporter in helper/askui-helper.ts to alway take screenshots instead of only when a step failed.

-- CODE language-ts line-numbers -- xRayReporter = new AskUIXRayStepReporter({ withScreenshots: 'always', });

Full report:

-- CODE language-ts line-numbers -- { "tests": [ { "testKey": "TEST1", "start": "2024-01-10T15:17:05.998Z", "steps": [ { "status": "PASS", "evidences": [ { "data": "iVBORw0KGgoAAAANSUhEUgAAC ...", "filename": "before.png", "contentType": "image/png" }, { "data": "AAAAesCAYAAACEMRk8ABJYYklEQVR ...", "filename": "after.png", "contentType": "image/png" } ] } ], "finish": "2024-01-10T15:17:10.852Z", "status": "FAIL" } ] }

Upload over XRAY REST-API

Getting a XRAY-compatible JSON is nice, but useless without pushing it to XRAY 😉. The recommended way is to use XRAYs-REST-API.

In this Gitlab-CI-example the first job run-askui-job runs your tests inside a Gitlab-Pipeline and makes the report availaible under xray-report/report.json.

The next job transfer-report-to-xray-job requests a token from the Jira/XRAY-Server and then uploads the report via API.

Do not forget to set the Gitlab-Variables for your Jira/XRAY-Instance: CLIENT_ID, CLIENT_SECRET, JIRA_SERVER_URL, PROJECT_KEY

-- CODE language-yaml line-numbers -- # Example how to make xray-report/report.json available run-askui-job: script: - npm run askui artifacts: paths: - xray-report/report.json expire_in: 1 week # Expects the file xray-report/report.json to be # made available through another step transfer-report-to-xray-job: script: - echo "Transfer report to XRAY" - token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "${CLIENT_ID}","client_secret": "${CLIENT_SECRET}" }' $JIRA_SERVER_URL/api/v2/authenticate) - # XRAY-JSON - curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" --data @"xray-report/report.json" "${JIRA_SERVER_URL}/api/v2/import/execution" - echo "done" variables: CLIENT_ID: $CLIENT_ID CLIENT_SECRET: $CLIENT_SECRET JIRA_SERVER_URL: $JIRA_SERVER_URL PROJECT_KEY: $PROJECT_KEY needs: - run-askui-job


With our AskUIXRayStepReporter you can create XRAY-compatible JSON with stepreporting and screenshots on fail-only/always. For a complete example including all the setup files, please check our XRAY reporter example repository on Github.

Johannes Dienst
April 22, 2024
On this page