When you use Jira as a testing team it is highly likely that you also use XRAY for testmanagement. So when you run UI-tests with AskUI you want to integrate the test run reports into xray automatically on each run.
With the latest version of askui-reporters we added the AskUIXRayStepReporter that can create XRAY-proprietory-json reports, which offers step-level-reporting with screenshots!
In this post we will show you how to setup this reporter and get the reports into your Jira/XRAY via REST-API.
Prerequisites
Installation and Configuration of the AskUIXRayStepReporter
The AskUIXRayStepReporter already comes bundled with the latest AskUI version. But by default our Allure Reporter is active.
In the following steps we will show you how to activate and configure the AskUIXRayStepReporter instead.
Enable and Configure the AskUIXRayStepReporter in `helper/askui-helper.ts`
First you have to modify your helper/askui-helper.ts file to use the AskUIXRayStepReporter.
Afterwards it should look like this.
**Do not forget to change <your workspace id> and <your access token>!**
-- CODE language-ts line-numbers --
import { UiControlClient, UiController } from 'askui';
import { AskUIXRayStepReporter } from '@askui/askui-reporters';
// Client is necessary to use the askui API
let aui: UiControlClient;
jest.setTimeout(60 * 1000 * 60);
let xRayReporter: AskUIXRayStepReporter;
beforeAll(async () => {
xRayReporter = new AskUIXRayStepReporter({
// Uncomment the next line if you want screenshots also when steps pass
// withScreenshots: 'always',
});
aui = await UiControlClient.build({
credentials: {
workspaceId: '',
token: '',
},
reporter: xRayReporter,
});
await aui.connect();
});
beforeEach(async () => {
xRayReporter.createNewTestEntry(global.testName);
});
afterEach(async () => {
xRayReporter.finishTestEntry(global.testStatus);
});
afterAll(async () => {
await xRayReporter.writeReport();
aui.disconnect();
});
export { aui };
Enable the `jest-xray-environment` in `jest.config.ts`
Our Reporter has to use a special jest-environment, so it can include the name of each test from the it-block in the report.json. You configure the jest-environment in jest.config.ts like this.
-- CODE language-ts line-numbers --
import type { Config } from '@jest/types';
const config: Config.InitialOptions = {
preset: 'ts-jest',
testEnvironment: '@askui/askui-jest-xray-environment',
setupFilesAfterEnv: ['./helpers/askui-helper.ts'],
sandboxInjectedGlobals: [
'Math',
],
reporters: [ "default" ]
};
// eslint-disable-next-line import/no-default-export
export default config;
An Example Report
Here is an example report.json
from our example repository. As you can see the testKey
property is the description from the it
-block. Also in this case we configured the xRayReporter
in helper/askui-helper.ts
to alway take screenshots instead of only when a step failed.
-- CODE language-ts line-numbers --
xRayReporter = new AskUIXRayStepReporter({
withScreenshots: 'always',
});
Full report:
-- CODE language-ts line-numbers --
{
"tests": [
{
"testKey": "TEST1",
"start": "2024-01-10T15:17:05.998Z",
"steps": [
{
"status": "PASS",
"evidences": [
{
"data": "iVBORw0KGgoAAAANSUhEUgAAC ...",
"filename": "before.png",
"contentType": "image/png"
},
{
"data": "AAAAesCAYAAACEMRk8ABJYYklEQVR ...",
"filename": "after.png",
"contentType": "image/png"
}
]
}
],
"finish": "2024-01-10T15:17:10.852Z",
"status": "FAIL"
}
]
}
Upload over XRAY REST-API
Getting a XRAY-compatible JSON is nice, but useless without pushing it to XRAY 😉. The recommended way is to use XRAYs-REST-API.
In this Gitlab-CI-example the first job run-askui-job
runs your tests inside a Gitlab-Pipeline and makes the report availaible under xray-report/report.json
.
The next job transfer-report-to-xray-job
requests a token from the Jira/XRAY-Server and then uploads the report via API.
Do not forget to set the Gitlab-Variables for your Jira/XRAY-Instance: CLIENT_ID, CLIENT_SECRET, JIRA_SERVER_URL, PROJECT_KEY
-- CODE language-yaml line-numbers --
# Example how to make xray-report/report.json available
run-askui-job:
script:
- npm run askui
artifacts:
paths:
- xray-report/report.json
expire_in: 1 week
# Expects the file xray-report/report.json to be
# made available through another step
transfer-report-to-xray-job:
script:
- echo "Transfer report to XRAY"
- token=$(curl -H "Content-Type: application/json" -X POST --data '{ "client_id": "${CLIENT_ID}","client_secret": "${CLIENT_SECRET}" }' $JIRA_SERVER_URL/api/v2/authenticate)
- # XRAY-JSON
- curl -H "Content-Type: application/json" -X POST -H "Authorization: Bearer $token" --data @"xray-report/report.json" "${JIRA_SERVER_URL}/api/v2/import/execution"
- echo "done"
variables:
CLIENT_ID: $CLIENT_ID
CLIENT_SECRET: $CLIENT_SECRET
JIRA_SERVER_URL: $JIRA_SERVER_URL
PROJECT_KEY: $PROJECT_KEY
needs:
- run-askui-job
Conclusion
With our AskUIXRayStepReporter
you can create XRAY-compatible JSON with stepreporting and screenshots on fail-only/always. For a complete example including all the setup files, please check our XRAY reporter example repository on Github.