AskUI Blog Categorie

Follow the askui blog for product news, feature announcements and technical posts about automation.

Headline Common Terms in Computer Vision written over blue sky from someone standing in the desert on a road leading to mountains.

Common Terms in Computer Vision

When you read about Artificial Intelligence, Machine Learning and Computer Vision you often come across terms that seem to be common. But do you know all of them and what they actually mean? In this blog we will list the most common terms that we use in our day-to-day work.

Johannes Dienst
June 12, 2024
A person in a sitting yoga pose with arms above head watching the sunrise. The person is center of the picture. Headline is expect() and and() and or() subtitles are on the left and right of the person.

How to expect() - The Pitfalls

Selecting elements visually is pretty cool and all. But once you start to build a serious workflow it is not enough. You also need: Checking if elements are actually there. Check for multiple elements to be there. Check if either a element is there or another one is there. This is where expect(), and() and or() get on stage to shine.

Johannes Dienst
May 29, 2024
Five filled teabags in different shapes and sizes hanging in the air with their strings reaching outside the top of the image. The background is black and a title text Convenience Methods Update 0.18.0 in the foreground.

Convenience Methods - Update TypeScript ADK 0.18.0

In an earlier blog post, we explained how convenience methods could be used to write more expressive AskUI code. Those were based on common use cases we noticed and already brought improvements. But we also discovered they needed to be more streamlined and include relations, to be useful in every scenario. That is why we returned to the drawing board to redesign the API and include more elements like switches and checkboxes. This update will be beneficial for your efficiency!

Johannes Dienst
May 10, 2024
AskUI Logo with text AskUI underneath on the left side. Cucumber Logo with text Cucumber on the right side of it. Both divided by a diagonal line. Left is turquoise and right side is rose.

Use AskUI and Cucumber Together

By defining the behavior of a system in a structured format like Gherkin, Behavior-Driven Development (BDD) enables teams to bridge the gap between stakeholders, testers, and developers, avoiding misunderstandings and reducing rework. In this blog post, we'll show you how to set up Cucumber in conjunction with AskUI to define AskUI workflows using BDD principles.

Johannes Dienst
April 29, 2024
Tablet with diagram left. Clipboard with XRAY logo on the right. Apple keyboard top right. Title in the bottom reading JSON Report.

Create XRAY JSON Report And Upload it via API

When you use Jira as a testing team it is highly likely that you also use XRAY for testmanagement. So when you run UI-tests with AskUI you want to integrate the test run reports into xray automatically on each run. With the latest version of `askui-reporters` we added the `AskUIXRayStepReporter` that can create XRAY-proprietory-json reports, which offers step-level-reporting with screenshots!

Johannes Dienst
April 22, 2024
Left side AskUI logo with subtitle AskUI. Right side Playwright logo with title Playwright on the top. Background is left side teal colored right side is rose colored.

Use Playwright and AskUI Together

Playwright is the de facto standard when it comes to reliably testing browser applications across browsers and operating systems. But what if your workflow under test leaves your browser and requires you to automate on a desktop application? AskUI is not confined to a specific application because it uses visual selectors and real human-user-like input with keypresses and natural mouse movement. This makes it an ideal tool to accompany Playwright.

Johannes Dienst
April 22, 2024
Left side AskUI logo with subtitle AskUI. Right side Selenium logo with title Selenium on the top. Background is left side teal colored right side is rose colored.

Use Selenium and AskUI Together

When you automate browser applications there is no way around an automation tool as Selenium. It is optimized for browser interaction and executes blazingly fast. However sometimes you need to leave the confines of your browser to interact with another application or your operating system. This is not possible with Selenium alone.AskUI shines in this regard as it is not limited to a browser with its visual selectors and real input commands like keypresses and mouse movement.

Johannes Dienst
April 10, 2024
Five filled teabags in different shapes and sizes hanging in the air with their strings reaching outside the top of the image. The background is black and a title text Convenience Methods is in the foreground.

Better AskUI Workflows with Convenience Methods in AskUI SDK (ADK)

AskUI interacts with your operating system like a real human user. This directly leads to code that specifies WHAT you have to do in a specific sequence to achieve a workflow. Unfortunately this also means a lot of repetitive code. This is why we implemented some convenience methods to help you ease these pain points.

Johannes Dienst
March 13, 2024

Developing an Automated UI Controller using GPT Agents and GPT-4 Vision

The emergence of Large Language Models (LLMs) like ChatGPT has ushered in a new era in text generation and AI advancements. While tools such as AutoGPT have aimed at task automation, their reliance on underlying DOM/HTML code poses challenges for desktop applications built on .NET/.SAP. The GPT-4 Vision (GPT-4v) API addresses this limitation by focusing on visual inputs, eliminating the need for HTML code.

Murali Kondragunta
February 23, 2024