UI Testing remains one of the most feared challenges for business owners and companies. It can be considered one of the most difficult disciplines in software testing and has a complicated reputation. But things have changed and some fears are just myths that can be debunked easily. In this article we present the four most common myths we hear from customers.
Myth 1: Visual changes are not detected
The myth: Visual changes are not detected
The truth: So called invisible changes to your code can make your UI test crash, such as adding div elements or replacing IDs. Secondly visual changes such as button colors or text cuts are not detected. Are these elements really new though? Of course not. When you paint your bike and take it to a friend, you will be asked “did you paint your bike?” rather than “did you get a new bike?”. Truth is invisible changes to the appearance of your UI should not lead to crashes and visual changes should be detected.
The solution: Research and modern technologies. We can use deep learning and image recognition to teach artificial intelligence how to detect UI elements. As a matter of fact we already did.
Think about babies learning how objects look like. You would never teach a baby that a chair has 4 legs á 50cm height, brown color and a 30cm backrest attached to it. A baby sees hundreds of different chairs and learns about the similarities they have. Same thing applies to teaching an AI how UI elements look like. You show it hundreds of different buttons for example until it’s able to detect a button on its own. An AI can be taught to
An all-device-coverage is possible and already here. With AskUI, all you need is screenshots of the application you want to test. No matter which format, color or device you are using and how often you change it – our AI will detect it.
Myth 2: UI Testing is expensive
The myth: UI Testing is too expensive compared to its use.
The truth: If you’re a small business owner you would take this myth for a fact. The status quo of UI testing is awful. There are three main factors that cause UI testing to be as expensive as it is right now. The first one is the cost of maintainability. This includes the cost of fixing your test code and instructions when your UI changes. Second is the cost of developers who have to be paid salaries not only to write your tests but also to maintain your tools and frameworks. Talking about these, thirdly we have the hosting cost. This not only includes the cost of using an internal or external server structure but also the cost of these tools and frameworks.
Several different frameworks and tools have to be used to achieve a somewhat acceptable coverage and all these tools and frameworks have to be administered by developers too. Setting up and maintaining these – that’s way too expensive for business owners, which is why a huge majority of companies still test their UIs manually.
Is UI testing too expensive for its use? A flawless and clean web appearance is one of the most important factors for customers. They only see the user interface of your application – the use of UI testing should never be underestimated. Testing happy paths is the minimum you have to do, but more on that later. Truth is though, we live in the 2020s and UI testing should be automated for a reasonable price.
The solution: Well what is a reasonable price for UI testing? Consider the status quo we just explained. A proper UI testing tool should cover all these expenses at once: one tester should be able to cover your entire UI testing with just one tool that can run your test in a cloud environment. If the status quo applies to your business, make sure to cut your expenses by ⅔ because one tool is enough to cover it all.
Myth 3: UI tests are slow
The myth: UI tests are slow.
The truth: There is no denying that UI tests are one of the slowest in all of software testing. As a rule of thumb, you should try to cover your UI tests in less than 60 minutes. In some cases UI tests can take several hours though. Comparing these numbers to unit tests shows why there is no denying to this myth, unit tests are estimated to take about 10ms per test and 10 seconds for the entire execution.
Just because this myth is true, doesn’t mean it is a bad thing or there is no solution to it. We already explained that there is no way around UI testing, it literally tests the most important aspects for your customer. This alone would justify longer durations. But there are some tricks and best practices to speed up your UI tests.
The solution: If you want to speed up UI tests you have to consider a risk evaluation. Which parts of your UI really need to be tested as much as possible and which parts can be covered with unit tests?
Make a list of all the happy paths in your application that have to work every time all the time. If you have an online shop, one happy path is the checkout process. Customers always need to be able to select an item in your shop, add it to the cart, order and pay for it. On the other hand, you can probably ignore static parts of your website that never change.
Another rule of thumb is the shift left approach. Everything you can automate faster with unit tests should be tested with unit tests. UI tests should be seen as a tool to test your entire aggregation, from one end to the other. Split your tests up to get the most of your time.
Myth 4: An all-device-coverage is not possible
The myth: UI tests can’t cope with the broad variety of devices and responsive web design.
The truth: It is true that most UI testing tools are specialized for a certain context, for example mobile or web based – rarely both. The reason for this is the disadvantage of these tools: they rely on code-based selectors that have trouble finding elements if they are at different positions because of display format, size or appearance. If any of these characteristics change because of responsive web design, most code-based testing tools do indeed struggle to offer an all-device-coverage.
The solution: Yet again: using modern technologies and research. It’s indeed impossible to cover all devices with code-based tools. The solution has to be switching to visual tools that are independent of these environments. Yet again, if a tool can handle the element-detecting based on a visual input such as screenshots, responsive web design isn’t that much of a challenge anymore.
- Visual tools cope better with responsive web design and element changes than code-based tools
- Consider your happy paths and the shift left approach to speed up your UI tests
- You don’t need several different tools and frameworks because there are tools that do it all. Save your time and money.