In this blog post we are going to cover everything you need to know about Unit Tests. This is the first of two blog posts covering Unit Testing. By the end of this post you will have an overview of all the important characteristics, techniques and patterns.
Let’s start with a quote by Martin Fowler:
“Every fool can write code that a computer can understand. Good programmers write code that humans understand.”
Analogous, good Unit Tests are comprehensible to other people, not just machines. Making code and in this case Unit Tests comprehensible to everyone is what we want to call the Art of Unit Testing.
Why do you need to test?
- Write working code
- Keep code working
- Develop faster
- Find bugs before production
- Force us to write testable code
- Humans do mistakes
There are six reasons, the most obvious answer (1) is that you want your code to work properly. But you need to make sure that it (2) keeps working, even when you change the code. Even the tiniest changes can lead to a series of unintended errors. You don’t want your users to find bugs, you want to (3) find bugs before production. The release cycles of updates heavily increased over the last years which means that the (4) development-process needs to be done more quickly. A well written test is the foundation to this. (5) Writing testable code from the beginning is an investment that pays off in the long run, the amount of work just piles up over time. Last but not least, (6) humans make mistakes. Making mistakes is not a problem, not finding them is a problem.
How Are Unit Tests defined?
There are various different definitions of Unit Tests, so let’s stick to Martin Fowler one more time, who identified three core characteristics of Unit Tests that every definition shares: Firstly there is a notion that unit tests are low-level, focusing on a small part of the software system. Secondly unit tests are usually written by the programmers themselves using their language dependent unit testing framework, like JUnit, MSTest or NUnit. Thirdly unit tests are expected to be significantly faster than other kinds of tests.
Elliotte Rusty Harold stripped the definition of Unit Tests down to: “verifying that a known fixed input produces a known fixed output”.
But first step back. What is a unit in general? A unit is normally a method, constructor or deconstructor.
Units that are to be tested often rely on other units to fulfill their behavior. When testing a unit, all dependencies should be replaced by mocks. A mock object is a simulated object which simulates the behavior of the dependencies. You only want to test the functionality of this unit and not the functionality of the other ones.
The test calls the unit and it will call our mocks. At the end, we verify that the mocks are called with the expected parameters. (See image below)
What Is the FIRST-Principle?
When it comes to the properties of Unit Testing, Brett Schuchert and Tim Ottinger came up with the FIRST Principle: Fast, Independent, Repeatable, Self-Checking and Timely. Writing your Unit Tests with these properties in mind keeps your code clean.
Your Unit Test has to be fast. Most projects cover hundreds of thousands of tests. A test that takes 0.5 or even 0.25 seconds to complete is unacceptably slow. Time is money, make your tests as fast as possible.
An independent test does not rely on any subset in a specific order. You can run every test without constraint. Passing or failing should not depend on the order in which the test was run. Tim Ottinger also refers to the “I” in FIRST as isolated.
Repeatable means that tests must be able to be run repeatedly without intervention. They must not depend upon a specific initial state and they must not leave any problems behind that would prevent them from being re-run. They have to be run repeatedly in any order at any time.
Self-checking or self-validating means that a test can automatically detect if it was passed or failed. There is no additional instance needed to confirm or validate the result.
Tests are written at the right time, immediately before the code that makes the tests pass. Writing the test first makes a difference, don’t write them after you write your code.
Do You Know Your Test Coverage?
Let’s talk about Code Coverage or Test Coverage as some like to call it. Coverage shows you which lines and branches of the code were (or were not) covered by the tests. It is also a metric which helps you to find out the percentage of your covered (executed) code by the tests. E.g.: It tells you that your codebase consists of 10 lines, 8 lines were being covered by your tests, so your coverage is 80%. While this gives you no information about the quality of your software or how good your tests really are. The coverage of the complete code base should be higher than 80%.
Code coverage is useful for core refactoring, too. Execute it before you start, check which code branches are covered, add the missing tests and start refactoring your tests.
What Is the Best Way to Write Unit Tests?
Last but not least: how should your Unit Tests be written? What are the Do’s and Don’ts, what do the naming conventions and building patterns look like and how is the code structured? Let’s start by prefacing some general tips before you actually start writing your code.
In the FIRST-Principle we already came across the term clean code. The term deviates from the Clean Code book by Robert Cecil Martin. In a nutshell, clean code is any code that can be understood intuitively – by others, not just by yourself. Next up is Dependency Injection (DI) – it’s not necessary but highly recommended to isolate your dependencies in the code. This allows you to easily inject your mocks to your class and only test the unit.
Some general Do’s you should consider:
- One assert per test method
- Share setup and fixture
- Multiple test classes per class
Avoid these Don’ts:
- Generate random input
- Access network or file system
- Names constants from model code.
- Using description annotations
First code example: to avoid god testing classes you can split your test class in multiple test classes. Then test only a single method in your test class, as shown in the next code example.
Second code example: Never ever (!) use conditions or loops in test methods. Each test method should only test one test case. Split them in multiple test methods.
Third code example: Warning! When you call static methods from external dependencies, the behavior is not only changed in this test method it is also changed in all other test methods. If you run them at the same time it can result in flaky tests.
Have You Already Thought About Naming Your Test Methods?
Talking about intuitive, clean and easy to understand code (for everybody), let’s have a look at a few common naming conventions for test methods. There are some general recommendations regarding test naming (Stefanovskiy):
- Test name should express a specific requirement
- Test name could include the expected input or state and the expected result for that input or state
- Test name should be presented as a statement or fact of life that expresses workflows and outputs
- Test name could include the name of the tested method or class
An example might be:
Renaming of method name is necessary, when renaming the origin method
Renaming of method name is necessary, when renaming the origin method
- test[Feature being tested]
test-prefix is duplicated information in combination of an @Test-annotation
- Feature to be tested
Expected result is not defined
Long name through should and when
Long name through should and expect
Given, When and then are duplicated
It does not matter which naming convention you choose, but your team should use one consistently. Personally I prefer a combination of 1 and 5 <MethodName>_Should_<ExpectedBehavior>_When_<StateUnderTest>.
Do You Know These Test Doubles?
When writing your code, knowing the following terms is crucial:
The most basic term you have to know is the Dummy. A Dummy is a placeholder required to pass the unit. The Dummy should not be accessed during the test. The simplest example might be null:
In automated testing it is common to use objects that implement the production interface and behave like it. Fakes are objects that have working implementations, but not the same as the production one. Fakes simplify the complexity of the external dependency setup. Fakes are frequently used for database access, openid connect endpoints or external APIs.
Stub is an object that holds predefined data and uses it to answer calls during tests. It provides indirect input to the unit from a dependency. Classic examples are objects, exceptions or primitive values which are returned from a method call.
Compared to a stub, allow mocks to verifying method invocation of the unit. An example is the service method invoking the repository method.
A Spy uses the original implementation and is the most complex version of a test double. It is a wrapper of the original implementation, so it records the number of calls and not the input. For example:
How Are Test Methods Structured?
There are two common practices to structure your test method. First the AAA-Pattern and the BDD-Pattern. The BDD-Pattern is not covered here.
The AAA-Pattern is commonly used in unit tests. This Acronym stands for arrange, act and assert (Bill Wake). The test method code is structured in three blocks, as shown in the following example.
Now let’s come to a conclusion. Unit Testing is an art that is easy to learn (and not too hard to master) if you follow the instructions given in this overview. Now that you know about all the characteristics, basic coding examples and patterns, we suggest you to always question your code (as Fowler would say): is my code comprehensible to everyone?
This blog is based on a Unit Test input Dominik Klotz gave for the Karlsruher Testing Community.
Working in What-You-See-Is-What-You-Get editors (WYSIWYG) can be very tiring and also taxing on your wrists. Things like highlighting the same thing over and over again can often not be done with search-and-replace, because there is just no button for it 😤 In this tutorial I will show you how I automated this task with AskUI and also how to work around some kinks of WYSIWIG-Editors like microanimations and strange behaviour.
Embrace Immersive and Interactive User Experiences: A Guide to Automating WebGL Websites with AskUI As web developers increasingly seek to captivate users with immersive experiences, WebGL-based websites have gained popularity. By harnessing the power of canvas to draw graphics and animations directly, they offer unparalleled creative control and dynamism in crafting user interfaces. The result? A more flexible and engaging platform for stunning graphical compositions, delivering real-time interactions that captivate visitors.
Visual Regression Testing (VRT) is a useful method to guard yourself against unwanted layout and visual changes that can occur with new version of a Graphical User Interface (GUI).AskUI is not a testing tool in the original sense but an automation tool for all GUIs that is platform-independent. Wouldn't it be nice to combine VRT with AskUI, so you do not have to maintain two different projects.Luckily you can do this because Jest - AskUIs favourite runner - can be extended to provide VRT, while AskUI delivers the screenshots to test on.