We’re continuing with our series of blogs about everything related to testing. In this blog, we’re focusing on real examples.

While the examples in this post are written using JUnit 5 and AssertJ, the lessons are applicable to any other unit testing framework.

JUnit is the most popular testing framework for Java. AssertJ is a Java library that helps developers write more expressive tests.

Basic test structure

The first example of a test we will look at is a simple calculator for adding 2 numbers.

class CalculatorShould {

     @Test // 1
     void sum() {
         Calculator calculator = new Calculator(); // 2
         int result = calculator.sum(1, 2); // 3
         assertThat(result).isEqualTo(3); // 4
     }
}

I prefer using the ClassShould naming convention when writing tests to avoid repeating should or test in every method name. You can read more about it here.

What does the test above do?

Let’s break the test line by line:

  1. @Test annotation lets JUnit framework know which methods are meant to be run as tests. It is perfectly normal to have private methods in the test class which are not tests.
  2. This is the arrange phase of our test, where we prepare the testing environment. All we need for this test is to have a Calculator instance.
  3. This is the act phase where we trigger the behaviour we want to test.
  4. This is the assert phase in which we inspect what happened and if everything resolved as expected. assertThat(result) method is part of the AssertJ library and has multiple overloads.

Each overload returns a specialized Assert object. The returned object has methods that make sense for the object we passed to the assertThat method. In our case, that object is AbstractIntegerAssert with methods for testing Integers. isEqualTo(3) will check if result == 3. If it is, the test will pass and fail otherwise.

We won’t focus on any implementations in this blog post.

Another way of thinking about Arrange, Act, Assert is Given, When, Then.

After we write our sum implementation, we can ask ourselves some questions:

  • How can I improve on this test?
  • Are there more test cases I should cover?
  • What happens if I add a positive and a negative number? Two negative numbers? One positive and one negative?
  • What if I overflow the integer value?

Let’s add these cases and improve on the existing test name a little bit.

We will not allow overflows in our implementation. If sum overflows, we will throw an ArithmeticException instead.

  class CalculatorShould {

     private Calculator calculator = new Calculator();

     @Test
     void sumPositiveNumbers() {
         int sum = calculator.sum(1, 2);
         assertThat(sum).isEqualTo(3);
     }

     @Test
     void sumNegativeNumbers() {
         int sum = calculator.sum(-1, -1);
         assertThat(sum).isEqualTo(-2);
     }

     @Test
     void sumPositiveAndNegativeNumbers() {
         int sum = calculator.sum(1, -2);
         assertThat(sum).isEqualTo(-1);
     }

     @Test
     void failWithArithmeticExceptionWhenOverflown() {
         assertThatThrownBy(() -> calculator.sum(Integer.MAX_VALUE, 1))
             .isInstanceOf(ArithmeticException.class);
     } 

}

JUnit will create a new instance of CalculatorShould before running each @Test method. That means that each CalculatorShould will have a different calculator so we don’t have to instance it in every test.

shouldFailWithArithmeticExceptionWhenOverflown test uses a different kind of an assert. It checks that a piece of code failed. assertThatThrownBy method will run the lambda we provided and make sure that it failed. As we already know, all assertThat methods return a specialized Assert allowing us to check which type of exception occurred.

This is an example of how we can test that our code fails when we expect it to. If at any point we refactor Calculator and it does not throw ArithmeticException on an overflow, our test will fail.

Read Writing Testable Code

ObjectMother design pattern

The next example is a validator class for ensuring a Person instance is valid.

class PersonValidatorShould {

    private PersonValidator validator = new PersonValidator();

    @Test
    void failWhenNameIsNull() {
        Person person = new Person(null, 20, new Address(...), ...);

        assertThatThrownBy(() -> validator.validate(person))
            .isInstanceOf(InvalidPersonException.class);
    }

    @Test
    void failWhenAgeIsNegative() {
        Person person = new Person("John", -5, new Address(...), ...);

        assertThatThrownBy(() -> validator.validate(person))
            .isInstanceOf(InvalidPersonException.class);

    }
}

ObjectMother design pattern is often used in tests that create complex objects to hide the instantiation details from the test. Multiple tests might even create the same object but test different things on it.

Test #1 is very similar to test #2. We can refactor PersonValidatorShould by extracting the validation as a private method then pass illegal Person instances to it expecting them all to fail in the same fashion.

 class PersonValidatorShould {

     private PersonValidator validator = new PersonValidator();

     @Test
     void failWhenNameIsNull() {
         shouldFailValidation(PersonObjectMother.createPersonWithoutName());
     }

     @Test
     void failWhenAgeIsNegative() {
         shouldFailValidation(PersonObjectMother.createPersonWithNegativeAge());
     }

     private void shouldFailValidation(Person invalidPerson) {
         assertThatThrownBy(() -> validator.validate(invalidPerson))
             .isInstanceOf(InvalidPersonException.class);
   
     }
 }

Testing randomness

How are we supposed to test randomness in our code? 

Let’s suppose we have a PersonGenerator that has generateRandom to generate random Person instances. 

We start by writing the following:

class PersonGeneratorShould {

     private PersonGenerator generator = new PersonGenerator();

     @Test
     void generateValidPerson() {
         Person person = generator.generateRandom();
         assertThat(person).
    }
}

And then we should ask ourselves:

  • What am I trying to prove here? What does this functionality need to do? 
  • Should I just verify that the generated person is a non-null instance?
  • Do I need to prove it is random?
  • Does the generated instance have to follow some business rules?

We can simplify our test using Dependency Injection.

  public interface RandomGenerator {
     String generateRandomString();
     int generateRandomInteger();
}

The PersonGenerator now has another constructor that also accepts an instance of that interface as well. By default, it uses the JavaRandomGenerator implementation that generates random values using java.Random.

However, in the test, we can write another, more predictable implementation.

@Test
 void generateValidPerson() {
     RandomGenerator randomGenerator = new PredictableGenerator("John Doe", 20);
     PersonGenerator generator = new PersonGenerator(randomGenerator);
     Person person = generator.generateRandom();
     assertThat(person).isEqualTo(new Person("John Doe", 20));
}

This test proves that the PersonGenerator generates random instances as specified by the RandomGenerator without getting into any details of the RandomGenerator

Testing the JavaRandomGenerator does not really add any value since it is a simple wrapper around java.Random. By testing it, you would essentially be testing java.Random from the Java standard library. Writing obvious tests will only lead to additional maintenance with little if any benefits.

To avoid writing implementations for testing purposes, such as PredictableGenerator, you should use a mocking library such as Mockito.

When we wrote PredictableGenerator, we actually stubbed the RandomGenerator class manually. You could have also stubbed it using Mockito:

@Test
 void generateValidPerson() {
     RandomGenerator randomGenerator = mock(RandomGenerator.class);
     when(randomGenerator.generateRandomString()).thenReturn("John Doe");
     when(randomGenerator.generateRandomInteger()).thenReturn(20);

     PersonGenerator generator = new PersonGenerator(randomGenerator);
     Person person = generator.generateRandom();
     assertThat(person).isEqualTo(new Person("John Doe", 20));
 }

This way of writing tests is more expressive and leads to fewer implementations for specific tests.

Mockito is a Java library for writing mocks and stubs. It is very useful when testing code that depends on external libraries you cannot easily instantiate. It allows you to write behaviour for these classes without implementing them directly.

Mockito also allows another syntax for creating and injecting mocks to reduce boilerplate when we have more than one test similar to what we are used to:

@ExtendWith(MockitoExtension.class) // 1
 class PersonGeneratorShould {

     @Mock // 2
     RandomGenerator randomGenerator;

     @InjectMocks // 3
     private PersonGenerator generator;

     @Test
     void generateValidPerson() {
         when(randomGenerator.generateRandomString()).thenReturn("John Doe");
         when(randomGenerator.generateRandomInteger()).thenReturn(20);

         Person person = generator.generateRandom();
         assertThat(person).isEqualTo(new Person("John Doe", 20));
     }
}

1. JUnit 5 can use “extensions” to extend its capabilities. This annotation allows it to recognize mocks through annotations and inject them properly.

2. @Mock annotation creates a mocked instance of the field. This is the same as writing mock(RandomGenerator.class) in our test method body.

3. @InjectMocks annotation will create a new instance of PersonGenerator and inject mocks in the generator instance.

For more details on JUnit 5 extensions see here.

For more details on Mockito injection see here.

There is one pitfall to using @InjectMocks. It may remove the need to declare an instance of the object manually, but we lose the compile-time safety of the constructor. If at any point in time someone adds another dependency to the constructor, we would not get the compile-time error here. This could lead to failing tests that are not easy to detect. I prefer to use @BeforeEach to setup the instance manually:

@ExtendWith(MockitoExtension.class)
class PersonGeneratorShould {

     @Mock
     RandomGenerator randomGenerator;

     private PersonGenerator generator;

     @BeforeEach
     void setUp() {
         generator = new PersonGenerator(randomGenerator);
     }

     @Test
     void generateValidPerson() {
         when(randomGenerator.generateRandomString()).thenReturn("John Doe");
         when(randomGenerator.generateRandomInteger()).thenReturn(20);

         Person person = generator.generateRandom();
         assertThat(person).isEqualTo(new Person("John Doe", 20));
     }
}

Testing time-sensitive processes

A piece of code is often dependent on timestamps and we tend to use methods such as System.currentTimeMillis() to get the current epoch timestamp.

While this looks fine, it is hard to test and prove if our code works correctly when the class makes decisions for us internally. An example of such a decision would be determining what the current day is.

class IndexerShould {
     private Indexer indexer = new Indexer();
     @Test
     void generateIndexNameForTomorrow() {
         String indexName = indexer.tomorrow("my-index");
         // this test would work today, but what about tomorrow?
        assertThat(indexName)
           .isEqualTo("my-index.2022-02-02");
     }
}

We should use Dependency Injection again to be able to ‘control’ what the day is when generating the index name. 

Java has a Clock class to handle use-cases such as this. We can pass an instance of a Clock to our Indexer to control the time. The default constructor could use Clock.systemUTC() for backwards compatibility. We can now replace System.currentTimeMillis() calls with clock.millis().

By injecting a Clock we can enforce a predictable time in our classes and write better tests.

Testing file-producing methods

  • How should we test classes which write their output to files? 
  • Where should we store these files for them to work on any OS?
  • How can we make sure that the file does not already exist?

When dealing with files, it can be difficult to write tests if we try to tackle these concerns ourselves as we will see in the following example. The test that follows is an old test of dubious quality. It should test if a DogToCsvWriter serializes and writes dogs to a CSV file:

class DogToCsvWriterShould {

     private DogToCsvWriter writer = new DogToCsvWriter("/tmp/dogs.csv");
     
     @Test
     void convertToCsv() {
         writer.appendAsCsv(new Dog(Breed.CORGI, Color.BROWN, "Monty"));
         writer.appendAsCsv(new Dog(Breed.MALTESE, Color.WHITE, "Zoe"));

         String csv = Files.readString("/tmp/dogs.csv");

         assertThat(csv).isEqualTo("Monty,corgi,brownnZoe,maltese,white");
     }
}

The serialization process should be decoupled from the writing process, but let’s focus on fixing the test.

The first problem with the test above is that it won’t work on Windows since Windows users won’t be able to resolve the path /tmp/dogs.csv. Another issue is that it won’t work if the file already exists since it is not deleted when the test above executes. It might work ok in a CI/CD pipeline, but not locally if run multiple times.

JUnit 5 has an annotation you can use to get a reference to a temporary directory that gets created and deleted by the framework for you. While the mechanism of creating and deleting temporary files varies from framework to framework, the ideas remain the same. 

class DogToCsvWriterShould {

     @Test
     void convertToCsv(@TempDir Path tempDir) {
         Path dogsCsv = tempDir.resolve("dogs.csv");
         DogToCsvWriter writer = new DogToCsvWriter(dogsCsv);
         writer.appendAsCsv(new Dog(Breed.CORGI, Color.BROWN, "Monty"));
         writer.appendAsCsv(new Dog(Breed.MALTESE, Color.WHITE, "Zoe"));

         String csv = Files.readString(dogsCsv);

         assertThat(csv).isEqualTo("Monty,corgi,brownnZoe,maltese,white");
     }
}

With this small change, we are now sure that the test above will work on Windows, macOS and Linux without having to worry about absolute paths. It will also delete the created files after the test so we can now run it multiple times and get predictable results each time. 

Command vs query testing

What is the difference between a command and a query?

  • Command: we instruct an object to perform an action that produces an effect without returning a value (void methods)
  • Query: we ask an object to perform an action and return a result or an exception

So far, we have tested mainly queries where we called a method that returned a value or has thrown an exception in the act phase. How can we test void methods and see if they interact correctly with other classes? Frameworks provide a different set of methods for writing these kinds of tests.

The assertions we wrote thus far for queries were beginning with assertThat. When writing command tests, we use a different set of methods because we are no longer inspecting the direct results of methods as we did with queries. We want to ‘verify’ interactions our method had with other parts of our system.

@ExtendWith(MockitoExtension.class)
 class FeedMentionServiceShould {

     @Mock
     private FeedRepository repository;

     @Mock
     private FeedMentionEventEmitter emitter;

     private FeedMentionService service;

     @BeforeEach
     void setUp() {
         service = new FeedMentionService(repository, emitter);
     }

     @Test
     void insertMentionToFeed() {
         long feedId = 1L;
         Mention mention = ...;

         when(repository.upsertMention(feedId, mention))
             .thenReturn(UpsertResult.success(feedId, mention));

         FeedInsertionEvent event = new FeedInsertionEvent(feedId, mention);
         mentionService.insertMentionToFeed(event);

         verify(emitter).mentionInsertedToFeed(feedId, mention);
         verifyNoMoreInteractions(emitter);
     }
}

In this test, we first mocked our repository to respond with a UpsertResult.success when asked to upsert mention in our feed. We are not concerned with testing the repository here. The repository methods should be tested in the FeedRepositoryShould. By mocking this behaviour, we didn’t actually call the repository method. We simply told it how to respond next time it is called.

We then told our mentionService to insert this mention in our feed. We know that it should emit the result only if it successfully inserted the mention in the feed. By using the verify method we can make sure that the method mentionInsertedToFeed was called with our mention and feed and wasn’t called again using verifyNoMoreInteractions.

Final thoughts

Writing quality tests comes from experience, and the best way to learn is by doing. The tips written in this blog come from practice. It is hard to see some of the pitfalls if you never encountered them and hopefully, these suggestions should make your code design more robust. Having reliable tests will increase your confidence to change things without breaking a sweat each time you have to deploy your code.

Skip to content