Development is one of the key activities that build, enable and ultimately power the overall mobile environment. Today we are sharing some key insight into an important aspect of development – testing framework – from one of our most experienced developers here at Smaato.
Developers know that choosing the right testing framework for the job can be really complicated. The jUnit Framework is an excellent Java testing tool, but for specific tests that require sample data, better tools - like the Groovy-based Spock - are available. This post describes how to use Spock and how it handles test data. It also explores how Spock tests can be executed with jUnit tests and maven.
How jUnit runs parameterized tests
To have a test that works with a set of test data, you need to define a data method which returns all samples in a Collection. Each sample is used to instantiate a new test class. The constructor needs suitable parameters for the test samples. Then all test methods are executed, as you can see in the code samples below:
Here we can see some major problems with jUnit parameterized tests:
- Constructor and attributes are required (boiler code)
- A separate runner is required
- A test is executed for every data sample (wasteful for tests without sample data)
- Only one set of sample data is allowed per test class
Certain test frameworks like JUnitParams offer different approaches that make tests more readable. But I was unable to find a pure Java-based approach that fit the bill, so I looked around for a solution and found Spock.
How to use Spock (a short introduction)
Spock is a Groovy-based test framework. It uses the jUnit TestRunner, which means you can code in your favorite IDE and build tools to write and execute Spock specifications.
These maven dependencies are required at setup:
We also need a /src/test/groovy directory in our project, which will contain all Groovy-related files.
With Spock, you write specifications that describe the features your system has (or should have).
Each specification must extend from the spock.lang.Specification and should be named with “Specification” as a postfix. This detail will be important later in the build process.
Feature methods are named with String literals - this way, you can write more readable names. Each feature has the same phases as any unit test:
- Setup (optional)
- Clean up (optional)
Here’s a short “hello world” example of how to implement these phases with Spock:
In this example, we use three blocks to define our feature method:
- Given – an alias for setup of this feature method (Spock has more fixture methods like setupSpec which are executed once before the first feature method).
- When – a description of the stimulus to our software under test; the execution phase.
- Then - a description of the expected response; the verification phase.
More blocks can be found in Spock documentation, which I recommend to you for review.
Groovy and jUnit tests together with maven
When writing tests in jUnit and specifications in Spock, we want to execute them both to test our system. In maven, we use the gmavenplus and surefire plugins to do that. The gmavenplus plugin compiles all Groovy files it finds in src/test/groovy:
The surefire plugin is configured to execute specifications together with all other tests:
To set our Java version to 1.8, we write the following plugin configuration:
How Spock handles parameterized specification methods
Spock uses data tables to define test data for a feature method. The nice thing here is that you can have multiple methods with their own test data, unlike jUnit tests which have to share test data. To define a data table, we use the Where block and it must be the last block of a method.
In this example, I introduced a minor bug in the test data to demonstrate how Spock shows failing tests:
Above, we see a failing test and the failing assertion stacktrace, which raises a number of interesting points:
- The printout of Spock assertion errors are a lot better than jUnit or hamcrest.
- We do not see the result of the second data sample.
To fix this issue, we use the Spock.lang.Unroll annotation. This enables the visibility of each iteration as a separate feature. If we use this together with test data in the method name, then we receive a clear overview of what happened.
It's clear that jUnit is an excellent test framework for all kind of Java tests, but there are some test scenarios where it's not a good fit. For tests with test data, the jUnit Parameterized solution has some disadvantages that can be solved by using Spock, a Groovy-based test framework. You can use multiple specification methods and each of them is directly connected to their own test data. It's possible to use jUnit and Spock tests together, so you should consider this option as part of your overall testing strategy.
Do you know of better ways to write tests with test data? If so, please leave a comment on this blog and we can discuss further.