Running the tests
Tests within the code are grouped using Tag annotations. There are three groups: 'Unit', 'Component' and 'Integration'. To just run the unit tests, for example, type:
./mvnw test -Dgroups="Unit"
mvnw.cmd test -Dgroups="Unit"
Similarly, for the Component tests, you should use:
./mvnw test -Dgroups="Component"
mvnw.cmd test -Dgroups="Component"
and the equivalent for running the Integration tests.
A test report may be created from the
test package in Java using the Surefire Plugin.
The report file (as either XML or HTML) generated for each of the test classes can be found under
Run the following command to create the report:
To generate the Serenity report:
./mvnw clean verify
mvnw.cmd clean verify
The report can be viewed under
Code coverage report
The JaCoCo Code Coverage tool allows identifying how much of the code is run during the test suite execution. It is by no means an identifier of code and test quality in its own right but can be helpful for checking if logic paths have test coverage, especially for code updates. This library is run in the API pipeline, with output reports being made available to the CI tool being used.
To generate the JaCoCo report, please execute the following command:
The generated report can be viewed under
Dependency checker report
The OWASP Dependency-Check is a Software Composition Analysis tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies. Dependency-check has a command line interface, a Maven plugin, an Ant task, and a Jenkins plugin. The core engine contains a series of analyzers that inspect the project dependencies, collect pieces of information about the dependencies (referred to as evidence within the tool). To generate the dependency checker report please execute:
./mvnw clean install -Powasp-dependency-check
mvnw.cmd clean install -Powasp-dependency-check
The generated report can be viewed under -
Optional: Mutation testing
PIT mutation testing is used to generate mutation tests (see https://pitest.org/ for details). The mutation coverage goal analyses all classes in the codebase that match the target tests and target class filters. To run it:
The generated report can be viewed under –
Running tests in parallel threads - Using forks
Choosing the right forking strategy and parallel execution settings can have a substantial impact on the memory requirements, and the execution time of the build system. Using multiple forks can be a good alternative to running all tests in a single JVM, and can reduce the risk of certain types of errors.
Using Forked Test Execution, new JVM processes are spun up to execute the tests, up to a configurable maximum number of processes. This creates better separation between tests, which can improve their reliability.
Maven is set to spawn new processes by the
forkCount configuration element, as shown here below. This can either be a number (the maximum number of forks) or a multiplier (the number of forks per CPU).
The current configuration uses the value of "2", which means 2 forked processes per CPU:
Access token configuration
In order to make the authenticated requests (pass Bearer Token into headers) for the functional tests, please change the value of
serenity.conf file config to 'true'.
Also, to get the right access token please provide values for following variables in the
In case the Authorization Token is not required the
generate.auth0.token value is false
The more threads are used, the higher the chances of sporadic, hard-to-reproduce test failures due to timeouts and other related issues. The added performance gain of each thread also tends to drop off for higher numbers of threads.