Developers write unit tests to get fast feedback on code changes. So it’s frustrating when the number of unit tests in your project grows and suddenly you’re wasting hours every week waiting for them to run.
So much for fast feedback.
Imagine what you could achieve if you got back even a fraction of that time. Maybe get another feature out the door? In this article you’ll learn the simple technique of disabling test reports, guaranteed to speed up tests in Gradle.
An unoptimised project
Let’s explore the slow test problem with a big old Java project comprising 100,000 unit tests.
$ cd big-old-java-project/
$ ./gradlew test
BUILD SUCCESSFUL in 24s
3 actionable tasks: 1 executed, 2 up-to-date
With 100,000 unit tests, Gradle takes 24 seconds to run its test task.
It would be easy to write this off as something you can’t do anything about.
But that’s not how we do things around here! Are you up for a challenge?
Because if you run these tests 10 times a day and each time takes 24 seconds, how long would that be per year?
24s * 10 * 5 days * 52 weeks / 60s / 60m / 8 hour day = 2 days
Yes. We’re wasting lots of time waiting for tests to run, so let’s do something about it.
Take a closer look at the console output during execution to see what’s going on.
$ ./gradlew test
<===========-<===========--> 85% EXECUTING [12s]
> :test
> IDLE
While the tests are running they get stuck at around the 12 second mark. Yes. Literally nothing seems to be happening.
Of course that’s not really the case.
To discover what Gradle is actually doing pass the --info command line option for detailed log output.
$ ./gradlew test --info
...
Generating HTML test report...
Finished generating test html results (4.941 secs) into: C:\workspace\gradle-hero\optimisation\big-old-java-project\build\reports\tests\test
<===========--> 85% EXECUTING [23s]
> :test
Ah! Generating HTML test report!
Those pretty HTML test reports are created after every test run in build/reports/tests/test.
This test report goes on forever.
For 100,000 tests, that’s one big old test report.
And can you believe that generating the test reports takes just as long as running the tests themselves?
In this project, generating test reports takes as long as running tests.
The good news is it’s easy to disable HTML test reports.
In the Gradle build script, locate the test task and set reports.html.required to false.
Here’s how that looks in the Kotlin build.gradle.kts.
tasks.withType<Test>().configureEach {
reports.html.required = false
}
Now run again….
$ ./gradlew test
BUILD SUCCESSFUL in 17s
3 actionable tasks: 1 executed, 2 up-to-date
That’s 7 seconds shaved off the test time!
The HTML reports are no longer generated, but there’s something else. If you look closely in the build directory, you’ll notice a mysterious test-results directory.
ls build/test-results/test/
TEST-org.gradle.test.performance.mediummonolithicjavaproject.p0.Test0.xml
TEST-org.gradle.test.performance.mediummonolithicjavaproject.p0.Test1.xml
TEST-org.gradle.test.performance.mediummonolithicjavaproject.p0.Test10.xml
TEST-org.gradle.test.performance.mediummonolithicjavaproject.p0.Test11.xml
TEST-org.gradle.test.performance.mediummonolithicjavaproject.p0.Test12.xml
...
In here are 1000s of XML test reports, used by CI tools like Jenkins and GitHub to show info about failing tests.
We definitely don’t need these when running tests locally, so disable XML reports in a similar way by setting reports.junitXml.required to false.
tasks.withType<Test>().configureEach {
reports.html.required = false
reports.junitXml.required = false
}
Let’s run again….
$ ./gradlew test
BUILD SUCCESSFUL in 12s
3 actionable tasks: 1 executed, 2 up-to-date
And another 5 seconds saved!
Our original 24 second build now runs in 12 seconds.
That’s twice as fast!
For this specific project, that saves 1 day per year per developer. What impact could this optimisation have on your project?
Make it configurable
There’s still one important point to consider. With reports disabled, locally you can still see test failures in the console.
Viewing a test failure in the console.
But what if you still need the HTML or XML test reports in certain environments? For example, you might want to use XML test reports to show test results in the Jenkins UI.
Well, you can use a Gradle project property to configure that.
Remember, Gradle build scripts are just code, so a good old fashioned if statement can check for the presence of a createReports project property.
tasks.withType<Test>().configureEach {
if (!project.hasProperty("createReports")) {
reports.html.required = false
reports.junitXml.required = false
}
}
To run tests with reports, pass the project property on the command line using the -PpropertyName syntax.
$ ./gradlew test -PcreateReports
BUILD SUCCESSFUL in 24s
3 actionable tasks: 1 executed, 2 up-to-date
When you do this, the build time increases because the reports are generated again. This could be an ideal way to run tests in CI.
But for local development where real humans are involved, don’t pass the property to disable reports and enjoy supercharged test runs.
$ ./gradlew test
BUILD SUCCESSFUL in 12s
3 actionable tasks: 1 executed, 2 up-to-date
Final thoughts
The size of project you’re building will influence how much impact this optimisation has. The more tests, the more time you save.
More tests means more potential savings.
Try it out in your Gradle project! You can let me know if it makes a difference by leaving a comment below.
Stop reading Gradle articles like these
This article helps you fix a specific problem, but it doesn't teach you the Gradle fundamentals you need to really help your team succeed.
Instead, follow a step-by-step process that makes getting started with Gradle easy.
