Tom Gregory

Scaling developer productivity

How to Measure Code Coverage Using SonarQube and Jacoco

How to measure code coverage using SonarQube and Jacoco
By Tom GregoryPosted on

Last Updated on April 21, 2023

Code coverage is a metric that teams use to measure the quality of their tests, and it represents the percentage of production code that has been tested. Discover how to apply the Gradle Jacoco plugin to your project and run a SonarQube scan to generate a code coverage report.

Contents
1. Overview
2. A worked example
3. Setting up a SonarQube instance
4. Using the Jacoco Gradle plugin
5. Using the SonarQube Gradle plugin
6. Viewing the SonarQube report details
7. SonarQube 9
8. Conclusion
9. Resources

UPDATED in November 2021 to reflect SonarQube LTS version switching to 8.9.

1. Overview

SonarQube is a tool which aims to improve the quality of your code using static analysis techniques to report:

  • code coverage
  • bugs
  • code smells
  • security vulnerabilities

The SonarQube server is a standalone service which allows you to browse reports from all the different projects which have been scanned. To scan a specific codebase you run the SonarQube scanner. This is a local process that analyses your code then sends reports to the SonarQube server.

Code coverage

The code coverage metric has to be computed outside of SonarQube using a different tool. The result is then ingested into SonarQube and shown within its web interface.

The tool we’ll look at today to calculate code coverage for a Java project is called Jacoco. Jacoco analyses the code and generates an XML report, which is later ingested by SonarQube.

2. A worked example

Let’s run through an example of exactly how Jacoco and SonarQube work together to calculate code coverage. To do this we’ll use the sonarqube-jacoco-code-coverage GitHub project, which you can clone to see it working with your own eyes.

We’ll set up:

  1. an instance of SonarQube running in Docker
  2. a Java project with a class and some unit tests
  3. a Gradle build which generates a Jacoco code coverage report then runs the SonarQube scanner against the Java project

ℹ️ Gradle is a popular build tool for building Java applications. It can also generate a Jacoco code coverage report and run the SonarQube scanner, as you’ll see later in the article. Learn more with this beginner tutorial.

The project has 2 classes:

  1. A MathService class which has 2 methods, multiply and subtract
  2. A MathServiceTest class which only tests 1 out of the 2 methods in MathService

So we’re hoping that SonarQube will highlight the fact that we’re missing a test here i.e. only one of the methods in MathService has been tested. Any guesses for what percentage code coverage SonarQube will report in this case? 🤔

ℹ️ Prerequisites: to follow along with this example, you’ll need installed locally both Docker and Java 8+. It’s been tested with Java 8, 11, and 17.

3. Setting up a SonarQube instance

An official Docker image exists for SonarQube, making this really easy to get up and running using Docker Compose. Just add the following docker-compose.yml file to your project:

version: "3"
services:
  sonarqube:
    image: sonarqube:lts
    ports:
      - 9000:9000
    environment:
      - SONAR_FORCEAUTHENTICATION=false
  • uses the lts (long term support) version of the SonarQube Docker image, which is currently SonarQube version 8.9.
  • SonarQube is configured to start on port 9000
  • SonarQube authentication is disabled to simplify this demo. Always use authentication in a production environment.

To run Docker Compose from Gradle, just add this plugin to your build.gradle:

plugins {
    id 'com.avast.gradle.docker-compose' version '0.14.3'
    // any other plugins
}

ℹ️ if you’re following along with the example GitHub repository rather than applying this to your own project, the previous and following steps are already configured for you.

Now we can run ./gradlew composeUp to start the SonarQube Docker container through Gradle:

$ ./gradlew composeUp

> Task :composeUp
sonarqube uses an image, skipping
...
BUILD SUCCESSFUL in 19s
1 actionable task: 1 executed

That was successful, but we can double check everything is OK by seeing what Docker containers are running with docker ps:

$ docker ps
CONTAINER ID   IMAGE           COMMAND          CREATED              STATUS              PORTS                                       NAMES
35459fcff334   sonarqube:lts   "./bin/run.sh"   About a minute ago   Up About a minute   0.0.0.0:9000->9000/tcp, :::9000->9000/tcp   39b4a0caef82cbb00dd981c87ae00
c20_sonarqube-jacoco-code-coverage__sonarqube_1

This shows that SonarQube is running on localhost:9000. It might take a minute for SonarQube to start up, but eventually within a browser you’ll be able to reach the main SonarQube dashboard.

This is correctly reporting we currently have 0 projects analysed. Let’s fix that! 👌

4. Using the Jacoco Gradle plugin

Before we get onto actually scanning our code with SonarQube, let’s set up the Jacoco Gradle plugin. This will generate the test coverage statistics for our Java code. Just add the following plugin definition and configuration to build.gradle:

plugins {
    id 'jacoco'
    // any other plugins
}

jacocoTestReport {
    reports {
        xml.enabled true
    }
}
test.finalizedBy jacocoTestReport
  • we apply the jacoco plugin, a core Gradle plugin
  • we configure the plugin to output test reports in the XML format required by SonarQube
  • we use finalizedBy to ensure that the test report is always generated after tests are run

Now let’s run ./gradlew test. Notice we have a new file reports/jacoco/test/jacocoTestReport.xml created in the build directory.

Jacoco XML report file location

This contains the code coverage information that SonarQube will pick up during its scan.

5. Using the SonarQube Gradle plugin

Now that we’ve got our test code coverage data being generated by Jacoco, it’s time to hook all this up by running a SonarQube scan. This will report on the code coverage as well as run a full scan of our code. To do this we’ll use the SonarQube Gradle plugin which adds the sonarqube task to our build.

We can include it in our build.gradle like this:

plugins {
    id 'org.sonarqube' version '3.3'
    // any other plugins
}

We also need to include a configuration to tell the SonarQube scanner where to find the SonarQube server:

sonarqube {
    properties {
        property 'sonar.host.url', 'http://localhost:9000'
    }
}

Lastly, to ensure the Jacoco test report is always created when we run the sonarqube task let’s setup the following dependsOn relationship:

tasks.named('sonarqube').configure {
    dependsOn test
}

Now we just need to run the sonarqube Gradle task to run a scan, with ./gradlew sonarqube.

$ ./gradlew sonarqube

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.2/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 10s
5 actionable tasks: 1 executed, 4 up-to-date

Success!

6. Viewing the SonarQube report details

We can head back to SonarQube at localhost:9000/projects to see the test code coverage report.

We can see a reported code coverage of 66.7%. Awesome! Click on the sonarqube-jacoco-code-coverage link and we’ll try to drill into exactly how this was calculated.

This is a more detailed view of the report. Click on the 66.7% link.

We now see information about what class has been analysed, in this case the MathService. Click on the link to see even more details:

We can now see the class itself, where green highlights code that is properly tested and red code that isn’t. Let’s zoom in a bit:

We can see that SonarQube is telling us that:

  • the class MathService is covered by tests (green mark)
  • the multiply method is covered by tests (green mark)
  • the subtract method is not covered by tests (red mark)

That makes 2 out of 3, hence the 66.7% being reported by SonarQube. So there’s definitely room for improvement!

7. SonarQube 9

The above example uses the LTS version of SonarQube, currently 8.9. The same steps can be used to run against SonarQube 9.

You can view the SonarQube version at the bottom of any page on the web UI.

8. Conclusion

You’ve seen that it’s really easy to setup code coverage reporting in a Gradle project using Jacoco and SonarQube. Try it out on your own project to see how you measure up.

Note that this isn’t the only metric you should use to measure your test quality, but it can be a helpful indicator. Code may have a high code coverage percentage, but it might be brittle and difficult to maintain.

To continue learning, check out these super-relevant articles:

Get the Build Boss newsletter

Found this website helpful? Subscribe for updates.

✅ All of my latest articles each week
✅ Access to video tutorials
✅ Exclusive tips and offers not found on my website

Please wait...

Thank you, your sign up request was successful! I'll be in touch soon. Tom.

9. Resources

GITHUB REPOSITORY
Follow along with this article by checking out the accompanying GitHub repository. This uses the LTS version of SonarQube (currently version 8.9).

GRADLE
Jacoco Plugin docs
SonarQube Plugin docs
Tutorial for complete beginners

SONARQUBE
General documentation

VIDEO
If you prefer to learn in video format, check out this accompanying video on the Tom Gregory Tech YouTube channel.

How to Measure Code Coverage Using SonarQube and Jacoco

Related Posts

  • Integration Test Code Coverage with SonarQube and Jacoco
  • 13 Simple Ways To Edit Code Faster in IntelliJ IDEA
  • SonarQube branch analysis
  • How and when to use a Prometheus gauge

31 thoughts on “How to Measure Code Coverage Using SonarQube and Jacoco”

  1. Hai team,

    I am building the java gradle project. after scanning the code by Sonarqube the quality gate result gets passed. But However I am getting the mentioned error “The Project Analysis has failed. More details available on the Background Tasks page” Could someone help me to resolve the error.

    Reply

  2. I am getting an error when I execute the command,
    ./gradlew sonarqube
    The error : FAILURE: Build failed with an exception.

    * What went wrong:
    ‘java.io.File org.gradle.api.tasks.testing.JUnitXmlReport.getDestination()’

    Can you please suggest a fix??

    Reply

  3. Hi Tom Gregory, I’m trying to setup code coverage reporting in a Maven project using Jacoco and SonarQube. Do you have a similar tutorial for that. If yes, please share the link. I found your content very easy to understand and excellent website for beginners. Thanks in advance!

    Reply

    1. Hi Kasturi. I don’t have a tutorial for Jacoco + Maven as I mostly focus on the Gradle build tool. Thanks for the suggestion though!

      Reply

  4. Hi,

    I am running the app in Window 10. I can login to the SonarQube console. However, when I ran “./gradlew sonarqube -PsonarPassword=password”, I got
    Execution failed for task ‘:sonarqube’.
    > Not authorized. Please check the properties sonar.login and sonar.password.

    The password is correct and I used it to login to the console. I tried to replace the sonar.login with the token without avail.

    Reply

    1. Hi ct. Sorry to hear you’re having an issue. I presume you’re using the sample project I supplied?

      Can you please try temporarily hard-coding the password in the build.gradle?

      sonarqube {
    properties {
        property 'sonar.host.url', 'http://localhost:9000'
        property 'sonar.login', 'admin'
        property 'sonar.password', 'your-password'
    }
}

      Let me know how you get on.

      Reply

      1. I got the same problem on Windows 10, I have tried hard-coding the password, But it didn’t work. However, when I typed the command `gradlew sonarqube` under the project’s directory, It can be successful.
        So still don’t know what step goes wrong.
        Hope for your reply. tks a lot

      2. Hi Lewis. So you’re getting a Not authorized error? When you start SonarQube and login you should be prompted to set a new password. This new password should be passed when you run the snoarqube Gradle task.

        If you’re not setting a password, I’m not sure how the sonarqube task is authorizing with SonarQube.

        One thing to try is to run docker pull sonarqube:lts to make sure you’re using an up-to-date image.

      3. Quick update. I’ve simplified the demo by using the SONAR_FORCEAUTHENTICATION environment variable to allow anonymous SonarQube analysis. Remember to use authentication in a production environment.

      4. Hi, I have solved this problem the next day, My way is by removing the two properties of `sonar.login` and `sonar.password` from `build.gradle`. I don’t know why this could work, but I guess it’s something in conflict with the Jenkins SonarQube Scanner plugin on Windows 10.
        Anyway thanks a lot.

  5. Hello,
    I have a big problem on my side… I am using the new way of doing things with SonarQube 8.9 … I have a coverage rate on my local project, but as soon as sending my result on sonar, 0% on the coverage rate. Can you help me please ?

    Reply

    1. Hi Vanessa. Are you generating the xml report as suggested in the article? Does it look correct?

      Reply

  7. How do we publish integration tests coverage report to Sonarqube. I have some UI tests and Jacoco Report (.exec file) is generated after execution. Now I want to push that file to sonarqube. What is the configuration required in this case?

    Reply

    1. Hi Saurabh. This isn’t something I’ve done before but I will investigate when I get some time. In the meantime, please let us know if you make any progress on this.

      Reply

  8. Could SonarQube, Jacoco or any other tool would automatically run tests whenever I push to my repository, providing me with information if tests passed and code coverage information without running it on the local machine.

    Reply

    1. Hi Akash,

      I know that SonarQube has integration with version control system such as GitLab (see docs). You can analyse branches and merge requests to see if they meet your standards, and also tag merge requests with the results.

      As far as running tests goes, that has to be outside SonarQube and Jacoco. Consider using something like GitLab pipelines or Jenkins for that.

      Also bear in mind that the features mentioned above are only available in paid version of SonarQube.

      Reply

  9. Hi @Tom,

    I ran your example. It is working fine and you explained it very nice.
    So how do we generate pdf report using sonar result? Is there any plugin?

    Thanks.

    Reply

    1. Hi Erandika. Thanks for the feedback. I know that Gradle Enterprise offers PDF reporting. Another option might be to use the Web API to get the information you need then format it into a report.

      Reply

  10. could not see the code coverage as of running through this today. Is sonarqube or jacoco broken? I was trying to fix why it wasn’t working in a pipeline for work, but I can’t even get it to work using this demo.

    Reply

    1. Hi Kevin. Sorry you couldn’t get the example working. I tried it a few weeks ago without issue. I will be taking a look later today, so please bare with me.

      Reply

      2. Hi again Kevin. I’ve just tried running the example from the GitHub repository and I’m getting the 66.7% test coverage as shown in this article.

        Can you please provide some more details about the problem you’re having? Are you managing to log into the SonarQube UI? If so, are you seeing that the project has been analysed?

        The version of SonarQube used in the project is the lts (long term support version) and the Jacoco plugin comes with the version of Gradle in the project (6.4.1).

      3. Tom,

        I was able to get it to work on my end. It was partly user error! It had to do with the java that I was using. I got it working in the end. Thank you for running through it again and verifying though! I’m currently trying to integrate the xml reporting in as that’s what broke code coverage for a work project. Thanks for providing this tutorial. It has been helpful in me figuring out how all of this works!

  11. Could it be related to this:
    Property ‘sonar.jacoco.reportPath’ is no longer supported. Use JaCoCo’s xml report and sonar-jacoco plugin.
    Property ‘sonar.jacoco.reportPaths’ is no longer supported. Use JaCoCo’s xml report and sonar-jacoco plugin.

    Reply

  12. Dear Tom,
    Nice and easy explained.
    However i get 0% coverage, 100% unit test
    If i run the same example against an external sonarqube scanner i have also 0 %.
    This is the logging:

    build 24-Mar-2020 18:13:42 INFO: parsing [/ec/local/citnet/bamboo-agent-home/xml-data/build-dir/EACDEVOPS-EACDEVOPSPLAN1-CHEC/sonarqube-jacoco-code-coverage/build/test-results/test]
    build 24-Mar-2020 18:13:42 INFO: Sensor SurefireSensor [java] (done) | time=31ms
    build 24-Mar-2020 18:13:42 INFO: Sensor JaCoCoSensor [java]
    build 24-Mar-2020 18:13:42 INFO: Sensor JaCoCoSensor [java] (done) | time=1ms
    build 24-Mar-2020 18:13:42 INFO: Sensor JavaXmlSensor [java]
    build 24-Mar-2020 18:13:42 INFO: Sensor JavaXmlSensor [java] (done) | time=1ms
    build 24-Mar-2020 18:13:42 INFO: Sensor HTML [web]
    build 24-Mar-2020 18:13:42 INFO: Sensor HTML [web] (done) | time=26ms
    build 24-Mar-2020 18:13:42 INFO: Sensor JaCoCo XML Report Importer [jacoco]
    build 24-Mar-2020 18:13:42 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=3ms
    build 24-Mar-2020 18:13:42 INFO: ————- Run sensors on project
    build 24-Mar-2020 18:13:42 INFO: Sensor Dependency-Check [dependencycheck]
    build 24-Mar-2020 18:13:42 INFO: Process Dependency-Check report
    build 24-Mar-2020 18:13:42 INFO: Dependency-Check XML report does not exists. Please check property sonar.dependencyCheck.reportPath:…
    build 24-Mar-2020 18:13:42 INFO: Analysis skipped/aborted due to missing report file
    build 24-Mar-2020 18:13:42 INFO: Dependency-Check HTML report does not exists. Please check property sonar.dependencyCheck.htmlReportPath:…
    build 24-Mar-2020 18:13:42 INFO: HTML-Dependency-Check report does not exist.
    build 24-Mar-2020 18:13:42 INFO: Process Dependency-Check report (done) | time=4ms
    build 24-Mar-2020 18:13:42 INFO: Sensor Dependency-Check [dependencycheck] (done) | time=4ms
    build 24-Mar-2020 18:13:42 INFO: Sensor Zero Coverage Sensor
    build 24-Mar-2020 18:13:42 INFO: Sensor Zero Coverage Sensor (done) | time=11ms
    build 24-Mar-2020 18:13:42 INFO: Sensor Java CPD Block Indexer
    build 24-Mar-2020 18:13:42 INFO: Sensor Java CPD Block Indexer (done) | time=19ms
    build 24-Mar-2020 18:13:42 INFO: SCM Publisher is disabled
    build 24-Mar-2020 18:13:42 INFO: 1 file had no CPD blocks
    build 24-Mar-2020 18:13:42 INFO: Calculating CPD for 0 files
    build 24-Mar-2020 18:13:42 INFO: CPD calculation finished
    build 24-Mar-2020 18:13:42 INFO: Analysis report generated in 122ms, dir size=78 KB
    build 24-Mar-2020 18:13:42 INFO: Analysis report compressed in 11ms, zip size=13 KB
    build 24-Mar-2020 18:13:42 INFO: Analysis report uploaded in 28ms
    build 24-Mar-2020 18:13:42 INFO: ANALYSIS SUCCESSFUL, you can browse https://webgate.ec.europa.eu/CITnet/sonarqube/dashboard?id=EACDEVOPS-SRCKEY
    build 24-Mar-2020 18:13:42 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
    build 24-Mar-2020 18:13:42 INFO: More about the report processing at https://webgate.ec.europa.eu/CITnet/sonarqube/api/ce/task?id=AXENiSBOgY0MYh9regFH
    build 24-Mar-2020 18:13:42 INFO: Analysis total time: 5.861 s
    build 24-Mar-2020 18:13:42 INFO: ————————————————————————
    build 24-Mar-2020 18:13:42 INFO: EXECUTION SUCCESS
    build 24-Mar-2020 18:13:42 INFO: ————————————————————————
    build 24-Mar-2020 18:13:42 INFO: Total time: 13.805s
    build 24-Mar-2020 18:13:42 INFO: Final Memory: 33M/349M
    build 24-Mar-2020 18:13:42 INFO: ————————————————————————
    simple 24-Mar-2020 18:13:42 Finished task ‘sonarqube source scanning’ with result: Success
    s

    Kind Rg
    Stefan

    Reply

    1. Hi Stefan. Thanks for emailing this question to me. I’m adding my response here in case it’s useful for anyone.

      I think the problem is with the latest version of Sonarqube, as specified in docker-compose.yml. For some reason it’s not generating the code coverage stats correctly. I have updated the GitHub repository and blog post to specify the version of lts (long term support) instead of latest

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Scroll to top