Building a Spring Boot application in Docker and Jenkins (part 2 of microservice devops series)

Building a Spring Boot application in Docker and Jenkins

Last Updated on November 4, 2020

Welcome to the second of this three part series where you’ll learn how to take a Spring Boot microservice from inception to deployment, using all the latest continuous integration best practices.

In this second article we’ll be updating the Spring Boot service we built in Part 1 and getting it running inside a Docker container. Then we’ll setup our Jenkins instance to work with Docker, and create a pipeline to build the image and push it to Docker Hub.

This series of articles includes:

Did you read Part 1 already? If not, check it out as we’ll be building on top of the example from that article.

UPDATED in September 2020 to use the latest version of Jenkins (2.249.1)

1. Running a Spring Boot application in Docker

In the previous article we created this Spring Boot API application to model a theme park, including getting and creating rides. 🎢

Let’s take things forward and get the application running in Docker. Some reasons we might want to do this include:

  • more easily package and deploy the code (e.g. with a Docker image Java comes pre-installed)
  • use modern container orchestration services like Kubernetes or AWS ECS
  • make development simpler, by allowing us to more easily start up multiple associated services or microservices

Dockerfile

A Docker image is created from a Dockerfile. It’s just a series of instructions describing how to create an image. On a high level, let’s think about what we’ll need in the image to run our Spring Boot application:

  1. have Java installed (we’re using Java 11 right now)
  2. copy the Spring Boot jar file into the container
  3. execute the Spring Boot jar file with the java command

Let’s go ahead then and create a Dockerfile in the root of the Spring Boot project:

FROM openjdk:11
ARG JAR_FILE
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]
  • the FROM instruction tells Docker that we’re going to extend an existing base image, in this case an image for OpenJDK 11
  • the ARG instruction defines a variable that can be passed in at build time, in this case to provide the name of the Spring Boot jar file
  • the COPY instruction allows us to copy a file into the image, in this case the Spring Boot jar file
  • the ENTRYPOINT instruction describes what should be run when the container starts up, in this case we want to execute the jar file with the java command

Building a Docker image in Gradle

Let’s extend the build.gradle from part 1 so that we can build and run our Docker image by executing a Gradle task. The best plugin I’ve found to do this is the Palantir Gradle Docker plugin, which exposes useful tasks like docker to build the image and dockerRun to run it.

It consists of several plugins, which you can pick and choose depending on what functionality you need. Let’s apply the following two Gradle plugins in our build.gradle, so we can build images and run containers.

plugins {
    ...
    id 'com.palantir.docker' version '0.25.0'
    id 'com.palantir.docker-run' version '0.25.0'
}
Building the image

As you probably know, some Gradle plugins require additional configuration to set them up. We’ll configure the com.palantir.docker plugin with this configuration block:

String imageName = "tkgregory/spring-boot-api-example:$version"

docker {
    name imageName
    files "build/libs/${bootJar.archiveFileName.get()}"
    buildArgs([JAR_FILE: bootJar.archiveFileName.get()])
}
  • we’re defining an imageName variable as we’ll need to reference it in multiple places (the tkgregory prefix you can change later on when we cover pushing the image to your own Docker Hub account)
  • we configure the Docker plugin name property to create an image with the specified name
  • the files property exposes the specified file to be available for COPY instructions in the Dockerfile. We’re referencing the archiveFileName from the Spring Boot Gradle plugin to get the correct jar file name.
  • we’re passing the JAR_FILE build argument to the Dockerfile, so it can execute the COPY instruction

Run ./gradlew assemble docker and the image will build:

Info: the first time you execute the docker Gradle task it may take some time as Docker needs to download the large base image.

We can verify that our image has been built by running docker images:

Running the image

Next up, let’s configure the com.palantir.docker-run plugin:

dockerRun {
    name project.name
    image imageName
    ports '8080:8080'
    clean true
}
  • the name of our container can be the same as our project e.g. spring-boot-api-example
  • the image refers to the image which we will have built from our previous configuration. We’re reusing the imageName variable.
  • the ports we configure are 8080 on the host and 8080 on the container, which is where our Spring Boot application is listening by default
  • clean true means that when we stop our container it will be automatically deleted, saving us a bit of time

Now let’s run ./gradlew assemble docker dockerRun to run a container from our image:

We can run docker ps to validate that the container is running:

Awesome! Let’s hit the application’s http://localhost:8080/actuator/health endpoint just to make sure:

If you want to stop the container, you can just run ./gradlew dockerStop.

GitHub repository
If you want to jump straight to the good stuff, the code above is available for you in the spring-boot-api-example repository.

2. Pushing to Docker hub

In a real-world scenario we’re going to need to push our Docker image to a remote repository in order to then pull it onto a server or cloud service to run it. Our setup, from the point of view of our continuous integration service (i.e. Jenkins) might look like this:

Docker registries

In Docker lingo, a repository is where you can store one or more versions of a specific Docker image. A registry, on the other hand, is a remote service that stores a collection of repositories.

Docker Hub is a Docker registry providing free storage of public and private images (you only get to store 1 private image with a free account).

If you’re using AWS, then you might want to consider AWS ECR (Elastic Container Registry) instead, as you’ll be able to keep everything within your private AWS infrastructure.

Docker login and Docker push

To use Docker Hub, go ahead and setup an account. Before you can push images you’ll need to login with the command docker login --username=<username>. Enter your password when prompted.

Info: if you’re using Git Bash on Windows, like me, you’ll need to prefix the command with winpty

Edit the build.gradle of the Spring Boot API application and replace tkgregory in the imageName variable with your own Docker Hub username i.e. String imageName = "<your-docker-hub-username>/spring-boot-api-example:$version"

To push the image to Docker Hub just run ./gradlew docker dockerPush and wait for it to upload:

Now over in Docker Hub you’ll be able to see your image in your own account repository at https://hub.docker.com/r/<your-docker-hub-username>/spring-boot-api-example:

A word of warning
Note that by default when you push an image to Docker Hub it will be public for the world to see. See this guide to learn about setting up private repositories.

OK, so that’s our application all setup. Time to jump through a few hoops to get things working with Docker in Jenkins.

3. Configuring Jenkins to be able to run Docker in Docker

In part 1 we made use of the jenkins-demo GitHub repository (the theme-park-job branch) to run an instance of Jenkins locally using our own custom Dockerfile. We’re going to make some modifications to:

  • install Docker in the Jenkins Docker image
  • hook up the Docker command in the Jenkins image to our Docker installation on the host. This will allow us to run ‘Docker in Docker’.
Installing Docker in Jenkins (Docker in Docker)

Update the Dockerfile for Jenkins to the one below, which now includes installing Docker from https://get.docker.com:

FROM jenkins/jenkins:2.225

USER root
RUN curl -sSL https://get.docker.com/ | sh
RUN usermod -a -G docker jenkins
USER jenkins

COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt

COPY seedJob.xml /usr/share/jenkins/ref/jobs/seed-job/config.xml

ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false
Updating the Gradle Docker plugin

Now in the build.gradle, we need to make sure we’re using the latest version of the docker and docker-run plugins. At the time of writing this is 0.25.0:

plugins {
    ...
    id 'com.palantir.docker' version '0.25.0'
    id 'com.palantir.docker-run' version '0.25.0'
    ...
}

docker-run plugin version
The latest version of the docker-run plugin supports passing custom arguments to the Docker command

Making /var/run/docker.sock available to Jenkins

In order that when we run Docker commands in Jenkins they communicate with the Docker process running on the host, we have to use a file /var/run/docker.sock. This file must be mounted as a volume inside the Jenkins container.

Update the dockerRun configuration in build.gradle to as below:

Process process
if (DefaultNativePlatform.getCurrentOperatingSystem().isWindows()) {
    process = "docker run --rm -v /var/run/docker.sock:/var/run/docker.sock alpine stat -c %g /var/run/docker.sock".execute()
} else if (DefaultNativePlatform.getCurrentOperatingSystem().isLinux()) {
    process = "stat -c %g /var/run/docker.sock".execute()
} else {
    throw new GradleException("Unsupported operating system. No way to find group of /var/run/docker.sock.")
}
def out = new ByteArrayOutputStream()
process.waitForProcessOutput(out, System.err)
String dockerSockGroupId = out.toString().trim()

dockerRun {
    name "${project.name}"
    image "${project.name}:${project.version}"
    ports '8080:8080'
    clean true
    daemonize false
    arguments '-v', '//var/run/docker.sock:/var/run/docker.sock', '--group-add', dockerSockGroupId
}

You’ll also have to include this import at the top of the file:

import org.gradle.nativeplatform.platform.internal.DefaultNativePlatform
  • in the first 9 lines we’re determining if we’re on a Linux or Windows machine. Depending on the outcome, we run a specific command to find out which group owns /var/run/docker.sock and then store that value as dockerSockGroupId.
  • dockerRun has been updated to pass 2 new arguments
    • the -v argument mounts //var/run/docker.sock inside the container so that Jenkins can use the Docker process of the host
    • the --group-add argument makes sure the container is also run with the group that owns /var/run/docker.sock, so that Jenkins has the required access

Docker host in Windows
If you’re using Docker for Windows, like me, the host from the container’s point of view is not Windows but a Linux VM running inside Windows. For a full explanation of Docker in Docker see my article Running Docker in Docker on Windows (Linux containers).

At this point you can run Jenkins using ./gradlew docker dockerRun, but there are still a few tweaks to add in the following section. Don’t forget to first stop the Spring Boot API application as it’s also running on port 8080.

To double check that you can run Docker in Docker, run docker exec jenkins-demo docker ps:

4. Building the Docker Spring Boot API application in Jenkins

In part 1 we had the application building in this Jenkins pipeline, consisting of checkout, build, and test stages:

Let’s extend this pipeline to include stages to build the Docker image and push it to Docker Hub.

Updating the Jenkinsfile

Remember that the pipeline above is stored in a file called Jenkinsfile in our Spring Boot API application repository? Add the additional Build Docker image and Push Docker image stages shown below:

pipeline {
    agent any

    triggers {
        pollSCM '* * * * *'
    }
    stages {
        stage('Build') {
            steps {
                sh './gradlew assemble'
            }
        }
        stage('Test') {
            steps {
                sh './gradlew test'
            }
        }
        stage('Build Docker image') {
            steps {
                sh './gradlew docker'
            }
        }
        stage('Push Docker image') {
            environment {
                DOCKER_HUB_LOGIN = credentials('docker-hub')
            }
            steps {
                sh 'docker login --username=$DOCKER_HUB_LOGIN_USR --password=$DOCKER_HUB_LOGIN_PSW'
                sh './gradlew dockerPush'
            }
        }
    }
}
  • for the Build Docker image stage we’re using the Gradle Docker plugin to build the image
  • for the Push Docker image stage we’re grabbing the docker-hub credentials from Jenkins and storing it as an environment variable. We’ll cover how to add the credentials shortly. We then use this variable to run a docker login command, and once we’re logged in we can push the image to Docker Hub using the dockerPush Gradle command.

Credentials in pipelines
When you store a username with password type credential in an environment variable in a Jenkins pipeline, the username and password are automatically separated out into two additional variables. Read more about this here in How To Secure Your Gradle Credentials In Jenkins.

If you’re following along with this example by building up your own project, make the change above and push it. Alternatively, the change is also available in Jenkinsfile-docker, a custom Jenkinsfile I’ve pushed just for you (I’ll show you how to use it in the next section).

Adding the new Docker job definition

Lastly, in createJobs.groovy of the jenkins-demo project add the following job definition:

pipelineJob('theme-park-job-docker') {
    definition {
        cpsScm {
            scm {
                git {
                    remote {
                        url 'https://github.com/tkgregory/spring-boot-api-example.git'
                    }
                    branch 'master'
                    scriptPath('Jenkinsfile-docker')
                }
            }
        }
    }
}

This will create a job based on the Jenkinsfile-docker file discussed earlier. If you want this to refer to your own repository, change the url and scriptPath appropriately. Also, don’t forget to change the location in seedJob.xml to point to your own appropriate git URL and branch to fetch the createJobs.groovy file.

Now all we have to do is run ./gradlew docker dockerRun to start up Jenkins:

Adding credentials to Jenkins

Let’s add our Docker Hub credentials into Jenkins, by navigating through the somewhat long-winded following series of pages…

In the left hand navigation menu select Manage Jenkins then Manage Credentials:

Manage Jenkins

Select the Jenkins credential store:

Select Global credentials:

Click Add Credentials:

Finally we’re here! Now add a Username with password credential, entering your Docker Hub username, password, and an id of docker-hub. Click OK to save your credentials in Jenkins.

Running the new Docker pipeline

Now for the final act! 🎭 Build the seed-job, then build the newly created theme-park-job-docker:

Awesome! We now have a Jenkins job that builds, tests, and pushes an image for our Spring Boot API application.

4. Final thoughts

Having a pipeline that will automatically take code as soon as it’s pushed and put it in a Docker image in Docker Hub is a very good place to be.

In the final article in this series, we’ll be putting the cherry on top by taking our Docker image and deploying it into AWS.

Get the newsletter

Found this article helpful? Subscribe for monthly updates.

✅ All of my latest articles for the month
✅ Access to video tutorials
✅ Exclusive tips not found on my website

5. Resources

GITHUB
Check out the sample Spring Boot application for the theme park API. It now contains the Jenkinsfile-docker file which includes Docker build and push stages in the pipeline.
Here you can find the code for the jenkins-demo project, for bringing up an instance of Jenkins. The theme-park-job-docker branch contains the updated version of createJobs.groovy as well as the modifications needed to run Docker in Docker.

RELATED ARTICLES
Discover more details about how to secure credentials in Jenkins in this article.

VIDEO
If you prefer to learn in video format then check out the accompanying video below. It’s part of the Tom Gregory Tech YouTube channel.

Building a Spring Boot application in Docker and Jenkins (part 2 of microservice devops series)

14 thoughts on “Building a Spring Boot application in Docker and Jenkins (part 2 of microservice devops series)

  1. Hi gregory, i have a problem when i run ./gradlew docker dockerRun on jenkins-demo, that gives me the next response “> Task :docker FAILED
    The command ‘/bin/sh -c usermod -a -G docker jenkins’ returned a non-zero code: 6” and “FAILURE: Build failed with an exception.

    * What went wrong:
    Execution failed for task ‘:docker’.
    > Process ‘command ‘docker” finished with non-zero exit value 6″ and i don’t know what to do, what’s happening here, i’m gonna be grateful with your response. salutes from colombia.

    1. Hi Santiago. I believe you’re seeing an error on the theme-park-job-docker branch of the jenkins-demo repository? RUN usermod -a -G docker jenkins is part of the Dockerfile, which is building fine for me when I run ./gradlew docker.

      A couple of ideas:

      • try running docker build .. Do you get the same error?
      • have you modified the Dockerfile in any way?
      • what OS are you building from?

      Let me know how you get on, as I’m keen to resolve your issue.

      1. Hello gregory, i resolved the issue modified the DockerFile, i think something was wrong with “RUN usermod -a -G docker jenkins” so i changed the lines 1 and 6 and then modified with…
        ” FROM jenkins/jenkins:lts
        USER root
        RUN apt-get update
        RUN curl -fsSL https://get.docker.com/ | sh
        RUN usermod -aG docker jenkins
        USER jenkins”

        Also thank you very much for your help, i appreciate it.

  2. Hi Gregory,

    I’m afraid, there is a bug in your build.gradle that took me quite a while to pin down:
    ” if (DefaultNativePlatform.getCurrentOperatingSystem().isWindows()) {
    process = “docker run –rm -v /var/run/docker.sock:/var/run/docker.sock alpine stat -c %g /var/run/docker.sock”.execute()
    } else {
    process = “stat -c %g /var/run/docker.sock”.execute()
    } ”

    I’m working with a Mac and get the following error:
    ” stat: illegal option — c
    usage: stat [-FlLnqrsx] [-f format] [-t timefmt] [file …]
    > Task :dockerRun FAILED
    docker: Error response from daemon: Unable to find group.
    FAILURE: Build failed with an exception. ”

    This is because for the stat command “c” is not a valid option. You want to use “f” here. But nevertheless, changing this only sets the group of /var/run/docker.sock in your host system (for me: “daemon”) but not the group of this file in the docker image (for me: “root”).
    I ended up fixing this by hard coding dockerSockGroupId = “root” for unix systems. Clearly not the best solution.
    Do you have a better way to solve this?

    Cheers,
    Matthias

    1. Hi Matthias,

      Sorry about the issue you found. I’ll be honest I hadn’t tested it on a Mac as I don’t have access to one. Not a great reason, but there you go 😉

      I’ve made the support explicit to Linux for use of the stat -c %g /var/run/docker.sock command. If you try with any other OS except Windows and Linux the build will fail now.

      Obviously that’s no good for Mac users. Anyone trying out this this example would greatly appreciate a solution if you have time. You can make a PR for this file if you are able.

      If not, I can always hard code the dockerSockGroupId to root for Mac. What do you think?

      Tom

  3. Thanks Tom. Brilliant work!! I am running this on a MacOs and had to make certain changes as well. What process id are we expecting to see. My code is always returning thr value as 1. Also, I am running into an issue where the “exec docker” part of the gradlew execution commands does not return to the command prompt as you demonstrated. Any idea?

    1. Hi Raman. Thanks for visiting the site.

      You just need to determine the id of the group that owns `/var/run/docker.sock`. Can you try hard coding dockerSockGroupId as root as suggested by Matthias?

      When you say “exec docker” do you mean this command isn’t working docker exec jenkins-demo docker ps? Please let me know the error as well as any other problems you’ve had.

      Thanks, Tom

  4. Thanks Tom!! That was real quick and I really appreciate it. Are you suggesting to set the value as this?
    String dockerSockGroupId = “root”

    After I added the following code, I get a value of 1 assigned to dockerSockGroupId which seems like the wrong value.

    else if (DefaultNativePlatform.getCurrentOperatingSystem().isMacOsX()) {
    process = “stat -f %g /var/run/docker.sock”.execute()
    }

    Also, when I run the gradlew assemble, it comes back to the prompt in not time and I do see the jar file in the ditrribution folder. On the other hand, when I run ./gradlew docker or ./gradlew asemble docker, it just get to this state and never comes out.

    + exec /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/bin/java -Xmx64m -Xdock:name=Gradle -Xdock:icon=/Users/aastha/s2/jenkins-setup/media/gradle.icns -Dorg.gradle.appname=gradlew -classpath /Users/aastha/s2/jenkins-setup/gradle/wrapper/gradle-wrapper.jar org.gradle.wrapper.GradleWrapperMain docker
    1

    > Configure project :
    rootProcess[pid=17328, exitValue=0]
    66% EXECUTING [10s]
    > :docker

  5. Nevermind, I found the issue. On Mac, the docker container gets locked and the only way to get around that is to restart Docker which is painful to say the least..

  6. Anyone else faced this issue the moment you include the gradle plugins?

    Error:
    Illegal null value provided in this collection: [inspect, –format={{.State.Running}}, null]

    id ‘com.palantir.docker’ version ‘0.25.0’
    id ‘com.palantir.docker-run’ version ‘0.25.0’

    gradle -version

    ————————————————————
    Gradle 6.6.1
    ————————————————————

    Build time: 2020-08-25 16:29:12 UTC
    Revision: f2d1fb54a951d8b11d25748e4711bec8d128d7e3

    Kotlin: 1.3.72
    Groovy: 2.5.12
    Ant: Apache Ant(TM) version 1.10.8 compiled on May 10 2020
    JVM: 14.0.1 (Oracle Corporation 14.0.1+14)
    OS: Mac OS X 10.15.7 x86_64

    1. Hi Sriram. I haven’t seen this issue before. Can you try running the Gradle command with –stacktrace to see where the error happens?

      What version of Docker are you using?

    2. I got something like this on gradle 6.6.1 (different project, but also using com.palantir.docker-run). The error went away when I switched back to gradle 6.5.

Leave a Reply to Sriram Cancel reply

Your email address will not be published. Required fields are marked *

Scroll to top

Get the newsletter

Found this article helpful? Subscribe for monthly updates.

✅ All of my latest articles for the month
✅ Access to video tutorials
✅ Exclusive tips not found on my website