The Unknown Maven Repository System Powering Java

The Unknown Maven Repository System Powering Java

When a Java developer hears Maven they probably think of the popular build tool. But there’s another side to Maven: a vital repository system serving up dependencies needed to build Java applications.

In this article you’ll discover why the Maven repository system is so important and how to use it effectively in your own project.

In today’s world of cheap cloud storage a repository might not sound like anything special, but the Maven repository format was designed pre-AWS to store all kinds of libraries loved by Java developers. And it’s still in use today.

These libraries are stored as jar file artifacts, and can be pulled as dependencies into:

  • Maven builds
  • Gradle builds
  • SBT builds
  • and even Ant builds

Whatever your choice of build tool, the code from the library becomes available from within your application code so you don’t have to reinvent the wheel.

The Before Maven period

History can roughly be divided into two periods. There is the painful pre-2004 Before Maven period and the mostly happy post-2004 After Maven period.

pre and post Maven periods

Before Maven most Java engineers were using the Ant build tool, and storing any required libraries in the version control system.

<project name="HelloWorld" basedir="." default="main">
    ...
    <property name="lib.dir" value="lib"/>
    <path id="classpath">
        <fileset dir="${lib.dir}" includes="**/*.jar"/>
    </path>
    ...
    <target name="compile">
        <mkdir dir="${classes.dir}"/>
        <javac srcdir="${src.dir}" destdir="${classes.dir}" classpathref="classpath"/>
    </target>
    ...
</project> <!-- code snippet from Ant tutorial https://ant.apache.org/manual/tutorial-HelloWorldWithAnt.html -->

Yes, every single library and transitive dependency was downloaded and stored in the project along with the source code! 😲

Better dependency handling with Maven

When Jason van Zyl built Maven, part of the Apache Software Foundation, he included a ground-breaking new system for downloading artifacts from a remote repository.

On the opposite side Maven also included a way to publish to a repository in a custom format. This format made it easy for consumers to fetch any transitive dependencies.

Maven repository system overview
Jason van Zyl’s Maven repository system

The first Maven repository is what’s known today as Maven Central. It’s a community-lead repository that anyone’s free to publish artifacts to.

By pulling dependencies from a central location you not only save on version control storage space, but it also means dependencies can be managed more effectively from your build tool.

To understand how all this works, let’s get into some implementation details of the Maven repository system.

The Maven repository format

To build an application using the Maven build tool requires a pom.xml file (stands for Project Object Model). It contains an essential group id, artifact id, and version, known as the coordinates of the project.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.tomgregory</groupId>
    <artifactId>maven-demo</artifactId>
    <version>1.0-SNAPSHOT</version>
    ...
</project>

When the Maven build tool builds a project, generates a jar file, and publishes it to a remote Maven repository, the coordinates are used to calculate the jar file URL.

For example, here’s how the URL looks for the Apache commons-lang3 artifact within the Maven Central repository.

https://repo1.maven.org/maven2/org/apache/commons/commons-lang3/3.12.0/commons-lang3-3.12.0.jar

It consists of a:

  • repository prefix https://repo1.maven.org/maven2
  • group id org/apache/commons
  • artifact id commons-lang3
  • version 3.12.0
  • file name commons-lang3-3.12.0.jar

If we browse the directory containing the jar file, we see other files have also been published.

Contents of Maven Central artifact directory
Contents of commons-lang3 directory in Maven Central

For example, there’s the commons-lang3-3.12.0.pom XML file which importantly contains a list of dependencies of this artifact.

Transitive dependencies of commons-lang3
Transitive dependencies of commons-lang3

That means when a build tool downloads this artifact, it can also download transitive dependencies at the same time. We’ll get into that shortly with a worked example in Maven and Gradle.

Now that you understand the Maven repository format, let’s learn about Maven Central in more detail.

Maven Central: the home of open source Java

Maven Central is a repository hosted at https://repo1.maven.org/maven2 containing over 8 million artifacts.

The maven2 in the URL refers to Maven version 2 released in 2005. Even though version 3 of the Maven build tool was released in 2010, it still uses the repository version 2. If you think this is a confusing naming convention, I completely agree!

But the other confusing thing is that the Maven Central repository is actually operated by a company called Sonatype. You might know them just from the strange name or from their popular repository tool Nexus, which stores private artifacts in the Maven repository format.

Searching artifacts in Maven Central

If you go to https://search.maven.org you can search for any library you like, see all available versions, see the transitive dependencies of a particular version, and view the directory structure as seen earlier.

Searching spring-boot-starter-web at https://search.maven.org
Searching spring-boot-starter-web at https://search.maven.org

An alternative way to search Maven artifacts is at https://mvnrepository.com.

This is a search engine which not only indexes Maven Central, but other repositories too. If we search for the same artifact here we get similar information, but I think it’s easier to browse transitive dependencies through this site.

Transitive dependencies at mvnrepository.com
Viewing transitive dependencies of spring-boot-starter-web at https://mvnrepository.com

The site also gives you the actual code needed to depend on an artifact in a variety of build tools.

Code to depend on artifact

Let’s see how this looks for Maven and Gradle.

Add a dependency in Maven

So with Maven I can copy the XML snippet provided by mvnrepository.com and paste it into my pom.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    ...
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <version>2.6.4</version>
        </dependency>
    </dependencies>
</project>

When I run mvn package to build this project, Maven will:

  1. download the artifact from Maven Central if it doesn’t already exist locally
  2. download any transitive dependencies of the artifact
  3. add the artifacts to the Java compile classpath
Result of mvn package
Result of running mvn package

Within the IDE my code now has access to code from the spring-boot-starter-web library, and its dependencies. Nice!

Access to code from spring-boot-starter-web in IDE

Add a dependency in Gradle

For Gradle, it’s just a one line code snippet, which I paste into the Gradle build script.

plugins {
    id 'java'
}

repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.springframework.boot:spring-boot-starter-web:2.6.4'
}

Running ./gradlew build also causes the artifact and its transitive dependencies to be downloaded and added to the compile classpath.

Your personal private Maven repository

But what happens when you don’t want anyone else seeing your precious code in your published artifact?

Well, you need to use a private repository. There are several options, including:

  • Sonartype’s Nexus
  • Artifactory
  • AWS CodeArtifact

CodeArtifact is my favourite. It’s a fully managed AWS service, so you can just click a button to create a repository and start publishing to it.

Creating a repository with AWS CodeArtifact
It’s easy to create a new private Maven repository with AWS CodeArtifact

CodeArtifact provides instructions for publishing to and depending on your private repository in a variety of build tools, including Maven and Gradle.

Here’s how to setup publishing in Gradle’s Groovy build script.

...
publishing {
  publications {
      mavenJava(MavenPublication) {
          groupId = '<groupId>'
          artifactId = '<artifactId>'
          version = '<version>'
          from components.java
      }
  }
  repositories {
      maven {
          url '<your-private-repo-url>'
          credentials {
              username "aws"
              password System.env.CODEARTIFACT_AUTH_TOKEN
          }
      }
  }
}

With this in place, when I run ./gradlew publish I end up with an artifact in my private repository using the Maven format.

That artifact can then be used in whatever Java projects I like by referencing the private repository in the Gradle build script.

...
repositories {
    maven {
        url '<your-private-repo-url>'
        credentials {
            username "aws"
            password System.env.CODEARTIFACT_AUTH_TOKEN
        }
    }
}

Then adding an appropriate dependency.

...
dependencies {
    implementation '<groupId>:<artifactId>:<version>'
}

Final thoughts

That was an overview of the Maven repository system. I hope you can now see Maven as more than just a build tool.

As mentioned, Maven repositories are essential for building Java projects with Gradle. If you want to get started with Gradle or just understand the fundamentals, check out this awesome course. 👇

Get going with Gradle course
Gradle icon

Want to learn more about Gradle?
Check out the full selection of Gradle tutorials.

The Unknown Maven Repository System Powering Java

Leave a Reply

Your email address will not be published.

Scroll to top

FREE GRADLE COURSE!

Getting started with Gradle just got A LOT easier!

Get Going with Gradle is the fastest way to a working knowledge of Gradle.

✅ Work effectively in basic Gradle projects
✅ Know how to setup Java projects in Gradle
✅ Understand the Gradle fundamentals

And it's yours today for FREE!

Where shall I send your access details?