mvnd is amazing!

In this post, I write about my thoughts on mvnd, a daemon for Apache Maven, and how it drastically improved my development speed.

As with most of my recent Java development endeavours and discoveries, this one also started with the CommandAPI. The CommandAPI is getting larger and larger and now comprises of 43 different Maven modules! As the CommandAPI gets bigger, the time it takes to compile the CommandAPI gets longer and I was thinking of ways to speed it up without resorting to switching the entire codebase to Gradle which is notably known for being much faster than Maven builds.

Initial benchmarks

Starting with some preliminary information, these tests were run on my main machine which has an AMD Ryzen 9 5900X, and I have 64GB of available RAM. Running a full build of the CommandAPI for Bukkit is performed using this command, which cleans the project, then compiles and packages it (into .jar files). The -P Platform.Bukkit tells Maven to use the Platform.Bukkit profile which will only build Bukkit-specific modules:

mvn clean package -P Platform.Bukkit

Running this takes approximately 3 minutes on my machine. Not too bad, but we could do better! The mvn command has a -T flag which lets you specify the number of threads to run a build against. My processor has 12 cores, so I try breaking things up into 6 threads by using the following command:

mvn clean package -P Platform.Bukkit -T 6

This now takes approximately 1 minute 15 seconds. Using similar options and values (such as -T 1C to allocate one core per thread) seems to all yield similar results.

Reducing the build complexity

There are some fairly slow tasks that Maven does that we don’t need to do for every single run and are only really important when we’re making full release builds. Notably:

• Generating JavaDocs
• Running tests

Skipping JavaDoc generation is trivial, using the -Dmaven.javadoc.skip flag. Skipping tests can be done using -DskipTests, but because we’re now skipping the tests, we can save ourselves even more time by skipping compiling the tests in the first place! This can be done by also using the -Dmaven.test.skip=true flag.

Author’s note:

Although skipping tests is the quickest way to do things, ideally we’d want to improve the general speed of running tests overall. There are various ways to do that using parallel tests, forking multiple JVMs and all sorts of funky things, but in the context of the CommandAPI, forking multiple anything causes considerable slowdowns due to the massive initial time penalty of “starting up” a new test run using MockBukkit. For all intents and purposes, we’ll not delve into parallel tests in this post and just stick with skipping tests.

So, back to the benchmarks! Using no threads with the above flags now gives us this command, which takes approximately 40 seconds:

mvn clean package -P Platform.Bukkit -Dmaven.javadoc.skip -DskipTests -Dmaven.test.skip=true

And as expected, using multiple threads drastically improves things with this command, which takes approximately 25 seconds:

mvn clean package -P Platform.Bukkit -Dmaven.javadoc.skip -DskipTests -Dmaven.test.skip=true -T 6

But can we go even faster?

Introducing mvnd

mvnd is Maven, but faster. It uses the build techniques known from Gradle and Takari (some other smart build system built upon Maven that I know little about and won’t explain in any more detail than that) to make Maven go speed. Notably:

• mvnd uses a daemon to run builds - it has a background process that it hooks into to build things. The use of a daemon avoid the need to start up a new JVM every time something has to be built - it just uses the existing one.
• mvnd uses GraalVM instead of a normal JVM. GraalVM is effectively a native VM is generally just straight up faster.
• mvnd builds modules in parallel using multiple CPU cores. In short, it uses the -T flag by default and we don’t have to deal with it.

mvnd can be downloaded from their GitHub repository’s releases page here.

So, enough about mvnd and the technical stuff - let’s see the numbers because that’s all we care about! Taking our previous fastest command:

mvn clean package -P Platform.Bukkit -Dmaven.javadoc.skip -DskipTests -Dmaven.test.skip=true -T 6

we can transform this into the much faster mvnd equivalent by replacing mvn with mvnd, and omitting the -T flag because it uses parallel builds automatically:

mvnd clean package -P Platform.Bukkit -Dmaven.javadoc.skip -DskipTests -Dmaven.test.skip=true

Because mvnd runs a daemon the first time, I’ve decided to ignore the initial run, and instead test against a subsequent run (i.e. I only care about the time taken for the second build to run). Running the new mvnd command a second time takes approximately 14.5 seconds! Evidently, there has been some improvement to the build process!

The mvnd command also outputs a short bottleneck summary report which you can probably utilize to improve future builds, but I personally wasn’t able to find ways to reduce concurrency (lots of modules depending on other modules can’t be built concurrently!)

[INFO] ------------------------------------------------------------------------
[INFO] Average project wall time: 0.33s
[INFO] Total concurrency: 20%
[INFO] Bottleneck projects that decrease concurrency: (run build with -Dsmartbuilder.profiling=true for further details)
[INFO]  - dev.jorel:commandapi-bukkit-core:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-1.19.1:9.0.4
[INFO]  - dev.jorel:commandapi-core:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-test-impl-1.19.4:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-vh:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-test-impl:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-test-tests:9.0.4
[INFO]  - dev.jorel:commandapi-bukkit-nms-dependency:9.0.4
[INFO]  - dev.jorel:commandapi-preprocessor:9.0.4
[INFO]  - dev.jorel:commandapi-codecov:9.0.4
[INFO]  - dev.jorel:commandapi:9.0.4
[INFO] ------------------------------------------------------------------------

Summary

Is there even anything I need to say here? mvnd is amazing! Of course, this was also helped by skipping JavaDocs and tests which greatly improved build times, but knowing that I have been able to go from building things in ~3 minutes to building things in ~15 seconds just by adding some simple flags and adding one letter to my main build command is something worth praising.