How we handle binary compatibility at Apache Commons

undefined

As a maintainer of the free and open source software project Apache Commons, I review pull requests on GitHub. Since the libraries we produce in components like Commons Lang, Commons IO, and Commons VFS are used directly and transitively in countless applications, open and closed source alike, we want to be careful when releasing new versions to cause the least headaches as possible. One way to enforce our least-headache policy is a recurring reason PRs are rejected: The PR breaks binary compatibility. I’ll outline here how we deal with this issue while still allowing our APIs to evolve.

What is binary compatibility?

Maintaining binary compatibility, in a nutshell, is the ability to drop a new version of a class file (in a jar for example) on top of your application and have it run without the Java Virtual Machine throwing an error trying to call the updated code.

At the JLS level, this means binary compatibility is preserved when a binary that previously linked without error continues to link without error.

Binary compatibility is… well, a binary quality: A release either is or isn’t binary compatible with its previous version.

If you remove a class or method and your app tries to call it, you break binary compatibility and your app blows up.

If you add or remove a parameter to a method, you break binary compatibility and, your app blows up.

This is all detailed in the Java Language Specification.

Note that this is different from source compatibility and behavioral compatibility.

Benefiting from binary compatibility

Why do this? To avoid Jar hell and needlessly breaking applications. See also DLL hell.

Defining guidelines at Apache Commons

Our guideline at Apache Commons on binary compatibility is:

Don’t break binary compatibility in a minor release (e.g. 2.0 to 2.1) or a maintenance release (e.g. 2.1.2 to 2.1.3).

For a major release (e.g. 2.2 to 3.0), you may break binary compatibility; and if you do, you must change the Java package name and Maven coordinate’s artifact ID.

For example, when Apache Commons Lang 3.0 was released, we updated the Java package from org.apache.commons.lang to org.apache.commons.lang3 and the Maven artifact ID from commons-lang to commons-lang3.

In the case of Apache Commons Lang, the 1.x and 2.x series of releases live in the same package and Maven coordinates.

This system allows, and this is key, for version 2 and 3 to co-exist in the same class loader. While the same is true for version 1 and 3, it is not for 1 and 2.

What am I allowed to change?

Almost everything, according to the Java Language Specification, you may:

  • Rewrite the body of methods, constructors, and initializers (like static blocks).
  • Rewrite code in the above that previously threw exceptions to no longer do so.
  • Add fields, methods, and constructors.
  • Delete items declared private.
  • Reorder fields, methods, and constructors.
  • Move a method higher in a class hierarchy.
  • Reorder the list of direct super-interfaces in a class or interface.
  • Insert new class or interface types in a type hierarchy.
  • Add generics (since the compiler erases them).

In addition, our guidelines allow for updating package private element.

Checking for binary compatibility

When a release candidate is put up for a vote, it is proposed along with a web site generated by Maven which contains a report on binary compatibility. There are several tools you can use to do this, we usually use the JApiCmp Maven plugin. For example, this is the report for the current version of Apache Commons Lang.

A pattern I like using is providing a default Maven goal which includes JApiCmp which I invoke for builds on Travis-CI and GitHub Actions. For example:

<defaultGoal>clean verify apache-rat:check japicmp:cmp checkstyle:check spotbugs:check javadoc:javadoc</defaultGoal>

Wrapping up

All of this to say, as a library author:

  • Don’t give your users headaches or put their apps in Jar hell by maintaining binary compatibility within a major release line.
  • Use a build tool to fail a build that breaks binary compatibility.
  • When you must break binary compatibility, change the Java package name and Maven coordinates (this works for Ivy, Gradle, and so on).

Explaining Terms at Apache

At Apache, the term project refers to a whole top level project (TLP).

Apache Commons is a TLP.

Apache Commons is made up of over 20 components.

Apache Commons components produce one or or more jar files. The components are listed on the Commons’ project main page.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s