Apache Commons logo

Introduction

This document contains a mixture of information, advice and examples. It is intended to be a recommendation of best practices for Commons components. The instructions provided here are consistent with, but not a replacement for the ASF Release Guidelines.

The Apache Commons project uses the Nexus installation at the Apache Software Foundation to stage Maven artifacts before the release and later publish them; the Commons Parent POM contains all the necessary configuration. This guide covers deploying to Nexus using Maven. For alternatives see the ASF wide documentation. If this is the first time you are publishing to the ASF's Nexus instance you'll need to prepare your development environment.

The examples below assume that preparation is being made to release version 1.2 of component Foo.

Build Environments

Commons components are expected to use Maven to build the project website. Components may choose to use either Maven or Ant to build the actual jar and tar/zip files to be distributed, although it is recommended that Maven be used for this. Both approaches are covered below. The version of Maven used is assumed to be Maven 3 throughout. At a minimum, Commons releases must include full source distributions packaged in tar and zip archives.

This document assumes that the release is being prepared on a linux/unix system, and that steps are being executed from the command-line. The principles are the same, however, for a release prepared on other operating systems.

Preparation

Select a Release Manager

A commons committer (normally one of the development team) should post an email to the development list proposing that a release be made and nominating a release manager. Typically, the proposer volunteers as the release manager and it passes by lazy consensus.

If the release manager has not yet performed a Commons release, they may need to add their public key to the Commons KEYS file, which is located at: https://dist.apache.org/repos/dist/release/commons/KEYS. Note that the SVN directory contains the current releases of all "proper" Commons components, so use a command such as the following to check it out without downloading every Commons project:

        svn co --depth=files https://dist.apache.org/repos/dist/release/commons/
      

The public key should also be uploaded to a public keyserver.

A release plan should also be prepared, in which the tasks remaining to be done before the release are listed. It may be useful to prepare draft release notes before proposing the release, so that others can see what changes are expected for the new release.

Consider a Release Branch

Consider whether a release branch is needed before preparing for the new release. In general, Commons components are small enough that there is no need for a release branch, but if active development will continue on the next version while a release is being made then trunk should be branched to allow this. If a release branch is taken then work will be required to merge any changes back into the trunk.

The following is one way to create a release branch:

        cd to the project's base directory in your subversion working copy.
        svn update trunk
        svn cp trunk branches/foo-1.2-release
        svn commit -m "Creating foo-1.2-release branch" branches/foo-1.2-release
      

Note that the "svn update" step is necessary in order to ensure that the directory being copied does not have a mix of files at various revisions; even if the files haven't changed since the last svn update this can cause "svn log -v" on the new directory to report files as having been (R)eplaced.

Alternatively, use "svn cp URLsrc URLtag"

        svn cp -m "Creating foo-1.2-release branch" \
            https://svn.apache.org/repos/asf/commons/proper/foo/trunk \
            https://svn.apache.org/repos/asf/commons/proper/foo/branches/foo-1.2-release
      

which will copy files internally within the repository without using the local working copy: this always ensures a clean copy is made.

The description below assumes a release is being prepared using the code in trunk. The process is nearly identical when preparing from a release branch: only the directory in which the work is performed is different.

Check Compatibility

Consult the Commons Versioning Guidelines and check that the current level of compatibility is suitable for the proposed version number. Check the current level of compatibility in the code. Tools like Clirr and JDiff are very useful. Both support Ant and Maven.

Check Javadocs And Code Style

Ensure all new classes and methods have @since tags. Compatibility reports produced in the last section may prove helpful.

Ensure no errors or warnings are reported by the Javadoc tool. Check that the Javadocs have the correct version number.

If the component uses checkstyle, findbugs or PMD tools, examine the reports and fix all problems.

Configure the Build to Generate a Complete Set of Release Artifacts

For builds using Maven, the contents of the source and binary distributions are configured in assembly descriptors. By convention, these are stored in src/main/assembly and named src.xml and bin.xml, respectively. Names and locations for these files are specified in the maven-assembly-plugin configuration in the pom. Make sure that all (and only) files that should be included in the source and binary distributions are included in the fileset elements of the descriptors. Look at some recently released components' descriptors for comparison. At a minimum, both source and binary distributions must include the release notes, license and notice files.

Update the version numbers in pom.xml and build.xml to the new release version, in this example, 1.2. For Maven builds, make sure that the build properties are properly set. Review the properties section of the pom:

      <properties>
        <commons.componentid>foo</commons.componentid>
        <commons.release.version>1.2</commons.release.version>
        <commons.rc.version>RC1</commons.rc.version>

        <-- properties not related to versioning -->
        <commons.jira.id>FOO</commons.jira.id>
        <commons.jira.pid>007</commons.jira.pid>
        <maven.compile.source>1.3</maven.compile.source>
        <maven.compile.target>1.3</maven.compile.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
      <properties> 

Make sure that the release version is set to the new release and that the compile and source targets are set correctly. Generate and check in a new download page for the component:

      mvn commons:download-page
      svn commit -m "Updated download page in preparation for 1.2 release." src/site/xdoc/download_foo.xml 
Note that the target directory must exist beforehand, and that for a multi-module project the -N (non-recursive) parameter should be specified on the mvn command line.

When using Ant, typically the Ant "dist" target produces the source and binary distributions. Included files are specified in the target or sub-targets and should be verified similarly to above.

Test the "Ant dist" or "mvn assembly:assembly" command and inspect the tars/zips and jars produced. Verify that

  1. "Ant dist" or "mvn site" succeeds from the unpacked source distribution.
  2. The jar manifests include LICENSE and NOTICE files and the contents of these files are correct.
  3. The jar manifests contain X-Compile-Source-JDK and X-Compile-Target-JDK entries set to the correct values.
  4. The jar manifests include correctly set OSGi bundle properties (ask for help verifying these on commons-dev if necessary).
  5. The jar manifests include the following required properties, set to the correct values(*):
      Extension-Name: org.apache.commons.foo
      Specification-Title: Apache Commons Foo
      Specification-Vendor: The Apache Software Foundation
      Specification-Version: 1.2
      Implementation-Vendor-Id: org.apache
      Implementation-Title: org.apache.commons.foo
      Implementation-Vendor: The Apache Software Foundation
      Implementation-Version: 1.2 
      Built-By: <your apache ID>
  6. "mvn site" generates the documentation correctly and all links are working.
Do not proceed with tagging or cutting RCs until you have a fully working build that produces a good set of distribution artifacts. If your component supports both Ant and Maven builds, make sure that the build succeeds using both on all supported JDK versions. If you have access to multiple platforms, test the build(s) on as many as you can.

(*) The default configuration of the build will use the name of the current logged in user as value for the Built-By MANIFEST header. Use -Duser.name=<your apache ID> to change this.

Creating a Release Candidate

Resolve Bugs

Resolve all bugs on that version! They can be resolved by:

  • fixing
  • marking them as INVALID or WONTFIX
  • changing their fix version to another unreleased version

Prepare Release Notes

Each component should have a file RELEASE-NOTES.txt in the base directory of the component. This file should be updated for the release and checked in prior to tagging or rolling the release candidate. As noted above, this file should be included in both the source and binary release distributions. The release notes should contain a description of all the changes since the previous release. Any compatibility issues with the last release (whether binary or semantic) should be highlighted. If there are no compatibilty issues, this too should be mentioned. An introduction to the release may also be given, describing the component and the release in general terms. The release notes should contain the minimum target Java version for the component.

Components that use the Maven changes plugin and changes.xml to track changes can generate a "starter" release notes document by supplying a custom Velocity template to the Maven announcements plugin:

    mvn changes:announcement-generate
    mv target/announcement/foo-release-notes.vm RELEASE-NOTES.txt 

The Commons Parent pom has a "release-notes" profile which automatically sets the output file, so you can just run the following instead:

mvn changes:announcement-generate -Prelease-notes [-Dchanges.version=nnn]

The commons parent project ships with a default template. If you want to configure your own you need to set the following property in the configuration for the maven-changes-plugin in the pom:

    <template>foo-release-notes.vm</template>
    
and the Velocity template, foo-release-notes.vm needs to be defined in src/changes. If you want to put it into a different directory you also want to set the templateDirectory property.

The release notes should be a plain text file. Take care to ensure that the format allows easy reading on a wide variety of platforms. Long lines may need to be broken manually to allow them to be easily read easily without word wrap.

Check the commit log

Different components have their own ways of creating the change log. The most common, and recommended, way, is to record all significant changes in JIRA tickets and include entries in the Maven change-log file, changes.xml. If your component has a change-log you can skip this step.

Here's a way to get the information directly from svn if no change log has been maintained for the component:

Get a list of all the commits since the last release by following these steps.
Assuming that, as part of the last release, a directory {project-base}/tags/foo-1.1 had been created:

      cd {project-base}/tags

      svn log --stop-on-copy foo-1.1
      # The last revision NNNN reported in the log output is the revision that was
      # copied to create the tag. If this is a true tag directory, then of course
      # there will only be one revision listed by the log command..

      cd ..
      svn log -v -rNNNN:HEAD trunk > commits-since-last-release.txt
        

This will result in a file that contains info on each commit that affected at least one file within the trunk directory since the last release. Note that if a commit affected a group of files of which some were outside the trunk directory, then they will be included with the associated commit message but can be ignored.

Using "svn diff" instead of "svn log -v" will result instead in a file that shows the actual diffs for each file instead of the commit messages. This may be more convenient when the commit messages are not sufficiently detailed to be able to build the release notes directly from them.

Inspect the list of changes and enter relevant information into the release notes; this may require inspecting the actual changes using "svn diff". Enhancements and new features need to be collated by topic. Bugs fixed should be listed separately together with a short summary of the bug.

Please remember to spell check the release notes. Please break lines at 80 characters.

IMPORTANT! At the current time, selecting by date in subversion within the ASF repository isn't reliable. The reason is that when converting a date to a revision number, subversion assumes that revision N has an earlier date than revision N+M, and that it can therefore perform a binary search on revision numbers to locate one with the required date. However CVS imports of data that retain the original date information from CVS break this assumption. And unfortunately because revisions are repository-wide information, this affects date-based searches even in directories unrelated to the ones that CVS stuff was imported into. So while dates are reported correctly in "svn log" output, only revision numbers should be used with the -r option. See issue #752 in the subversion issue tracker at tigris.org.

Test Against Listed Dependencies

If you are using Maven to execute the unit tests associated with the component then there is nothing to do here; Maven will automatically perform the tests using the library versions specified in the pom.xml file.

It's also vital to ensure that you test against the correct version of the Java libraries. If Maven requires a later version of Java, then use the appropriate java-1.x profile. See Testing with different Java versions for further details.

If you are using Ant to execute unit tests, then ensure the Ant build.xml file references the same library versions as are listed as dependencies in the pom.xml file then execute the unit tests.

Check the Apache License

Check the Apache Licenses page for current information. Check that each distribution contains a copy of the license. Check that the jar contains a copy of the license in the META-INF directory.

Check that the years in the copyright statement in the NOTICE file are correct.

Developer documentation on how to apply the Apache License can be found in Applying the Apache License, Version 2.0 and ASF Source Header and Copyright Notice Policy

Some of the important items from the aforementioned documents:

Do I have to have a copy of the license in each source file? Maven builds can use the RAT plugin to generate a license report.

Only one full copy of the license is needed per distribution. Each source file only needs to contain the boilerplate notice at:

http://www.apache.org/legal/src-headers.html#headers

In my current source files I have attribution notices for other works. Do I put this in each source file now?

No. The new license allows for a NOTICE file that contains such attribution notices (including the Apache attribution notice). See

http://www.apache.org/licenses/example-NOTICE.txt

for an example that provides all of the notices applicable to the httpd-2.0 distribution.

Check NOTICE.txt

The component should contain a NOTICE.txt (next to the LICENSE.txt). If this is not present, it must be created. The basic content (excepting external attributes notes) should be:

    Apache Commons {Foo}
    Copyright {earliest}-{latest} The Apache Software Foundation

    This product includes software developed at
    The Apache Software Foundation (http://www.apache.org/).
        

Verify {latest} is the current year.

The NOTICE.txt must be distributed along with the LICENSE.txt. Check that the distribution build correctly adds this file to the distributions and that the copyright start and end dates are correct.

Tag the Release Candidate

Once all the preparations agreed in the release plan has been completed, create a Release Candidate. Before creating the tag from which the release candidate will be generated, run the distribution build and double check that everything is still fine.

Make sure that the build version number corresponds to the release version. For example, commons-foo-1.2. What you are preparing at this point is a candidate for release, which we will vote on and then push directly to the mirrors. Clean build, run the unit tests and check that the javadocs have the correct version number. Check in all changes.

Now create the tag for the release candidate. There are two options how to do that, you can either use the Maven release plugin or create the tag manually. In either case, record the svn revision number, you'll need it for the release vote mail.

Manual Method

Create a clean SVN workspace for the release candidate:

      svn co https://svn.apache.org/repos/asf/commons/proper/foo/trunk foo-1.2-RC1
      

Edit the version fields in the POMs to remove the -SNAPSHOT, for example using Maven's version plugin.

      mvn versions:set -DnewVersion=1.2 -DgenerateBackupPoms=false
      

Edit the SCM entries in the parent POM. Note: use the final tag, without any RC suffix. Do svn diff and svn status to check that the changes are OK. These should only show changes to the POMs.

Create the RC tag, by copying the tag workspace to SVN as below. The tag name should include the component name, as this makes it easier to distinguish checkouts. Try to follow the existing naming convention so the tag names will sort in a reasonable order. For example NET uses NET_M_N_RC1, and LANG has a similar convention. For an example of how not to do it, see https://svn.apache.org/repos/asf/commons/proper/io/tags/ The IO tags don't sort properly.

      svn copy foo-1.2-RC1 -m "Creating foo-1.2-RC1 tag" https://svn.apache.org/repos/asf/commons/proper/foo/tags/foo-1.2-RC1
      

Do NOT check the updated workspace back into the development trunk. It's important to only have SNAPSHOT versions in trunk; only tags should have non-SNAPSHOT versions. Also, by leaving trunk unchanged, nothing needs to be reverted if the RC vote fails and the Rc needs to be re-rolled.

Maven Release Plugin

When using the release plugin, please verify that your poms will not lose content when they are rewritten during the release process.

Inside the branch you are cutting the release from start with

      mvn release:prepare -DdryRun=true
        

Diff the original file pom.xml with the one called pom.xml.tag to see if the license or any other info has been removed. This has been known to happen if the starting <project> tag is not on a single line. The only things that should be different between these files are the <version> and <scm> elements. Any other changes, you must backport yourself to the original pom.xml file and commit before proceeding with the release.

Remember to do mvn release:clean before you start the real release process. If everything is ok, run:

  
      mvn release:prepare
      

The Maven release:prepare goal updates the trunk tag to the next SNAPSHOT release. If the RC vote fails, this will need to be reverted before re-rolling the RC. (the manual method described above avoids this problem)

Create the Release Candidate

Build distributions from a fresh checkout of the RC tag. Build the code with the target version of Java if possible.
If the version of Maven you are using requires a more recent version of Java than the code you are building, then use the appropriate java-1.x profile to ensure that compilation and testing is done with the correct Java version. This will catch any accidental use of methods etc. that are not in the target Java version libraries. See Testing with different Java versions for further details. (the compiler.source and compiler.target versions only affect source syntax and class file format)

To create all distributables and upload the Maven artifacts to the ASF Nexus instance run

        mvn deploy -Prelease [-Pjava-1.x]
      
If you want to do a test deployment first, you can add the following option:
        mvn deploy -Prelease -Ptest-deploy [-Pjava-1.x]
      
The "test-deploy" profile uploads the artifacts to target/deploy instead of Nexus so one can check all the required files have been created with the expected contents. Best to run "mvn clean" before doing the live deployment. Again remember to use -Duser.name=<your apache ID> to make sure the Built-By MANIFEST header is correct.

This will PGP-sign all artifacts and upload them to a new staging repository, Nexus itself will create MD5 and SHA1 checksums for all files that have been uploaded.

A known problem is that gpg signing may fail if the gpg plugin tries to read the passphrase interactively. It works if you specify the passphrase when invoking mvn (-Dgpg.passphrase=***) or run gpg-agent before starting mvn.

Unfortunately this uploads more than should be part of the Maven repository, in particular the binary and source distribution .tar.gz and .zip files are there as well. Before you close the staging repository you must remove the tarballs/zips together with their PGP signatures and checksums using the Nexus web interface. While you are at it you can also remove the checksums Nexus created for the PGP signatures of the remaining artifacts, they are not needed at all.

If anybody knows how we can avoid uploading the distributions, please lend a hand. The manual step is not only annoying, uploading the files also wastes time and bandwidth.

Once the unneeded files have been deleted you can now close the staging repository.

The tarballs and zips need to go to https://dist.apache.org/repos/dist/dev/commons/. If this is the first release of foo after svnpubsub has been enabled you have to create the directory structure for your component.

        svn mkdir -m "Creating initial directory structure for foo" https://dist.apache.org/repos/dist/dev/commons/foo
        svn mkdir -m "Creating initial directory structure for foo" https://dist.apache.org/repos/dist/dev/commons/foo/binaries
        svn mkdir -m "Creating initial directory structure for foo" https://dist.apache.org/repos/dist/dev/commons/foo/source
      

Then check out https://dist.apache.org/repos/dist/dev/commons/foo and copy the tarballs, checksums and PGP signatures to the appropriate directories. You can find those artifacts inside your local Maven repository. The procedure will be something like (where release_path points to your working copy of the dist repository):

        version=1.2
        repo_path=~/.m2/repository/org/apache/commons/commons-foo/${version}
        release_path=~/foo-release
        mkdir -p ${release_path}/binaries
        mkdir -p ${release_path}/source
        cp ${repo_path}/*.bin.zip* ${release_path}/binaries
        cp ${repo_path}/*.bin.tar.gz* ${release_path}/binaries
        cp ${repo_path}/*.src.zip* ${release_path}/source
        cp ${repo_path}/*.src.tar.gz* ${release_path}/source
        cp RELEASE-NOTES.txt ${release_path}
      

svn add the files and commit them. Again, record the revision number for the vote email.

Create the Release Candidate Website

The new website should be published in your home directory on people.apache.org. For example:

      mvn site [-Pjacoco] [-Pcobertura]
    
The optional "jacoco" and "cobertura" profiles are used to select the appropriate code coverage tool if required. Then tar or zip the site in target/site and scp and unpack the files on people.apache.org in a directory ~/public_html/foo-1.2-RC1.

Note that when verifying this candidate site you need to be careful of absolute URLs; following these will lead to the currently published site, not to the equivalent page on the new site being evaluated.

Voting On Release

[VOTE] Release Foo 1.2 based on RC1

Once the release candidate has been created and uploaded, now it's time to propose the release VOTE.

Post a [VOTE] Release Foo 1.2 based on RC1 email to dev@commons.apache.org. This should include links to minimally the distributions, site, tag and KEYS. Here is an example release VOTE message body:

  We have fixed quite a few bugs and added some significant enhancements since Foo 1.1 was released,
  so I would like to release Foo 1.2.

  Foo 1.2 RC1 is available for review here:
    https://dist.apache.org/repos/dist/dev/commons/foo/ (svn revision XYZ)

  Maven artifacts are here:
    https://repository.apache.org/content/repositories/orgapachecommons-ABC/org/apache/commons/commons-foo/1.2/

  Details of changes since 1.1 are in the release notes:
    https://dist.apache.org/repos/dist/dev/commons/foo/RELEASE-NOTES.txt
    http://people.apache.org/~luckyrm/foo-1.2-RC1/changes-report.html

  I have tested this with JDK 1.3, 1.4, 1.5 and 1.6 using maven2.

  The tag is here:
    http://svn.apache.org/repos/asf/commons/proper/foo/tags/FOO_1_2_RC1/ (svn revision)
  N.B. the SVN revision is required because SVN tags are not immutable.

  Site:
    http://people.apache.org/~luckyrm/foo-1.2-RC1/
  (note some *relative* links are broken and the 1.2 directories are
  not yet created - these will be OK once the site is deployed)

  Clirr Report (compared to 1.1):
    http://people.apache.org/~luckyrm/foo-1.2-RC1/clirr-report.html

  RAT Report:
    http://people.apache.org/~luckyrm/foo-1.2-RC1/rat-report.html

  KEYS:
  https://www.apache.org/dist/commons/KEYS
          
  Please review the release candidate and vote.
  This vote will close no sooner that 72 hours from now, i.e. after 0400 GMT 31-March 2010

  [ ] +1 Release these artifacts
  [ ] +0 OK, but...
  [ ] -0 OK, but really should fix...
  [ ] -1 I oppose this release because...

  Thanks!

  Lucky RM 

Votes from members of the Commons PMC are binding, however votes from other committers, users and contributors are welcomed. If the [VOTE] is successful then continue. Release VOTEs should be left open for a minimum of 72 hours so community members have ample opportunity to download, review and test the release candidate. It is a good practice to, as above, specify the closing time of the VOTE in the VOTE message.

If the vote fails, then fix the problem and create a new release candidate (including creating another tag; tags are cheap!). Then call another vote. Creating a perfect release isn't easy, and it is quite common for the first few release candidates to fail, particularly on simple issues like missing license files. Don't take negative feedback on RCs personally. The release belongs to the community and we are all accountable for anything wrong or lacking in the code we release. That's why suggestions for improvement are more often than not accompanied by patches and/or commits to fix problems. Always start a new VOTE thread to vote on a new RC.

Once a vote is successful, post a [RESULT] Release Foo 1.2 email to dev@commons.apache.org as a reply to the original posting. This email should contain a summary of the voters/votes that were cast, eg

        The following people voted on release Foo 1.2:
        Bob +1
        Sue +1
        Sam +0
        Sandy +1 (non-binding) 

Note that binding the VOTEs recorded need to be clearly designated in the RESULT. This may be done by either stating only the binding votes (and indicating that to be the case) or by adding (non-binding) to those which are not. It is best practice to indicate how each person voted. This allows any mistakes to be caught and corrected early. If you do vote, please check the results to ensure that your vote has been correctly tallied.

During the course of the VOTE, make sure that one or more of the reviewers have verified the signatures and hash files included with the release artifacts. If no one specifically mentions having done that during the VOTE, ask on the dev list and make sure someone does this before you proceed with the release.

Things To Look For When Inspecting A Release Candidate

There are a number of common things that releases fail on.

API Changes

Accidental non-compatible API changes in a minor release. The clirr report generated by Maven is very useful in spotting these.

Javadoc

  • Ensure that the Javadoc tool reports no warnings or errors.
  • Ensure that all new classes and methods have appropriate @since Javadoc tags.

Code Style

Many projects enforce coding styles using the CheckStyle or PMD tools. If your project does this, don't forget to check the output and fix any problems.

Class File Format

Building on a more recent JVM than the code will run on. Java class file format has changed a number of times over the years, and code compiled with a modern JVM may fail to load in an older JVM with the error message "invalid class file format" unless the code is compiled with appropriate options set. If you are using Maven, then ensure that your pom has maven.compile.target set to the minimum JVM version your binary is intented to support. If you are using Ant, then ensure that the javac task has xml attribute "target" is set to the appropriate JVM version.

Maven Build

The Maven build has been modified to include two non standard attributes in the jar's manifest to indicate the maven.compile.source and maven.compile.target properties used to create the jar. This serves two purposes:

  • To provide comfort to users regarding JVM compatibility.
  • Enable releases to be checked more easily for JVM compatibility.

The entries created in the manifest will look something like the following:

      X-Compile-Source-JDK: 1.3
      X-Compile-Target-JDK: 1.3
    
These values should be checked to ensure that the release has been built for the appropriate JVM. If they are not present or no values are specified then the Build-Jdk entry should be checked to ensure the release has been built with the appropriate JDK.

Licensing

The NOTICE.txt file must be included in both the distribution tars/zips and the included jars.

Maven builds default to including this, so no further effort is required for those projects.

What next?

Vote succeded

If the vote succeeded, please see the page Publishing the Release

Vote failed

If the vote failed, there are various items to tidy up.

  • Drop the Nexus Staging repository.
  • Delete and files in the dist/dev area.
  • Do not delete the SVN tag yet unless the release has been completely abandoned. It can be useful for reviewers to have access to the previous tag
If a new RC is to be created, restart the process as described above.

Feedback

Feedback - yes please!

Comments, critiques and error reports - post them any and all to the dev mailing list at commons.apache.org. Please prefix with [doc].