IntroductionThis document contains a mixture of information, advice and examples. It is intended to be a recommendation of best practices for Commons components. The instructions provided here are consistent with, but not a replacement for the ASF Release Guidelines. The Apache Commons project uses the Nexus installation at the Apache Software Foundation to stage Maven artifacts before the release and later publish them; the Commons Parent POM contains all the necessary configuration. This guide covers deploying to Nexus using Maven. For alternatives see the ASF wide documentation. If this is the first time you are publishing to the ASF's Nexus instance you'll need to prepare your development environment. The examples below assume that preparation is being made to release version 1.2 of component Foo. Build EnvironmentsCommons components are expected to use Maven to build the project website. Components may choose to use either Maven or Ant to build the actual jar and tar/zip files to be distributed, although it is recommended that Maven be used for this. Both approaches are covered below. The version of Maven used is assumed to be Maven 3 throughout. At a minimum, Commons releases must include full source distributions packaged in tar and zip archives. This document assumes that the release is being prepared on a linux/unix system, and that steps are being executed from the command-line. The principles are the same, however, for a release prepared on other operating systems. PreparationSelect a Release ManagerA commons committer (normally one of the development team) should post an email to the development list proposing that a release be made and nominating a release manager. Typically, the proposer volunteers as the release manager and it passes by lazy consensus. Update KEYS file if necessaryIf the release manager has not yet performed a Commons release, they may need to add their public key to the Commons KEYS file, which is located at: https://dist.apache.org/repos/dist/release/commons/KEYS. Note that the SVN directory contains the current releases of all "proper" Commons components, so use a command such as the following to check it out without downloading every Commons project: svn co --depth=files https://dist.apache.org/repos/dist/release/commons/ The public key should also be uploaded to a public keyserver. Create a Release PlanA release plan should also be prepared, in which the tasks remaining to be done before the release are listed. It may be useful to prepare draft release notes before proposing the release, so that others can see what changes are expected for the new release. Using a Release Branch
Consider whether a release branch is needed before preparing for the new release. In general,
Commons components are small enough that there is no need for a release branch, but if
active development will continue on the next version while a release is being made then
trunk should be branched to allow this. Creating a release branch with SVN The description below assumes a release is being prepared using the code in trunk for SVN based components. The process is nearly identical when preparing from a release branch: only the directory in which the work is performed is different. cd to the project's base directory in your subversion working copy. svn update trunk svn cp trunk branches/foo-1.2-release svn commit -m "Creating foo-1.2-release branch" branches/foo-1.2-release Note that the "svn update" step is necessary in order to ensure that the directory being copied does not have a mix of files at various revisions; even if the files haven't changed since the last svn update this can cause "svn log -v" on the new directory to report files as having been (R)eplaced. Alternatively, use "svn cp URLsrc URLtag" svn cp -m "Creating foo-1.2-release branch" \ https://svn.apache.org/repos/asf/commons/proper/foo/trunk \ https://svn.apache.org/repos/asf/commons/proper/foo/branches/foo-1.2-release which will copy files internally within the repository without using the local working copy: this always ensures a clean copy is made. Creating a release branch with git All git specific commands in the description below assume that you're on the release branch. git checkout -b release Check Compatibility
Consult the Commons Versioning Guidelines and check
that the current level of compatibility is suitable for the proposed version number.
Check the current level of compatibility in the code. Tools like
Clirr,
JDiff, and japicmp
are very useful, and support Apache Maven (Clirr and JDiff support Apache Ant as well;
japicmp could be invoked using Ant's Check DependenciesEnsure that there are no dependencies with a SNAPSHOT version. Ensure that dependencies (and build plugins) are updated to the latest suitable version. For example: mvn versions:display-dependency-updates -U mvn versions:display-plugin-updates -U Check Javadocs And Code Style
Ensure all new classes and methods have Ensure no errors or warnings are reported by the Javadoc tool. Check that the Javadocs have the correct version number. If the code is targetted at Java 7 or before, check that the Javadoc can still be created using Javadoc 8. Javadoc 8 is a lot stricter about the syntax. Ideally fix all the syntax errors. However if this is not feasible in the time-scale, consider adding a temporary profile to the pom to disable the stricter checking: <profile> <!-- Temporary hack to suppress Javadoc 8 errors --> <!-- to re-enable the checks, build with -P-javadoc_8 --> <id>javadoc_8</id> <activation> <jdk>[1.8,)</jdk> </activation> <properties> <!-- Disable Xdoclint, until JavaDoc issues are fixed --> <additionalparam>-Xdoclint:none</additionalparam> </properties> </profile> If the component uses checkstyle, findbugs or PMD tools, examine the reports and fix all problems. Configure the Build and Release Plugin to Generate a Complete Set of Release ArtifactsFor builds using Maven, the contents of the source and binary distributions are configured in assembly descriptors. By convention, these are stored in src/main/assembly and named src.xml and bin.xml, respectively. Names and locations for these files are specified in the maven-assembly-plugin configuration in the pom. Make sure that all (and only) files that should be included in the source and binary distributions are included in the fileset elements of the descriptors. Look at some recently released components' descriptors for comparison. At a minimum, both source and binary distributions must include the release notes, license and notice files.
Update the version numbers in pom.xml and build.xml to the new release version, in this example, 1.2.
For Maven builds, make sure that the build properties are properly set. Review the <properties> <commons.componentid>foo</commons.componentid> <commons.release.version>1.2</commons.release.version> <commons.rc.version>RC1</commons.rc.version> <!-- properties not related to versioning --> <commons.jira.id>FOO</commons.jira.id> <commons.jira.pid>007</commons.jira.pid> <maven.compile.source>1.8</maven.compile.source> <maven.compile.target>1.8</maven.compile.target> <!-- Encoding of Java source files: ensures that the compiler and the javadoc generator use the right encoding. Subprojects may overwrite this, if they are using another encoding. --> <commons.encoding>iso-8859-1</commons.encoding> <!-- used in this pom to provide the Javadoc HTML file encoding --> <commons.docEncoding>${commons.encoding}</commons.docEncoding> <!-- Define source encoding for filtering; used by general plugins --> <project.build.sourceEncoding>${commons.encoding}</project.build.sourceEncoding> <!-- This is used by reporting plugins --> <project.reporting.outputEncoding>${commons.encoding}</project.reporting.outputEncoding> <!-- Commons Release Plugin --> <!-- Previous version of the component (used for reporting binary compatibility check)--> <commons.bc.version>1.5</commons.bc.version> <commons.release.isDistModule>true</commons.release.isDistModule> <commons.distSvnStagingUrl>scm:svn:https://dist.apache.org/repos/dist/dev/commons/${commons.componentid}</commons.distSvnStagingUrl> <properties>
Ensure your Release Manager details are defined in your Maven <settings> ... <profiles> ... <profile> <id>active-profile</id> <properties> <commons.releaseManagerName>Your Name</commons.releaseManagerName> <commons.releaseManagerKey>Your Signing Key Hex ID</commons.releaseManagerKey> </properties> </profile> </profiles> <activeProfiles> <!-- define active profile name --> <activeProfile>active-profile</activeProfile> </activeProfiles> ... </settings> Make sure that the release version is set to the new release and that the compile and source targets are set correctly. Generate and check in a new download page for the component: mvn [-N] commons-build:download-page [-Dcommons.release.version=m.n] svn commit -m "Updated download page in preparation for 1.2 release." src/site/xdoc/download_foo.xml git commit -am "Updated download page in preparation for 1.2 release." src/site/xdoc/download_foo.xml -N
(non-recursive) parameter should be specified on the mvn command line.
The release version can be overridden with -Dcommons.release.version=m.n if you want to create the new file
without updating the pom (e.g. before creating the tag)
When using Ant, typically the Ant "dist" target produces the source and binary distributions. Included files are specified in the target or sub-targets and should be verified similarly to above. Test the "Ant dist" or "mvn assembly:assembly" command and inspect the tars/zips and jars produced. Verify that
(*) The default configuration of the build will use the name of the current logged in user as value for the Creating a Release CandidateResolve BugsResolve all bugs on that version! They can be resolved by:
Check the commit log
Different components have their own ways of creating the change log.
The most common, and recommended, way, is to record all significant
changes in JIRA tickets and include entries in the Maven change-log
file, Here's a way to get the information directly from svn if no change log has been maintained for the component:
Get a list of all the commits since the last release by following these steps.
cd {project-base}/tags svn log --stop-on-copy foo-1.1 # The last revision NNNN reported in the log output is the revision that was # copied to create the tag. If this is a true tag directory, then of course # there will only be one revision listed by the log command.. cd .. svn log -v -rNNNN:HEAD trunk > commits-since-last-release.txt This will result in a file that contains info on each commit that affected at least one file within the trunk directory since the last release. Note that if a commit affected a group of files of which some were outside the trunk directory, then they will be included with the associated commit message but can be ignored. Using "svn diff" instead of "svn log -v" will result instead in a file that shows the actual diffs for each file instead of the commit messages. This may be more convenient when the commit messages are not sufficiently detailed to be able to build the release notes directly from them. Inspect the list of changes and enter relevant information into the release notes; this may require inspecting the actual changes using "svn diff". Enhancements and new features need to be collated by topic. Bugs fixed should be listed separately together with a short summary of the bug. Please remember to spell check the release notes. Please break lines at 80 characters. IMPORTANT! At the current time, selecting by date in subversion within the ASF repository isn't reliable. The reason is that when converting a date to a revision number, subversion assumes that revision N has an earlier date than revision N+M, and that it can therefore perform a binary search on revision numbers to locate one with the required date. However CVS imports of data that retain the original date information from CVS break this assumption. And unfortunately because revisions are repository-wide information, this affects date-based searches even in directories unrelated to the ones that CVS stuff was imported into. So while dates are reported correctly in "svn log" output, only revision numbers should be used with the -r option. See issue #752 in the subversion issue tracker at tigris.org. Prepare Release NotesEach component should have a file RELEASE-NOTES.txt in the base directory of the component. This file should be updated for the release and checked in prior to tagging or rolling the release candidate. As noted above, this file should be included in both the source and binary release distributions. The release notes should contain a description of all the changes since the previous release. Any compatibility issues with the last release (whether binary or semantic) should be highlighted. If there are no compatibilty issues, this too should be mentioned. An introduction to the release may also be given, describing the component and the release in general terms. The release notes should contain the minimum target Java version for the component. Components that use the Maven changes plugin and changes.xml to track changes can generate a "starter" release notes document by supplying a custom Velocity template to the Maven announcements plugin: mvn changes:announcement-generate mv target/announcement/foo-release-notes.vm RELEASE-NOTES.txt The Commons Parent pom has a "release-notes" profile which automatically sets the output file, so you can just run the following instead: mvn changes:announcement-generate -Prelease-notes [-Dchanges.version=nnn] The commons parent project ships with a
default
template. If you want to configure your own you need to set
the following property in the <template>foo-release-notes.vm</template> foo-release-notes.vm needs to be defined in
src/changes . If you want to put it into a different directory
you also want to set the templateDirectory property.
The release notes should be a plain text file. Take care to ensure that the format allows easy reading on a wide variety of platforms. Long lines may need to be broken manually to allow them to be easily read easily without word wrap. Test Against Listed Dependencies
If you are using Maven to execute the unit tests associated with the component then
there is nothing to do here; Maven will automatically perform the tests using the
library versions specified in the
It's also vital to ensure that you test against the correct version of the Java libraries.
If Maven requires a later version of Java, then use the appropriate
If you are using Ant to execute unit tests, then ensure the Ant Check the Apache LicenseCheck the Apache Licenses page for current information. Check that each distribution contains a copy of the license. Check that the jar contains a copy of the license in the META-INF directory. Check that the years in the copyright statement in the NOTICE file are correct. Developer documentation on how to apply the Apache License can be found in Applying the Apache License, Version 2.0 and ASF Source Header and Copyright Notice Policy Some of the important items from the aforementioned documents: Do I have to have a copy of the license in each source file? Maven builds can use the RAT plugin to generate a license report. Only one full copy of the license is needed per distribution. Each source file only needs to contain the boilerplate notice at:
In my current source files I have attribution notices for other works. Do I put this in each source file now? No. The new license allows for a NOTICE file that contains such attribution notices (including the Apache attribution notice). See
for an example that provides all of the notices applicable to the httpd-2.0 distribution. Update the download xml file
The download page on the website is created from the file Update the other autogenerated filesVarious files are autogenerated using the commons build plugin.
Check NOTICE.txtThe component should contain a NOTICE.txt (next to the LICENSE.txt). If this is not present, it must be created. The basic content (excepting external attributes notes) should be: Apache Commons {Foo} Copyright {earliest}-{latest} The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). Verify The NOTICE.txt must be distributed along with the LICENSE.txt. Check that the distribution build correctly adds this file to the distributions and that the copyright start and end dates are correct. Tag the Release CandidateOnce all the preparations agreed in the release plan has been completed, create a Release Candidate. Before creating the tag from which the release candidate will be generated, run the distribution build and double check that everything is still fine.
Make sure that the build version number corresponds to the release version. For example,
Now create the tag for the release candidate. There are two options how to do that, you can either use the Maven release plugin or create the tag manually. In either case, record the svn revision number, you'll need it for the release vote mail. Tag Name ConventionThe naming of release candidates' tag for "Commons" projects should be done according to the following convention: <name>-<x.y[.z]>[-beta<m>]-rc<n>
where the Manual MethodManual Method (SVN)Create a clean SVN workspace for the release candidate: svn co https://svn.apache.org/repos/asf/commons/proper/foo/trunk foo-1.2-RC1 Edit the version fields in the POMs to remove the -SNAPSHOT, for example using Maven's version plugin. mvn versions:set -DnewVersion=1.2 -DgenerateBackupPoms=false Edit the SCM entries in the POM. Note: use the final tag, without any RC suffix.
Do Create the RC tag, by copying the tag workspace to SVN as below. The tag name should include the component name, as this makes it easier to distinguish checkouts. Try to follow the existing naming convention so the tag names will sort in a reasonable order. For example NET uses NET_M_N_RC1, and LANG has a similar convention. For an example of how not to do it, see https://svn.apache.org/repos/asf/commons/proper/io/tags/ The IO tags don't sort properly. svn copy foo-1.2-RC1 -m "Creating foo-1.2-RC1 tag" https://svn.apache.org/repos/asf/commons/proper/foo/tags/foo-1.2-RC1 Do NOT check the updated workspace back into the development trunk. It's important to only have SNAPSHOT versions in trunk; only tags should have non-SNAPSHOT versions. Also, by leaving trunk unchanged, nothing needs to be reverted if the RC vote fails and the Rc needs to be re-rolled. Manual Method (Git)After editing the version fields in the POMs: git commit -am "Update version numbers for Commons Foo release 1.2" git tag -s commons-foo-1.2-rc1 -m "Tag Commons Foo release 1.2 RC1" git push --tags If this is the first release using git and there has never been a release branch, git will ask you to set the upstream branch: git push -u origin/release Maven Release PluginWhen using the release plugin, please verify that your poms will not lose content when they are rewritten during the release process. Inside the branch you are cutting the release from start with mvn release:prepare -DdryRun=true Diff the original file Remember to do mvn release:prepare The Maven release:prepare goal updates the trunk tag to the next SNAPSHOT release. If the RC vote fails, this will need to be reverted before re-rolling the RC. (the manual method described above avoids this problem) TODO: Does the Maven Release plugin work with git?Create the Release Candidate with the Commons Release Plugin.The Commons Release Plugin, as of commons-parent-46 is included but disabled. Note, as of version 1.2 of the plugin and version 46 of the parent, we have throughly tested single module builds, but multi-module builds remain an open problem (which may work, but at the time of writing have not been fully vetted). To enable the plugin, one needs to configure theproperties section of the pom as follows:
<properties> <commons.release.isDistModule>true</commons.release.isDistModule> <commons.distSvnStagingUrl>scm:svn:https://dist.apache.org/repos/dist/dev/commons/foo</commons.distSvnStagingUrl> </properties> foo represents our component. Furthermore, during a release build, we ask that the svn
folder be empty. We then generate our release build by runnning:
mvn -Duser.name=${my_apache_id} -Prelease -Ptest-deploy clean package site deploy target/commons-release-plugin/scm . And, to perform a full release, we run:
mvn -Duser.name=${my_apache_id} -Prelease clean package site deploy RELEASE-NOTES.txt from the
root of the project.
After finishing the build you will need to "close" the staging repository in the
Apache Nexus instance. You can then continue down
to preparing the vote below to prepare the vote.
Further, the commons-release-plugin, as of 1.4, can automatically generate the vote email for you. When you've
finished staging the artifacts, run
mvn commons-release:vote-txt target/vote.txt that will need to be altered for the vote email, but it
gives you a considerable head start on writing the email.
Create the Release Candidate Manually
Build distributions from a fresh checkout of the RC tag.
Build the code with the target version of Java if possible. To create all distributables and upload the Maven artifacts to the ASF Nexus instance run mvn clean deploy -Prelease -Duser.name=<your apache ID> [-Pjava-1.x] mvn clean deploy -Prelease -Duser.name=<your apache ID> -Ptest-deploy [-Pjava-1.x] [-Dgpg.skip] -Duser.name=<your apache ID> to make sure the Built-By MANIFEST header is correct.
This will PGP-sign all artifacts and upload them to a new staging repository, Nexus itself will create MD5 and SHA1 checksums for all files that have been uploaded. A known problem is that gpg signing may fail if the gpg plugin tries to read the passphrase interactively.
It works if you specify the passphrase when invoking mvn ( Version 1.6 of the GPG plugin supports storage of the password using
Maven password encryption
This uses the server id of Unfortunately this uploads more than should be part of the Maven repository, in particular the binary and source
distribution If anybody knows how we can avoid uploading the distributions, please lend a hand. The manual step is not only annoying, uploading the files also wastes time and bandwidth. Once the unneeded files have been deleted you can now close the staging repository. The tarballs and zips need to go to https://dist.apache.org/repos/dist/dev/commons/. If this is the first release of foo after svnpubsub has been enabled you have to create the directory structure for your component. svn mkdir -m "Creating initial directory structure for foo" https://dist.apache.org/repos/dist/dev/commons/foo Then check out version=1.2 repo_path=~/.m2/repository/org/apache/commons/commons-foo/${version} release_path=~/foo-release mkdir -p ${release_path}/FOO_1_2_RC1 cp ${repo_path}/*-bin.zip* ${release_path}/FOO_1_2_RC1 cp ${repo_path}/*-bin.tar.gz* ${release_path}/FOO_1_2_RC1 cp ${repo_path}/*-src.zip* ${release_path}/FOO_1_2_RC1 cp ${repo_path}/*-src.tar.gz* ${release_path}/FOO_1_2_RC1 cp RELEASE-NOTES.txt ${release_path}/FOO_1_2_RC1
Create the Release Candidate WebsiteThe new website should be published in your home directory on people.apache.org. For example: mvn site [-Pjacoco] [-Pcobertura] ~/public_html/foo-1.2-RC1 .
Note that when verifying this candidate site you need to be careful of absolute URLs; following these will lead to the currently published site, not to the equivalent page on the new site being evaluated. Voting On ReleaseUsing commons-release-pluginThe vote email should have been mostly generated for you by the plugin. See the section above pertaining to the commons-release-plugin for more details.[VOTE] Release Foo 1.2 based on RC1Once the release candidate has been created and uploaded, now it's time to propose the release VOTE.
Post a We have fixed quite a few bugs and added some significant enhancements since Foo 1.1 was released, so I would like to release Foo 1.2. Foo 1.2 RC1 is available for review here: https://dist.apache.org/repos/dist/dev/commons/foo/FOO_1_1_RC1 (svn revision XYZ) The tag is here: http://svn.apache.org/repos/asf/commons/proper/foo/tags/FOO_1_2_RC1/ (svn revision) N.B. the SVN revision is required because SVN tags are not immutable. Maven artifacts are here: https://repository.apache.org/content/repositories/orgapachecommons-NNNN/org/apache/commons/commons-foo/1.2/ These are the Maven artifacts and their hashes commons-xyz-1.2-test-sources.jar (SHA1: abcd...) commons-xyz-1.2-sources.jar (SHA1: def...) ... (no need for .asc hashes!) I have tested this with JDK 1.6, 7, 8 ... using Maven 3.x.x. Details of changes since 1.1 are in the release notes: https://dist.apache.org/repos/dist/dev/commons/foo/FOO_1_1_RC1/RELEASE-NOTES.txt http://home.apache.org/~luckyrm/foo-1.2-RC1/changes-report.html Site: http://home.apache.org/~luckyrm/foo-1.2-RC1/ (note some *relative* links are broken and the 1.2 directories are not yet created - these will be OK once the site is deployed) Clirr Report (compared to 1.1): http://home.apache.org/~luckyrm/foo-1.2-RC1/clirr-report.html Note that Clirr reports several errors. These are considered OK for the reasons stated below. These exceptions are also noted in the Changes and Release Notes. Errors reported: - methods added to interface: OK because that does not affect binary compatibility. - etc. RAT Report: http://home.apache.org/~luckyrm/foo-1.2-RC1/rat-report.html KEYS: https://www.apache.org/dist/commons/KEYS Please review the release candidate and vote. This vote will close no sooner that 72 hours from now, i.e. sometime after 04:00 UTC 31-March 2010 [ ] +1 Release these artifacts [ ] +0 OK, but... [ ] -0 OK, but really should fix... [ ] -1 I oppose this release because... Thanks! Lucky RM
Votes from members of the Commons PMC are binding, however votes from other committers, users and
contributors are welcomed.
If the If the vote fails, then fix the problem and create a new release candidate (including creating another tag; tags are cheap!). Then call another vote. Creating a perfect release isn't easy, and it is quite common for the first few release candidates to fail, particularly on simple issues like missing license files. Don't take negative feedback on RCs personally. The release belongs to the community and we are all accountable for anything wrong or lacking in the code we release. That's why suggestions for improvement are more often than not accompanied by patches and/or commits to fix problems. Always start a new VOTE thread to vote on a new RC.
Once a vote is successful, post a The following people voted on release Foo 1.2: Bob +1 Sue +1 Sam +0 Sandy +1 (non-binding)
Note that binding the VOTEs recorded need to be clearly designated in the RESULT.
This may be done by either stating only the binding votes (and indicating that to be the case)
or by adding During the course of the VOTE, make sure that one or more of the reviewers have verified the signatures and hash files included with the release artifacts. If no one specifically mentions having done that during the VOTE, ask on the dev list and make sure someone does this before you proceed with the release. Things To Look For When Inspecting A Release CandidateThere are a number of common things that releases fail on. DownloadsCheck the validity of what you download:
There should be at least one SHA file:
Licensing
Maven builds default to including this, so no further effort is required for those projects. API ChangesAccidental non-compatible API changes in a minor release. The clirr report generated by Maven is very useful in spotting these. Javadoc
Code StyleMany projects enforce coding styles using the CheckStyle or PMD tools. If your project does this, don't forget to check the output and fix any problems. Class File FormatBuilding on a more recent JVM than the code will run on. Java class file format has changed a number of times over the years, and code compiled with a modern JVM may fail to load in an older JVM with the error message "invalid class file format" unless the code is compiled with appropriate options set. If you are using Maven, then ensure that your pom has maven.compile.target set to the minimum JVM version your binary is intented to support. If you are using Ant, then ensure that the javac task has xml attribute "target" is set to the appropriate JVM version. Maven Build
The Maven build has been modified to include two non standard attributes
in the jar's manifest to indicate the
The entries created in the manifest will look something like the following: X-Compile-Source-JDK: 1.3 X-Compile-Target-JDK: 1.3 Build-Jdk entry should
be checked to ensure the release has been built with the appropriate JDK.
What next?Vote succeededIf the vote succeeded, please see the page Publishing the Release Vote failedIf the vote failed, there are various items to tidy up.
FeedbackFeedback - yes please! Comments, critiques and error reports - post them any and all to the dev mailing list at commons.apache.org. Please prefix with [doc]. |