Apache Commons - Building Components

Introduction

This document provides information on setting up and executing builds for Apache Commons components and web sites. The contents include best practices and conventions that commons components should follow. This information should be useful to committers setting up new components as well as to volunteers who want to build commons components or web sites from source or to contribute patches to the build artifacts.

The assumption throughout is that we are working with a component named "Apache Commons Foo" that uses maven as its build system, both for distribution binaries and for the web site. Contributions of instructions covering ant-based builds are welcome!

Some knowledge of maven is assumed in this document. Please refer to the maven user guide and reference for more information. Most commons components now use version 1 of maven, so this is the maven version assumed throughout this document. Updates covering maven 2 are welcome!

Local environment setup

To build Apache Commons components or web sites, you need to have maven installed and the project source files checked out locally. You may also need to have some local build properties specified. This section describes the local setup required.

Getting and installing maven

Follow the maven download instructions to download and install maven. You will need Maven version 1.0.2.

You need the maven-xdoc-plugin version 1.9.2 to build a Apache Commons web site. If you don't have that version, you need to upgrade. To automatically install the plugin, type the following on a single line:

      maven plugin:download
        -Dmaven.repo.remote=http://www.ibiblio.org/maven
        -DgroupId=maven
        -DartifactId=maven-xdoc-plugin
        -Dversion=1.9.2

For a manual installation, you can download the plugin here: http://www.apache.org/dyn/closer.cgi/java-repository/maven/plugins/maven-xdoc-plugin-1.9.2.jar

Checking out the commons sources

Follow the commons subversion instructions to check out the commons source files. If you want to build commons component web sites, you need to have the commons-build module checked out as a peer to whatever component(s) you are trying to generate the site for. If you check out the full set of "trunks" using the svn externals "trunks-proper" or "trunks-sandbox" this will happen automatically.

Configuring local properties

As a general rule, dependencies on local properties (things not specified in project.xml or project.properties) should be avoided. There are a few things, however, that are specific to individual developers that may need to be set locally. Generally, only committers need to set local properties. As documented in the maven properties reference, maven looks for property values first in project.properties in the project's base directory (where project.xml lives), then in build.properties if that exists in the same directory, then in ${user.home}/build.properties and finally in System properties. Values are successively overridden if they occur in multiple places in this list.

It may be convenient for committers to set the following properties in ${user.home}/build.properties, which will make them apply to all maven projects that they work on.

      maven.username (apache user name, for site deployment)
      maven.repo.apache.releases.username   (apache user name, for releases)
      maven.repo.apache.releases.privatekey (path to pgp signing key)

If you work only on Commons projects, you can add maven.remote.group=commons otherwise this will need to be specified for each project that you want to use maven to deploy.

Maven POM (project.xml) configuration

This section describes the key elements in the maven POM and how the sample project.xml file needs to be modified or customized for different components.

Top level elements

The following top-level elements of the POM should be changed to reflect the individual component. Elements not listed below should keep the values in the sample file.

Element	Value	Example (value to replace)
name	The capitalized name of the component	Foo
groupId	commons-component (where "component" is the uncapitalized name of the component)	commons-foo
artifactId	commons-component (where "component" is the uncapitalized name of the component)	commons-foo
currentVersion	Current version under development. Development versions should always use the suffix "-SNAPSHOT". Maven handles these SNAPSHOTs in a special way.	1.0-SNAPSHOT
inceptionYear	Year when the component was created	2005
shortDescription	Apache Commons <capitalized name>	Apache Commons Foo
description	Brief description of what the component is	A Foo thing
logo	Path to component logo image	/images/logo.png

Addtional POM elements

In addition to the basic project information in the preceeding section, the following POM elements need to be customized or at least verified for individual commons components.

Element	Comments
versions	This list should include an entry for each official release of the component. The `id` and `name` for each version should be the release number, e.g., "1.2". The `tag` should be the name of the svn tag corresponding to the sources used to build the release.
developers	This list should include all committers who work on the component. Each committer's first commit to the component should include adding him/herself to the developers list.
contributors	Most commons components have done away with @author tags, so the way we acknowledge contributors is by adding them to this list, which will appear on the maven-generated web site. Make sure that you have the correct spelling of contributors' names and do not include email addresses unless contributors specifically authorize this.
dependencies	Minimize these ;-) and group them with comments designating them as compile-time, run-time and test-only if these distinctions apply.
build	The sample file assumes the standard directory layout, with source code in `src/jave` and tests in `src/test`. It also assumes that test classes have names that end in "Test", e.g. "FooManagerTest.java" and that abstract test classes have names that end in "AbstractTest", e.g., "FooFooerAbstractTest." Do not remove the `resources` element that includes `NOTICE.TXT`.
reports	Reports published to the live web site should be limited to the short list in the sample POM. The `maven-changes-plugin` should only be used if the component maintains a `changes.xml` file in `/xdocs` following the format described in the maven changes plugin documentation

Property configuration

There is very little that needs to be customized for individual components in the sample project.properties file. The table below calls out a few properties which individual components may wish to modify. The deployment and site generation properties (which should not be changed) are discussed in other sections below.

Property	Comments
maven.test.skip	Set to `true` to skip unit tests during build. Useful when testing build changes, but should not be checked in with value set to `true.`
maven.test.failure	Set this to `false` to allow the build to succeed when unit tests fail. This should not be checked in with the value set to `false.`
maven.checkstyle.properties	If the component uses the checkstyle plugin, this property should be set to the location of the checkstyle rules file.
maven.javadoc.links	Components should add links to any javadoc resources beyond the jdk here.

Generating and publishing the web site

Generating the site

Use maven site:generate to generate the component web site locally. Then review the generated html in target/docs. Navigation is specified in xdocs/navigation.xml. The sample navigation file refers to entities defined in ../../commons-build/menus/menus.dtd, so, as mentioned above, commons-build has to be checked out as a peer to the component whose web site you are building. The menu structure in the sample is standard for commons components, though some elements (e.g, User's Guide) may not be present for all components. At the very least, each component must include a link to the javadoc for the most recent release. Links to Javadoc for the last release and for the current development branch should also be included.

Updating the public site

Once you have successfully generated the web site, reviewed it locally and checked in any xdocs or configuration changes, you can deploy it using maven site:sshdeploy. For this to work, the following conditions need to be satisfied:

The <siteAddress> and <siteDirectory> POM elements have to have the values people.apache.org and /www/commons.apache.org/foo/ respectively (will be the case if you do not change the settings in project.xml.sample.)
You have to be a member of the commons unix group (should happen when you are set up as a commons committer).
You have an ssh key set up. Follow the instructions on the apache SSH Access page to get this set up correctly.

Assuming the second condition above is met, you can also deploy sites manually using scp or by tarring / zipping and extracting the content to the path above on people.apache.org. In either case, The content will be rsynced within a few hours to the production web server(s). Note that because of the rsync delay, your changes will not appear immediately on the live site.

Deploying jars

Maven's jar plugin supports automated deployment of jars to local or remote maven repositories. This powerful feature makes distribution easy, but also makes inadvertent deployment of development jars possible. The setup in project.properties.sample ensures that this does not happen. This section explains how to deploy jars using maven at apache.

Repository structure

There are three maven repositories that apache committers should be aware of. The table below describes each of these repositories.

External Repository URL	Function
http://people.apache.org/repository/	deployment location for apache "snapshot" jars. This repository exists for internal apache use only. External projects should not use this repository for dependency resolution and apache projects should not release code with dependencies on snapshot jars.
http://www.apache.org/dist/java-repository/	deployment location for released apache jars. Should not be accessed for dependency resolution by maven builds. Contents are rsynced to ibiblio.
http://www.ibiblio.org/maven/	Default remote repository used by maven. Apache release jars are rsynced every few hours from java-repository to this repository.

Deploying release jars

The instructions here automate the process described in step 8 of the Apache Commons Release Instructions. It is essential that only release jars be deployed in this way and that the artifactId in project.xml be set correctly.

To deploy a release jar to java-repository, you need to have an ssh key set up (see site deployment section above) and you need to set the maven.repo.list property to refer to java-repository. The safest way to do this (to minimize risk of inadvertent production deployment) is to rely on the setup in project.properties.sample, which leaves java-repository off of the repo list, so you have to overrride the list on the command line:

maven -Dmaven.repo.list=apache.releases jar:deploy

For this to work, apache.releases has to be configured as in project.properties.sample to point to java-repository. Jars deployed to java-repository will be rsynced to ibiblio within a few hours. Immediately after deployment, you should download the jar and verify that:

the name of the jar is correct (matches release)
the md5 checksum of the jar matches the release version
the md5 checksum of the jar is correct
the manifest of the jar is correct

As stated in the release instructions, please make sure to update the currentVersion to end in -SNAPSHOT after pushing out the release jar.

Deploying snapshot jars

Assuming the setup in project.properties.sample, and correctly set currectVersion (should end in -SNAPSHOT) the following command can be used to deploy a "snapshot" or "development build" jar to the apache snapshot repository:

maven jar:deploy-snapshot

Scripting in maven.xml

To customize the behavior of maven builds, jelly-script can be added to maven.xml The maven scripting reference provides details and examples. For ease of maintanance and to make upgrades easier, scripting should be kept to an absolute minimum. If a plugin does not quite do what we need, it is better to try to get the plugin extended to perform the function than to script around the problem. That said, there are some standard customizations for commons components shown in maven.xml.sample The pre- and post-goals defined there copy the resources required to ensure that all web sites have a common look and feel and that required licenses and notices are included in release distributions.