The configuration definition file

In previous chapters we have already seen how specific configuration classes like PropertiesConfiguration or XMLConfiguration can be used to load configuration data from a single source. This may be sufficient for small applications, but if requirements for configuration become more complex, additional support for managing a set of different configuration sources is desired. This is the domain of DefaultConfigurationBuilder which allows combining multiple configuration sources. The properties defined in these sources can then be accessed as if they were defined in a single configuration file. The sources to be loaded have to be defined in a XML document with a specific structure, a so-called configuration definition file. The following listing shows a simple example of such a definition file:

<?xml version="1.0" encoding="ISO-8859-1" ?>

<configuration>
  <properties fileName="usergui.properties"/>
</configuration>

A configuration definition file can contain an arbitrary number of elements declaring the configuration sources to load. The <properties> element is one of these; it is used to include properties files. For this example we store the definition file in the same directory as the properties file and call it config.xml. The properties file used in this example is the same as in the section about properties files.

Setting up a DefaultConfigurationBuilder

Now we have to create a DefaultConfigurationBuilder object and let it read this definition file. This is quite simple: Just create a new instance and set the name of the definition file (DefaultConfigurationBuilder is derived from XMLConfiguration, so all options for specifying the document to load are available here, too). The combined configuration collecting all sources defined in the configuration definition file can then be obtained by calling the getConfiguration() method:

DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder();
builder.setFile(new File("config.xml"));
Configuration config = builder.getConfiguration(true);

Now the config object can be accessed in the usual way to query configuration properties, e.g. by using methods like getString(), or getInt(). We will see in a moment how properties defined in different configuration sources are accessed.

Overriding properties

Using DefaultConfigurationBuilder to collect configuration sources does not make much sense if there is only a single source to be loaded. So let's add another one! This time we will embed a XML file: gui.xml which is shown in the next listing:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<gui-definition>
  <colors>
    <background>#808080</background>
    <text>#000000</text>
    <header>#008000</header>
    <link normal="#000080" visited="#800080"/>
  </colors>
  <rowsPerPage>15</rowsPerPage>
</gui-definition>

To make this XML document part of our global configuration we have to modify our configuration definition file to also include the new file. For XML documents the element <xml> can be used so that we have now:

<?xml version="1.0" encoding="ISO-8859-1" ?>

<configuration>
  <properties fileName="usergui.properties"/>
  <xml fileName="gui.xml"/>
</configuration>

The code for setting up the DefaultConfigurationBuilder object remains the same. From the Configuration object returned by the factory the new properties can be accessed in the usual way.

There is one problem with this example configuration setup: The color.background property is defined in both the properties and the XML file, and - to make things worse - with different values. Which value will be returned by a call to getString()?

The answer is that the configuration sources are searched in the order they are defined in the configuration definition file. Here the properties file is included first, then comes the XML file. Because the color.background property can be found in the properties file the value specified there will be returned (which happens to be #FFFFFF).

It might not be obvious why it makes sense to define the value of one and the same property in multiple configuration sources. But consider the following scenario: An application comes with a set of default properties and allows the user to override some or all of them. This can now easily be realized by saving the user's settings in one file and the default settings in another. Then in the configuration definition file the file with the user settings is included first and after that the file with the default values. The application code that queries these settings needs no be aware whether a property was overridden by the user. DefaultConfigurationBuilder takes care that properties defined in the first file (the user file) are found; other properties which the user has not changed will still be returned from the second file (the defaults file).

Optional configuration sources

The example above with two configuration sources - one for user settings and one with default values - raises an interesting question: What happens if the user has not defined specific properties yet? Or what if a new user starts our application for the first time and thus no user specific properties exist?

The default behavior of DefaultConfigurationBuilder is to throw a ConfigurationException exception if one of the sources defined in the configuration definition file cannot be loaded. For our example this behavior is not desired: the properties file with specific user settings is not required. If it cannot be loaded, the example application should still work because a complete set of configuration properties is defined in the second file.

DefaultConfigurationBuilder supports such optional configuration sources. For this purpose in the definition of a configuration source the config-optional attribute can be placed. An example of this is shown below:

<?xml version="1.0" encoding="ISO-8859-1" ?>

<configuration>
  <properties fileName="usersettings.properties" config-optional="true"/>
  <properties fileName="default.properties"/>
</configuration>

In this configuration definition file the first properties file with user specific settings is marked as optional. This means that if it cannot be loaded, DefaultConfigurationBuilder will not throw an exception, but only write a warning message to its logger. Note that the config-optional attribute is absent for the second properties file. Thus it is mandatory, and the getConfiguration() method of DefaultConfigurationBuilder would throw an exception if it could not be found.

Union configuration

In an earlier section about the configuration definition file for DefaultConfigurationBuilder it was stated that configuration files included first can override properties in configuration files included later, and an example use case for this behavior was given. There may be cases when there are other requirements.

Let's continue the example with the application that somehow process database tables and that reads the definitions of the affected tables from its configuration. This example and the corresponding XML configuration files were introduced in the section about XMLConfiguration. Now consider that this application grows larger and must be maintained by a team of developers. Each developer works on a separated set of tables. In such a scenario it would be problematic if the definitions for all tables would be kept in a single file. It can be expected that this file needs to be changed very often and thus can be a bottleneck for team development when it is nearly steadily checked out. It would be much better if each developer had an associated file with table definitions and all these information could be linked together at the end.

DefaultConfigurationBuilder provides support for such a use case, too. It is possible to specify in the configuration definition file that from a set of configuration sources a logic union configuration is to be constructed. Then all properties defined in the provided sources are collected and can be accessed as if they had been defined in a single source. To demonstrate this feature let us assume that a developer of the database application has defined a specific XML file with a table definition named tasktables.xml:

<?xml version="1.0" encoding="ISO-8859-1" ?>

<config>
  <table tableType="application">
    <name>tasks</name>
    <fields>
      <field>
        <name>taskid</name>
        <type>long</type>
      </field>
      <field>
        <name>name</name>
        <type>java.lang.String</type>
      </field>
      <field>
        <name>description</name>
        <type>java.lang.String</type>
      </field>
      <field>
        <name>responsibleID</name>
        <type>long</type>
      </field>
      <field>
        <name>creatorID</name>
        <type>long</type>
      </field>
      <field>
        <name>startDate</name>
        <type>java.util.Date</type>
      </field>
      <field>
        <name>endDate</name>
        <type>java.util.Date</type>
      </field>
    </fields>
  </table>
</config>

This file defines the structure of an additional table, which should be added to the so far existing table definitions. To achieve this the configuration definition file has to be changed: A new section is added that contains the include elements of all configuration sources which are to be combined.

<?xml version="1.0" encoding="ISO-8859-1" ?>
<!-- Configuration definition file that demonstrates the
     override and additional sections -->

<configuration>
  <override>
    <properties fileName="usergui.properties"/>
    <xml fileName="gui.xml"/>
  </override>

  <additional>
    <xml fileName="tables.xml"/>
    <xml fileName="tasktables.xml" config-at="tables"/>
  </additional>
</configuration>

Compared to the older versions of this file some changes have been done. One major difference is that the elements for including configuration sources are no longer direct children of the root element, but are now contained in either an <override> or <additional> section. The names of these sections already imply their purpose.

The override section is not strictly necessary. Elements in this section are treated as if they were children of the root element, i.e. properties in the included configuration sources override properties in sources included later. So the <override> tags could have been omitted, but for the sake of clearity it is recommended to use them if there is also an <additional> section.

It is the <additional> section that introduces a new behavior. All configuration sources listed here are combined to a union configuration. In our example we have put two xml elements in this area that load the available files with database table definitions. The syntax of elements in the additional section is analogous to the syntax described so far. In this example the config-at attribute is introduced. It specifies the position in the logic union configuration where the included properties are to be added. Here it is set for the second element to the value tables. This is because the file starts with a <table> element, but to be compatible with the other table definition file it should be accessible under the key tables.table.

After these modifications have been performed, the configuration obtained from DefaultConfigurationBuilder allows access to three database tables. A call of config.getString("tables.table(2).name"); results in a value of tasks. In an analogous way it is possible to retrieve the fields of the third table.

Note that it is also possible to override properties defined in an additional section. This can be done by placing a configuration source in the override section that defines properties that are also defined in one of the sources listed in the additional section. The example does not make use of that. Note also that the order of the override and additional sections in a configuration definition file does not matter. Sources in an override section are always treated with higher priority (otherwise they could not override the values of other sources).

Configuration definition file reference

Configuration definition files are XML documents telling DefaultConfigurationBuilder which configuration sources to load and how to process them in order to create the resulting combined configuration.

Overall structure of a configuration definition file

A configuration definition file for DefaultConfigurationBuilder can contain three sections, all of which are optional. A skeleton looks as follows:

<?xml version="1.0" encoding="ISO-8859-1" ?>

<configuration systemProperties="path to property file">
  <header>
    <!-- Meta data about the resulting combined configuration -->
  </header>
  <override>
    <!-- Configuration declarations with override semantics -->
  </override>
  <additional>
    <!-- Configuration declarations that form a union configuration -->
  </additional>
</configuration>

Declaring configuration sources

The override and additional sections have already been introduced when the basics of DefaultConfigurationBuilder were discussed. They contain declarations for the configuration sources to be embedded. For convenience reasons it is also possible to declare configuration sources outside these sections; they are then treated as if they were placed inside the override section.

Each declaration of a configuration source is represented by an XML element whose name determines the type of the configuration source. Attributes or nested elements can be used to provide additional configuration options for the sources to be included (e.g. a name of a file to be loaded or a reloading strategy). Below is a list of all tags which can be used out of the box:

properties

With this element properties files can be included. The name of the file to load is specified using the fileName attribute. Which configuration class is created by this tag depends on the extension of the file to load: If the extension is ".xml", a XMLPropertiesConfiguration object is created, which is able to process the XML properties format introduced in Java 5.0. Otherwise a PropertiesConfiguration object is created, the default reader for properties files.

xml

The xml element can be used to load XML configuration files. It also uses the fileName attribute to determine the name of the file to load and creates an instance of XMLConfiguration.

jndi

As the name implies, with this element JNDI resources can be included in the resulting configuration. Under the hood this is done by an instance of the JNDIConfiguration class. The prefix attribute can be used to select a subset of the JNDI tree.

plist

The plist element allows to embedd configuration files in the NeXT / OpenStep or Mac OS X format. Again the name of the file to load is specified through the fileName attribute. If a XML file is specified, a XMLPropertyListConfiguration object is created to process the file. Otherwise this task is delegated to a PropertyListConfiguration instance.

system

With this element an instance of SystemConfiguration is added to the resulting configuration allowing access to system properties. Note: Using this element system properties are directly made available. Alternatively the interpolation features introduced in version 1.4 (see Variable Interpolation for more details) can be used for referencing system properties.

configuration

The configuration tag allows other configuration definition files to be included. This makes it possible to nest these definition files up to an arbitrary depth. In fact, this tag will create another DefaultConfigurationBuilder object, initialize it, and obtain the CombinedConfiguation from it. This combined configuration will then be added to the resulting combined configuration. Like all file-based configurations the fileName attribute can be used to specify the configuration definition file to be loaded. This file must be an XML document that conforms to the format described here. Some of the most important settings are copied from the original DefaultConfigurationBuilder object to the newly created builder:

• the base path under which configuration files are searched
• some flags, e.g. for controlling delimiter parsing or throwing exceptions on missing properties
• the logger
• the configuration and error listeners

ini

This tag can be used to include an ini file into the resulting combined configuration. Behind the scenes an instance of HierarchicalINIConfiguration is used to load the ini file.

env

With this tag direct access to environment properties can be enabled. This works in the same way as the <system> tag for Java system properties.

In the declaration of a configuration source it is possible to set properties on the corresponding configuration objects. Configuration declarations are indeed Bean declarations. That means they can have attributes matching simple properties of the configuration object to create, and sub elements matching complex properties. The following example fragment shows how complex initialization can be performed in a configuration declaration:

  <properties fileName="test.properties" throwExceptionOnMissing="true">
    <reloadingStrategy refreshDelay="10000"
    config-class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/>
  </properties>
  <xml fileName="test.xml" delimiterParsingDisabled="true">
    <expressionEngine config-class="org.apache.commons.configuration.tree.DefaultExpressionEngine"
      propertyDelimiter="/" indexStart="[" indexEnd="]"/>
  </xml>

In this example a configuration source for a properties file and one for an XML document are defined. For the properties source the throwExceptionOnMissing property is set to true, which means that it should throw an exception if a requested property is not found. In addition it is assigned a reloading strategy, which is declared and configured in a sub element. The XML configuration source is initialized in a similar way: a simple property is set, and an expression engine is assigned. More information about the format for declaring objects and initializing their properties can be found in the section about bean declarations.

In addition to the attributes that correspond to properties of the configuration object to be created, a configuration declaration can have a set of special attributes that are evaluated by DefaultConfigurationBuilder when it creates the objects. These attributes are listed in the following table:

Note: In older versions of Commons Configuration the attributes config-at and config-optional were named at and optional respective. They have been renamed in order to avoid possible name clashes with property names for configuration sources. However, for reasons of backwards compatibility, the old attribute names can still be used.

Another useful feature is the built-in support for interpolation (i.e. variable substitution): You can use variables in your configuration definition file that are defined in declared configuration sources. For instance, if the name of a configuration file to be loaded is defined by the system property CONFIG_FILE, you can do something like this:

<configuration>
  <!-- Load the system properties -->
  <system/>
  <!-- Now load the config file, using a system property as file name -->
  <properties fileName="${CONFIG_FILE}"/>
</configuration>

Note that you can refer only to properties that have already been loaded. If you change the order of the <system> and the <properties> elements in the example above, an error will occur because the ${CONFIG_FILE} variable will then be undefined at the moment it is evaluated.

<configuration systemProperties="systemProperties.xml">
  <!-- Load the system properties -->
  <system/>
  <!-- Now load the config file, using a system property as file name -->
  <properties fileName="${CONFIG_FILE}"/>
</configuration>

This example differs from the previous one by the systemProperties attribute added to the root element. It causes the specified to be read and all properties defined therein to be added to the system properties. So properties like CONFIG_FILE can be defined in a properties file and are then treated as if they were system properties.

The header section

In the header section properties of the resulting combined configuration object can be set. The main part of this section is a bean declaration that is used for creating the resulting configuration object. Other elements can be used for customizing the Node combiners used by the override and the union combined configuration. The following example shows a header section that uses all supported properties:

  <header>
    <result delimiterParsingDisabled="true" forceReloadCheck="true">
      <nodeCombiner config-class="org.apache.commons.configuration.tree.OverrideCombiner"/>
      <expressionEngine config-class="org.apache.commons.configuration.tree.xpath.XPathExpressionEngine"/>
    </result>
    <combiner>
      <override>
        <list-nodes>
          <node>table</node>
          <node>list</node>
        </list-nodes>
      </override>
      <additional>
        <list-nodes>
          <node>table</node>
        </list-nodes>
      </additional>
    </combiner>
  </header>

The result element points to the bean declaration for the resulting combined configuration. In this example we set some attributes and initialize the node combiner (which is not necessary because the default override combiner is specified), and the expression engine to be used. Note that the config-class attribute makes it possible to inject custom classes for the resulting configuration or the node combiner.

The combiner section allows nodes to be defined as list nodes. This can be necessary for certain node combiner implementations to work correctly. More information can be found in the section about Node combiners.

Note: From time to time the question is raised whether there is a document type definition or a schema defining exactly the structure of a configuration definition file. Frankly, the answer is no. This is due to the fact that the format is extensible. As will be shown below, it is possible to register yout own tags in order to embed custom configuration sources.

An example

After all that theory let's go through a more complex example! We start with the configuration definition file that looks like the following:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<!-- Test configuration definition file that demonstrates complex initialization -->
<configuration>
  <header>
    <result delimiterParsingDisabled="true" forceReloadCheck="true">
      <expressionEngine config-class="org.apache.commons.configuration.tree.xpath.XPathExpressionEngine"/>
    </result>
    <combiner>
      <additional>
        <list-nodes>
          <node>table</node>
        </list-nodes>
      </additional>
    </combiner>
  </header>
  <override>
    <properties fileName="user.properties" throwExceptionOnMissing="true"
      config-name="properties" config-optional="true">
      <reloadingStrategy refreshDelay="10000"
      config-class="org.apache.commons.configuration.reloading.FileChangedReloadingStrategy"/>
    </properties>
    <xml fileName="settings.xml" config-name="xml"/>
  </override>
  <additional>
    <xml config-name="tab1" fileName="table1.xml" config-at="database.tables"/>
    <xml config-name="tab2" fileName="table2.xml" config-at="database.tables"
        validating="true"/>
  </additional>
</configuration>

This configuration definition file includes four configuration sources and sets some properties for the resulting CombinedConfiguration. Of special interest is the forceReloadCheck property, which enables a special check for detecting property changes in the contained configuration sources. If this property is not set, reloading won't work. Because we have configured a reloading strategy for one of the included configuration sources we have to set this flag so that this reloading strategy can function properly. More details about this topic can be found in the Javadocs for CombinedConfiguration. We also set some properties for the configurations to be loaded; for instance we declare that one of the XML configurations should be validated.

With the following code we can create a DefaultConfigurationBuilder and load this file:

DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder();
builder.setFile(new File("configuration.xml"));
CombinedConfiguration cc = builder.getConfiguration(true);

It would have been possible to specify the location of the configuration definition file in multiple other ways, e.g. as a URL. The boolean argument in the call to getConfiguration() determines whether the configuration definition file should be loaded. For our simple example we want this to happen, but it would also be possible to load the file manually (by calling the load() method), and after that updating the configuration. (Remember that DefaultConfigurationBuilder is derived from XMLConfiguration, that means you can use all methods provided by this class to alter its data, e.g. to add further configuration sources.) If the configuration's data was manually changed, you should call getConfiguration() with the argument false. XMLConfiguration also provides the registerEntityId() method that can be used to define the location of DTD files (refer to the section Validation of XML configuration files for more details). This method is available for DefaultConfigurationBuilder, too. The entities registered here will be passed to the loaded child XML configurations. So you can register the DTDs of all child XML configurations globally at the configuration builder.

In the header section we have chosen an XPATH expression engine for the resulting configuration. So we can query our properties using the convenient XPATH syntax. By providing the config-name attribute we have given all configuration sources a name. This name can be used to obtain the corresponding sources from the combined configuration. For configurations in the override section this is directly possible:

Configuration propertiesConfig = cc.getConfiguration("properties");
Configuration xmlConfig = cc.getConfiguration("xml");

Configurations in the additional section are treated a bit differently: they are all packed together in another combined configuration and then added to the resulting combined configuration. So in our example the combined configuration cc will contain three configurations: the two configurations from the override section, and the combined configuration with the additional configurations. The latter is stored under a name determined by the ADDITIONAL_NAME constant of DefaultConfigurationBuilder. The following code shows how the configurations of the additional section can be accessed:

CombinedConfiguration ccAdd = (CombinedConfiguration)
  cc.getConfiguration(DefaultConfigurationBuilder.ADDITIONAL_NAME);
Configuration tab1Config = ccAdd.getConfiguration("tab1");
Configuration tab2Config = ccAdd.getConfiguration("tab2");

Extending the configuration definition file format

If you have written a custom configuration class, you might want to declare instances of this class in a configuration definition file, too. With DefaultConfigurationBuilder this is now possible by registering a ConfigurationProvider.

ConfigurationProvider is an inner class defined in DefaultConfigurationBuilder. Its task is to create and initialize a configuration object. Whenever DefaultConfigurationBuilder encounters a tag in the override or the additional section it checks whether for this tag a ConfigurationProvider was registered. If this is the case, the provider is asked to create a new configuration instance; otherwise an exception will be thrown.

So for adding support for a new configuration class you have to create an instance of ConfigurationProvider (or a derived class) and register it at the configuration builder using the addConfigurationProvider() method. This method expects the name of the associated tag and the provider instance as arguments.

If your custom configuration class does not need any special initialization, you can use the ConfigurationProvider class directly. It is able of creating an instance of a specified class (which must be derived from AbstractConfiguration). Let's take a look at an example where we want to add support for a configuration class called MyConfiguration. The corresponding tag in the configuration definition file should have the name myconfig. The code for registering the new provider and loading the configuration definition file looks as follows:

DefaultConfigurationBuilder builder = new DefaultConfigurationBuilder();
DefaultConfigurationBuilder.ConfigurationProvider provider = new
  DefaultConfigurationBuilder.ConfigurationProvider(MyConfiguration.class);
builder.addConfigurationProvider("myconfig", provider);

builder.setFileName("configuration.xml");
Configuration config = builder.getConfiguration();

If your configuration provider is registered this way, your configuration definition file can contain the myconfig tag just as any other tag for declaring a configuration source:

<configuration>
  <additional>
    <xml fileName="settings.xml"/>
    <myconfig delimiterParsingDisabled="true"/>
  </additional>
</configuration>

As is demonstrated in this example, it is possible to specify attributes for initializing properties of your configuration object. In this example we set the default delimiterParsingDisabled property inherited from AbstractConfiguration. Of course you can set custom properties of your configuration class, too.

If your custom configuration class is a file-based configuration, you should use the FileConfigurationProvider class instead of ConfigurationProvider. FileConfigurationProvider is another inner class of DefaultConfigurationBuilder that knows how to deal with file-based configurations: it ensures that the correct base path is set and takes care of invoking the load() method.

If your custom configuration class requires special initialization, you need to create your own provider class that extends ConfigurationProvider. Here you will have to override the getConfiguration(ConfigurationDeclaration) method, which is responsible for creating the configuration instance (all information necessary for this purpose can be obtained from the passed in declaration object). It is recommended that you call the inherited method first, which will instantiate and initialize the new configuration object. Then you can perform your specific initialization.