Handling of missing properties

What is a configuration object supposed to do if you pass in a key to one of its get methods that does not map to an existing property? Well, the default behavior as implemented in AbstractConfiguration is to return null if the return value is an object type. For primitive types as return values returning null (or any other special value) is not possible, so in this case a NoSuchElementException is thrown:

// This will return null if no property with key "NonExistingProperty" exists
String strValue = config.getString("NonExistingProperty");

// This will throw a NoSuchElementException exception if no property with
// key "NonExistingProperty" exists
long longValue = config.getLong("NonExistingProperty");

For object types like String, BigDecimal, or BigInteger this default behavior can be changed: If the setThrowExceptionOnMissing() method is called with an argument of true, these methods will behave like their primitive counter parts and also throw an exception if the passed in property key cannot be resolved.

Note: Unfortunately support for the throwExceptionOnMissing property is not always consistent: The methods getList() and getStringArray() do not evaluate this flag, but return an empty list or array if the requested property cannot be found. Maybe this behavior will be changed in a future major release.

List handling

With getList() and getStringArray() the Configuration interface defines methods for dealing with properties that have multiple values. When a configuration source (e.g. a properties file, an XML document, or a JNDI context) is processed the corresponding Configuration implementation detects such properties with multiple values and ensures that the data is correctly stored.

When modifying properties the addProperty() and setProperty() methods of AbstractConfiguration also implement special list handling. The property value that is passed to these methods can be a list or an array resulting in a property with multiple values.

For some use cases it makes sense to treat string values in a special way. For instance, some configuration formats do not support lists directly. As an example, properties files can be used: they just consist of key-value pairs, and there is no default syntax to assign multiple values to a key. AbstractConfiguration supports a mechanism to split strings at specific characters, so that effectively multiple values are generated from a single string. To achieve this, an AbstractConfiguration instance collaborates with an object implementing the ListDelimiterHandler interface. This interface defines methods for dealing with objects that may contain multiple values. Examples of such complex objects are arrays, lists, and strings containing a specific list delimiter. Depending on a concrete implementation of the ListDelimiterHandler interface, these objects are treated in a special way.

Per default, AbstractConfiguration uses an instance of the DisabledListDelimiterHandler class as list handler. This class can deal with lists and arrays, but it does not recognize any list delimiter characters in strings. (Hence the name, as splitting of strings at list delimiters is disabled.)

An alternative implementation of ListDelimiterHandler is DefaultListDelimiterHandler. This class actually supports string splitting; the list delimiter character has to be passed to the constructor. Whenever a property is added to the configuration (or the configuration is loaded from its source), it is checked whether the property value is a string and - if so - if it contains the list delimiter character. In this case, the string is split, and its single parts are added one by one. By using the setListDelimiterHandler() method you can set a specific list handler implementation. Here are some examples:

// Enable list delimiter handling using a slash as delimiter character
config.setListDelimiterHandler(new DefaultListDelimiterHandler('/'));
// Now add some properties
config.addProperty("greeting", "Hello, how are you?");
config.addProperty("colors.pie",
  new String[] { "#FF0000", "#00FF00", "#0000FF" });
config.addProperty("colors.graph", "#808080/#00FFCC/#6422FF");

// Access data
String salut = config.getString("greeting");
List<Object> colPie = config.getList("colors.pie");
String[] colGraph = config.getStringArray("colors.graph");

String firstPieColor = config.getString("colors.pie");

In this example splitting of strings is enabled using the slash character as delimiter. Because this character is not contained in the greeting property it won't be split, but remains a single string. In contrast, the string passed as value for the colors.graph property contains the delimiter character and thus will result in a property with three values. Note that the version of the getList() method used here returns lists of type Object. This is because the concrete class of the values of a property is not known. For instance, if you call addProperty("answer", 42), an Integer object will be stored in the configuration. However, there are overloaded versions of getList() supporting a data type conversion to a specific target class (see below).

Of interest is also the last line of the example fragment. Here the getString() method is called for a property that has multiple values. This call will return the first value of the list.

One word of warning at the end: Be careful when changing the list delimiter handler on a configuration which already contains values. List handling is typically applied already when properties are added to the configuration. If later another handler is set which processes lists differently, results may be unexpected; some operations may even cause exceptions!

Variable Interpolation

If you are familiar with Ant or Maven, you have most certainly already encountered the variables (like ${token}) that are automatically expanded when the configuration file is loaded. Commons Configuration supports this feature as well, here is an example (we use a properties file in this example, but other configuration sources work the same way; you can learn more about properties files in the chapter Properties files):

application.name = Killer App
application.version = 1.6.2

application.title = ${application.name} ${application.version}

If you now retrieve the value for the application.title property, the result will be Killer App 1.6.2. So per default variables are interpreted as the keys of other properties. This is only a special case, the general syntax of a variable name is ${prefix:name}. The prefix tells Commons Configuration that the variable is to be evaluated in a certain context. We have already seen that the context is the current configuration instance if the prefix is missing. The following table lists a few of the prefixes supported by default. (See the next section for more details.)

user.file = ${sys:user.home}/settings.xml

action.key = ${const:java.awt.event.KeyEvent.VK_CANCEL}

java.home = ${env:JAVA_HOME}

Below is some more information related to variable interpolation users should be aware of:

• If a variable cannot be resolved, e.g. because the name is invalid or an unknown prefix is used, it won't be replaced, but is returned as is including the dollar sign and the curly braces.
• References to variables can be nested; a variable can refer to a variable which in turn can have references to other variables and so on.
• Cyclic references are detected. In this case, no interpolation is done.
• Variable interpolation happens when properties are queried from the configuration, not at creation time. Therefore, the mechanism is quite dynamic: changes on one property can impact the value of another property that references the first one.
• Variable interpolation is done by all property access methods. One exception is the generic getProperty() method which returns the raw property value.

Default interpolation lookups

Commons configuration comes with a number of default prefix interpolators in addition to the few demonstrated above. The full list can be found in the documentation of the ConfigurationInterpolator.getDefaultPrefixLookups() method. This method returns a map of standard interpolation lookup objects keyed by prefix, which are enabled by default in all interpolation operations performed through the Configuration interface. Prior to version 2.8.0, this map was constant. However, starting in version 2.8.0, the lookups included in this map can be optionally configured via system property, allowing users greater control over the types of interpolation performed in their applications. The system property in question is named org.apache.commons.configuration2.interpol.ConfigurationInterpolator.defaultPrefixLookups and is expected to contain a comma-separated list of names from the DefaultLookups enum. For example, launching an application with the system property given below will only enable the sys and env lookups.

org.apache.commons.configuration2.interpol.ConfigurationInterpolator.defaultPrefixLookups=SYSTEM_PROPERTIES,ENVIRONMENT

It is also important to note that starting in version 2.8.0, several previously enabled default lookups were changed to be disabled by default. These include the dns, url, and script lookups. These lookups are still present in the library but must be explicitly enabled by either

1. listing them in the system property described above (along with all other enabled lookups), or
2. adding them programmatically using the techniques laid out in the next section.

Customizing interpolation

This sub section goes a bit behind the scenes of interpolation and explains some approaches for adding your own interpolation facilities. Under the hood the implementation of interpolation relies on objects implementing the Lookup interface. Lookup defines a simple lookup() method that must be implemented by custom implementations; it expects the name of a variable as argument and returns the corresponding value. The standard prefixes for variables we have covered so far are indeed provided by special classes implementing Lookup.

It is now possible to create your own implementation of Lookup and configure a configuration builder (builders are introduced in the next chapter of this user's guide) to make it available for all configuration objects it creates under a custom prefix. We will show how this can be achieved. The first step is to create a new class implementing Lookup, which must define the lookup() method. As an example we implement a rather dull lookup object that simply returns a kind of "echo" for the variable passed in:

import org.apache.commons.configuration2.interpol.Lookup;

public class EchoLookup implements Lookup
{
    @Override
    public Object lookup(String varName)
    {
        return "Value of variable " + varName;
    }
}

Now we want this class to be called for variables with the prefix echo. For this purpose the EchoLookup class has to be registered at the ConfigurationInterpolator instance of our configuration with the desired prefix. Each Configuration object is associated with a ConfigurationInterpolator object that handles variable interpolation. It manages the Lookup objects that should be used to resolve variables.

There are multiple ways to make a Lookup object known to a ConfigurationInterpolator. The most direct way is to call the interpolator's registerLookup() method passing in the Lookup and the desired prefix:

// simple, but not recommended approach
ConfigurationInterpolator interpolator = config.getInterpolator();
interpolator.registerLookup("echo", new EchoLookup());

This approach works, but has some drawbacks, especially when used with advanced features like reloading of configurations. The recommended way is to set custom lookup objects via the builder which creates the configuration object; this ensures that every Configuration instance created via the builder has a correctly initialized ConfigurationInterpolator object. To achieve this, create a map using the variable prefixes as keys and the associated Lookup objects as values. This map can then be passed to the setPrefixLookup() method of a parameters object for the builder. Note that the lookups for the default prefixes are added explicitly; omitting a lookup would remove support for the corresponding prefix:

// Create a map with defaults and one additional lookup
Map<String, Lookup> lookups = new HashMap<String, Lookup>(
    ConfigurationInterpolator.getDefaultPrefixLookups());
lookups.put("echo", new EchoLookup());

// Configure builder with lookups
Parameters params = new Parameters();
BasicConfigurationBuilder<PropertiesConfiguration> builder =
        new BasicConfigurationBuilder<PropertiesConfiguration>(
                PropertiesConfiguration.class)
                .configure(params.basic()
                        .setPrefixLookups(lookups);
PropertiesConfiguration config = builder.getConfiguration();

Using Expressions

In addition to the simple lookup mechanisms previously described, Commons Configuration provides ExprLookup which uses Apache Commons Jexl to allow expressions to be resolved wherever a StrLookup is allowed. This example shows an alternate way of obtaining a system property if the ExprLookup is configured.

user.file = ${expr:System.getProperty("user.home")}/settings.xml

ExprLookup is not enabled by default, it must be manually added or configured via DefaultConfigurationBuilder. Builds that use Maven 2 and reference Commons Configuration will not include a dependency on Jexl, so if this feature is used the dependency on Jexl must be manually added to the project.

When using DefaultConfigurationBuilder adding ExprLookup is straightforward.

<configuration>
  <header>
    <result/>
    <lookups>
      <lookup config-prefix="expr"
              config-class="org.apache.commons.configuration2.interpol.ExprLookup">
        <variables>
          <variable name="System" value="Class:java.lang.System"/>
          <variable name"net" value="Class:java.net.InetAddress"/>
          <variable name="String" value="Class:org.apache.commons.lang.StringUtils"/>
        </variables>
      </lookup>
    </lookups>
  </header>
  <override>
    <xml fileName="${expr:System.getProperty('basePath') +
         String.lowercase(net.localHost.hostName) + '/testMultiConfiguration_default.xml'}"
         config-name="defaultConfig" delimiterParsingDisabled="true">
    </xml>
  </override>
</configuration>

The example above shows how to invoke static methods during expression evaluation. The next example shows mixing expression evaluation with a subordinate lookup to obtain the "basePath" system property. Note the difference in how the system property was obtained in the previous example.

<configuration>
  <header>
    <result/>
    <lookups>
      <lookup config-prefix="expr"
              config-class="org.apache.commons.configuration2.interpol.ExprLookup">
        <variables>
          <variable name"net" value="Class:java.net.InetAddress"/>
          <variable name="String" value="Class:org.apache.commons.lang.StringUtils"/>
        </variables>
      </lookup>
    </lookups>
  </header>
  <override>
    <xml fileName="${expr:$[sys:basePath] +
         String.lowercase(net.localHost.hostName) + '/testMultiConfiguration_default.xml'}"
         config-name="defaultConfig" delimiterParsingDisabled="true">
    </xml>
  </override>
</configuration>

Data type conversions

As was already mentioned when discussing the Configuration interface, there are various get() methods returning property values in different target data types. If necessary, a data type conversion is performed. Take a look at the following example:

config.addProperty("answer", "42");
int theAnswer = config.getInt("answer");

Here a string value is assigned to the key answer. Then the getInt() method is called to query this key as an integer value. getInt() triggers a type conversion automatically. If such a conversion is not possible, a runtime exception of type ConversionException is thrown.

The Configuration interface defines a bunch of methods returning property values in different data types. All of these methods work in the same way: they obtain the actual value of the property and attempt a data type conversion if necessary. In addition, there are generic methods for accessing properties in specific data types:

<T> get(Class<T> cls, String key);

Obtains the value of the specified property and tries to convert it to the specified target type. If the key cannot be resolved, result is null, or - if the throwExceptionOnMissing flag is set - an exception is thrown.

<T> get(Class<T> cls, String key, T defValue);

Obtains the value of the specified property and tries to convert it to the specified target type. If the key cannot be resolved, the specified default value is returned (which may be null).

Generic conversion methods are also available for obtaining arrays or collections. For instance, it is possible to obtain the value of a property as an array of int or a list of java.lang.Long objects. For arrays these conversions are directly supported by the generic get() methods: if the target type passed to the method is an array class, an array conversion is done automatically. So to obtain an array of int from a configuration, the following code can be used:

int[] result = config.getInt(int[].class, "myIntArrayKey");

For conversions to collections specific methods are provided (this is necessary because the element type of the collection cannot be determined automatically as in case of arrays). These are the following ones:

<T> List<T> getList(Class<T> cls, String key);

Returns a list of the specified element class.

<T> Collection<T> getCollection(Class<T> cls, String key, Collection<T> target);

This method is similar to getList(), but it allows specifying the target collection. This is useful if the result should be stored in a different collection type, e.g. a set to remove duplicates.

These methods obtain all values stored for the property with the passed in key. For each value a conversion to the desired target type is performed. Then the resulting values are stored in the target array or collection. There are also variants supporting default values.

It is worth to mention that data type conversion plays well together with variable interpolation: Before a data type conversion is attempted interpolation is handled first. Then the resulting object is converted if necessary.

Customizing data type conversions

Commons Configuration supports a number of conversions to standard types out of the box. In addition to the result types of the get() methods in the Configuration interface, some more types are supported. These are listed in the documentation of the DataConfiguration class. DataConfiguration is a decorator providing additional data type conversion methods for more target types; these additional methods are implemented on top of the generic conversion methods provided by AbstractConfiguration.

If your application deals with special objects, there may be the requirement to extend the conversion capabilities offered by the library. To support this use case, there is the ConversionHandler interface. Each instance of a configuration class derived from AbstractConfiguration is associated with such a ConversionHandler object. The interface defines methods for converting an object to a target class, and for converting values to arrays or collections of a given element type. The data type conversion methods implemented in AbstractConfiguration delegate to this handler object.

The ConversionHandler interface gives a developer full control over the whole data conversion process. By implementing the different conversion methods, it is possible to integrate existing conversion libraries (e.g. the converters offered by Commons BeanUtils) with Commons Configuration.

If the conversion facilities provided by Commons Configuration are not to be fully replaced, but only extended by some custom types, the class DefaultConversionHandler is a good starting point. As the name implies, an instance of this class is used per default for doing type conversions. DefaultConversionHandler has some protected methods which can be overridden by subclasses in order to extend data type conversions:

convertValue()

This is the main conversion method for single values. It is called when a single value is to be converted to a target type and also for the single elements of arrays or collections. If a conversion to another target class is to be supported, this is the method to override. A custom implementation typically checks whether the desired target class is one of the classes it supports. If this is the case, the passed in source object is converted, and the result is returned. Otherwise, the super method is called to handle standard conversions.

isComplexObject()

This boolean method determines whether a passed in object contains multiple values. When doing a conversion to a target class DefaultConversionHandler checks whether the source object is a complex object like an array or a collection. If this is the case, it first extracts the value to be converted from the source object before it calls convertValue().

extractConversionValue()

extractConversionValue() is called if a complex object (i.e. an object containing multiple values) is to be converted to a single object (i.e. not an array or a collection). In this case, it is not directly obvious how the multiple values should be handled. This situation occurs for example if a client calls getInt() on a property which actually has multiple values. The default implementation of extractConversionValue() simply returns the first value available. If an application needs a different behavior, it can provide a custom implementation of this method.

convert()

This method contains the logic for converting a single value. It delegates to the other methods discussed here. It first checks whether the object to be converted is a complex one. If so, it extracts the value to be converted. Eventually, it delegates to convertValue() for performing the actual conversion. So in order to gain more control over the whole conversion process, this method is a good candidate for overriding.

After a custom ConversionHandler implementation has been created, it can be installed at a configuration instance by using the setConversionHandler() method:

CustomConversionHandler handler = new CustomConversionHandler(...);
config.setConversionHandler(handler);

Another feature of DefaultConversionHandler is that is allows defining the format for the conversion of Date and Calendar objects. This can be done by calling setDateFormat() with the corresponding date pattern. The expected string argument must be a pattern string compatible with the java.text.SimpleDateFormat class. If no date format was set, the default pattern yyyy-MM-dd HH:mm:ss is used.

Encoded Properties

Sometimes property values cannot be stored in plain text in configuration files. For instance, security-related information like database passwords should be encrypted. Commons Configuration does not provide algorithms for encrypting (or otherwise encoding) properties. However, there is a generic mechanism for automatically reading encoded properties and transforming them into plain text before they are handed over to the caller. A key role in this mechanism plays the ConfigurationDecoder interface.

ConfigurationDecoder defines a single method decode() which expects a string as input and returns a decoded string. It should be easy for an application to provide a custom implementation for the encoding algorithm it uses. The ImmutableConfiguration interface defines two overloaded methods for querying the values of encoded properties:

    String getEncodedString(String key, ConfigurationDecoder decoder);

    String getEncodedString(String key);

Both methods operate on string properties. Basically, the string value for the passed in key is retrieved by delegating to getString(). This value is then passed to a ConfigurationDecoder to obtain the plain text value. One of these methods expects the ConfigurationDecoder to be used as argument. The other variant makes use of a decoder associated with this configuration. For this purpose AbstractConfiguration offers a property named configurationDecoder. Making use of this property simplifies access to encoded properties: When the central configuration object is created the decoder is initialized. Other parts of the application do not need any knowledge about the decoding algorithm to be applied; rather, it is sufficient to call the simple variant of getEncodedString() to obtain a property value which can be processed immediately.