Commons Configuration – Combined Configurations

Combined Configuration

The CombinedConfiguration class provides an alternative for handling multiple configuration sources. Its API is very similar to the CompositeConfiguration class, which was discussed in the previous section. There are the following differences however:

A CombinedConfiguration is a truly hierarchical configuration. This means that all the enhanced facilities provided by HierarchicalConfiguration (e.g. expression engines) can be used.
A CombinedConfiguration is not limited to implementing an override semantics for the properties of the contained configurations. Instead it has the concept of so-called node combiners, which know how properties of multiple configuration sources can be combined. Node combiners are discussed later in detail. For instance, there is a node combiner implementation available that constructs a union of the contained configurations.
Contained configurations can be assigned a name. They can later be accessed by their name.
Each contained configuration can have an optional prefix. Its properties are then added under this prefix to the combined configuration.
There is no concept of an in memory configuration.

How it works

A CombinedConfiguration provides a logic view on the properties of the configurations it contains. This view is determined by the associated node combiner object. Because of that it must be re-constructed whenever one of these contained configurations is changed.

To achieve this, a CombinedConfiguration object registers itself as an event listener at the configurations that are added to it. It will then be notified for every modification that occurs. If such a notification is received, the internally managed view is invalidated. When a property of the combined configuration is to be accessed, the view is checked whether it is valid. If this is the case, the property's value can be directly fetched. Otherwise the associated node combiner is asked to re-construct the view.

Node combiners

A node combiner is an object of a class that inherits from the abstract NodeCombiner class. This class defines an abstract combine() method, which takes the root nodes of two hierarchical configurations and returns the root node of the combined node structure. It is up to a concrete implementation how this combined structure will look like. Commons Configuration ships with three concrete implementations OverrideCombiner, MergeCombiner and UnionCombiner, which implement an override, merge, and union semantics respectively.

Constructing a combination of multiple node hierarchies is not a trivial task. The available implementations descend the passed in node hierarchies in a recursive manner to decide, which nodes have to be copied into the resulting structure. Under certain circumstances two nodes of the source structures can be combined into a single result node, but unfortunately this process cannot be fully automated, but sometimes needs some hints from the developer. As an example consider the following XML configuration sources:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

and

<configuration>
  <database>
    <tables>
      <table>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

These two configuration sources define database tables. Each source defines one table. When constructing a union for these sources the result should look as follows:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
      </table>
      <table>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

As you can see, the resulting structure contains two table nodes while the nodes database and tables appear only once. For a human being this is quite logic because database and tables define the overall structure of the configuration data, and there can be multiple tables. A node combiner however does not know anything about structure nodes, list nodes, or whatever. From its point of view there is no detectable difference between the tables nodes and the table nodes in the source structures: both appear once in each source file and have no values. So without any assistance the result constructed by the UnionCombiner when applied on the two example sources would be a bit different:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

Note that the table node would be considered a structure node, too, and would not be duplicated. This is probably not what was desired. To deal with such situations it is possible to tell the node combiner that certain nodes are list nodes and thus should not be combined. So in this concrete example the table node should be declared as a list node, then we would get the expected result. We will see below how this is done. Note that this explicit declaration of a list node is necessary only in situations where there is ambiguity. If in one of our example configuration sources multiple tables had been defined, the node combiner would have concluded itself that table is a list node and would have acted correspondingly.

The examples that follow are provided to further illustrate the differences between the combiners that are delivered with Commons Configuration. The first two files are the files that will be combined.

testfile1.xml

testfile2.xml

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default="2">1</level>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user>Admin</user>
          <passwd type="secret"/>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id="1">
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id="1" type="half">
      <Name>My Channel</Name>
    </Channel>
    <Channel id="2">
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id="3" type="half">
      <Name>Test Channel</Name>
    </Channel>
    <Channel id="4">
      <Name>Channel 4</Name>
    </Channel>
  </Channels>
</config>

<config>
  <base>
    <services>
      <security>
        <login>
          <user type="default">scotty</user>
          <passwd>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <gui>
    <bgcolor>black</bgcolor>
    <fgcolor>blue</fgcolor>
    <level min="1">4</level>
  </gui>
  <net>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
    <service>
      <url type="2">http://service2.org</url>
      <url type="2">http://service3.org</url>
    </service>
  </net>
  <database>
    <tables>
      <table id="2">
        <name>tasks</name>
        <fields>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id="1">
      <Name>Channel 1</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id="2" type="full">
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id="3" type="full">
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
    <Channel id="4" type="half">
      <Name>Test Channel 1</Name>
    </Channel>
    <Channel id="4" type="full">
      <Name>Test Channel 2</Name>
    </Channel>
  </Channels>
</config>

The first listing shows the result of using the OverrideCombiner.

OverrideCombiner Results Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2' min='1'>1</level>
    <fgcolor>blue</fgcolor>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user type='default'>Admin</user>
          <passwd type='secret'>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
    </Channel>
    <Channel id='2'>
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
  </Channels>
</config>

The features that are significant in this file are:

In the gui section each of the child elements only appears once. The level element merges the attributes from the two files and uses the element value of the first file.
In the security section the user type attribute was obtained from the second file while the user value came from the first file. Alternately, the password type was obtained from the first file while the value came from the second.
Only the data from table 1 was included.
Channel 1 in the first file completely overrode Channel 1 in the second file.
Channel 2 in the first file completely overrode Channel 2 in the second file. While the attributes were merged in the case of the login elements the type attribute was not merged in this case.
Again, only Channel 3 from the first file was included.

How the Channel elements ended up may not at first be obvious. The OverrideCombiner simply noticed that the Channels element had three child elements named Channel and used that to determine that only the contents of the Channels element in the first file would be used.

The next file is the the result of using the UnionCombiner

UnionCombiner Results

Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2'>1</level>
    <bgcolor>black</bgcolor>
    <fgcolor>blue</fgcolor>
    <level min='1'>4</level>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
      <url type='2'>http://service2.org</url>
      <url type='2'>http://service3.org</url>
    </service>
    <server></server>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user>Admin</user>
          <passwd type='secret'></passwd>
          <user type='default'>scotty</user>
          <passwd>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1' id='2'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
        <name>tasks</name>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
    </Channel>
    <Channel id='2'>
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
    <Channel id='1'>
      <Name>Channel 1</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id='2' type='full'>
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id='3' type='full'>
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
  </Channels>
</config>

The feature that is significant in this file is rather obvious. It is just a simple union of the contents of the two files.

Finally, the last file is the result of using the MergeCombiner

MergeCombiner Results

Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2' min='1'>1</level>
    <fgcolor>blue</fgcolor>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user type='default'>Admin</user>
          <passwd type='secret'></passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
      <table id='2'>
        <name>tasks</name>
        <fields>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id='2' type='full'>
      <MoreChannelData>more test 2 data</MoreChannelData>
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
    <Channel id='3' type='full'>
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
  </Channels>
</config>

The features that are significant in this file are:

In the gui section the elements were merged.
In the net section the elements were merged, with the exception of the urls.
In the security section the user and password were merged. Notice that the empty value for the password from the first file overrode the password in the second file.
Both table elements appear
Channel 1 and Channel 2 were merged
Both Channel 3 elements appear as they were determined to not be the same.

When merging elements attributes play a critical role. If an element has an attribute that appears in both sources, the value of that attribute must be the same for the elements to be merged.

Merging is only allowed between a single node in each of the files, so if an element in the first file matches more than one element in the second file no merging will take place and the element from the first file (and its contents) are included and the elements in the second file are not. If the element is marked as a list node then the elements from the second file will also be included.

Constructing a CombinedConfiguration

To create a CombinedConfiguration object you specify the node combiner to use and then add an arbitrary number of configurations. We will show how to construct a union configuration from the two example sources introduced earlier:

// Load the source configurations
Parameters params = new Parameters();
FileBasedConfigurationBuilder<XMLConfiguration> builder1 =
    new FileBasedConfigurationBuilder<XMLConfiguration>(XMLConfiguration.class)
    .configure(params.xml()
        .setFileName("table1.xml"));
FileBasedConfigurationBuilder<XMLConfiguration> builder2 =
    new FileBasedConfigurationBuilder<XMLConfiguration>(XMLConfiguration.class)
    .configure(params.xml()
        .setFileName("table2.xml"));

// Create and initialize the node combiner
NodeCombiner combiner = new UnionCombiner();
combiner.addListNode("table");  // mark table as list node
            // this is needed only if there are ambiguities

// Construct the combined configuration
CombinedConfiguration cc = new CombinedConfiguration(combiner);
cc.addConfiguration(builder1.getConfiguration(), "tab1");
cc.addConfiguration(builder2.getConfiguration());

Here we also specified a name for one of the configurations, so it can later be accessed by cc.getConfiguration("tab1");. Access by index is also supported. After that the properties in the combined configuration can be accessed as if it were a normal hierarchical configuration.

Dealing with changes

There is nothing that prevents you from updating a combined configuration, e.g. by calling methods like addProperty() or clearProperty(). However, this will not have the expected effect!

Remember that a CombinedConfiguration is just a view over a set of other configurations processed by a NodeCombiner. The combiner sets up a nodes structure consisting of ImmutableNode objects. Some of these nodes are likely to be shared with the child configurations. Because the nodes are immutable updates on the combined configuration cause nodes to be replaced in the hierarchy, but this does not affect any of the child configurations. When now one of the child configurations is changed, the combined configuration is re-constructed, and all the changes made before on it are lost! With other words, changes on a combined configuration are only temporary.

The recommended approach is to treat a combined configuration as immutable and to perform updates on selected child configurations only. It is in the responsibility of an application anyway to decide which child configuration is affected by a change; there is no easy way to determine the target configuration of a change automatically.

If an editable combined configuration is really needed, a possible solution is to create a CombinedConfiguration as usual, and then copy it into another hierarchical configuration. Hierarchical configuration classes typically have constructors that copy the content of another configuration. The following example shows how a combined configuration is copied into a XMLConfiguration; it is then even possible to save the content of the original configuration as an XML document:

// Set up the combined configuration, e.g. like in the example before
CombinedConfiguration cc = ...;

// Create an XMLConfiguration with the content of the combined configuration
XMLConfiguration config = new XMLConfiguration(cc);

In this scenario, the CombinedConfiguration object is used only temporarily to apply the node combiner to the data contained in the child configurations. The resulting nodes structure is then passed to the XMLConfiguration. This is now a full-blown, editable configuration. However, the connection to the child configurations does no longer exist.