Commons Configuration – Combined Configurations

Combined Configuration

The CombinedConfiguration class provides an alternative for handling multiple configuration sources. Its API is very similar to the CompositeConfiguration class, which was discussed in the last section. There are the following differences however:

A CombinedConfiguration is a truly hierarchical configuration. This means that all the enhanced facilities provided by HierarchicalConfiguration (e.g. expression engines) can be used.
A CombinedConfiguration is not limited to implementing an override semantics for the properties of the contained configurations. Instead it has the concept of so-called node combiners, which know how properties of multiple configuration sources can be combined. Node combiners are discussed later in detail. For instance, there is a node combiner implementation available that constructs a union of the contained configurations.
Contained configurations can be assigned a name. They can later be accessed by their name.
Each contained configuration can have an optional prefix. Its properties are then added under this prefix to the combined configuration.
There is no concept of an in memory configuration. Changes to a combined configuration are handled in a different way.

How it works

A CombinedConfiguration provides a logic view on the properties of the configurations it contains. This view is determined by the associated node combiner object. Because of that it must be re-constructed whenever one of these contained configurations is changed.

To achieve this, a CombinedConfiguration object registers itself as an event listener at the configurations that are added to it. It will then be notified for every modification that occurs. If such a notification is received, the internally managed view is invalidated. When a property of the combined configuration is to be accessed, the view is checked whether it is valid. If this is the case, the property's value can be directly fetched. Otherwise the associated node combiner is asked to re-construct the view.

Node combiners

A node combiner is an object of a class that inherits from the abstract NodeCombiner class. This class defines an abstract combine() method, which takes the root nodes of two hierarchical configurations and returns the root node of the combined node structure. It is up to a concrete implementation how this combined structure will look like. Commons Configuration ships with three concrete implementations OverrideCombiner, MergeCombiner and UnionCombiner, which implement an override, merge, and union semantics respectively.

Constructing a combination of multiple node hierarchies is not a trivial task. The available implementations descend the passed in node hierarchies in a recursive manner to decide, which nodes have to be copied into the resulting structure. Under certain circumstances two nodes of the source structures can be combined into a single result node, but unfortunately this process cannot be fully automated, but sometimes needs some hints from the developer. As an example consider the following XML configuration sources:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

and

<configuration>
  <database>
    <tables>
      <table>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

These two configuration sources define database tables. Each source defines one table. When constructing a union for these sources the result should look as follows:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
      </table>
      <table>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

As you can see, the resulting structure contains two table nodes while the nodes database and tables appear only once. For a human being this is quite logic because database and tables define the overall structure of the configuration data, and there can be multiple tables. A node combiner however does not know anything about structure nodes, list nodes, or whatever. From its point of view there is no detectable difference between the tables nodes and the table nodes in the source structures: both appear once in each source file and have no values. So without any assistance the result constructed by the UnionCombiner when applied on the two example sources would be a bit different:

<configuration>
  <database>
    <tables>
      <table>
        <name>users</name>
        <fields>
          <field>
            <name>user_id</name>
          </field>
          ...
        </fields>
        <name>documents</name>
        <fields>
          <field>
            <name>document_id</name>
          </field>
          ...
        </fields>
      </table>
    </tables>
  </database>
</configuration>

Note that the table node would be considered a structure node, too, and would not be duplicated. This is probably not what was desired. To deal with such situations it is possible to tell the node combiner that certain nodes are list nodes and thus should not be combined. So in this concrete example the table node should be declared as a list node, then we would get the expected result. We will see below how this is done. Note that this explicit declaration of a list node is necessary only in situations where there is ambiguity. If in one of our example configuration sources multiple tables had been defined, the node combiner would have concluded itself that table is a list node and would have acted correspondingly.

The examples the follow are provided to further illustrate the differences between the combiners that are delivered with Commons Configuration. The first two files are the files that will be combined.

testfile1.xml

testfile2.xml

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default="2">1</level>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user>Admin</user>
          <passwd type="secret"/>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id="1">
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id="1" type="half">
      <Name>My Channel</Name>
    </Channel>
    <Channel id="2">
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id="3" type="half">
      <Name>Test Channel</Name>
    </Channel>
    <Channel id="4">
      <Name>Channel 4</Name>
    </Channel>
  </Channels>
</config>

<config>
  <base>
    <services>
      <security>
        <login>
          <user type="default">scotty</user>
          <passwd>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <gui>
    <bgcolor>black</bgcolor>
    <fgcolor>blue</fgcolor>
    <level min="1">4</level>
  </gui>
  <net>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
    <service>
      <url type="2">http://service2.org</url>
      <url type="2">http://service3.org</url>
    </service>
  </net>
  <database>
    <tables>
      <table id="2">
        <name>tasks</name>
        <fields>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id="1">
      <Name>Channel 1</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id="2" type="full">
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id="3" type="full">
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
    <Channel id="4" type="half">
      <Name>Test Channel 1</Name>
    </Channel>
    <Channel id="4" type="full">
      <Name>Test Channel 2</Name>
    </Channel>
  </Channels>
</config>

The first listing shows the result of using the OverrideCombiner.

OverrideCombiner Results Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2' min='1'>1</level>
    <fgcolor>blue</fgcolor>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user type='default'>Admin</user>
          <passwd type='secret'>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
    </Channel>
    <Channel id='2'>
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
  </Channels>
</config>

The features that are significant in this file are:

In the gui section each of the child elements only appears once. The level element merges the attributes from the two files and uses the element value of the first file.
In the security section the user type attribute was obtained from the second file while the user value came from the first file. Alternately, the password type was obtained from the first file while the value came from the second.
Only the data from table 1 was included.
Channel 1 in the first file completely overrode Channel 1 in the second file.
Channel 2 in the first file completely overrode Channel 2 in the second file. While the attributes were merged in the case of the login elements the type attribute was not merged in this case.
Again, only Channel 3 from the first file was included.

How the Channel elements ended up may not at first be obvious. The OverrideCombiner simply noticed that the Channels element had three child elements named Channel and used that to determine that only the contents of the Channels element in the first file would be used.

The next file is the the result of using the UnionCombiner

UnionCombiner Results

Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2'>1</level>
    <bgcolor>black</bgcolor>
    <fgcolor>blue</fgcolor>
    <level min='1'>4</level>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
      <url type='2'>http://service2.org</url>
      <url type='2'>http://service3.org</url>
    </service>
    <server></server>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user>Admin</user>
          <passwd type='secret'></passwd>
          <user type='default'>scotty</user>
          <passwd>BeamMeUp</passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1' id='2'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
        <name>tasks</name>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
    </Channel>
    <Channel id='2'>
      <MoreChannelData>more test 2 data</MoreChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
    <Channel id='1'>
      <Name>Channel 1</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id='2' type='full'>
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id='3' type='full'>
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
  </Channels>
</config>

The feature that is significant in this file is rather obvious. It is just a simple union of the contents of the two files.

Finally, the last file is the result of using the MergeCombiner

MergeCombiner Results

Notes

<config>
  <gui>
    <bgcolor>green</bgcolor>
    <selcolor>yellow</selcolor>
    <level default='2' min='1'>1</level>
    <fgcolor>blue</fgcolor>
  </gui>
  <net>
    <proxy>
      <url>http://www.url1.org</url>
      <url>http://www.url2.org</url>
      <url>http://www.url3.org</url>
    </proxy>
    <service>
      <url>http://service1.org</url>
    </service>
    <server>
      <url>http://appsvr1.com</url>
      <url>http://appsvr2.com</url>
      <url>http://testsvr.com</url>
      <url>http://backupsvr.com</url>
    </server>
  </net>
  <base>
    <services>
      <security>
        <login>
          <user type='default'>Admin</user>
          <passwd type='secret'></passwd>
        </login>
      </security>
    </services>
  </base>
  <database>
    <tables>
      <table id='1'>
        <name>documents</name>
        <fields>
          <field>
            <name>docid</name>
            <type>long</type>
          </field>
          <field>
            <name>docname</name>
            <type>varchar</type>
          </field>
          <field>
            <name>authorID</name>
            <type>int</type>
          </field>
        </fields>
      </table>
      <table id='2'>
        <name>tasks</name>
        <fields>
          <field>
            <name>taskid</name>
            <type>long</type>
          </field>
          <field>
            <name>taskname</name>
            <type>varchar</type>
          </field>
        </fields>
      </table>
    </tables>
  </database>
  <Channels>
    <Channel id='1' type='half'>
      <Name>My Channel</Name>
      <ChannelData>test 1 data</ChannelData>
    </Channel>
    <Channel id='2' type='full'>
      <MoreChannelData>more test 2 data</MoreChannelData>
      <Name>Channel 2</Name>
      <ChannelData>test 2 data</ChannelData>
    </Channel>
    <Channel id='3' type='half'>
      <Name>Test Channel</Name>
    </Channel>
    <Channel id='3' type='full'>
      <Name>Channel 3</Name>
      <ChannelData>test 3 data</ChannelData>
    </Channel>
  </Channels>
</config>

The features that are significant in this file are:

In the gui section the elements were merged.
In the net section the elements were merged, with the exception of the urls.
In the security section the user and password were merged. Notice that the empty value for the password from the first file overrode the password in the second file.
Both table elements appear
Channel 1 and Channel 2 were merged
Both Channel 3 elements appear as they were determined to not be the same.

When merging elements attributes play a critical role. If an element has an attribute that appears in both sources, the value of that attribute must be the same for the elements to be merged.

Merging is only allowed between a single node in each of the files, so if an element in the first file matches more than one element in the second file no merging will take place and the element from the first file (and its contents) are included and the elements in the second file are not. If the element is marked as a list node then the elements from the second file will also be included.

Constructing a CombinedConfiguration

To create a CombinedConfiguration object you specify the node combiner to use and then add an arbitrary number of configurations. We will show how to construct a union configuration from the two example sources introduced earlier:

// Load the source configurations
XMLConfiguration conf1 = new XMLConfiguration("table1.xml");
XMLConfiguration conf2 = new XMLConfiguration("table2.xml");

// Create and initialize the node combiner
NodeCombiner combiner = new UnionCombiner();
combiner.addListNode("table");  // mark table as list node
            // this is needed only if there are ambiguities

// Construct the combined configuration
CombinedConfiguration cc = new CombinedConfiguration(combiner);
cc.addConfiguration(conf1, "tab1");
cc.addConfiguration(conf2);

Here we also specified a name for one of the configurations, so it can later be accessed by cc.getConfiguration("tab1");. Access by index is also supported. After that the properties in the combined configuration can be accessed as if it were a normal hierarchical configuration

Dealing with changes

There is nothing that prevents you from updating a combined configuration, e.g. by calling methods like addProperty() or removeProperty(). The problem is that depending on the used node combiner it might no be clear, which of the contained configurations will be modified or whether one is modified at all.

Typical node combiners work by copying parts of the node structures of the source configurations into the target structure and linking them together using special link nodes. So updates of the combined node structure will either effect nodes from one of the contained configuration (then the changes are directly visible in this configuration) or one of the link nodes (then they cannot really be saved).

It is also possible that a change is done at the combined node structure, which is not compatible with the current node combiner. Imagine that an OverrideCombiner is used and that a property should be removed. This property will then be removed from one of the contained configurations. Now it may happen that this removed property had hidden property values of other contained configurations. Their values won't become visible automatically, but only after the combined view was re-constructed.

Because of that it is recommended that changes are not done at the combined configuration, but only at contained configurations. This way the correct configuration to be updated can unambiguously be identified. Obtaining the configuration to be updated from the combined configuration is easy when it was given a name.