I'm still working on testing the output from development versions of the LCFG server by taking a known input and generating XML profiles which can be compared against known "good" output. The test suite seemed to be mostly running fairly well on my test server, telford, but I've been seeing a few oddities related to the last-modified-file which I couldn't explain. This afternoon I decided to try running an experimental version of the server on my desktop instead and I couldn't get it to do anything sensible at all for ages. Eventually I tracked this down to a couple of symlinks in the input "releases" directory for the develop and default releases which were absolute paths. This worked on telford as that is acting as a full test LCFG slave server and has all the data directories installed but, of course, didn't on my desktop. Changing those links to relative links now gives nice predictable output. I've updated my input data collection scripts so it won't happen again. I think there's a lesson here about being careful over where tests are run to be sure they are really doing something useful.
Recently I have been having lots of "fun" working out how to compare LCFG XML profiles generated by different LCFG servers to see if they are functionally equivalent. As a first step I removed the contents of all the nodes which are obviously server-dependent, these are: published_at, published_by and server_version. This really is only the beginning of the job though, nearly every component and package node has a derivation attribute which holds a list of paths that are dependent on the paths to the server input directories. I came up with a cunning scheme to reduce these paths to the shortest form, this takes in all the lists of directories involved, sorts them by depth so that the most specific is removed and converts them into regular expressions to handle the release and host name format strings.
After this I really thought I had cracked the problem but this turned up some issues with the LCFG server which lead to making a code change. It turned out that the LCFG servers generated lists of nodes for spanning maps in an unpredictable order which varied between hosts. This doesn't really bother the clients using the data but it doesn't fit well alongside normal LCFG taglists where the order is considered important and is intended to be maintained. The result is that we now sort the taglists generated from spanning maps when they are added to a subscribing profile.
The first stage of the LCFG server refactoring project is to develop a test suite to ensure we don't introduce any bugs or changes to the overrall behaviour. This will be based around comparing the generated output from the new and old server code given the same input and configuration data. I've put together a wiki page which collects the various ideas and thoughts.