The Demonstration Aggregator shows how its possible to deploy an aggregator setup that pulls in courses from several providers, and offers a user interface with basic features such as searching, browsing, bookmarking, tags and so on. It also demonstrates some value-added aspects such as geocoding the course venues and displaying them on Google Maps.
The aggregator has two major components: the aggregator daemon, and the aggregator website. As you can see in the diagram below (click for a larger picture):

The daemon is a threaded Java application launched using a scheduler (cron) that reads the target schedule to see which providers to poll, and then launches harvesters to pull in their data. The data is combined, addresses are geocoded, and the data written into a MySQL database. The implementation uses Apache XMLBeans, but otherwise has minimal dependencies and is very lightweight.
At the other end, we have a Ruby on Rails web application that operates on the same database, mainly in a read-only capacity, but also we have the interface here for adding more targets to the schedule, bookmarking and other user-specific actions. The web application’s main function is to allow providers to see how their feeds look for users, how searchable they are, how their category tags look alongside those of other providers and so on - basically, human-usability validation of the feed rather than technical validation.
Implementation-wise, the web application is deployed on Mongrel, provides search features via Ferret (the Ruby version of Lucene) and uses a lot of Ajax via Prototype and other standard libraries. As with the rest of the XCRI sites, we use OpenID for login.
The current setup is managed using Capistrano automated deployment, and is placed on a proxy setup behind Apache, which serves all static files. As demand grows we can use the load balancing proxy and mongrel_cluster to increase the number of Mongrel servers and keep things nice and snappy.
What we don’t have yet, and would be nice, is an external geocoding web service. UK postcode-geocode maps are all copyrighted (!) and so we can’t just use Yahoo or Google. Apparently UKOLN and EDINA are on the case…
You can see the aggregator web in action and browse the source code.
If you come across any issues, report them on the forum.