I spent some time with a customer this week that’s using Terracotta and the Master-Worker pattern to build a scalable workflow processing platform. Very cool stuff.

The system consists of a set of application nodes (each a separate JVM running on a set of servers). The nodes are clustered with Terracotta and have two shared queues (which are “roots” in Terracotta terms). Incoming events arrive on the Message Queue and will be picked up on one of the application nodes. The message is handed to a factory that knows how to build a work plan for that event message. The plans can be either serial or parallel. The work items are then placed onto a Work Queue to be executed. Workers on the application nodes pick the work items off the queue and execute them as necessary.

The events in question are coming from the front half of the application (running on Mule) which is receiving events from RFID, GUIs, and automated systems in a variety of formats, providing some data conversion and translation and ultimately dropping the message onto the workflow grid.

The customer has a specific app in mind that will run on this system but they are also hoping to use it as a general platform for other applications that have similar patterns. It seems to me that with the rising prominence of BPM right now that there has to be a reasonably large set of applications that could benefit from this general model.

Even cooler, this customer is considering open-sourcing the application and making it available for general use. I think this would be fantastic. They are not sure whether there is enough demand out there for this kind of system for it to be worth their while. If you’d be interested in such a system, please drop a note in the comments.