DataSmasher is a distributed, multi-threaded, job execution framework. Jobs are a generic concept, they can compile code, crunch data, search data and the list goes on... DataSmasher exists because I wanted to learn more about distributed and multi-threaded software development and doesn't have a big plan (yet), but I want to keep it alive for now (it's quite fun).

I hope you like it. Any feedback, requests, critiques or general “Hey, how's it going?” comments are welcomed±.

Have a look at the Wiki pages for more information, documentation and general ideas (which I will add soon!)

± This isn't a complete list of accepted comments ;)

Coming soon...

• Simple to use, create a job, subscribe to its events and Enqueue in the JobManager
• Jobs are completely generic
• Modular architecture, should be very very simple to extend the distribution model±, job management or anything else
• Web interface

• Just a few days worth of work, very very early stages and basic!
• Too generic, needs specific implementations of distributed models to be useful

± Just extend the RemoteJobConsumer class

• Blog post about the tech side!
• Add Tests! (Yep, should have done first)
• Fix node addresses, at the moment always returns 127.0.0.1 Next task!
• Make the server suck less! (many aspects of it are hardcoded) Next task!
• Error handling, especially sudden disconnections, definitely has to be improved
• Select serialisation or marshalling of Jobs depending on the remote node platform
• C++ and maybe Java (Android) clients
• Limit the local number of jobs depending on the current platform (LocalJobConsumer is independent of this, just give it a max number of threads)
• RemoteJobConumers will have to accept more than 1 connection and 1 job
• Make nodes independent of WebService to discover other nodes in the cluster (NodeGraph)
• Create Job dependency manager