Nice article on Wired.com that revolves around the open source Apache incubator project Mesos.
Mesos is considered to be cluster management software. It is designed to take your entire data center and virtualize the resources for applications running under Mesos.
You don't have to set up a dedicated cluster of machines for a single purpose like running Hadoop. Mesos allows you to set up multiple Hadoop cluster instances over the same hardware set to make more effecient use of resources.
Google's system is referred to as Borg, which is fitting. It is being upgraded soon to Omega. The underlying goal is the same, to effeciently use computing resources for all of the required tasks.
Mesos is being supported by a number of engineers and companies including Twitter and some former Google engineers who worked on Borg.
- Craig
