Tuesday, February 5, 2013

Testing MapReduce with MRUnit By Mansoor Ashraf


Here is the home page for MRUnit. http://mrunit.apache.org/

I ran across this blog today and thought it was really worthwhile.I don't know how many of you are into writing MR jobs, but having MRUnit is an invaluable piece of kit to have available.

Many MR jobs are very time consuming and even running small jobs can be problematic in terms of time spinning up a job and reviewing the results, but there may not be resources available to run your job and test the results. On top of that, there can be additional expenses involved if you're running something like Elastic Map Reduce on AWS.

MRUnit is also a good learning tool, giving you a change try try out MR features or test new algorithms in a quick and safe environment. Unit tests in general allow other people to come up to speed on how your code functions quickly, provides usage documentation, and some protection against your code being broken.

Go ahead and give it at try.

  - Craig