Running Benchmark tests for Performance testing in Hadoop

Benchmark tests are widely used for performance testing in Hadoop Cluster. Benchmark application tests are already written and readily available to use from hadoop-mapreduce-client-jobclient-<hadoop-distribution-tests.jar> file in Hadoop distribution.

You just need to run and test the performance of your cluster using various tests from this jar file. Let’s discuss a few common benchmark tests. One of the benchmark tests to check the I/O performance of your cluster is TestDFSIO. You can run it for reading or writing operation.

The command to run the write test using TestDFSIO for 10 files each of size 100 MB is:
hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar TestDFSIO -write -nrFiles 10 -fileSize 100
You can see the command execution in screenshot below:

You will get the final results of this commands like this:

Similarly, you can run various tests in Hadoop cluster. Another Benchmark test that you can run in your Hadoop cluster is: mrbench. This test creates many small jobs and tests their working in your cluster.

To run the mrbench test, use the given command.
hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar mrbench

The results of this benchmark test will be shown like this:

In the series of hadoop benchmark application test, another test is: testbigmapoutput. It creates and test A map/reduce program that works on a very big non-splittable file and does identity map/reduce on that.

The command to execute testbigmapoutput in your Hadoop cluster is:
hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar testbigmapoutput -input /inputbigmap -output /outputbigmap -create 100
You will get the result parameters of your cluster as an performance outcome of this test as shown:

Close Menu