title: Hadoop Benchmark: TestDFSIO
body: <p>I am testing my hadoop configuration with the apache provided benchmark file TestDFSIO. I'm running it according to this tutorial (resource 1):</p>
  
  <p><a href="http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/#testdfsio" rel="nofollow">http://www.michael-noll.com/blog/2011/04/09/benchmarking-and-stress-testing-an-hadoop-cluster-with-terasort-testdfsio-nnbench-mrbench/#testdfsio</a></p>
  
  <p>The usage of the test is as follows: </p>
  
  <pre><code>TestDFSIO.0.0.4
  Usage: hadoop jar $HADOOP_HOME/hadoop-*test*.jar TestDFSIO 
             -read | -write | -clean 
             [-nrFiles N] [-fileSize MB]
             [-resFile resultFileName] [-bufferSize Bytes]
  </code></pre>
  
  <p>I'm a little confused about some of the flags, specifically, <strong>what is the buffer size flag for?</strong> Also, while navigating hdfs when the job completed successfully (I first performed a write TestDFSIO), I couldn't find the filename I supposedly created by choosing a resultFileName. <strong>Why can't I find the file by the resultFileName I used?</strong></p>
  
  <p>I had also looked at this page (resource 2) (specifically <strong>page 25</strong>):</p>
  
  <p><a href="http://wr.informatik.uni-hamburg.de/_media/research/labs/2009/2009-12-tien_duc_dinh-evaluierung_von_hadoop-report.pdf" rel="nofollow">http://wr.informatik.uni-hamburg.de/_media/research/labs/2009/2009-12-tien_duc_dinh-evaluierung_von_hadoop-report.pdf</a></p>
  
  <p>As one of the parameters of their test, they were using block sizes of 64MB and 128MB. I tried putting '64MB' (converted to bytes) after the bufferSize flag, but this led to a failed job, which leads me to believe I do not understand what the buffersize flag is for, and how to use different block sizes for testing. <strong>How do you change the block size of the test (as per resource 2)?</strong> </p>