title: MapReduce Output ArrayWritable
body: <p>I'm trying to get an output from an ArrayWritable in a simple MapReduce-Task. I found a few questions with a similar problem, but I can't solve the problem in my own code. So I'm looking forward to your help. Thanks :)!</p>
  
  <p><strong>Input:</strong> Textfile with some sentence.</p>
  
  <p><strong>Output</strong> should be:</p>
  
  <pre><code>&lt;Word, &lt;length, number of same words in Textfile&gt;&gt;
   Example: Hello  5  2 
  </code></pre>
  
  <p>The output that I get in my Job is:</p>
  
  <pre><code>hello WordLength_V01$IntArrayWritable@221cf05
  test WordLength_V01$IntArrayWritable@799e525a
  </code></pre>
  
  <p>I think the problem is in the subclass from IntArrayWritable, but I don't get the right correction to fix this. By the way we have Hadoop 2.5 and I use the following code to get this result:</p>
  
  <p><strong>Main Method:</strong></p>
  
  <pre><code>public static void main(String[] args) throws Exception {
      Configuration conf = new Configuration();
      Job job = Job.getInstance(conf, "word length V1");
  
      // Set Classes
      job.setJarByClass(WordLength_V01.class);
      job.setMapperClass(MyMapper.class);
      // job.setCombinerClass(MyReducer.class);
      job.setReducerClass(MyReducer.class);
  
      // Set Output and Input Parameters
      job.setMapOutputKeyClass(Text.class);
      job.setMapOutputValueClass(IntWritable.class);
  
      job.setOutputKeyClass(Text.class);
      job.setOutputValueClass(IntArrayWritable.class);
  
      // Number of Reducers
      job.setNumReduceTasks(1);
  
      // Set FileDestination
      FileInputFormat.addInputPath(job, new Path(args[0]));
      FileOutputFormat.setOutputPath(job, new Path(args[1]));
  
      System.exit(job.waitForCompletion(true) ? 0 : 1);
  }
  </code></pre>
  
  <p><strong>Mapper:</strong></p>
  
  <pre><code>public static class MyMapper extends Mapper&lt;Object, Text, Text, IntWritable&gt; {
  
      // Initialize Variables
      private final static IntWritable one = new IntWritable(1);
      private Text word = new Text();
  
      // Map Method
      public void map(Object key, Text value, Context context) throws IOException, InterruptedException {
  
          // Use Tokenizer
          StringTokenizer itr = new StringTokenizer(value.toString());
  
          // Select each word
          while (itr.hasMoreTokens()) {
              word.set(itr.nextToken());
  
              // Output Pair
              context.write(word, one);
          }
      }
  }
  </code></pre>
  
  <p><strong>Reducer:</strong></p>
  
  <pre><code>public static class MyReducer extends Reducer&lt;Text, IntWritable, Text, IntArrayWritable&gt; {
  
      // Initialize Variables
      private IntWritable count = new IntWritable();
      private IntWritable length = new IntWritable();
  
      // Reduce Method
      public void reduce(Text key, Iterable&lt;IntWritable&gt; values, Context context) throws IOException, InterruptedException {
  
          // Count Words
          int sum = 0;
          for (IntWritable val : values) {
              sum += val.get();
          }
  
          count.set(sum);
  
          // Wordlength
          length.set(key.getLength());
  
          // Define Output
          IntWritable[] temp = new IntWritable[2];
          IntArrayWritable output = new IntArrayWritable(temp);
  
          temp[0] = count;
          temp[1] = length;
  
          // Output
          output.set(temp);
          context.write(key, new IntArrayWritable(output.get()));
      }
  }
  </code></pre>
  
  <p><strong>SubClass</strong></p>
  
  <pre><code>public static class IntArrayWritable extends ArrayWritable {
      public IntArrayWritable(IntWritable[] intWritables) {
          super(IntWritable.class);
      }
  
      @Override
      public IntWritable[] get() {
          return (IntWritable[]) super.get();
      }
  
      @Override
      public void write(DataOutput arg0) throws IOException {
          for(IntWritable data : get()){
              data.write(arg0);
          }
      }
  }   
  </code></pre>
  
  <p>I used the following links to find a solution:</p>
  
  <ul>
  <li><a href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/Writable.html" rel="nofollow noreferrer">Interface Writable (hadoop.apache.org)</a></li>
  <li><a href="https://hadoop.apache.org/docs/current/api/org/apache/hadoop/io/ArrayWritable.html" rel="nofollow noreferrer">Class ArrayWritable (hadoop.apache.org)</a></li>
  <li><a href="https://stackoverflow.com/questions/7919035/serialization-using-arraywritable-seems-to-work-in-a-funny-way">stackoverflow.com (1)</a></li>
  <li><a href="https://stackoverflow.com/questions/15810550/output-a-list-from-a-hadoop-map-reduce-job-using-custom-writable">stackoverflow.com (2)</a></li>
  </ul>
  
  <p>I'm really thankful for any idea!</p>
  
  <p><strong>-------- Solution --------</strong></p>
  
  <p>New SubClass:</p>
  
  <pre><code>public static class IntArrayWritable extends ArrayWritable {
  
      public IntArrayWritable(IntWritable[] values) {
          super(IntWritable.class, values);
      }
  
      @Override
      public IntWritable[] get() {
          return (IntWritable[]) super.get();
      }
  
      @Override
      public String toString() {
          IntWritable[] values = get();
          return values[0].toString() + ", " + values[1].toString();
      }
  }
  </code></pre>
  
  <p>New Reduce Method:</p>
  
  <pre><code>public void reduce(Text key, Iterable&lt;IntWritable&gt; values,
              Context context) throws IOException, InterruptedException {
  
          // Count Words
          int sum = 0;
          for (IntWritable val : values) {
              sum += val.get();
          }
  
          count.set(sum);
  
          // Wordlength
          length.set(key.getLength());
  
          // Define Output
          IntWritable[] temp = new IntWritable[2];
          temp[0] = count;
          temp[1] = length;
  
          context.write(key, new IntArrayWritable(temp));
  }
  </code></pre>