 <blockquote> <p>From what I can tell, there doesn't seem to be a way using streaming to send certain records in output file A and the rest of the records in output file B.</p> </blockquote> <p>By using a custom <a href="http://hadoop.apache.org/mapreduce/docs/r0.21.0/mapred_tutorial.html#Partitioner" rel="nofollow">Partitioner</a>, you can specify which key goes to which reducer. By default the <a href="http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/HashPartitioner.html" rel="nofollow">HashPartitioner</a> is used. Looks like the only other Partitioner Streaming supports is <a href="http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/lib/KeyFieldBasedPartitioner.html" rel="nofollow">KeyFieldBasedPartitioner</a>.</p> <p>You can find more details about the KeyFieldBasedPartitioner in the context of Streaming <a href="http://hadoop.apache.org/common/docs/current/streaming.html#Hadoop+Partitioner+Class" rel="nofollow">here</a>. You need not know Java to configure the KeyFieldBasedPartitioner with Streaming.</p> <blockquote> <p>Is there a way to do a map/reduce job to re-split every log files correctly?</p> </blockquote> <p>You should be able to write a MR job to re-split the files, but I think Partitioner should solve the problem.</p>