title: Multiple inputs into MapReduce job
body: <p>I'm trying to write a MapReduce job which takes a number of delimited input sources. All sources contain the same information, but it may be in different columns and the separator may be different per source. The sources are parsed in the mapper by a configuration file. This configuration file allows users to confine these different separators and column mappings.</p>
  
  <p>For example, input1 is parsed using configuration properties</p>
  
  <pre><code>input1.separator=,
  input1.id=1
  input1.housename=2
  input1.age=15
  </code></pre>
  
  <p>where 1, 2 and 15 are the columns in input1 which relate to those properties.</p>
  
  <p>So, the mapper needs to know which configuration properties to use for each input source. I can't hard code this as other people will be running my job and will want to add new inputs without requiring a compiler.</p>
  
  <p>The obvious solution is to extract the file name from the splits and apply configuration that way.</p>
  
  <p>For example, assume I'm inputting two files, "source1.txt" and "source2.txt". I could write my configuration like</p>
  
  <pre><code>source1.separator=,
  source1.id=2
  ...
  source2.separator=|
  source2.id=4
  ...
  </code></pre>
  
  <p>The mapper would get the file name from the splits, and then read the configuration properties with the same prefix.</p>
  
  <p>However, if I'm pointing to folders in a Hive warehouse, I can't use this. I could extract bits of the path and use those, but I don't really feel that's an elegant or sturdy solution. Is there an easier way to do this?</p>