title: Input split for Map function in Hadoop
body: <p>This is my first implementation in Hadoop. I am trying to implement my algorithm for probabilistic dataset in Map Reduce. In my dataset, last column will have some id(number of unique id's in the dataset is equal to the number of nodes in my cluster). I have to divide my dataset based on this column value and each set of records should be processed by each nodes in my cluster. </p>
  
  <p>For example, if i have three nodes in my cluster, for the below dataset, one node should process all the records with id=1, another one with id=2, another one with id=3</p>
  
  <pre><code>name time  dept  id
  --------------------
   b1  2:00pm z1   1
   b2  3:00pm z2   2
   c1  4:00pm y2   1
   b3  3:00pm z3   3
   c4  4:00pm x2   2
  </code></pre>
  
  <p>My map function should take each split as an input and process it in parallel in each node.</p>
  
  <p>I am just trying to understand, which approach is possible to do in Hadoop. Either to input this dataset as a input for my map function and pass an additional argument with map to split the data based on id value. 
  Or split the data beforehand to "n"(number of nodes) subsets and load it in to the nodes, if this is the correct approach, how it is possible to split the data based on value and load in different nodes. Because, what i understood from my readings is that hadoop split the data in to blocks based on the specified size. How can we specify a particular condition while loading. Just to add up, I am writing my program in python.</p>
  
  <p>Someone please advise. Thanks</p>