title: Why does real world MapReduce jobs tend to have very small dataset sizes?
body: <p>All the papers I have read suggest real world mapreduce jobs tend to operate on relatiely small data set sizes (mostly map only, tend to operate on KB-16GB for vast majority of jobs). If anyone working in production world could talk about how and why smaller data set tends to be the case, I would understand better. For small dataset (&lt;128MB), are the files tend to be fragmented or contigous because it has some implication on the splits and number of map tasks spawned ? And if hadoop lets mapreduce to operate only on a section of file ?</p>
  
  <p>Any pointers is much appreciated.</p>