title: Is Hadoop a good solution for small data?
body: <p>We have a set of data in MongoDB that we are map reducing (twice) we're going to be using Mongo's map reduce for now, but I'm thinking about how to scale and improve performance in the future and am thinking about Hadoop.</p>
  
  <p>Most of the stuff I'm reading about Hadoop talks about big data, terabytes of the stuff, when we're going to be dealing with megabytes, 10s maybe 100s of thousands of records. (There may be many of these running concurrently though so whilst a single task is small total could be large).</p>
  
  <p>We really want to get insane performance out of small data rather than make it possible to do big data. i.e. get map reduce results that take 10s of seconds in MongoDB to take seconds or sub second in Hadoop. </p>
  
  <p>Is this possible?<br>
  Is Hadoop a good fit for this?<br>
  If not what other technologies are there that will make this possible?</p>
  
  <p>Details of exact problem this is needed for and my solution to date, can be found in this question: <a href="https://stackoverflow.com/questions/18793035/linear-funnel-from-a-collection-of-events-with-mongodb-aggregation-is-it-possib">Linear funnel from a collection of events with MongoDB aggregation, is it possible?</a></p>