 <p>If you want a really real-time(around 200ms) response for your search application, for both the first time search query response and further suggested search response, Hadoop is not a good choice, not even Hive, HBase, or even Impala (or Apache Drill, Google Dremel like system). </p> <p>Hadoop is a batch processing system which is good for write once, read multiple times use cases. And the strength of it is good at scalability and I/O throuput. The trend I saw is that many organizations are using that as a data warehouse for offline data mining and BI analysis purpose as a replacement for data warehosue based on relational databases.</p> <p>Hive and HBase all provides extra features on top of Hadoop, but neither of those could possibly reach 200ms real time for average complex query workload.</p> <p>Check the high level view of how 'real-time' each possible solution can really reach, on <a href="http://incubator.apache.org/drill/" rel="nofollow">Apache Drill</a> homepage. CloudEra Impala, or Apache Drill, which borrows the idea from Google Dremel, have the intention to enhance the query speed on top of Hadoop by doing query optimization, column based storage, massive parallelism of I/O. I believe these 2 are still in early stage to achieve the goals they claim. Some <a href="http://architects.dzone.com/articles/meet-impala-open-source-real" rel="nofollow">initial performance benchmarking result of Impala</a> I found.</p> <p>If you decide to go with Hadoop or related solution stack, there are possible ways to load data from MySQL to Hadoop using <a href="http://sqoop.apache.org/" rel="nofollow">Sqoop</a> or other customized data load applications leveraging Hadoop Distributed File System API. But if you will have new data coming into MySQL time to time, then you need to schedule a job to run periodically to do delta load from MySQL to Hadoop. </p> <p>On the other hand, the workload to build a Hadoop cluster and find or build suitable data loading tool from MySQL to Hadoop might be a huge workload. Also you need to find a suitable extra layer for runtime data access and build code around that, no mater Impala or other things. To solve your own problem, it's probably better to build your own customized solution, like using in memory cache for hot records with the meta data in your database, along with some index mechanism to quickly locate the data you need for suggested search query calculation. If the memory on one machine cannot hold enough records, a memory cache grid or cluster component comes in handy like <a href="http://memcached.org/" rel="nofollow">Memcached</a> or Reddis, <a href="http://ehcache.org/" rel="nofollow">EhCache</a>, etc. </p>