 <p>IMHO, Hadoop all by itself won't be able to solve your problem. First of all Hadoop(HDFS to be precise) is a FS and doesn't provide columnar storage wherein you can query for a particular field. Data inside HDFS is stored as flat files and you have to traverse across the data in order to reach upto the point where the data of interest resides.</p> <p>Having said that, there are some workarounds, like making use of <strong><a href="http://hive.apache.org" rel="nofollow">Hive</a></strong>. Hive is another member of the Hadoop family which provides warehousing capability on top of your existing Hadoop cluster. It allows us to map HDFS files as Hive tables which can be queried conveniently. Also, it provides a SQL like interface to query these tables. But, Hive is not a good fit if you have real time needs.</p> <p>I feel something like <strong><a href="https://github.com/cloudera/impala" rel="nofollow">Imapala</a></strong> would be more useful to you which allows to query our BigData keeping the real-time-ness in mind.</p> <p>The reason for whatever I have mentioned above is that your use case requires more than just scalability provided by Hadoop. Along with the ability to distribute the load, your solution should be able to cater the needs you have specified above. It is more than just distributing your data over a group of machines and running raw querying over it. Your users would require <strong>real-time response</strong> along with the <strong>smart suggestions</strong> feature you have mentioned in your question.</p> <p>You actually need a smarter system than just a Hadoop cluster. Do have a look at <strong><a href="http://mahout.apache.org/" rel="nofollow">Apache Mahout</a></strong>. It is an awesome tool that provides the feature of <strong>Recommendation Mining</strong> and can be used with Hadoop easily. You can find more its home page. I will definitely help you in adding that <strong>smart suggestions</strong> feature to your system.</p> <p>You might wanna have a look at another tool of the Hadoop family, the <strong><a href="http://hbase.apache.org/" rel="nofollow">HBase</a></strong>, which is a distributed, scalable, big data store. It acts like a database, but it is not a relational database. It also runs on an existing Hadoop cluster and provides real-time random read/write capabilities. Read a bit about it and see if it fits somewhere.</p> <p>Last but not the least, it all depends on your needs. An exact decision can be made only after giving different things a try and doing a comparative study. We can just suggest you based on our experiences, but a fair decision can be made only after testing a few tools and finding which one fits best into your requirements :)</p>