Hadoop Scalability

How Big is Your Elephant?

In order to augment some research we’re doing, we’d like to determine the sizes of Hadoop clusters that organizations are commonly deploying. Why? Well, organizations with 100+ nodes have different needs than those with five, for example.

Configuration Hadoop Hive Scalability

Give Your MySQL Account Access to Hive

One area of the Apache Hive documentation that’s not entirely explicit is in regard to the database privileges needed for its metastore[1]. Developers often become accustomed to creating a database account that has all privileges granted. But in the Real World, end users of Hive must configure it to point to a metastore RDBMS account […]

Hadoop Hive

Using Hive with Existing Files on S3

One feature that Hive gets for free by virtue of being layered atop Hadoop is the S3 file system implementation. The upshot being that all the raw, textual data you have stored in S3 is just a few hoops away from being queried using Hive’s SQL-esque language. Imagine you have an S3 bucket un-originally named […]