title: Determining the number of buckets in Hive Table
body: <p>I have a question regarding the number of buckets to be used. I understand the uses of bucketing and how it positively impacts SMB joins and sampling. But what if the data volume spikes exponentially? </p>
  
  <p>Let's say looking at the initial data volume, I decide to use 4 buckets and partitioned by day. When I insert into this table it would take 4 reducers at some point (Last job in the insertion query). This is fine. But lets say the data volume suddenly spikes a whole lot for some partitions. It would still take 4 reducers which is not optimal and it is also possible it could fail with OOM.</p>
  
  <p>I could decide on using more buckets initially but that would start creating too many small files until I reach the high volume, as each bucket goes into a file. </p>
  
  <p>Is it possible to have more than one file for a bucket value? </p>
  
  <p>Your inputs are appreciated.</p>
  
  <p>K</p>