title: Elasticsearch / Kibana field data too large
body: <p>I have a small ELK cluster that is in testing. The kibana web interface is extremely slow and throws a lot of errors. </p>
  
  <p>Kafka => 8.2<br>
  Logstash => 1.5rc3  (latest)<br>
  Elasticsearch => 1.4.4 (latest)<br>
  Kibana => 4.0.2 (latest)  </p>
  
  <p>The elasticsearch nodes have 10GB of ram each on Ubuntu 14.04. I'm pulling in between 5GB and 20GB of data per day. </p>
  
  <p>Running even a simple query, with only 15 minutes of data in the kibana web interface takes several minutes, and often throws errors. </p>
  
  <pre><code>[FIELDDATA] Data too large, data for [timeStamp] would be larger than limit of [3751437926/3.4gb]]
  </code></pre>
  
  <p><img src="https://i.stack.imgur.com/EdVOB.png" alt="enter image description here"></p>
  
  <p>These errors about the shard failures only appear in kibana. According to all other plugins(head, kopf), the elasticsearch shards are perfectly fine, and the cluster is green.  </p>
  
  <p>I've checked with the google group, IRC and looked at stack overflow. It seems the only solution is to increase the ram. I've increased the ram on my nodes twice. While that seems to fix it for a day or two, the problem quickly returns. 
  Other solutions such as cleaning the cache have no long term improvements. </p>
  
  <pre><code>curl -XPUT 'http://elastic.example.com:9200/cache/clear?filter=true'
  curl -XPOST 'http://elastic.example.com:9200/_cache/clear' -d '{ "fielddata": "true" }'
  </code></pre>
  
  <p>According to the KOPF plugin, the amount of heap space routinely approaches 75% on a completely idle cluster. (I'm the only one in the company using it). 3 Nodes with 10GB of ram should be more than enough for the amount of data that I have. </p>
  
  <p>I have also tried adjusting the breakers as <a href="http://evertrue.github.io/blog/2014/11/16/3-performance-tuning-tips-for-elasticsearch/" rel="nofollow noreferrer">suggested by this blog.</a> </p>
  
  <pre><code>PUT /_cluster/settings -d '{ "persistent" : { "indices.breaker.fielddata.limit" : "70%" } }'
  PUT /_cluster/settings -d '{ "persistent" : {  "indices.fielddata.cache.size" : "60%" } }'
  </code></pre>
  
  <p>How can I prevent these errors , and fix the extreme slowness in kibana? </p>
  
  <p><a href="https://github.com/elastic/kibana/issues/3221" rel="nofollow noreferrer">https://github.com/elastic/kibana/issues/3221</a><br>
  <a href="https://stackoverflow.com/questions/27296340/elasticsearch-getting-too-many-results-need-help-filtering-query">elasticsearch getting too many results, need help filtering query</a><br>
  <a href="http://elasticsearch-users.115913.n3.nabble.com/Data-too-large-error-td4060962.html" rel="nofollow noreferrer">http://elasticsearch-users.115913.n3.nabble.com/Data-too-large-error-td4060962.html</a>   </p>
  
  <p><strong>Update</strong></p>
  
  <p>I have about 30 days of indexes from logstash. 2x Replication so that is 10 shards per day. </p>
  
  <p><img src="https://i.stack.imgur.com/rjNmp.png" alt="enter image description here"></p>
  
  <p><strong>Update2</strong></p>
  
  <p>I've increased the ram of each node to 16GB, (48GB total) and I've also upgraded to 1.5.2.</p>
  
  <p><img src="https://i.stack.imgur.com/Qojt8.png" alt="enter image description here"></p>
  
  <p>This appears to fix the issue for a day or two, however the problem returns. </p>
  
  <p><img src="https://i.stack.imgur.com/J45rD.png" alt="enter image description here"></p>
  
  <p><strong>Update3</strong></p>
  
  <p><a href="https://www.elastic.co/blog/support-in-the-wild-my-biggest-elasticsearch-problem-at-scale" rel="nofollow noreferrer">This blog article from an elastic employee has good tips</a> explaining what can cause these issues. </p>