{"action":"opened","issue":{"url":"https://api.github.com/repos/aol/moloch/issues/458","repository_url":"https://api.github.com/repos/aol/moloch","labels_url":"https://api.github.com/repos/aol/moloch/issues/458/labels{/name}","comments_url":"https://api.github.com/repos/aol/moloch/issues/458/comments","events_url":"https://api.github.com/repos/aol/moloch/issues/458/events","html_url":"https://github.com/aol/moloch/issues/458","id":146916956,"number":458,"title":"BytesRefHash$MaxBytesLengthExceededException errors","user":{"login":"josh-lane","id":3082157,"avatar_url":"https://avatars.githubusercontent.com/u/3082157?v=3","gravatar_id":"","url":"https://api.github.com/users/josh-lane","html_url":"https://github.com/josh-lane","followers_url":"https://api.github.com/users/josh-lane/followers","following_url":"https://api.github.com/users/josh-lane/following{/other_user}","gists_url":"https://api.github.com/users/josh-lane/gists{/gist_id}","starred_url":"https://api.github.com/users/josh-lane/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/josh-lane/subscriptions","organizations_url":"https://api.github.com/users/josh-lane/orgs","repos_url":"https://api.github.com/users/josh-lane/repos","events_url":"https://api.github.com/users/josh-lane/events{/privacy}","received_events_url":"https://api.github.com/users/josh-lane/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-04-08T12:28:55Z","updated_at":"2016-04-08T12:28:55Z","closed_at":null,"body":"**Elasticsearch version**:  ES 1.7.3\r\n\r\n**Moloch version**:  0.12.1\r\n\r\n**OS name and version**:  Ubuntu 12.04.5\r\n\r\n**Provide logs, stack traces and steps to reproduce**:  Sending Moloch captures to ES in normal operation and see multiple errors like this:\r\n\r\njava.lang.IllegalArgumentException: Document contains at least one immense term in field=\"rawus\" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[-removed-]...', original message: bytes can be at most 32766 in length; got 41548\r\n        at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:687)\r\n        at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:359)\r\n        at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:318)\r\n        at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:241)\r\n        at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:465)\r\n        at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1526)\r\n        at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1252)\r\n        at org.elasticsearch.index.engine.InternalEngine.innerIndex(InternalEngine.java:432)\r\n        at org.elasticsearch.index.engine.InternalEngine.index(InternalEngine.java:364)\r\n        at org.elasticsearch.index.shard.IndexShard.index(IndexShard.java:511)\r\n        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardIndexOperation(TransportShardBulkAction.java:413)\r\n        at org.elasticsearch.action.bulk.TransportShardBulkAction.shardOperationOnPrimary(TransportShardBulkAction.java:148)\r\n        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase.performOnPrimary(TransportShardReplicationOperationAction.java:574)\r\n        at org.elasticsearch.action.support.replication.TransportShardReplicationOperationAction$PrimaryPhase$1.doRun(TransportShardReplicationOperationAction.java:440)\r\n        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)\r\n        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)\r\n        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)\r\n        at java.lang.Thread.run(Thread.java:744)\r\nCaused by: org.apache.lucene.util.BytesRefHash$MaxBytesLengthExceededException: bytes can be at most 32766 in length; got 41548\r\n        at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:284)\r\n        at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:151)\r\n        at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:663)\r\n\r\nSome are nearly twice the max value shown.  "}}