title: Lucene on Maven - java.lang.IllegalArgumentException UTF8 encoding is longer than the max length 32766 error
body: <p>I am trying to index a large document that is over the limit of string length with Lucene Maven. Then, I receive this error.</p>
  
  <pre><code>Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="content" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped.  Please correct the analyzer to not produce such terms.  The prefix of the first immense term is: '[65, 32, 98, 101, 110, 122, 111, 100, 105, 97, 122, 101, 112, 105, 110, 101, 32, 91, 116, 112, 108, 93, 73, 80, 65, 99, 45, 101, 110, 124]...', original message: bytes can be at most 32766 in length; got 85391
  </code></pre>
  
  <p>The code is as below (It's a copy from the <a href="http://lucenetutorial.com/lucene-in-5-minutes.html" rel="nofollow noreferrer">http://lucenetutorial.com/lucene-in-5-minutes.html</a> with a very slight change to read the document from the file.):</p>
  
  <pre><code>File file = "doc.txt";
  
  StandardAnalyzer analyzer = new StandardAnalyzer();
  Directory index = new RAMDirectory();
  IndexWriterConfig config = new IndexWriterConfig(analyzer);
  IndexWriter w = new IndexWriter(index, config);
  Document doc = new Document();
  Scanner scanner = new Scanner(file))
       while (scanner.hasNextLine())
       {
            String line = scanner.nextLine();
            doc.add(new StringField("content", line, Field.Store.YES));
            w.addDocument(doc);
       }
  
  ...
  </code></pre>
  
  <p>There are other posts with the same issue as what I am having, but they are solutions for SOLR or Elasticsearch, not for Lucene on Maven, so I am not quite sure how to solve this problem.</p>
  
  <p>Can anyone direct me to the right place to solve this issue, please?</p>
  
  <p>Thank you in advance.</p>