title: Apache Lucene TokenStream contract violation
body: <p>Using Appache Lucene TokenStream to remove stopwords 
  causes an error:</p>
  
  <pre><code>TokenStream contract violation: reset()/close() call missing, reset() called multiple times, or subclass does not call super.reset(). Please see Javadocs of TokenStream class for more information about the correct consuming workflow.
  </code></pre>
  
  <p>I use this code:</p>
  
  <pre><code>public static String removeStopWords(String string) throws IOException {
      TokenStream tokenStream = new StandardTokenizer(Version.LUCENE_47, new StringReader(string));
      TokenFilter tokenFilter = new StandardFilter(Version.LUCENE_47, tokenStream);
      TokenStream stopFilter = new StopFilter(Version.LUCENE_47, tokenFilter, StandardAnalyzer.STOP_WORDS_SET);
      StringBuilder stringBuilder = new StringBuilder();
  
      CharTermAttribute token = tokenStream.getAttribute(CharTermAttribute.class);
  
      while(stopFilter.incrementToken()) {
          if(stringBuilder.length() &gt; 0 ) {
              stringBuilder.append(" ");
          }
  
          stringBuilder.append(token.toString());
      }
  
      stopFilter.end();
      stopFilter.close();
  
      return stringBuilder.toString();
  }
  </code></pre>
  
  <p>But as you can see i never call reset() or close().</p>
  
  <p>So why am i getting this error?</p>