title: Where do you close a writer in TwitterStream?
body: <p>I'm working on compiling a bunch of tweets for an information retrieval class. I'm trying this using both the REST API and the Streaming API through twitter4j. When using the Streaming API, I use the following modifications to <a href="https://github.com/yusuke/twitter4j/blob/master/twitter4j-examples/src/main/java/twitter4j/examples/stream/PrintSampleStream.java" rel="nofollow">this example</a>:</p>
  
  <pre><code>final LimitedFileWriter output = new LimitedFileWriter("Tweets","tweets");
  TwitterStream twitterStream = new TwitterStreamFactory(cb.build()).getInstance();
      StatusListener listener = new StatusListener() {
          @Override
          public void onStatus(Status status) {
              try{
                  output.write("@" + status.getUser().getScreenName() + " -- " + status.getText()+"\n");
              }
              catch(IOException e){
                  e.printStackTrace();
              }
          }
      }
  
      twitterStream.addListener(listener);
      twitterStream.sample("en");
      //output.close();
  </code></pre>
  
  <p>It seems I can't ever close my writer. The writer I am using simply wraps BufferedWriter, while keeping track of file size. If the file exceeds a certain size (128MB), the writer will close the current file and create a new file. Here are the relevant class functions:</p>
  
  <pre><code>public void write(String s) throws IOException
  {
      if(bytesWritten + s.getBytes(charset).length &gt;= MAXSIZE){
          output.close();
          bytesWritten = 0;
          fileNum++;
          String fileName = directory + "/" + baseName+fmt.format(fileNum);
          currentFile = new File(fileName);
          output = new BufferedWriter
              (new OutputStreamWriter(new FileOutputStream(fileName),charset));
      }
  
      output.write(s);
      bytesWritten += s.getBytes(charset).length;
  
  }
  
  public void close() throws IOException{
      output.close();
  }
  </code></pre>
  
  <p>If I try to close the writer after <code>twitterStream.sample()</code> (commented out), the program crashes because I am trying to write to a closed file. If my understanding is correct, this is because the TwitterStream class creates a new thread which runs concurrently with the <code>main</code> thread. Then, the <code>main</code> thread closes the stream and the <code>twitterStream</code> can no longer write to it.</p>
  
  <p>If that's the case, where should I close my writer?</p>