title: Properly streaming of both input and output in HTTP Servlet
body: <p>I'm trying to write servlet that will handle POST request and stream both input and output. I mean it should read one line of input, do some work on this line, and write one line of output. And it should be able to handle arbitrary long request (so also will produce arbitrary long response) without out of memory exceptions. Here's my first attempt:</p>
  
  <pre><code>protected void doPost(HttpServletRequest request, HttpServletResponse response) {
      ServletInputStream input = request.getInputStream();
      ServletOutputStream output = response.getOutputStream();
  
      LineIterator lineIt = lineIterator(input, "UTF-8");
      while (lineIt.hasNext()) {
          String line = lineIt.next();
          output.println(line.length());
      }
      output.flush();
  }
  </code></pre>
  
  <p>Now I tested this servlet using <code>curl</code> and it works, but when I've written client using Apache HttpClient both the client thread and server thread hangs. The client looks like that:</p>
  
  <pre><code>HttpClient client = HttpClientBuilder.create().build();
  HttpPost post = new HttpPost(...);
  
  // request
  post.setEntity(new FileEntity(new File("some-huge-file.txt")));
  HttpResponse response = client.execute(post);
  
  // response
  copyInputStreamToFile(response.getEntity().getContent(), new File("results.txt"));
  </code></pre>
  
  <p>The issue is obvious. Client does it's job sequentially in one thread - first it sends request completely, and only then starts reading the response. But server for every line of input writes one line of output, if client is not reading output (and sequential client isn't) then server is blocked trying to write to the output stream. This in turn blocks client trying to send the input to the server.</p>
  
  <p>I suppose <code>curl</code> works, because it somehow sends input and receives output concurrently (in separate threads?). So the first question is can Apache HttpClient be configured to behave similarly as <code>curl</code>?</p>
  
  <p>The next question is, how to improve the servlet so ill-behaving client won't cause server threads to be hanging? My first attempt is to introduce intermediate buffer, that will gather the output until the client finishes sending input, and only then servlet will start sending output:</p>
  
  <pre><code>ServletInputStream input = request.getInputStream();
  ServletOutputStream output = response.getOutputStream();
  
  // prepare intermediate store
  int threshold = 100 * 1024; // 100 kB before switching to file store
  File file = File.createTempFile("intermediate", "");
  DeferredFileOutputStream intermediate = new DeferredFileOutputStream(threshold, file);
  
  // process request to intermediate store
  PrintStream intermediateFront = new PrintStream(new BufferedOutputStream(intermediate));
  LineIterator lineIt = lineIterator(input, "UTF-8");
  while (lineIt.hasNext()) {
      String line = lineIt.next();
      intermediateFront.println(line.length());
  }
  intermediateFront.close();
  
  // request fully processed, so now it's time to send response
  intermediate.writeTo(output);
  
  file.delete();
  </code></pre>
  
  <p>This works, and ill-behaving client can use my servlet safely, but on the other hand for these concurrent clients like <code>curl</code> this solution adds an unnecessary latency. The parallel client is reading the response in separate thread, so it will benefit when the response will be produced line by line as the request is consumed.</p>
  
  <p>So I think I need a byte buffer/queue that:</p>
  
  <ul>
  <li>can be written by one thread, and read by another thread</li>
  <li>will initially be only in memory</li>
  <li>will overflow to disk if necessary (similarly as <code><a href="https://commons.apache.org/proper/commons-io/javadocs/api-2.4/org/apache/commons/io/output/DeferredFileOutputStream.html" rel="nofollow">DeferredFileOutputStream</a></code>).</li>
  </ul>
  
  <p>In the servlet I'll spawn new thread to read input, process it, and write output to the buffer, and the main servlet thread will read from this buffer and send it to the client.</p>
  
  <p>Do you know any library like to do that? Or maybe my assumptions are wrong and I should do something completely different...</p>