title: Fastest way to incrementally read a large file
body: <p>When given a buffer of MAX_BUFFER_SIZE, and a file that far exceeds it, how can one:</p>
  
  <ol>
  <li>Read the file in blocks of MAX_BUFFER_SIZE?</li>
  <li>Do it as fast as possible</li>
  </ol>
  
  <h2>I tried using NIO</h2>
  
  <pre><code>    RandomAccessFile aFile = new RandomAccessFile(fileName, "r");
      FileChannel inChannel = aFile.getChannel();
  
      ByteBuffer buffer = ByteBuffer.allocate(CAPARICY);
  
      int bytesRead = inChannel.read(buffer);
  
      buffer.flip();
  
          while (buffer.hasRemaining()) {
              buffer.get();
          }
  
          buffer.clear();
          bytesRead = inChannel.read(buffer);
  
      aFile.close();
  </code></pre>
  
  <h2>And regular IO</h2>
  
  <pre><code>    InputStream in = new FileInputStream(fileName);
  
      long length = fileName.length();
  
      if (length &gt; Integer.MAX_VALUE) {
          throw new IOException("File is too large!");
      }
  
      byte[] bytes = new byte[(int) length];
  
      int offset = 0;
  
      int numRead = 0;
  
      while (offset &lt; bytes.length
              &amp;&amp; (numRead = in.read(bytes, offset, bytes.length - offset)) &gt;= 0) {
          offset += numRead;
      }
  
      if (offset &lt; bytes.length) {
          throw new IOException("Could not completely read file " + fileName);
      }
  
      in.close();
  </code></pre>
  
  <p>Turns out that <strong>regular IO is about 100 times faster in doing the same thing as NIO</strong>. Am i missing something? Is this expected? Is there a faster way to read the file in buffer chunks?</p>
  
  <p>Ultimately i am working with a large file i don't have memory for to read it all at once. Instead, I'd like to read it incrementally in blocks that would then be used for processing.</p>