title: How to read a file using multiple threads in Java when a high throughput(3GB/s) file system is available
body: <p>I understand that for a normal Spindle Drive system, reading files using multiple threads is inefficient.</p>
  
  <p><strong>This is a different case, I have a high-throughput file systems available to me, which provides read speeds up to 3GB/s, with 196 CPU cores and 2TB RAM</strong></p>
  
  <p>A single threaded Java program reads the file with maximum 85-100 MB/s, so I have potential to get better than single thread. I have to read files as big as 1TB in size and I have enough RAM to load it.</p>
  
  <p>Currently I use the following or something similar, but need to write something with multi-threading to get better throughput:</p>
  
  <p>Java 7 Files: 50 MB/s</p>
  
  <pre><code>List&lt;String&gt; lines = Files.readAllLines(Paths.get(path), encoding);
  </code></pre>
  
  <p>Java commons-io: 48 MB/s</p>
  
  <pre><code>List&lt;String&gt; lines = FileUtils.readLines(new File("/path/to/file.txt"), "utf-8");
  </code></pre>
  
  <p>The same with guava: 45 MB/s</p>
  
  <pre><code>List&lt;String&gt; lines = Files.readLines(new File("/path/to/file.txt"), Charset.forName("utf-8"));
  </code></pre>
  
  <p>Java Scanner Class: Very Slow</p>
  
  <pre><code>Scanner s = new Scanner(new File("filepath"));
  ArrayList&lt;String&gt; list = new ArrayList&lt;String&gt;();
  while (s.hasNext()){
      list.add(s.next());
  }
  s.close();
  </code></pre>
  
  <p><em>I want to be able to load the file and build the same ArrayList, in the correct sorted sequence, as fast as possible.</em></p>
  
  <p>There is <a href="https://stackoverflow.com/questions/26226819/java-i-o-unexpected-performance-difference-between-sequentially-and-concurrentl">another question</a> that reads similar, but it is actually different, because of :
  The question is discussing about systems where multi-threaded file I/O is physically impossible to be efficient, but due to technological advancements, we now have systems that are designed to support high-throughput I/O , and so the limiting factor is CPU/SW , which can be overcome by multi-threading the I/O.</p>
  
  <p>The other question does not answer how to write code to multi-thread I/O.</p>