<p>"<a href="http://arxiv.org/abs/cs.PF/0502012" rel="nofollow">Sequential File Programming Patterns and Performance with .NET</a>" is a great article in I/O performance improvement.</p>

<p>In page 8 of <a href="http://arxiv.org/pdf/cs/0502012v1" rel="nofollow">this</a> PDF file, it shows that the bandwidth for buffer size bigger than eight bytes, is constant. Consider that the article has been written in 2004 and the hard disk drive is "<em>Maxtor 250 GB 7200 RPM SATA disk</em>" and the result should be different by latest I/O technologies.</p>

<p>If you are looking for the best performance take a look at <a href="http://www.pinvoke.net/default.aspx/kernel32.ReadFile" rel="nofollow">pinvoke.net</a> or the page 9 of the PDF file, the un-buffered file performance measurements shows better results:</p>

<blockquote>
  <p>In un-buffered I/O, the disk data moves directly between the
  application’s address space and the device without any intermediate
  copying.</p>
</blockquote>

<h2>Summary</h2>

<ul>
<li>For single disks, use the defaults of the .NET framework – they deliver excellent performance for sequential file access.</li>
<li>Pre-allocate large sequential files (using the SetLength() method) when the file is created. This typically improves speed by about 13% when compared to a fragmented file.</li>
<li>At least for now, disk arrays require un-buffered I/O to achieve the highest performance - buffered I/O can be eight times slower than un-buffered I/O. We expect this problem will be addressed in later releases of the .NET framework.</li>
<li>If you do your own buffering, use large request sizes (64&nbsp;KB is a good place to start). Using the .NET framework, a single processor can read and write a disk array at over 800 Mbytes/s using un-buffered I/O.</li>
</ul>
What is the best buffer size when using BinaryReader to read big files (>1 GB)?
<p>I'm reading binary files and here is a sample:</p>

<pre><code>public static byte[] ReadFully(Stream input)
{
    byte[] buffer = new byte[16*1024];
    int read;
    while ((read = input.Read(buffer, 0, buffer.Length)) &gt; 0)
    {
        ......
    }

}
</code></pre>

<p>Obviously the buffer size (16*1024) has a great role in performance. I've read that it depends on the I/O technology (<a href="http://en.wikipedia.org/wiki/Serial_ATA" rel="nofollow noreferrer">SATA</a>, <a href="http://en.wikipedia.org/wiki/Solid-state_drive" rel="nofollow noreferrer">SSD</a>, <a href="http://en.wikipedia.org/wiki/SCSI" rel="nofollow noreferrer">SCSI</a>, etc.) and also the fragment size of the partition which file exists on it (we can define during the formatting the partition).</p>

<p>But here is the question:
<strong>Is there any formula or best practice to define the buffer size?</strong> Right now, I'm defining based on trial-and-error.</p>

<p><strong>Edit:</strong>
I've tested the application on my server with different buffer sizes, and I get the best performance with 4095*256*16 (16&nbsp;MB)!!! 4096 is 4 seconds slower.</p>

<p>Here are some older posts which are very helpful but I can't still get the reason:</p>

<ul>
<li><p><em><a href="https://stackoverflow.com/questions/1238388/faster-unsafe-binaryreader-in-net">Faster (unsafe) BinaryReader in .NET</a></em></p></li>
<li><p><em><a href="https://stackoverflow.com/questions/1552107/optimum-file-buffer-read-size">Optimum file buffer read size?</a></em></p></li>
<li><p><em><a href="https://stackoverflow.com/questions/3033771/file-io-with-streams-best-memory-buffer-size">File I/O with streams - best memory buffer size</a></em></p></li>
<li><p><em><a href="https://stackoverflow.com/questions/236861/how-do-you-determine-the-ideal-buffer-size-when-using-fileinputstream">How do you determine the ideal buffer size when using FileInputStream?</a></em></p></li>
</ul>