title: Why are std::fstreams so slow?
body: <p>I was working on a simple parser and when profiling I observed the bottleneck is in... file read! I extracted very simple test to compare the performance of <code>fstreams</code> and <code>FILE*</code> when reading a big blob of data:</p>
  
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;chrono&gt;
  #include &lt;fstream&gt;
  #include &lt;iostream&gt;
  #include &lt;functional&gt;
  
  void measure(const std::string&amp; test, std::function&lt;void()&gt; function)
  {
      auto start_time = std::chrono::high_resolution_clock::now();
  
      function();
  
      auto duration = std::chrono::duration_cast&lt;std::chrono::nanoseconds&gt;(std::chrono::high_resolution_clock::now() - start_time);
      std::cout&lt;&lt;test&lt;&lt;" "&lt;&lt;static_cast&lt;double&gt;(duration.count()) * 0.000001&lt;&lt;" ms"&lt;&lt;std::endl;
  }
  
  #define BUFFER_SIZE (1024 * 1024 * 1024)
  
  int main(int argc, const char * argv[])
  {
      auto buffer = new char[BUFFER_SIZE];
      memset(buffer, 123, BUFFER_SIZE);
  
      measure("FILE* write", [buffer]()
      {
          FILE* file = fopen("test_file_write", "wb");
          fwrite(buffer, 1, BUFFER_SIZE, file);
          fclose(file);
      });
      measure("FILE* read", [buffer]()
      {
          FILE* file = fopen("test_file_read", "rb");
          fread(buffer, 1, BUFFER_SIZE, file);
          fclose(file);
      });
      measure("fstream write", [buffer]()
      {
          std::ofstream stream("test_stream_write", std::ios::binary);
          stream.write(buffer, BUFFER_SIZE);
      });
      measure("fstream read", [buffer]()
      {
          std::ifstream stream("test_stream_read", std::ios::binary);
          stream.read(buffer, BUFFER_SIZE);
      });
  
      delete[] buffer;
  }
  </code></pre>
  
  <p>The results of running this code on my machine are: </p>
  
  <pre><code>FILE* write 1388.59 ms
  FILE* read 1292.51 ms
  fstream write 3105.38 ms
  fstream read 3319.82 ms
  </code></pre>
  
  <p><code>fstream</code> write/read are about 2 times slower than <code>FILE*</code> write/read! And this while reading a big blob of data, without any parsing or other features of <code>fstreams</code>. I'm running the code on Mac OS, Intel I7 2.6GHz, 16GB 1600 MHz Ram, SSD drive. Please note that running again same code the time for <code>FILE* read</code> is very low (about 200 ms) probably because the file gets cached... This is why the files opened for reading are not created using the code.</p>
  
  <p>Why when reading just a blob of binary data using <code>fstream</code> is so slow compared to <code>FILE*</code>?</p>
  
  <p><strong>EDIT 1:</strong> I updated the code and the times. Sorry for the delay!</p>
  
  <p><strong>EDIT 2:</strong> I added command line and new results (very similar to previous ones!)</p>
  
  <pre><code>$ clang++  main.cpp -std=c++11 -stdlib=libc++ -O3
  $ ./a.out
  FILE* write 1417.9 ms
  FILE* read 1292.59 ms
  fstream write 3214.02 ms
  fstream read 3052.56 ms
  </code></pre>
  
  <p>Following the results for the second run:</p>
  
  <pre><code>$ ./a.out
  FILE* write 1428.98 ms
  FILE* read 196.902 ms
  fstream write 3343.69 ms
  fstream read 2285.93 ms
  </code></pre>
  
  <p>It looks like the file gets cached when reading for both <code>FILE*</code> and <code>stream</code> as the time reduces with the same amount for both of them.</p>
  
  <p><strong>EDIT 3:</strong> I reduced the code to this:</p>
  
  <pre><code>FILE* file = fopen("test_file_write", "wb");
  fwrite(buffer, 1, BUFFER_SIZE, file);
  fclose(file);
  
  std::ofstream stream("test_stream_write", std::ios::binary);
  stream.write(buffer, BUFFER_SIZE);
  </code></pre>
  
  <p>And started the profiler. It seems like <code>stream</code> spends lots of time in <code>xsputn</code> function, and the actual <code>write</code> calls have the same duration (as it should be, it's the same function...)</p>
  
  <pre><code>Running    Time     Self       Symbol Name
  3266.0ms   66.9%    0,0        std::__1::basic_ostream&lt;char, std::__1::char_traits&lt;char&gt; &gt;::write(char const*, long)
  3265.0ms   66.9%    2145,0          std::__1::basic_streambuf&lt;char, std::__1::char_traits&lt;char&gt; &gt;::xsputn(char const*, long)
  1120.0ms   22.9%    7,0                 std::__1::basic_filebuf&lt;char, std::__1::char_traits&lt;char&gt; &gt;::overflow(int)
  1112.0ms   22.7%    2,0                      fwrite
  1127.0ms   23.0%    0,0        fwrite
  </code></pre>
  
  <p><strong>EDIT 4</strong> For some reason this question is marked as duplicate. I wanted to point out that I don't use <code>printf</code> at all, I use only <code>std::cout</code> to write the time. The files used in the <code>read</code> part are the output from the <code>write</code> part, copied with different name to avoid caching</p>