title: What is causing slow file writes on Windows/NTFS?
body: <p>Here is a simple testrig demonstrating apparent slowness in 'write' file access on Windows/NTFS vs Linux/ext4:</p>
  <pre><code>#include &lt;stdio.h&gt;
  #include &lt;stdlib.h&gt;
  #include &lt;time.h&gt;
  #include &lt;unistd.h&gt;
  #include &lt;sys/time.h&gt;
  #include &lt;string.h&gt;
  
  static double time_ms(void)
  {
      struct timeval tv;
      struct timezone tz;
      gettimeofday(&amp;tv, &amp;tz);
      int time_now = (int)(tv.tv_sec * 1000 + tv.tv_usec / 1000);
      return (double)time_now / 1000.0;
  }
  
  int main(int argc, char *argv[])
  {
      argc--;
      argv++;
      int line = 0;
      double start = time_ms();
      if (argc &lt; 2)
          return 0;
      int step = atoi(argv[1]);
      int do_sync = (argc &gt; 2 &amp;&amp; strcmp(argv[2], &quot;sync&quot;) == 0);
      while(1)
      {
          FILE *f = fopen(&quot;test&quot;, argv[0]);
          if (step == 0)
              line++;
          else for (int j = 0; j &lt; step; j++)
          {
              fprintf(f, &quot;Adding line %08d at %ld\n&quot;, line, time(0));
              line++;
          }
          fclose(f);
  #ifdef SYNC
          if (do_sync)
              sync();
  #endif
          if (line % 1000 == 0)
          {
              double dt = time_ms() - start;
              if (dt)
                  printf(&quot;Written %s %d lines at %.2f lines/s\n&quot;, argv[0], line, line / dt);
          }
      }
  }
  </code></pre>
  <p>linux compilation:</p>
  <pre><code>gcc -o append append.c
  </code></pre>
  <p>and</p>
  <pre><code>gcc -o append append.c -DSYNC
  </code></pre>
  <p>(cross) compiled on linux with mxe:</p>
  <pre><code>x86_64-w64-mingw32.static-gcc -o append.exe append.c
  </code></pre>
  <p>This opens a file, writes a number of lines and closes it. the 'mode' and 'number of lines' can be given on the command line, and also, for linux, 'sync' can be given to perform a sync()</p>
  <p>run like this:</p>
  <pre><code>./append a 10
  ./append a 100 sync
  ./append w 1
  </code></pre>
  <p>results:</p>
  <pre><code>lines/sec       windows linux      nosync
  append 0        200      6,000    500,000
  append 1        150        300    240,000
  append 10       1500     3,000  1,800,000
  append 100      15000   30,000  5,000,000
  
  write 0         230       300     240,000
  write 1         180       280      30,000
  write 10        1700     2800     300,000
  write 100       17000   28000   2,000,000
  </code></pre>
  <p>These tests were run on different hardware for Linux and Windows (10), but the first two columns I believe are comparable -- this shows that Linux+sync() is running around the same speed as windows. but in &quot;normal&quot; operation, without the sync, Linux clearly runs along many thousand times faster as the file writes are being handed off to cache/os/ext4/...</p>
  <p>I realise that the <em>actual</em> final lines/s written is not as shown, but in the real application those 'slow' writes are blocking and causing noticeable stutter in the user interface.</p>
  <p>I have experimented with</p>
  <ul>
  <li>using open()/write() rather than streamed fopen()</li>
  <li>fopen r+ and fseek() rather than &quot;a&quot;</li>
  <li>&quot;native&quot; FileCreate and FileWrite</li>
  <li>compiling with VS</li>
  </ul>
  <p>and results vary, but still not anything like the Linux nosync results.</p>
  <p>The current &quot;solution&quot; I have is to write the files in a separate thread, which causes its own problems!</p>
  <p>Is there anything I can do to speed this up?</p>
  <p>Thanks,</p>