title: Writing data into file: fflush() takes a lot of time
body: <p>I have a requirement wherein I have to buffer a lot of data (in GBs), for future use.
  Since there isn't enough RAM available for buffering such huge amount of data, I decided to go for storing the data in a file.  </p>
  
  <p>Now the pitfall here is that while I am writing the data to the file, other threads might need that "buffered" data and so I have to flush the file stream every time I write something to it. Precisely, the data is video frames that I buffer as pre-recorded data (like a TiVo)
  and other threads may or may not want to write it at any given point in time, but when they do, they <code>fread</code> from the file and process the frames.  </p>
  
  <p>In the general case, the <code>fwrite</code>-<code>fflush</code> combo takes around 150 us but occasionally (and fairly regularly), the combo takes more than 1.5 seconds. I can't afford this as I have to process frames in real-time.  </p>
  
  <p>I have many questions here:  </p>
  
  <ol>
  <li><p>Is my approach of buffering data in the file correct? What alternatives do I have?  </p></li>
  <li><p>Any idea why the fwrite-fflush operation suddenly takes more time on some occasions? Note that it reverts back to 150 us after taking 1.5 seconds once.  </p></li>
  </ol>