title: When does Python write a file to disk?
body: <p>I have a library that interacts with a configuration file. When the library is imported, the initialization code reads the configuration file, possibly updates it, and then writes the updated contents back to the file (even if nothing was changed).</p>
  
  <p>Very occasionally, I encounter a problem where the contents of the configuration file simply disappear. Specifically, this happens when I run many invocations of a short script (using the library), back-to-back, thousands of times. It never occurs during the same directories, which leads me to believe it's a somewhat random problem--specifically a race condition with IO.</p>
  
  <p>This is a pain to debug, since I can never reliably reproduce the problem and it only happens on some systems. I have a suspicion about what might happen, but I wanted to see if my picture of file I/O in Python is correct.</p>
  
  <p>So the question is, <strong>when does a Python program actually write file contents to a disk?</strong> I thought that the contents would make it to disk by the time that the file closed, but then I can't explain this error. When python closes a file, does it flush the contents to the disk itself, or simply queue it up to the filesystem? <strong>Is it possible that file contents can be written to disk after Python terminates?</strong> And can I avoid this issue by using <code>fp.flush(); os.fsync(fp.fileno())</code> (where <code>fp</code> is the file handle)?</p>
  
  <p>If it matters, I'm programming on a Unix system (Mac OS X, specifically). <strong>Edit:</strong> Also, keep in mind that the processes are not running concurrently.</p>
  
  <p>Appendix: Here is the specific race condition that I suspect:</p>
  
  <ol>
  <li>Process #1 is invoked.</li>
  <li>Process #1 opens the configuration file in read mode and closes it when finished.</li>
  <li>Process #1 opens the configuration file in write mode, erasing all of its contents. The erasing of the contents is synced to the disk.</li>
  <li>Process #1 writes the new contents to the file handle and closes it.</li>
  <li>Process #1: Upon closing the file, Python tells the OS to queue writing these contents to disk.</li>
  <li>Process #1 closes and exits</li>
  <li>Process #2 is invoked</li>
  <li>Process #2 opens the configuration file in read mode, but new contents aren't synced yet. Process #2 sees an empty file.</li>
  <li>The OS finally finishes writing the contents to disk, after process 2 reads the file</li>
  <li>Process #2, thinking the file is empty, sets defaults for the configuration file.</li>
  <li>Process #2 writes its version of the configuration file to disk, overwriting the last version.</li>
  </ol>