title: Race-condition creating folder in Python
body: <p>I have a urllib2 caching module, which sporadically crashes because of the following code:</p>
  
  <pre><code>if not os.path.exists(self.cache_location):
      os.mkdir(self.cache_location)
  </code></pre>
  
  <p>The problem is, by the time the second line is being executed, the folder may exist, and will error:</p>
  
  <pre>  File ".../cache.py", line 103, in __init__
      os.mkdir(self.cache_location)
  OSError: [Errno 17] File exists: '/tmp/examplecachedir/'</pre>
  
  <p>This is because the script is simultaneously launched numerous times, by third-party code I have no control over.</p>
  
  <p>The code (before I attempted to fix the bug) can be found <a href="http://github.com/dbr/tvdb_api/commit/e7429cce89fb2406efce6d81336e2ffa01479976" rel="noreferrer">here, on github</a></p>
  
  <p>I can't use the <a href="http://docs.python.org/library/tempfile.html#tempfile.mkdtemp" rel="noreferrer">tempfile.mkstemp</a>, as it solves the race condition by using a randomly named directory (<a href="http://svn.python.org/projects/python/trunk/Lib/tempfile.py" rel="noreferrer">tempfile.py source here</a>), which would defeat the purpose of the cache.</p>
  
  <p>I don't want to simply discard the error, as the same error Errno 17 error is raised if the folder name exists as a file (a different error), for example:</p>
  
  <pre>$ touch blah
  $ python
  >>> import os
  >>> os.mkdir("blah")
  Traceback (most recent call last):
    File "", line 1, in 
  OSError: [Errno 17] File exists: 'blah'
  >>></pre>
  
  <p>I cannot using <code>threading.RLock</code> as the code is called from multiple processes.</p>
  
  <p>So, I tried writing a simple file-based lock (<a href="http://github.com/dbr/tvdb_api/blob/343df228d519ac5c8895a1e6bdab8d259a64cdb9/cache.py" rel="noreferrer">that version can be found here</a>), but this has a problem: it creates the lockfile one level up, so <code>/tmp/example.lock</code> for <code>/tmp/example/</code>, which breaks if you use <code>/tmp/</code> as a cache dir (as it tries to make <code>/tmp.lock</code>)..</p>
  
  <p>In short, I need to cache <code>urllib2</code> responses to disc. To do this, I need to access a known directory (creating it, if required), in a multiprocess safe way. It needs to work on OS X, Linux and Windows.</p>
  
  <p>Thoughts? The only alternative solution I can think of is to rewrite the cache module using SQLite3 storage, rather than files.</p>