title: Accessing a single file with multiple threads
body: <p>I need to access a file concurrently with multiple threads. This needs to be done concurrently, without thread serialisation for performance reasons.</p>
  <p>The file in particular has been created with the 'temporary' file attribute that encourages windows to keep the file in the system cache. This means most of the time the file read wont go near the disk, but will read the portion of the file from the system cache.</p>
  <p>Being able to concurrently access this file will significantly improve performance of certain algorithms in my code.</p>
  <p>So, there are two questions here:</p>
  <ol>
  <li>Is it possible for windows to concurrently access the same file from different threads?</li>
  <li>If so, how do you provide this ability? I've tried creating the temp file and opening the file again to provide two file handles, but the second open does not succeed.</li>
  </ol>
  <p>Here's the create:</p>
  <pre><code>FFileSystem := CreateFile(PChar(FFileName),
                            GENERIC_READ + GENERIC_WRITE,
                            FILE_SHARE_READ + FILE_SHARE_WRITE,
                            nil,
                            CREATE_ALWAYS,
                            FILE_ATTRIBUTE_NORMAL OR
                            FILE_FLAG_RANDOM_ACCESS OR
                            FILE_ATTRIBUTE_TEMPORARY OR
                            FILE_FLAG_DELETE_ON_CLOSE,
                            0);
  </code></pre>
  <p>Here's the second open:</p>
  <pre><code>FFileSystem2 := CreateFile(PChar(FFileName),
                            GENERIC_READ,
                            FILE_SHARE_READ,
                            nil,
                            OPEN_EXISTING,
                            FILE_ATTRIBUTE_NORMAL OR
                            FILE_FLAG_RANDOM_ACCESS OR
                            FILE_ATTRIBUTE_TEMPORARY OR
                            FILE_FLAG_DELETE_ON_CLOSE,
                            0);
  </code></pre>
  <p>I've tried various combinations of the flags with no success so far. The second file open always fails, with messages to the affect that the file cannot be accessed as it is in use by another process.</p>
  <p>Edit:</p>
  <p>OK, some more information (I was hoping to not get lost in the weeds here...)</p>
  <p>The process in question is a Win32 server process running on WinXP 64. It's maintaining large spatial databases and would like to keep as much of the spatial database as possible in memory in an L1/L2 cache structure. L1 already exists. L2 exists as a 'temporary' file that stays in the windows system cache (it's somewhat of a dirty trick, but gets around win32 memory limitations somewhat). Win64 means I can have lots of memory used by the system cache so memory used to hold the L2 cache does count towards process memory.</p>
  <p>Multiple (potentially many) threads want to concurrently access information contained in the  L2 cache. Currently, access is serialised, which means one thread gets to read it's data while most (or the rest) of the threads are blocked pending completion of that operation.</p>
  <p>The L2 cache file does get written to, but I'm happy to globally serialise/interleave read and write type operations as long as I can perform concurrent reads.</p>
  <p>I'm aware there are nasty potential thread concurrency issues, and I'm aware there are dozens of ways to skin this cat in other contexts. I have this particular context, and I'm trying to determine if there is a way to permit concurrent thread read access within the file and within the same process.</p>
  <p>Another approach I have considered would be two split the L2 cache into multiple temporary files, where each file serialises thread access the way the current single L2 cache file does.</p>
  <p>And yes, this somewhat desperate approach is because 64 bit Delphi wont be with us any time soon :-(</p>