title: largest memory map allocation size?
body: <p>My System:
  Physical memory: 3gb<br>
  Windows XP Service Pack 3 (32bit)
  Swap file size: 30gb<br></p>
  
  <p>Goal: To find the largest possible memory map size I can allocate on my machine. </p>
  
  <p>When I run the following code to allocate 2gb memory map file, the call fails.</p>
  
  <p>handle=CreateFileMapping(INVALID_HANDLE_VALUE,NULL,PAGE_READWRITE|SEC_COMMIT,0,INT_MAX,NULL);</p>
  
  <p>I've been very puzzled by this, because I can allocate a memory map file's up to the system swap file size of 30gb by constantly calling CreateFileMapping with 100mb at a time.</p>
  
  <p>After restarting the machine, and re-running the application that requests 2gb of memory mapped file to CreateFileMapping  it works and it returns a valid handle. So this leads me a bit confused what the hell is going on under the hood with windows?</p>
  
  <p>So the situation is this, I can create many small memory mapped files using up all the system page file (30gb), but when asking for a single allocation of 2gb the call fails. When restarting the machine and running the same application the call succeeds!</p>
  
  <p>Some notes:<br>
  1) The memory mapped file is not being loaded into the proccess virtual address space, there is no view yet to the file. <br>
  2) The OS can allocate small 100mb memory mapped files to 30gb of the systems page file!<br></p>
  
  <p>Right now the only conclusion I can come to, is that the Windows XP SP3 (32bit) virtual memory manager cannot successfully reserve the requested 2gb in the system page file, and then fails due to the system memory fragmentation (it seems like it needs to reserve a continues allocation of memory, even though the page file is 4kb each). After a restart I assume the system memory fragmentation is less, thus allowing the same call to succeed and allocate a memory mapped file of 2gb in size.</p>
  
  <p>I've run some experiments, after running the machine for a day I started a small application that would allocate a memory maped file of 300mb and then release it. It would then increase the size by 1mb and try again. Finally it stops at 700mb and reports (insufficient system resources). I would then go through and close down each application and this would in turn stop the error messages and it finally continues to allocate a memory mapped file of 3.5gb in size!</p>
  
  <p>So my question is what is going on here? There must be some type of memory fragmentation happening internally with the virtual memory manager, because allocating 100mbs memory mapped files will consume up to the 30gb of the system page file (commit limit).</p>
  
  <p><b>Update</b><br>
  Conclusion is if you're going to create a large memory mapped file backed by the system page file with INVALID_HANDLE_VALUE, then the system page file (swap file) needs to be resize to the required size and be in a non fragmented state for <b>large allocations > 2gb</b>! Though under heavy IO load it can still fail. To get around all these problems you can create your own file with the needed size (I did 1tb) and memory map to that file instead.</p>
  
  <p><b>Final Update</b><br>
  I ran the same tests on a Windows 7 box, and to my surprise it works every single time (up to the system page file size) without touching anything. So I guess this is just a bug, that large memory allocations can fail more often on Windows XP than Windows 7.</p>