title: "Cannot allocate memory" despite free reporting "available"
body: <p>This is a question for a linux kernel or sys admin guru.</p>
  
  <p>I got this error from qemu, trying to start a VM with 3GB memory:</p>
  
  <pre><code>ioctl(KVM_CREATE_VM) failed: 12 Cannot allocate memory
  failed to initialize KVM: Cannot allocate memory
  </code></pre>
  
  <p>To my knowledge this can be because there isn't enough memory or the commit limit is too low, but apparently not... There is 5.9 GB available by dumping caches and no commit limit:</p>
  
  <pre><code>$ free -m
                total        used        free      shared  buff/cache   available
  Mem:           7696        1287         135         139        6274        5973
  Swap:          2892         525        2367
  
  $ cat /proc/sys/vm/overcommit_memory 
  1
  </code></pre>
  
  <p>So then I wrote a c++ program to allocate successively bigger chunks. I found that it failed to allocate more than 2.1 GB. (N.B. it was compiled as 64-bit.) That's consistent with Qemu not starting up, but WHY???</p>
  
  <p>So then I modified it to write to the memory as well. This resulted in some caches being dumped, with free reporting this at about 2 GB allocated:</p>
  
  <pre><code>$ free -m
                total        used        free      shared  buff/cache   available
  Mem:           7696        2988         288         143        4420        4268
  Swap:          2892         525        2367
  </code></pre>
  
  <p>...and this when the program terminated:</p>
  
  <pre><code>$ free -m
                 total        used        free      shared  buff/cache   available
  Mem:           7696        1258        2253         147        4185       5994
  Swap:          2892         525        2367
  </code></pre>
  
  <p>Now I tried to start Qemu and magically it worked! Free reports:</p>
  
  <pre><code>$ free -m
                total        used        free      shared  buff/cache   available
  Mem:           7696        2438        4451         147         806        4834
  Swap:          2892         530        2362
  </code></pre>
  
  <p>So it looks like the kernel wasn't willing to free some caches when asked for a big chunk, but was willing to let them go in several steps. What's going on?</p>
  
  <p>I am running Debian testing:
  Linux - 4.11.0-1-amd64 #1 SMP Debian 4.11.6-1 (2017-06-19) x86_64 GNU/Linux</p>