title: C++ Memory mapped file data prefetching
body: <p>I'm dealing with very large memory mapped files (200+ GB) which can not be loaded fully into memory, and are accessed randomly. The mapped files are stored on a solid state drive array, but I still find that accessing the needed parts one at a time is very slow unless the same search has been done previously, and the pages used have already been read into memory.</p>
  
  <p>Adding multiple threads to read the variables in the mmap simultaneously improves the speed dramatically, and I was unable to reach an upper bound to the improvement in my testing, but having more than 1000 threads causes openmp to throw resource unavailable errors.</p>
  
  <p>I have also tried madvise to advise the kernel of the specific parts which will be needed (MADV_WILLNEED) but the kernel does not seem to act on the advise quick enough to make a difference.</p>
  
  <p>I'm looking for a way to simultaneously prefetch the parts of the data needed immediately prior to them actually being used. What would be the least resource intensive way to read a variable (or the memory page sized piece of the mapped file containing it) forcing it into memory without blocking on the read.</p>
  
  <p>If blocking can not be avoided, a way to run a much larger number of very light weight threads to do the reading would also work.</p>