title: Is Large Object Heap fragmentation on 64-bit platforms an issue at all compared to 32-bits, on .NET?
body: <p>I extensively use a nested data structure based on parallel arrays (<code>SCG.SortedList&lt;K,V&gt;</code>, looks like <code>Outer&lt;K, Inner&lt;K_delta,V&gt;&gt;</code>) and a calculation engine that copies the structures a lot. The size of the structures could be very different and many of them will be in LOH, because - even though they are already nested - either inner or outer ones will be long enough and the values are mostly doubles (for doubles the limit is 1000 elements per array after which it goes to LOH, at least on x86).</p>
  <p>The use case for the nested data structure is to accumulate high frequency data in a long running process. I could use it as a buffer to aggregate the data and keep only a minimum window required for aggregation (by dropping older inner lists and trimming the outer list), but this copying/trimming itself could create LOH fragmentation problem and take more memory instead of saving it.</p>
  <p>After reading deeper about LOH fragmentation I could reason that in my case I <em>will have LOH fragmentation</em>, because what happens frequently inside my process/calculations is exactly what is described in, e.g., a <a href="https://www.simple-talk.com/dotnet/.net-framework/the-dangers-of-the-large-object-heap/" rel="nofollow noreferrer">great article about LOH fragmentation by Andrew Hunter</a>: create, grow by random increment, copy arrays...</p>
  <p>However, I cannot <em>understand if LOH fragmentation on 64-bit platform is a problem at all?</em> In the last comment to the same article a commenter posits that on 64-bit platformss LOH fragmentation is almost a non-issue because address space is so large that it is hard to exhaust it in reality, while memory holes/empty pages are not backed by real memory. (Another option is that MS failed badly in designing GC).</p>
  <ul>
  <li><p><em>Could some expert please confirm if that statement in the comment is true for .NET managed memory?</em> Is it true that LOH fragmentation on 64-bit systems is not something one should worry about in most of the cases due to extremely large address space? While writing this question I have found <a href="https://stackoverflow.com/a/9908509/801189">a confirmation for C++</a>, so this question is for managed memory in .NET specifically.</p>
  </li>
  <li><p>The second question is <em>how it works for 32-bit processes running on 64-bits systems</em>? Does the same applies for both, or 32-bits processes could run out of memory pretty fast if there is LOH fragmentation? (e.g. for Excel add-ins I am limited to 32-bits architecture even on 64-bits systems, for a long time due to legacy add-ins around)</p>
  </li>
  </ul>
  <p>The comment from the linked article is below:</p>
  <blockquote>
  <p>Page Cache
  Posted by:    TruePath
  Posted on:    Thursday, November 24, 2011 at 6:50 AM
  Message:</p>
  <p>Either you are using an old
  OS/cpu that doesn't support 64 bit addressing (or a cheap version of
  win 7) or MS has radically failed to take proper advantage of 64 bit
  addressing.</p>
  <p>Win 7 ultimate lets a process address up to 192 gigs, snow leopard
  16TB and linux 32 exabytes. You are never going to realistically chew
  through that entire address space in the LOH. Thus the strategy that
  MS implemented for allocation sounds like the right one and it's only
  some artificial limit that is getting in the way.</p>
  <p>Remember just because you've allocated a page of memory doesn't mean
  the OS has to always back that up with a page of RAM. The OS is
  perfectly free to swap pages out to disk or leave blank pages
  unbacked. The LOH should be allocating an integer number of psges (4K)
  for each object. When an object in the LOH is freed by the GC the
  pages allocated to it should be returned to the OS so they no longer
  need any backing store and don't take up disk space or strain the page
  allocation structures.</p>
  <p>Indeed, the LOH on a 64 byte system should be substantially faster and
  less resource intensive than the normsl heaps since no objects are
  ever copied and all objects get an integer number of pages. Unlike the
  regulsr heap fragmentation isn't really an issue since this gets
  handled for you by the hardware supported page tables. Indeed it might
  be optimal to never GC the LOH until virtual memory limits are hit.</p>
  <p>True such a strategy would end up writing huge parts of your LOH out
  to disk but that happens in a second concurrent process. As long as
  you aren't allocating objects in the LOH and dirtying their pages
  faster than your disk can write them you shouldn't see any slowdown
  except a minor competion for disk IO and a small impact of larger OS
  page maps. The system won't thrash because most of those pages really
  are free and thus are never again read back from disk. Without a GC
  marking objects in the LOH no false page access by the GC occurs. Thus
  the LIFO algorithm employed by the page cache is a pretty decent GC
  that trades GC overhead for some disk writes and (maybe) an occasional
  read.</p>
  <p>I guess it would be superior to hold the GC metadata and any embedded
  pointers for objects in the LOH on a separate page from the rest of
  the data (it can precede it if you want and the rest of the page can
  be used for other heaps/metadata) so the GC can still free the pages
  in the LOH without triggering any unneeded page loads. Since objects
  in the LOH have few if any pointer members/elements (or are all
  pointers and must be scanned by the GC anyway) one can segregate those
  from the data and get the best of both worlds: no disk writes and no
  false page loads for the GC.</p>
  </blockquote>
  <p><strong>Update</strong>: Let it be <em>an assumption</em> that there is LOH fragmentation. The question is about the impact the fragmentation of memory address space has on actual memory on x64 platforms, and how it works in 32-bits processes running on x64.</p>
  <p>The question is not about how to avoid it/deal with it and what data structures to use (the article discusses this). I have done a lot of testing to find out that nested sorted lists are 5+ times faster than immutable structures and compress my data by c.40% by using key delta in the inner list (uint16 vs int64), while IntMap/AVL tree take 70/50 bytes overhead per key/value pair. With very realistic 10mn pairs I prefer nested SL. So the expected/possible LOH fragmentation is the price to pay for speed and naive memory compression. I cannot test at full scale yet how much memory is actually &quot;leaking&quot; due to fragmentation, but from what I have read I have reasons to doubt if there are any leaks at all.</p>