title: luajit Memory limit Workaround
body: <p>I've been working on a utility that makes judicious use of tables and I'm faced with a problem:</p>
  
  <p>The dreaded ~2GB mem limit for Luajit.  I'm using luajit 2.0.5.  I'm working with very large flat files.  I've tried forcing garbage collection from lua; it works some but continues to grow and eventually breaks with "not enough memory" </p>
  
  <p>I researched and found one way to get around this is use the luajit ffi interface to allocate memory outside of lua control.  Great!    How exactly?  </p>
  
  <p>I've done some research and found the following link:</p>
  
  <p><a href="https://www.develcuy.com/en/playing-luajit-ffi" rel="nofollow noreferrer">https://www.develcuy.com/en/playing-luajit-ffi</a></p>
  
  <p>This actually shows how to work with the C library; not exactly what I wanted as I was hoping to allocate memory from FFI C interface so that the script did not break. </p>
  
  <p>I looked a little more and found:</p>
  
  <p><a href="https://stackoverflow.com/questions/35155444/why-is-luajits-memory-limited-to-1-2-gb-on-64-bit-platforms">Why is LuaJIT&#39;s memory limited to 1-2 GB on 64-bit platforms?</a></p>
  
  <p>Which looks like RaptorJIT and OpenResty Luajit2 have patches to avoid this entirely. . . . and on top of that. . . looks like maybe the luaJIT beta has it as well.</p>
  
  <p>So the question is what is the smartest way to get past the 2GB limit without a re-write?  I finally have the code working the way I want and it's pretty fast and seems bug free--expect for that pesky 2GB limit which I didn't find till I tested with a 20GB input file.  :-(  </p>