title: How do process virtual machines implement callback support?
body: <p><strong>Preamble</strong>: This question is asked as a VM implementor, looking to support native callbacks (more specifically wndProc). For Process VMs  I mean for Smalltalk, Java, Python, etc.</p>
  
  <p>VMs are usually very careful of what gets into the stack. This is because during GC they will use the stack as a set of roots, and for each word in the stack they'll have to know whether it is a pointer to an object or not.</p>
  
  <p><strong>So here comes the problem</strong>: during a callback, the VM gets a bunch of stuff pushed into the stack. This may be just the arguments and return address, but might also contain much more things. Here is a little scheme, stack grows upwards:</p>
  
  <pre><code>oop &lt;- managed
  oop &lt;- managed
  -------------- &lt;- native code end
  ... bunch of native stuff ...
  -------------- &lt;- native code start
  oop &lt;- managed
  oop &lt;- managed
  </code></pre>
  
  <p>From the point after the callback is entered, as the callback programmer one can control what gets into the stack again. But the question is, how do you control what happens before? Being more specific, how do you detect the native code start limit?</p>
  
  <p>I'm guessing that the VM has a limit in the kind of callbacks, only allowing callbacks that will arrive when we call out, so the VM can save the limit of the stack at that moment. Is that correct? Is there any other alternative? I think using a separate stack could also be a solution, but I'm not sure of the performance implications.</p>