title: Available stack size is not used by R, returning "Error: node stack overflow"
body: <p>I have written a recursive code in R.
  Before invoking R, I set the stack size to 96 MB at the shell with:</p>
  
  <pre><code>ulimit -s 96000
  </code></pre>
  
  <p>I invoked R with maximum protection pointer stack size of 500000 with:</p>
  
  <pre><code>R --max-ppsize 500000
  </code></pre>
  
  <p>And I changed the maximum recursion depth to 500000:</p>
  
  <pre><code>options(expression = 500000)
  </code></pre>
  
  <p>I both used the binary R package at Arch Linux repositories (without memory profiling) and also a binary compiled by me with memory profiling option. Both are of version 3.4.2</p>
  
  <p>I used two versions of the code with and without gc().</p>
  
  <p>The problem is that R exits the code with "node stack overflow" error while only 16 MB of the 93 MB of total available stack is used and depth is just below one percent of the expressions option of 5e5:</p>
  
  <pre><code>      size    current  direction eval_depth
    93388800   16284704          1       4958
  
  Error: node stack overflow
  </code></pre>
  
  <p>The current stack usage change between the last two iterations were around 10K. The only passed and saved object is a numeric vector of 19 items.</p>
  
  <p>The recursive portion of the code is below:</p>
  
  <pre><code>network_recursive &lt;- function(called)
  {
      print(Cstack_info())
      callers &lt;- list_caller[[called + 1]] # get the callers of the called
      callers &lt;- callers[!bool[callers + 1]] # subset for nofriends - new friends
      new_friend_no &lt;- length(callers) # number of new friends
      print(list(called, callers) )
      if (new_friend_no &gt; 0) # if1 still new friends
      {
          friends &lt;&lt;- friends + new_friend_no # increment friend no
          print(friends)
          bool[callers + 1] &lt;&lt;- T # toggle friends
          sapply(callers, network_recursive) # recurse network control
      } # close if1
      print("end of recursion")
  }
  </code></pre>
  
  <p>What may be the reason for this stack overflow?</p>
  
  <p>Some notes on the R source code, related to the issue.</p>
  
  <p>The portion of the code that triggers the error is lines 5987-5988 from src/main/eval.c:</p>
  
  <pre><code>5975 #ifdef USE_BINDING_CACHE
  5976   if (useCache) {
  5977       R_len_t n = LENGTH(constants);
  5978 # ifdef CACHE_MAX
  5979       if (n &gt; CACHE_MAX) {
  5980       n = CACHE_MAX;
  5981       smallcache = FALSE;
  5982       }
  5983 # endif
  5984 # ifdef CACHE_ON_STACK
  5985       /* initialize binding cache on the stack */
  5986       vcache = R_BCNodeStackTop;
  5987       if (R_BCNodeStackTop + n &gt; R_BCNodeStackEnd)
  5988       nodeStackOverflow();
  5989       while (n &gt; 0) {
  5990       SETSTACK(0, R_NilValue);
  5991       R_BCNodeStackTop++;
  5992       n--;
  5993       }
  5994 # else
  5995       /* allocate binding cache and protect on stack */
  5996       vcache = allocVector(VECSXP, n);
  5997       BCNPUSH(vcache);
  5998 # endif
  5999   }
  6000 #endif
  </code></pre>