title: Saving within a foreach loop (R package)?
body: <p>I am using the foreach package to run a monte carlo simulation in parallel using HPC. I have no issue with foreach returning the results I want, but I am wondering if it is possible to periodically save the workspace. The reason I ask is that I have to specify a walltime (after which the job is terminated) and so I risk 1. setting the walltime too low and losing everything before the foreach loop has finished and saved my output or 2. setting a conservative walltime and wasting my quota. Ideally, I'd like to be able to periodically save my output, in which case I still have data if the job terminates.</p>
  
  <p>For example:</p>
  
  <pre><code>results &lt;- foreach (ii = 1:100, .combine = rbind) %dopar% {
   res1 &lt;- ii^2
   res2 &lt;- matrix(sample(1:9,9,replace = F),nrow = 3, ncol = 3)
   if (ii %% 10 == 0){
     save.image("temp_save.RData")
   }
   results &lt;- list(res1,res2)
  }
  save.image("final_save.RData")
  
  &gt; load("~/final_save.RData")
  &gt; results
             [,1]  [,2]     
  result.1   1     Integer,9
  result.2   4     Integer,9
  result.3   9     Integer,9
  result.4   16    Integer,9
  ...
  </code></pre>
  
  <p>but 'temp_save' is empty, presumably because 'results' is only generated once the foreach loop has finished. Is there any way to access and save these data before the foreach loop has completed? Thanks for your help.</p>