<p>I am the author of the R package <b>optimParallel</b>. It provides parallel versions of the gradient-based optimization methods of <code>optim()</code>. The main function of the package is <code>optimParallel()</code>, which has the same usage and output as <code>optim()</code>. Using <code>optimParallel()</code> can significantly reduce optimization times as illustrated in the following figure (<code>p</code> is the number of paramters).</p>

<p><a href="https://i.stack.imgur.com/XIjk9.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/XIjk9.png" alt="enter image description here"></a></p>

<p>See <a href="https://cran.r-project.org/package=optimParallel" rel="nofollow noreferrer">https://cran.r-project.org/package=optimParallel</a> and <a href="http://arxiv.org/abs/1804.11058" rel="nofollow noreferrer">http://arxiv.org/abs/1804.11058</a> for more information. </p>
Parallelized optimization in R
<p>I'm running R on linux box that has 8 multicore processors, and have an optimization problem I'd like to speed up by parallelizing the optimization routine itself. Importantly, this problem involves (1) multiple parameters, and (2) <em>inherently slow</em> model runs.  A fairly common problem! </p>

<p>Anyone know of a <em>parallelized optimizer</em> for such occasions?</p>

<p>More specifically, solvers like <code>nlm()</code> run multiple model evaluations (two per parameter value) each time the algorithm takes a step in parameter space, so parallelizing that instance of multiple model runs would greatly speed things up in these situations when more than a few parameter values are being fit. </p>

<p>It seems like code that makes use of the package <code>parallel</code> could be written in a way that the user would have to do <em>minimal</em> code modification to move from using <code>nlm()</code> or <code>optim()</code> to this parallelized optimization routine. That is, it seems one could rewrite these routines basically with no changes, except that the step of calling the model multiple times, as is common in gradient-based methods, would be done in parallel.</p>

<p>Ideally, something like nlmPara() would take code that looks like</p>

<pre><code>fit &lt;- nlm(MyObjFunc, params0);
</code></pre>

<p>and require only minor modifications, e.g., </p>

<pre><code>fit &lt;- nlmPara(MyObjFunc, params0, ncores=6);
</code></pre>

<p>Thoughts/suggestions?  </p>

<p>PS: I've taken steps to speed up those model runs, but they're slow for a variety of reasons (i.e. I don't need advice on speeding up the model runs! ;-) ). </p>