title: Parallel optimizations in SciPy
body: <p>I have a simple function</p>
  
  <pre><code>def square(x, a=1):
      return [x**2 + a, 2*x]
  </code></pre>
  
  <p>I want to minimize it over <code>x</code>, for several parameters <code>a</code>. I currently have loops that, in spirit, do something like this:</p>
  
  <pre><code>In [89]: from scipy import optimize
  
  In [90]: res = optimize.minimize(square, 25, method='BFGS', jac=True)
  
  In [91]: [res.x, res.fun]
  Out[91]: [array([ 0.]), 1.0]
  
  In [92]: l = lambda x: square(x, 2)
  
  In [93]: res = optimize.minimize(l, 25, method='BFGS', jac=True)
  
  In [94]: [res.x, res.fun]
  Out[94]: [array([ 0.]), 2.0]
  </code></pre>
  
  <p>Now, the function is already vectorized</p>
  
  <pre><code>In [98]: square(array([2,3]))
  Out[98]: [array([ 5, 10]), array([4, 6])]
  
  In [99]: square(array([2,3]), array([2,3]))
  Out[99]: [array([ 6, 12]), array([4, 6])]
  </code></pre>
  
  <p>Which means it would probably be much faster to run all the optimizations in parallel rather than looping. Is that something that's easily do-able with SciPy? Or any other 3rd party tool?</p>