<p dir="auto">Hello,</p>
<p dir="auto">I came across this minor inconsistency in shape when using different optimization methods provided by the <code class="notranslate">minimize</code> interface to optimize 1D scalar functions:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [1]: from scipy.optimize import minimize

In [2]: import numpy as np

In [3]: x = np.linspace(0, 10, 100)

In [4]: y = x + np.random.normal(size=len(x), scale=3)

In [5]: def affine_function(theta):
   ...:   return x * theta

In [5]: chi_sqr = lambda theta: np.sum((affine_function(theta) - y) ** 2)

In [6]: result_nm = minimize(chi_sqr, x0=1, method='Nelder-Mead')

In [6]: result_nm.x
Out[6]: array([ 1.17421875])

In [7]: result_p = minimize(chi_sqr, x0=1, method='Powell')

In [7]: result_p.x
Out[7]: array(1.1742389466088772)"><pre class="notranslate"><code class="notranslate">In [1]: from scipy.optimize import minimize

In [2]: import numpy as np

In [3]: x = np.linspace(0, 10, 100)

In [4]: y = x + np.random.normal(size=len(x), scale=3)

In [5]: def affine_function(theta):
   ...:   return x * theta

In [5]: chi_sqr = lambda theta: np.sum((affine_function(theta) - y) ** 2)

In [6]: result_nm = minimize(chi_sqr, x0=1, method='Nelder-Mead')

In [6]: result_nm.x
Out[6]: array([ 1.17421875])

In [7]: result_p = minimize(chi_sqr, x0=1, method='Powell')

In [7]: result_p.x
Out[7]: array(1.1742389466088772)
</code></pre></div>
<p dir="auto">I would expect <code class="notranslate">result_nm.x</code> and <code class="notranslate">result_p.x</code> to be of the same shape.</p>
<p dir="auto">PS: that divergence seems not exist when minimizing higher dimension scalar functions.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [8]: import scipy; scipy.__version__
Out[8]: '0.19.1'"><pre class="notranslate"><code class="notranslate">In [8]: import scipy; scipy.__version__
Out[8]: '0.19.1'
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="In [9]: import numpy; numpy.__version__
Out[9]: '1.13.1'"><pre class="notranslate"><code class="notranslate">In [9]: import numpy; numpy.__version__
Out[9]: '1.13.1'
</code></pre></div>
<p dir="auto">Cheers,<br>
Zé</p>
<p dir="auto">For 1D problems some optimizers return the result as a scalar (Powell, COBYLA), some<br>
as a list with one element.</p>
<p dir="auto">There are other optimizers that need the Jacobian, I did not check them.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="from scipy.optimize import minimize, rosen

o = ['Nelder-Mead', 'Powell', 'CG',  'BFGS', 'L-BFGS-B',
      'TNC', 'COBYLA', 'SLSQP']

for k in o:

    x0 = [1.3] #, 0.7, 0.8, 1.9, 1.2]
    res = minimize(rosen, x0, method=k, tol=1e-6)

    print(k, res.x)

print()

for k in o:

    x0 = 1.3 #, 0.7, 0.8, 1.9, 1.2]
    res = minimize(rosen, x0, method=k, tol=1e-6)

    print(k, res.x)

print()

for k in o:

    x0 = [1.3, 0.7]
    res = minimize(rosen, x0, method=k, tol=1e-6)

    print(k, res.x)
"><pre class="notranslate"><span class="pl-k">from</span> <span class="pl-s1">scipy</span>.<span class="pl-s1">optimize</span> <span class="pl-k">import</span> <span class="pl-s1">minimize</span>, <span class="pl-s1">rosen</span>

<span class="pl-s1">o</span> <span class="pl-c1">=</span> [<span class="pl-s">'Nelder-Mead'</span>, <span class="pl-s">'Powell'</span>, <span class="pl-s">'CG'</span>,  <span class="pl-s">'BFGS'</span>, <span class="pl-s">'L-BFGS-B'</span>,
      <span class="pl-s">'TNC'</span>, <span class="pl-s">'COBYLA'</span>, <span class="pl-s">'SLSQP'</span>]

<span class="pl-k">for</span> <span class="pl-s1">k</span> <span class="pl-c1">in</span> <span class="pl-s1">o</span>:

    <span class="pl-s1">x0</span> <span class="pl-c1">=</span> [<span class="pl-c1">1.3</span>] <span class="pl-c">#, 0.7, 0.8, 1.9, 1.2]</span>
    <span class="pl-s1">res</span> <span class="pl-c1">=</span> <span class="pl-en">minimize</span>(<span class="pl-s1">rosen</span>, <span class="pl-s1">x0</span>, <span class="pl-s1">method</span><span class="pl-c1">=</span><span class="pl-s1">k</span>, <span class="pl-s1">tol</span><span class="pl-c1">=</span><span class="pl-c1">1e-6</span>)

    <span class="pl-en">print</span>(<span class="pl-s1">k</span>, <span class="pl-s1">res</span>.<span class="pl-s1">x</span>)

<span class="pl-en">print</span>()

<span class="pl-k">for</span> <span class="pl-s1">k</span> <span class="pl-c1">in</span> <span class="pl-s1">o</span>:

    <span class="pl-s1">x0</span> <span class="pl-c1">=</span> <span class="pl-c1">1.3</span> <span class="pl-c">#, 0.7, 0.8, 1.9, 1.2]</span>
    <span class="pl-s1">res</span> <span class="pl-c1">=</span> <span class="pl-en">minimize</span>(<span class="pl-s1">rosen</span>, <span class="pl-s1">x0</span>, <span class="pl-s1">method</span><span class="pl-c1">=</span><span class="pl-s1">k</span>, <span class="pl-s1">tol</span><span class="pl-c1">=</span><span class="pl-c1">1e-6</span>)

    <span class="pl-en">print</span>(<span class="pl-s1">k</span>, <span class="pl-s1">res</span>.<span class="pl-s1">x</span>)

<span class="pl-en">print</span>()

<span class="pl-k">for</span> <span class="pl-s1">k</span> <span class="pl-c1">in</span> <span class="pl-s1">o</span>:

    <span class="pl-s1">x0</span> <span class="pl-c1">=</span> [<span class="pl-c1">1.3</span>, <span class="pl-c1">0.7</span>]
    <span class="pl-s1">res</span> <span class="pl-c1">=</span> <span class="pl-en">minimize</span>(<span class="pl-s1">rosen</span>, <span class="pl-s1">x0</span>, <span class="pl-s1">method</span><span class="pl-c1">=</span><span class="pl-s1">k</span>, <span class="pl-s1">tol</span><span class="pl-c1">=</span><span class="pl-c1">1e-6</span>)

    <span class="pl-en">print</span>(<span class="pl-s1">k</span>, <span class="pl-s1">res</span>.<span class="pl-s1">x</span>)</pre></div>