title: Maximum Likelihood in Julia
body: <p>I am trying to use maximum likelihood to estimate the normal linear model in Julia. I have used the following code to simulate the process with just an intercept and an anonymous function per the Optim documentation regarding values that do not change:</p>
  
  <pre><code>using Optim
  
  nobs = 500
  nvar = 1
  β = ones(nvar)*3.0
  x = [ones(nobs) randn(nobs,nvar-1)]
  ε = randn(nobs)*0.5
  y = x*β + ε
  
  function LL_anon(X, Y, β, σ)
    -(-length(X)*log(2π)/2 - length(X)*log(σ) - (sum((Y - X*β).^2) / (2σ^2)))
  end
  LL_anon(X,Y, pars) = LL_anon(X,Y, pars...)
  
  res2 = optimize(vars -&gt; LL_anon(x,y, vars...), [1.0,1.0]) # Start values: β=1.0, σ=1.0
  </code></pre>
  
  <p>This actually recovered the parameters and I received the following output:</p>
  
  <pre><code> * Algorithm: Nelder-Mead
   * Starting Point: [1.0,1.0]
   * Minimizer: [2.980587812647935,0.5108406803949835]
   * Minimum: 3.736217e+02
   * Iterations: 47
   * Convergence: true
     *  √(Σ(yᵢ-ȳ)²)/n &lt; 1.0e-08: true
     * Reached Maximum Number of Iterations: false
   * Objective Calls: 92
  </code></pre>
  
  <p>However, when I try and set nvar = 2, i.e. an intercept plus an additional covariate, I get the following error message:</p>
  
  <pre><code>MethodError: no method matching LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64, ::Float64)
  Closest candidates are:
    LL_anon(::Any, ::Any, ::Any, ::Any) at In[297]:2
    LL_anon(::Array{Float64,1}, ::Array{Float64,1}, ::Any, ::Any) at In[113]:2
    LL_anon(::Any, ::Any, ::Any) at In[297]:4
    ...
  
  Stacktrace:
   [1] (::##245#246)(::Array{Float64,1}) at .\In[299]:1
   [2] value!!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:9
   [3] initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:136
   [4] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
   [5] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##245#246}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
   [6] #optimize#148(::Array{Any,1}, ::Function, ::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
   [7] optimize(::Function, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:52
  </code></pre>
  
  <p>I'm not sure why adding an additional variable is causing this issue but it seems like a type instability problem.</p>
  
  <p>The second issue is that when I use my original working example and set the starting values to [2.0,2.0], I get the following error message:</p>
  
  <pre><code>log will only return a complex result if called with a complex argument. Try log(complex(x)).
  
  Stacktrace:
   [1] nan_dom_err at .\math.jl:300 [inlined]
   [2] log at .\math.jl:419 [inlined]
   [3] LL_anon(::Array{Float64,2}, ::Array{Float64,1}, ::Float64, ::Float64) at .\In[302]:2
   [4] (::##251#252)(::Array{Float64,1}) at .\In[304]:1
   [5] value(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\NLSolversBase\src\interface.jl:19
   [6] update_state!(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/solvers/zeroth_order\nelder_mead.jl:193
   [7] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::Optim.NelderMeadState{Array{Float64,1},Float64,Array{Float64,1}}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:51
   [8] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\optimize.jl:25
   [9] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##251#252}, ::Array{Float64,1}) at C:\Users\dolacomb\.julia\v0.6\Optim\src\multivariate/optimize\interface.jl:62
  </code></pre>
  
  <p>Again, I’m not sure why this is happening and since start values are very important I’d like to know how to overcome this issue and they are not too far off from the true values.</p>
  
  <p>Any help would be greatly appreciated!</p>