title: The optim function returns the initial values provided when gradient gr is used
body: <p>I am getting the initial values when I use the gr argument. I get sensible results without it. Why providing the gr argument causes the optim() function to return the initial values. Here is a replication of the problem.
  func_0 the function to optimize and func_1 is its gradient</p>
  
  <pre><code>func_0 &lt;- function(mu, mu0, Sigma_inv0, t_z){
      log_theta = mu - log(sum(exp(mu))) # mu is a Kx1 column vector
      f = t(log_theta)%*%t_z # t_z is a Kx1 column vector
      f = -(f - 0.5*t(mu - mu0)%*%Sigma_inv0%*%(mu - mu0))
      return(f)
      }
  
  func_1 &lt;- function(mu, mu0, Sigma_inv0, t_z){ 
      log_sum = log(sum(exp(mu)))
      theta = exp(mu - log_sum)
      df = t_z - sweep(as.matrix(theta), 1, sum(t_z), '*') - Sigma_inv0%*%(mu - mu0)
      return(df)
      }
  </code></pre>
  
  <p>The initial values are:</p>
  
  <pre><code>Sigma_inv0 = diag(1, 5)
  num_topics_  = 6
  mu0 = c(0,0,0,0,0)
  mu = c(0.5,0,0.75,0.25,1)
  t_z = c(5,4,7,2,10)
  </code></pre>
  
  <p>I apply optim() to these functions as:</p>
  
  <pre><code>optim(par = mu, fn = func_0,
    mu0 = mu0, t_z = t_z, Sigma_inv0 = Sigma_inv0,
    method = "CG")$par
  </code></pre>
  
  <p>And here is the sensible output:</p>
  
  <pre><code>$par
  [1] -0.02084741 -0.19948432  0.27185348 -0.65718095  0.60566111
  
  $value
  [1] 42.2506
  
  $counts
  function gradient 
       78       33 
  
  $convergence
  [1] 0
  
  $message
  NULL
  </code></pre>
  
  <p>Now the same function with the gradient applied:</p>
  
  <pre><code>optim(par = mu, fn = func_0, gr = func_1,
    mu0 = mu0, t_z = t_z, Sigma_inv0 = Sigma_inv0,
    method = "CG")$par
  
  $par
  [1] 0.50 0.00 0.75 0.25 1.00
  
  $value
  [1] 43.47864
  
  $counts
  function gradient 
      24        1 
  
  $convergence
  [1] 0
  
  $message
  NULL
  </code></pre>
  
  <p>What is wrong with using the gradient argument? I just get the initial values. I tried ridiculous initial values, and I still get my initial values returned to me.
  Thanks in advance!</p>
  
  <p>It has been suggested that the gradient may be wrong. I don't think that is the case. Here is a tentative check with a small mu.</p>
  
  <pre><code>&gt; optim(par = mu[1:2], fn = func_0,
  +       mu0 = mu0[1:2], t_z = t_z[1:2], Sigma_inv0 = Sigma_inv0[1:2,1:2],
  +       method = "CG")
  
  $par
  [1]  0.09111349 -0.09111587
  
  $value
  [1] 6.192819
  
  $counts
  function gradient 
       74       31 
  
  $convergence
  [1] 0
  
  $message
  NULL
  </code></pre>
  
  <p>Now, I can compute the gradient, which is zero for the optimum mu.</p>
  
  <pre><code>&gt; func_1(c(0.09111349, -0.09111587), mu0[1:2], Sigma_inv0[1:2,1:2], 
  t_z[1:2])
               [,1]
  [1,] 1.330154e-06
  [2,] 1.049846e-06
  </code></pre>
  
  <p>And this is almost zero.
  However, with a larger mu, the gradient is no longer close to zero. I confess that I do not know the inner work of optim().</p>