title: What is wrong with my Gradient Descent algorithm
body: <p>Hi I'm trying to implement Gradient Descent algorithm for a function: </p>
  
  <p><img src="https://i.stack.imgur.com/lGN65.png" alt="enter image description here"></p>
  
  <p>My starting point for the algorithm is w = (u,v) = (2,2). The learning rate is eta = 0.01 and bound = 10^-14. Here is my MATLAB code: </p>
  
  <pre><code>function [resultTable, boundIter] = gradientDescent(w, iters, bound, eta)
  % FUNCTION [resultTable, boundIter] = gradientDescent(w, its, bound, eta)
  % 
  % DESCRIPTION: 
  % - This function will do gradient descent error minimization for the
  % function E(u,v) = (u*exp(v) - 2*v*exp(-u))^2.
  %
  % INPUTS: 
  % 'w' a 1-by-2 vector indicating initial weights w = [u,v]
  % 'its' a positive integer indicating the number of gradient descent
  % iterations
  % 'bound' a real number indicating an error lower bound
  % 'eta' a positive real number indicating the learning rate of GD algorithm
  %
  % OUTPUTS: 
  % 'resultTable' a iters+1-by-6 table indicating the error, partial
  % derivatives and weights for each GD iteration
  % 'boundIter' a positive integer specifying the GD iteration when the error
  % function got below the given error bound 'bound'
  % 
  
  
  % The error function 
  E = @(u,v) (u*exp(v) - 2*v*exp(-u))^2;
  
  % Partial derivative of E with respect to u 
  pEpu = @(u,v) 2*(u*exp(v) - 2*v*exp(-u))*(exp(v) + 2*v*exp(-u));
  % Partial derivative of E with respect to v 
  pEpv = @(u,v) 2*(u*exp(v) - 2*v*exp(-u))*(u*exp(v) - 2*exp(-u));
  
  % Initialize boundIter
  boundIter = 0;
  % Create a table for holding the results
  resultTable = zeros(iters+1, 6);
  % Iteration number
  resultTable(1, 1) = 0;
  % Error at iteration i
  resultTable(1, 2) = E(w(1), w(2));
  % The value of pEpu at initial w = (u,v)
  resultTable(1, 3) = pEpu(w(1), w(2));
  % The value of pEpv at initial w = (u,v)
  resultTable(1, 4) = pEpv(w(1), w(2));
  % Initial u
  resultTable(1, 5) = w(1);
  % Initial v
  resultTable(1, 6) = w(2);
  
  % Loop all the iterations
  for i = 2:iters+1
  
      % Save the iteration number
      resultTable(i, 1) = i-1; 
      % Update the weights
      temp1 = w(1) - eta*(pEpu(w(1), w(2)));
      temp2 = w(2) - eta*(pEpv(w(1), w(2)));
      w(1) = temp1;
      w(2) = temp2;
      % Evaluate the error function at new weights
      resultTable(i, 2) = E(w(1), w(2));
      % Evaluate pEpu at the new point 
      resultTable(i, 3) = pEpu(w(1), w(2));
      % Evaluate pEpv at the new point
      resultTable(i, 4) = pEpv(w(1), w(2));
      % Save the new weights
      resultTable(i, 5) = w(1);
      resultTable(i, 6) = w(2);
      % If the error function is below a specified bound save this iteration
      % index
      if E(w(1), w(2)) &lt; bound
          boundIter = i-1;
      end
  
  end
  </code></pre>
  
  <p>This is an exercise in my machine learning course, but for some reason my results are all wrong. There must be something wrong in the code. I have tried debugging and debugging it and haven't found anything wrong...can someone identify what is my problem here?...In other words can you check that the code is valid gradient descent algorithm for the given function? </p>
  
  <p>Please let me know if my question is too unclear or if you need more info :) </p>
  
  <p>Thank you for your effort and help! =) </p>
  
  <p>Here is my results for five iterations and what other people got: </p>
  
  <blockquote>
    <p>PARAMETERS: w = [2,2], eta = 0.01, bound = 10^-14, iters = 5</p>
  </blockquote>
  
  <p><img src="https://i.stack.imgur.com/z5NbC.png" alt="enter image description here"></p>