title: Vectorizing a gradient descent algorithm
body: <p>I am coding gradient descent in matlab.
  For two features, I get for the update step:</p>
  
  <blockquote>
  <pre><code>temp0 = theta(1,1) - (alpha/m)*sum((X*theta-y).*X(:,1));
  temp1 = theta(2,1) - (alpha/m)*sum((X*theta-y).*X(:,2));
  theta(1,1) = temp0;
  theta(2,1) = temp1;
  </code></pre>
  </blockquote>
  
  <p>However, I want to vectorize this code and to be able to apply it to any number of features.
  For the vectorization part, it <a href="https://stackoverflow.com/questions/20735406/vectorization-of-a-gradient-descent-code">shows</a> that what I am trying to do is a matrix multiplication</p>
  
  <blockquote>
  <pre><code>theta = theta - (alpha/m) * (X' * (X*theta-y));
  </code></pre>
  </blockquote>
  
  <p>This is well seen, but when I tried, I realized that it doesn't work for gradient descent because the parameters are not updated simultaneously. </p>
  
  <p>Then, how can I vectorize this code and make sure the parameters and updated at the same time?</p>