title: what is the algorithm complexity difference between batch and stochasitc gradient descent
body: <p>Batch gradient descent is      : <code>θj:=θj+α∑(yi-h(xi))xji</code>     for every <code>j</code>.</p>
  
  <p>Stochastic gradient descent is :</p>
  
  <pre><code>loop
  {  
     for i=1 to m,
     {
         θj:=θj+α(yi-h(xi))xji   for every j.
     }
  }
  </code></pre>
  
  <p>I am curious about the complexity difference between two algorithms. I think the complexity is the same! Both are O(m*n), but batch gradient is m*n, and stochastic is n*m, just a different order, am i right?</p>