title: Clarification in the Theano tutorial
body: <p>I am reading <a href="http://nbviewer.ipython.org/github/craffel/theano-tutorial/blob/master/Theano%20Tutorial.ipynb" rel="noreferrer">this tutorial</a> provided on the <a href="http://deeplearning.net/software/theano/index.html" rel="noreferrer">home page of Theano documentation</a></p>
  
  <p>I am not sure about the code given under the gradient descent section.</p>
  
  <p><img src="https://i.stack.imgur.com/Vu9t4.png" alt="enter image description here"></p>
  
  <p><strong>I have doubts about the for loop</strong>.</p>
  
  <p>If you initialize the '<strong>param_update</strong>' variable to zero.</p>
  
  <pre><code>param_update = theano.shared(param.get_value()*0., broadcastable=param.broadcastable)
  </code></pre>
  
  <p>and then you update its value in the remaining two lines.</p>
  
  <pre><code>updates.append((param, param - learning_rate*param_update))
  updates.append((param_update, momentum*param_update + (1. - momentum)*T.grad(cost, param)))
  </code></pre>
  
  <p>Why do we need it?</p>
  
  <p>I guess I am getting something wrong here.
  Can you guys help me!</p>