title: Keras - coding a custom optimizer and attempting to compute a second gradient inside get_updates
body: <p>I am a researcher in optimization and I trying to write a custom optimizer. I have come across a problem. I have asked in many places and so far no response. </p>
  
  <p>Take any optimizer code, say just copy SGD. In the beginning of get_updates, you see</p>
  
  <pre><code>grads = self.get_gradients(loss, params)
  </code></pre>
  
  <p>now add the following line right after this one:</p>
  
  <pre><code>gradsb = self.get_gradients(loss, [tf.Variable(a) for a in params])
  </code></pre>
  
  <p>this should compute the gradients at a new tensor, with all the values the same as before</p>
  
  <p>now try to see what you get:</p>
  
  <pre><code>for a in gradsb:
     print(a)
  </code></pre>
  
  <p>you get a list of Nones (but if you print the list grads you see that they are still Tensors)</p>
  
  <p>Why?
  And how to circumvent this problem? This is important as I'd like to compute the gradients at another point for my algorithm.</p>