title: Can one only implement gradient descent like optimizers with the code example from processing gradients in TensorFlow?
body: <p>I was looking at the example code for processing gradients that TensorFlow has:</p>
  
  <pre><code># Create an optimizer.
  opt = GradientDescentOptimizer(learning_rate=0.1)
  
  # Compute the gradients for a list of variables.
  grads_and_vars = opt.compute_gradients(loss, &lt;list of variables&gt;)
  
  # grads_and_vars is a list of tuples (gradient, variable).  Do whatever you
  # need to the 'gradient' part, for example cap them, etc.
  capped_grads_and_vars = [(MyCapper(gv[0]), gv[1]) for gv in grads_and_vars]
  
  # Ask the optimizer to apply the capped gradients.
  opt.apply_gradients(capped_grads_and_vars)
  </code></pre>
  
  <p>however, I noticed that the <code>apply_gradients</code> function was derived from the <code>GradientDescentOptimizer</code>. Does that mean that using the example code from above, one can only implement gradient like descent rules (notice we could change the <code>opt = GradientDescentOptimizer</code> or <code>Adam</code> or any of the the other optimizers)? In particular, what does <code>apply_gradients</code> do? I definitively check the code in the <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/training/optimizer.py" rel="nofollow noreferrer">tf github page</a> but it was a bunch of python that had nothing to do with mathematical expressions, so it was hard to tell what that was doing and how it changed from optimizer to optimizer.</p>
  
  <p>For example, if I wanted to implement my own custom optimizer that might use gradients (or might not e.g. just change the weights directly with some rule, maybe more biologically plausible rule), its not possible with the above example code?</p>
  
  <hr>
  
  <p>In particular I wanted to implement a gradient descent version that is artificially restricted in a compact domain. In particular I wanted to implement the following equation:</p>
  
  <pre><code>w := (w - mu*grad + eps) mod B
  </code></pre>
  
  <p>in TensorFlow. I realized that the following is true:</p>
  
  <pre><code>w := w mod B - mu*grad mod B + eps mod B
  </code></pre>
  
  <p>so I thought that I could just implement it by doing:</p>
  
  <pre><code>def Process_grads(g,mu_noise,stddev_noise,B):
      return (g+tf.random_normal(tf.shape(g),mean=mu_noise,stddev=stddev_noise) ) % B
  </code></pre>
  
  <p>and then just having:</p>
  
  <pre><code>processed_grads_and_vars = [(Process_grads(gv[0]), gv[1]) for gv in grads_and_vars]
  # Ask the optimizer to apply the processed gradients.
  opt.apply_gradients(processed_grads_and_vars)
  </code></pre>
  
  <p>however, I realized that that wasn't good enough because I don't actually have access to <code>w</code> so I can't implement:</p>
  
  <pre><code>w mod B
  </code></pre>
  
  <p>at least not the way I tried. Is there a way to do this? i.e. to actually directly change the update rule? At least the way I tried?</p>
  
  <p>I know its sort of a hacky update rule, but my point is more to change the update equation than actually caring to much about that update rule (so don't get hung up on it if its a bit weird).</p>
  
  <hr>
  
  <p>I came up with super hacky solution:</p>
  
  <pre><code>def manual_update_GDL(arg,learning_rate,g,mu_noise,stddev_noise):
      with tf.variable_scope(arg.mdl_scope_name,reuse=True):
          W_var = tf.get_variable(name='W')
          eps = tf.random_normal(tf.shape(g),mean=mu_noise,stddev=stddev_noise)
          #
          W_new = tf.mod( W_var - learning_rate*g + eps , 20)
          sess.run( W_var.assign(W_new) )
  
  def manual_GDL(arg,loss,learning_rate,mu_noise,stddev_noise,compact,B):
      # Compute the gradients for a list of variables.
      grads_and_vars = opt.compute_gradients(loss)
      # process gradients
      processed_grads_and_vars = [(manual_update_GDL(arg,learning_rate,g,mu_noise,stddev_noise), v) for g,v in grads_and_vars]
  </code></pre>
  
  <p>not sure if it works but something like that should work in general. The idea is to just write down the equation one wants to use (<strong>in TensorFlow</strong>) for the learning rate and then update the weights manually using a session.</p>
  
  <p>Unfortunately, such a solution means we have to take care of the annealing (decaying learning rate manually which seems annoying). This solution probably has many other problems, feel free to point them out (and give solutions if you can).</p>
  
  <hr>
  
  <p>For this very simple problem I realized one can just do the normal optimizer update rule and then just take the mod of the weights and re-assign them to their value:</p>
  
  <pre><code>sess.run(fetches=train_step)
  if arg.compact:
      # apply w := ( w - mu*g + eps ) mod B
      W_val = W_var.eval()
      W_new = tf.mod(W_var,arg.B).eval()
      W_var.assign(W_new).eval()
  </code></pre>
  
  <p>but in this case its a coincidence that such a simple solution exists (unfortunately, bypasses the whole point of my question).</p>
  
  <p>Actually, this solutions slows down the code a lot. For the moment is the best that I've got.</p>
  
  <hr>
  
  <p>As a reference, I have seen this question: <a href="https://stackoverflow.com/questions/38431054/how-to-create-an-optimizer-in-tensorflow">How to create an optimizer in Tensorflow</a> , but didn't find it responded directly to my question.</p>