title: How to set adaptive learning rate for GradientDescentOptimizer?
body: <p>I am using TensorFlow to train a neural network. This is how I am initializing the <code>GradientDescentOptimizer</code>:</p>
  
  <pre><code>init = tf.initialize_all_variables()
  sess = tf.Session()
  sess.run(init)
  
  mse        = tf.reduce_mean(tf.square(out - out_))
  train_step = tf.train.GradientDescentOptimizer(0.3).minimize(mse)
  </code></pre>
  
  <p>The thing here is that I don't know how to set an update rule for the learning rate or a decay value for that. </p>
  
  <p>How can I use an adaptive learning rate here?</p>