title: how to get the 'mathematical' gradient of output w.r.t neural network parameters and compute cosine between two such gradients
body: <p>I have a neural network <code>out = f(s)</code>, where <code>s</code> is a batch of input with shape<code>[None, dim_s]</code>, while out is a scaler, <code>f</code> is simply a MLP. With <code>tf.gradient(out, tf.trainable_variables())</code> I can get gradient of <code>out</code> w.r.t neural network parameters of <code>f</code>, which is a list of gradient.</p>
  
  <p>Now, I have two different batch of <code>s</code>: <code>s1</code> and <code>s2</code>, then we can get two different the above gradients <code>G1</code> and <code>G2</code>.</p>
  
  <p>For example:</p>
  
  <pre><code>import tensorflow as tf
  import numpy as np
  
  batch_1=np.random.normal(0,1, [2, 3])
  batch_2=np.random.normal(0,1, [2, 3])
  x = tf.placeholder(tf.float32, shape=(None, 3))
  
  out = tf.layers.dense(x, 2, tf.tanh,
                                kernel_initializer=tf.random_normal_initializer(
                                    stddev=np.sqrt(1 / 100)))
  predictions = tf.layers.dense(out, 1, tf.tanh,
                                kernel_initializer=tf.random_normal_initializer(
                                    stddev=np.sqrt(1 / 100)))
  
  opt = tf.train.GradientDescentOptimizer(learning_rate=0.01)
  gradient_step = opt.compute_gradients(predictions, tf.trainable_variables())
  
  sess=tf.Session()
  sess.run(tf.global_variables_initializer())
  gradients_1 = sess.run(gradient_step, feed_dict={x: batch_1})
  gradients_2 = sess.run(gradient_step, feed_dict={x: batch_2})
  </code></pre>
  
  <p>If it is possible to compute cosine between <code>G1</code> and <code>G2</code>? Do I need to flatten both gradients first? Do <code>G1</code> and <code>G2</code> are the usual gradient in math? </p>