<p>There is another way you can visualize activations of your hidden layers, as described in the this paper : <a href="http://arxiv.org/pdf/1506.06579.pdf" rel="nofollow noreferrer">http://arxiv.org/pdf/1506.06579.pdf</a></p>

<p>Check the following post for how it is implemented in MNIST dataset</p>

<p><a href="https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4" rel="nofollow noreferrer">https://medium.com/@awjuliani/visualizing-neural-network-layer-activation-tensorflow-tutorial-d45f8bf7bbc4</a></p>

<p>Do let me know in the comments if you need any further clarification.</p>
Is there a way to reverse a graph in TensorFlow?
<p>As the title says, I'm looking for a way to reverse the flow of a TensorFlow graph. The reason for this, is that I want to visualize the hidden layers of the graph given a logit vector for the output of the trained graph.</p>

<p>For example, say that I have a fully connected graph given as follows (inspired by MNIST):</p>

<pre><code>inputs = tf.placeholder(dtype=tf.float32, shape=[None, 784])
hidden_w1 = tf.get_variable('w1', [784,100], initializer=tf.random_normal_initializer)
hidden_b1 = tf.get_variable('b1', [100], initializer=tf.random_normal_initializer)
a1 = tf.matmul(inputs, hidden_w1) + hidden_b1
z1 = tf.nn.relu(a1)

hidden_w2 = tf.get_variable('w2', [100,100], initializer=tf.random_normal_initializer)
hidden_b2 = tf.get_variable('b2', [100], initializer=tf.random_normal_initializer)
a2 = tf.matmul(z1, hidden_w2) + hidden_b2
z2 = tf.nn.relu(a2)

output_w = tf.get_variable('w3', [100,10], initializer=tf.random_normal_initializer)
output_b = tf.get_variable('b3', [10], initializer=tf.random_normal_initializer)
a3 = tf.matmul(z2, output_w) + output_b
output = tf.nn.relu(a3)

loss = tf.nn.softmax_cross_entropy_with_logits_v2(labels=..., logits=output)
train_op = tf.train.AdamOptimizer().minimize(loss)
</code></pre>

<p>Suppose I now train this graph and want to visualize hidden 1 when only the first output neuron is activated. The way I would do this would be to reverse the flow of the graph and feed a tensor <code>[1, 0, 0, 0, 0, 0, 0, 0, 0, 0]</code> from the output layer back through the reversed graph until I finally got the output of the hidden1 layer. I have tried to see if there is a way to do this in TensorFlow, but there seems to be little information about this. The way I would inutitively construct it is to add an operation sess.run_reverse() when running the graph as follows:</p>

<pre><code>with tf.Session() as sess:
while training:
    sess.run(train_op, feed_dict={inputs:...})

# finished training, reverse graph
category_to_visualize = tf.one_hot(indices=0, depth=10)
sess.run_reverse(hidden1, feed_dict={output:category_to_visualize})
</code></pre>

<p>If this sort of operation doesn't exist or even is possible to get, however, I would instead construct separate operators for reversing the flow of the graph as follows:</p>

<pre><code>output_reversed = tf.placeholder(dtype=tf.float32, shape=[1,10])
z3_reversed = tf.nn.relu(output_reversed)
a3_reversed = tf.matrix_inverse(output_w)*(z3_reversed - output_b)

z2_reversed = tf.nn.relu(a3_reversed)
a2_reversed = tf.matrix_inverse(hidden_w2)*(z2_reversed - hidden_b2)

z1_reversed = tf.nn.relu(a2_reversed)
a1_reversed = tf.matrix_inverse(hidden_w1)*(z1_reversed - hidden_b1)
</code></pre>

<p>I realize that there might be logical flaws to this method that wouldn't make it possible. A couple of things I've overlooked is singular matrices and undefined inversion of ReLu when input is below 0 (ReLu, though, can be replaced by sigmoid for theoretically defined inversion of the entire input space). The core idea, though, is to visualize a feature map given a category - something I believe should be possible if a few assumptions are allowed.</p>

<p>Anyways, please tell me if I'm thinking wrongly here, and if there is a way to reverse the graph!</p>