<p>I advise you to have a look at the <a href="https://github.com/marcotcr/lime" rel="nofollow noreferrer">LIME framework</a>:</p>

<blockquote>
  <p>Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin: <a href="https://arxiv.org/abs/1602.04938" rel="nofollow noreferrer">"Why Should I Trust You?": Explaining the Predictions of Any Classifier</a>. 2016.</p>
</blockquote>

<p>It also has <a href="https://www.youtube.com/watch?v=hUnRCxnydCc" rel="nofollow noreferrer">a youtube video explanation</a>.</p>

<p>The very gist / simplified extremely:</p>

<ol>
<li>Make the normal prediction</li>
<li>Remove some words. Make more predictions.</li>
<li>Look at the change in prediction. If the prediction changes more, the word was more important.</li>
</ol>
How to visualize the text analysis using CNN and tensorflow
<p>I am new to tensorflow, I am using CNN model descrined by  <a href="http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/" rel="nofollow noreferrer">http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/</a></p>

<p>I am getting around 60% accuracy for 5 class and 80% accuracy for 2 class classification</p>

<p>Now I want to visualize which word impact most to a particular classification, can anybody tell me how to do it</p>