<blockquote>
  <p>Is it possible to determine the features of the image from the hidden layers that will lead to "yes"? </p>
</blockquote>

<p>Yes, it is. Have a look at</p>

<blockquote>
  <p>Zeiler, M.D. and Fergus, R., 2014, September. <a href="https://arxiv.org/abs/1311.2901" rel="nofollow noreferrer">Visualizing and understanding convolutional networks</a>. In European Conference on Computer Vision (pp. 818-833). Springer International Publishing.</p>
</blockquote>

<h2>Summary</h2>

<p>There are three main ideas:</p>

<ol>
<li><em>Training data argmax method</em>: Pump your data through the network. Record for the neuron which you are interested which caused the highest activation.</li>
<li><em>Occlusion sensitivity analysis</em>: Cover a part of the image. Push the occluded image through the network. How did the score change? If it was about the same, the important features are likely not in that part of the image.</li>
<li><em>Gradient methods</em>: Train a "reconstruction network" which reconstructs the activation. Then set the neuron you are interested in to maximum activation, the rest to no activation. Reconstuct what could cause this behavior.</li>
</ol>
I am trying out a yes/no classification of an image using CNN.
<p>Is it possible to determine the features of the image from the hidden layers that will lead to "yes"?
Like suppose I train the CNN with 1000 images, then I would like to know from the intermediate hidden layers about which features actually are leading to the image being tagged with a yes finally.
Is it possible?
And also how many training examples are required to converge for a binary classification using CNN?</p>