<p>In my opinion, you are lacking some critical literature review.
Here are some good papers about RNN and CNN that can be used for image recognition appications :</p>

<p><a href="https://pdfs.semanticscholar.org/86ef/e7769f2b8a0e15ca213ab09881e6705caeb0.pdf" rel="nofollow noreferrer">https://pdfs.semanticscholar.org/86ef/e7769f2b8a0e15ca213ab09881e6705caeb0.pdf</a>
<a href="https://arxiv.org/pdf/1506.00019.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/1506.00019.pdf</a></p>

<p>What is a feature? A feature represents one of the elements of the input vector which will be used to train the model and produce output.</p>

<p>The feature set is to be determined depending on the application. 
Each element of the input vector is a different (dependent or independent) feature. </p>

<p>Look at this tutorial for example using the MNIST digit data set:
<a href="https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/recurrent_network.py" rel="nofollow noreferrer">https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/recurrent_network.py</a></p>

<p>It says:
'''
To classify images using a recurrent neural network, we consider every image
row as a sequence of pixels. Because MNIST image shape is 28*28px, we will then
handle 28 sequences of 28 steps for every sample.
'''</p>

<p>The RNN is built on sequences, hence if the image is 28 by 28 you can break it in 28 sequences of 28 features.</p>

<pre><code># Network Parameters
num_input = 28 # MNIST data input (img shape: 28*28)
timesteps = 28 # timesteps
</code></pre>

<p>This is what you see in the network parameters. The 28 <strong>features</strong> (num_input = 28) representing one sequence of the image. </p>

<p>To repeat again, each element of the input vector is considered a feature. Furthermore, is the analyst's responsibility to properly define these features.</p>
What is considered a feature in an Image in machine learning?
<p>I sort of understand what features are, say a ML algorithm that learns SPAM, certain keywords could be a feature? </p>

<p>But in the famous MNIST digits data set, I see a matrix of numbers, is the entire matrix one single feature? Or is a feature each number in the matrix?</p>