<p>You are looking for some kind of generative model. RNNs are commonly used, and there's a great blog post <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/" rel="nofollow noreferrer">here</a> demonstrating character-by-character text generation.</p>

<p>The same principle can be applied to any ordered sequence. You talk about an image as being a sequence of pixels, but images have an intrinsic 2D structure (3 if you include color) that would be lost if you took the exact same approach as text generation. A couple of ideas:</p>

<ol>
<li>Use tensorflow's <a href="https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/GridLSTMCell" rel="nofollow noreferrer">GridLSTMCell</a>s</li>
<li>Treat a column of pixels as a single element of the sequence and predict column-by-column (or row by row)</li>
<li>Combine idea 2 with some 1D convolutions along the column/row</li>
<li>Use features from a section of an image as the seed to a <a href="https://arxiv.org/abs/1406.2661" rel="nofollow noreferrer">generative adversarial network</a>. See <a href="https://github.com/jackd/gan_tf" rel="nofollow noreferrer">this repository</a> for basic implementations.</li>
</ol>
How to predict the next pixel given all previous pixels in an image using tensorflow
<p>Just as a project, I wanted to see if it would be possible to predict the next pixel in an image given all previous pixels.<br>
For example: lets say I have an image with <em>x</em> pixels. Given the first <em>y</em> pixels, I want to be able to somewhat accurately predict the <em>y+1th</em> pixel. How should I go about solving this problem.</p>