<h2>Dynamic placeholders</h2>

<p>Tensorflow allows to have <em>multiple</em> dynamic (a.k.a. <code>None</code>) dimensions in placeholders. The engine won't be able to ensure correctness while the graph is built, hence the client is responsible for feeding the correct input, but it provides a lot of flexibility.</p>

<p>So I'm going from...</p>

<pre class="lang-py prettyprint-override"><code>x = tf.placeholder(tf.float32, shape=[None, N*M*P])
y_ = tf.placeholder(tf.float32, shape=[None, N*M*P, 3])
...
x_image = tf.reshape(x, [-1, N, M, P, 1])
</code></pre>

<p>to...</p>

<pre class="lang-py prettyprint-override"><code># Nearly all dimensions are dynamic
x_image = tf.placeholder(tf.float32, shape=[None, None, None, None, 1])
label = tf.placeholder(tf.float32, shape=[None, None, 3])
</code></pre>

<p>Since you intend to reshape the input to 5D anyway, so why don't use 5D in <code>x_image</code> right from the start. At this point, the second dimension of <code>label</code> is arbitrary, but we <em>promise</em> tensorflow that it will match with <code>x_image</code>.</p>

<h2>Dynamic shapes in deconvolution</h2>

<p>Next, the nice thing about <a href="https://www.tensorflow.org/api_docs/python/tf/nn/conv3d_transpose" rel="noreferrer"><code>tf.nn.conv3d_transpose</code></a> is that its output shape can be dynamic. So instead of this:</p>

<pre class="lang-py prettyprint-override"><code># Hard-coded output shape
DeConnv1 = tf.nn.conv3d_transpose(layer1, w, output_shape=[1,32,32,7,1], ...)
</code></pre>

<p>... you can do this:</p>

<pre class="lang-py prettyprint-override"><code># Dynamic output shape
DeConnv1 = tf.nn.conv3d_transpose(layer1, w, output_shape=tf.shape(x_image), ...)
</code></pre>

<p>This way the transpose convolution can be applied to <em>any</em> image and the result will take the shape of <code>x_image</code> that was actually passed in at runtime. </p>

<p>Note that static shape of <code>x_image</code> is <code>(?, ?, ?, ?, 1)</code>.</p>

<h2>All-Convolutional network</h2>

<p>Final and most important piece of the puzzle is to make <strong>the whole network</strong> convolutional, and that includes your final dense layer too. Dense layer <em>must</em> define its dimensions statically, which forces the whole neural network fix input image dimensions.</p>

<p>Luckily for us, Springenberg at al describe a way to replace an FC layer with a CONV layer in <a href="https://arxiv.org/abs/1412.6806" rel="noreferrer">"Striving for Simplicity: The All Convolutional Net"</a> paper. I'm going to use a convolution with 3 <code>1x1x1</code> filters (see also <a href="https://stats.stackexchange.com/q/194142/130598">this question</a>):</p>

<pre class="lang-py prettyprint-override"><code>final_conv = conv3d_s1(final, weight_variable([1, 1, 1, 1, 3]))
y = tf.reshape(final_conv, [-1, 3])
</code></pre>

<p>If we ensure that <code>final</code> has the same dimensions as <code>DeConnv1</code> (and others), it'll make <code>y</code> right the shape we want: <code>[-1, N * M * P, 3]</code>.</p>

<h2>Combining it all together</h2>

<p>Your network is pretty large, but all deconvolutions basically follow the same pattern, so I've simplified my <em>proof-of-concept</em> code to just one deconvolution. The goal is just to show what kind of network is able to handle images of arbitrary size. Final remark: image dimensions can vary <em>between</em> batches, but within one batch they have to be the same.</p>

<p>The full code:</p>

<pre class="lang-py prettyprint-override"><code>sess = tf.InteractiveSession()

def conv3d_dilation(tempX, tempFilter):
  return tf.layers.conv3d(tempX, filters=tempFilter, kernel_size=[3, 3, 1], strides=1, padding='SAME', dilation_rate=2)

def conv3d(tempX, tempW):
  return tf.nn.conv3d(tempX, tempW, strides=[1, 2, 2, 2, 1], padding='SAME')

def conv3d_s1(tempX, tempW):
  return tf.nn.conv3d(tempX, tempW, strides=[1, 1, 1, 1, 1], padding='SAME')

def weight_variable(shape):
  initial = tf.truncated_normal(shape, stddev=0.1)
  return tf.Variable(initial)

def bias_variable(shape):
  initial = tf.constant(0.1, shape=shape)
  return tf.Variable(initial)

def max_pool_3x3(x):
  return tf.nn.max_pool3d(x, ksize=[1, 3, 3, 3, 1], strides=[1, 2, 2, 2, 1], padding='SAME')

x_image = tf.placeholder(tf.float32, shape=[None, None, None, None, 1])
label = tf.placeholder(tf.float32, shape=[None, None, 3])

W_conv1 = weight_variable([3, 3, 1, 1, 32])
h_conv1 = conv3d(x_image, W_conv1)
# second convolution
W_conv2 = weight_variable([3, 3, 4, 32, 64])
h_conv2 = conv3d_s1(h_conv1, W_conv2)
# third convolution path 1
W_conv3_A = weight_variable([1, 1, 1, 64, 64])
h_conv3_A = conv3d_s1(h_conv2, W_conv3_A)
# third convolution path 2
W_conv3_B = weight_variable([1, 1, 1, 64, 64])
h_conv3_B = conv3d_s1(h_conv2, W_conv3_B)
# fourth convolution path 1
W_conv4_A = weight_variable([3, 3, 1, 64, 96])
h_conv4_A = conv3d_s1(h_conv3_A, W_conv4_A)
# fourth convolution path 2
W_conv4_B = weight_variable([1, 7, 1, 64, 64])
h_conv4_B = conv3d_s1(h_conv3_B, W_conv4_B)
# fifth convolution path 2
W_conv5_B = weight_variable([1, 7, 1, 64, 64])
h_conv5_B = conv3d_s1(h_conv4_B, W_conv5_B)
# sixth convolution path 2
W_conv6_B = weight_variable([3, 3, 1, 64, 96])
h_conv6_B = conv3d_s1(h_conv5_B, W_conv6_B)
# concatenation
layer1 = tf.concat([h_conv4_A, h_conv6_B], 4)
w = tf.Variable(tf.constant(1., shape=[2, 2, 4, 1, 192]))
DeConnv1 = tf.nn.conv3d_transpose(layer1, filter=w, output_shape=tf.shape(x_image), strides=[1, 2, 2, 2, 1], padding='SAME')

final = DeConnv1
final_conv = conv3d_s1(final, weight_variable([1, 1, 1, 1, 3]))
y = tf.reshape(final_conv, [-1, 3])
cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=label, logits=y))

print('x_image:', x_image)
print('DeConnv1:', DeConnv1)
print('final_conv:', final_conv)

def try_image(N, M, P, B=1):
  batch_x = np.random.normal(size=[B, N, M, P, 1])
  batch_y = np.ones([B, N * M * P, 3]) / 3.0

  deconv_val, final_conv_val, loss = sess.run([DeConnv1, final_conv, cross_entropy],
                                              feed_dict={x_image: batch_x, label: batch_y})
  print(deconv_val.shape)
  print(final_conv.shape)
  print(loss)
  print()

tf.global_variables_initializer().run()
try_image(32, 32, 7)
try_image(16, 16, 3)
try_image(16, 16, 3, 2)
</code></pre>
Tensorflow Convolution Neural Network with different sized images
<p>I am attempting to create a deep CNN that can classify each individual pixel in an image. I am replicating architecture from the image below taken from <a href="https://github.com/dhasl002/Research-DeepLearning/blob/master/DEEP.pdf" rel="noreferrer">this</a> paper. In the paper it is mentioned that deconvolutions are used so that any size of input is possible. This can be seen in the image below. </p>

<p><a href="https://github.com/dhasl002/Research-DeepLearning" rel="noreferrer">Github Repository</a></p>

<p><a href="https://i.stack.imgur.com/VgITR.png" rel="noreferrer"><img src="https://i.stack.imgur.com/VgITR.png" alt="enter image description here"></a></p>

<p>Currently, I have hard coded my model to accept images of size 32x32x7, but I would like to accept any size of input. <strong>What changes would I need to make to my code to accept variable sized input?</strong></p>

<pre><code> x = tf.placeholder(tf.float32, shape=[None, 32*32*7])
 y_ = tf.placeholder(tf.float32, shape=[None, 32*32*7, 3])
 ...
 DeConnv1 = tf.nn.conv3d_transpose(layer1, filter = w, output_shape = [1,32,32,7,1], strides = [1,2,2,2,1], padding = 'SAME')
 ...
 final = tf.reshape(final, [1, 32*32*7])
 W_final = weight_variable([32*32*7,32*32*7,3])
 b_final = bias_variable([32*32*7,3])
 final_conv = tf.tensordot(final, W_final, axes=[[1], [1]]) + b_final
</code></pre>