<p>I want to explain with picture from <a href="https://arxiv.org/abs/1412.0767" rel="noreferrer">C3D</a>.</p>
<p>In a nutshell, <strong>convolutional direction</strong> &amp; <strong>output shape</strong> is important!</p>
<p><a href="https://i.stack.imgur.com/owWjX.png" rel="noreferrer"><img src="https://i.stack.imgur.com/owWjX.png" alt="enter image description here" /></a></p>
<p>↑↑↑↑↑ <em><strong>1D Convolutions - Basic</strong></em> ↑↑↑↑↑</p>
<ul>
<li>just <strong>1</strong>-direction (time-axis) to calculate conv</li>
<li>input = [W], filter = [k], output = [W]</li>
<li>ex) input = [1,1,1,1,1], filter = [0.25,0.5,0.25], output = [1,1,1,1,1]</li>
<li>output-shape is 1D array</li>
<li>example) graph smoothing</li>
</ul>
<h3>tf.nn.conv1d code Toy Example</h3>
<pre><code>import tensorflow as tf
import numpy as np

sess = tf.Session()

ones_1d = np.ones(5)
weight_1d = np.ones(3)
strides_1d = 1

in_1d = tf.constant(ones_1d, dtype=tf.float32)
filter_1d = tf.constant(weight_1d, dtype=tf.float32)

in_width = int(in_1d.shape[0])
filter_width = int(filter_1d.shape[0])

input_1d   = tf.reshape(in_1d, [1, in_width, 1])
kernel_1d = tf.reshape(filter_1d, [filter_width, 1, 1])
output_1d = tf.squeeze(tf.nn.conv1d(input_1d, kernel_1d, strides_1d, padding='SAME'))
print sess.run(output_1d)
</code></pre>
<p><a href="https://i.stack.imgur.com/hvMaU.png" rel="noreferrer"><img src="https://i.stack.imgur.com/hvMaU.png" alt="enter image description here" /></a></p>
<p>↑↑↑↑↑ <em><strong>2D Convolutions - Basic</strong></em> ↑↑↑↑↑</p>
<ul>
<li><strong>2</strong>-direction (x,y) to calculate conv</li>
<li>output-shape is <strong>2D</strong> Matrix</li>
<li>input = [W, H], filter = [k,k] output = [W,H]</li>
<li>example) <a href="https://en.wikipedia.org/wiki/Sobel_operator" rel="noreferrer">Sobel Egde Fllter</a></li>
</ul>
<h3>tf.nn.conv2d - Toy Example</h3>
<pre><code>ones_2d = np.ones((5,5))
weight_2d = np.ones((3,3))
strides_2d = [1, 1, 1, 1]

in_2d = tf.constant(ones_2d, dtype=tf.float32)
filter_2d = tf.constant(weight_2d, dtype=tf.float32)

in_width = int(in_2d.shape[0])
in_height = int(in_2d.shape[1])

filter_width = int(filter_2d.shape[0])
filter_height = int(filter_2d.shape[1])

input_2d   = tf.reshape(in_2d, [1, in_height, in_width, 1])
kernel_2d = tf.reshape(filter_2d, [filter_height, filter_width, 1, 1])

output_2d = tf.squeeze(tf.nn.conv2d(input_2d, kernel_2d, strides=strides_2d, padding='SAME'))
print sess.run(output_2d)
</code></pre>
<p><a href="https://i.stack.imgur.com/IvDQP.png" rel="noreferrer"><img src="https://i.stack.imgur.com/IvDQP.png" alt="enter image description here" /></a></p>
<p>↑↑↑↑↑ <em><strong>3D Convolutions - Basic</strong></em> ↑↑↑↑↑</p>
<ul>
<li><strong>3</strong>-direction (x,y,z) to calcuate conv</li>
<li>output-shape is <strong>3D</strong> Volume</li>
<li>input = [W,H,<strong>L</strong>], filter = [k,k,<strong>d</strong>] output = [W,H,M]</li>
<li><strong>d &lt; L</strong> is important! for making volume output</li>
<li>example) C3D</li>
</ul>
<h3>tf.nn.conv3d - Toy Example</h3>
<pre><code>ones_3d = np.ones((5,5,5))
weight_3d = np.ones((3,3,3))
strides_3d = [1, 1, 1, 1, 1]

in_3d = tf.constant(ones_3d, dtype=tf.float32)
filter_3d = tf.constant(weight_3d, dtype=tf.float32)

in_width = int(in_3d.shape[0])
in_height = int(in_3d.shape[1])
in_depth = int(in_3d.shape[2])

filter_width = int(filter_3d.shape[0])
filter_height = int(filter_3d.shape[1])
filter_depth = int(filter_3d.shape[2])

input_3d   = tf.reshape(in_3d, [1, in_depth, in_height, in_width, 1])
kernel_3d = tf.reshape(filter_3d, [filter_depth, filter_height, filter_width, 1, 1])

output_3d = tf.squeeze(tf.nn.conv3d(input_3d, kernel_3d, strides=strides_3d, padding='SAME'))
print sess.run(output_3d)
</code></pre>
<p><a href="https://i.stack.imgur.com/49cdt.png" rel="noreferrer"><img src="https://i.stack.imgur.com/49cdt.png" alt="enter image description here" /></a></p>
<p>↑↑↑↑↑ <em><strong>2D Convolutions with 3D input</strong></em>  - LeNet, VGG, ..., ↑↑↑↑↑</p>
<ul>
<li>Eventhough input is 3D ex) 224x224x3, 112x112x32</li>
<li>output-shape is not <strong>3D</strong> Volume, but <strong>2D</strong> Matrix</li>
<li>because filter depth = <strong>L</strong> must be matched with input channels = <strong>L</strong></li>
<li><strong>2</strong>-direction (x,y) to calcuate conv! not 3D</li>
<li>input = [W,H,<strong>L</strong>], filter = [k,k,<strong>L</strong>] output = [W,H]</li>
<li>output-shape is <strong>2D</strong> Matrix</li>
<li>what if we want to train N filters (N is number of filters)</li>
<li>then output shape is (stacked 2D) <strong>3D = 2D x N</strong> matrix.</li>
</ul>
<h3>conv2d - LeNet, VGG, ... for 1 filter</h3>
<pre><code>in_channels = 32 # 3 for RGB, 32, 64, 128, ... 
ones_3d = np.ones((5,5,in_channels)) # input is 3d, in_channels = 32
# filter must have 3d-shpae with in_channels
weight_3d = np.ones((3,3,in_channels)) 
strides_2d = [1, 1, 1, 1]

in_3d = tf.constant(ones_3d, dtype=tf.float32)
filter_3d = tf.constant(weight_3d, dtype=tf.float32)

in_width = int(in_3d.shape[0])
in_height = int(in_3d.shape[1])

filter_width = int(filter_3d.shape[0])
filter_height = int(filter_3d.shape[1])

input_3d   = tf.reshape(in_3d, [1, in_height, in_width, in_channels])
kernel_3d = tf.reshape(filter_3d, [filter_height, filter_width, in_channels, 1])

output_2d = tf.squeeze(tf.nn.conv2d(input_3d, kernel_3d, strides=strides_2d, padding='SAME'))
print sess.run(output_2d)
</code></pre>
<h3>conv2d - LeNet, VGG, ... for N filters</h3>
<pre><code>in_channels = 32 # 3 for RGB, 32, 64, 128, ... 
out_channels = 64 # 128, 256, ...
ones_3d = np.ones((5,5,in_channels)) # input is 3d, in_channels = 32
# filter must have 3d-shpae x number of filters = 4D
weight_4d = np.ones((3,3,in_channels, out_channels))
strides_2d = [1, 1, 1, 1]

in_3d = tf.constant(ones_3d, dtype=tf.float32)
filter_4d = tf.constant(weight_4d, dtype=tf.float32)

in_width = int(in_3d.shape[0])
in_height = int(in_3d.shape[1])

filter_width = int(filter_4d.shape[0])
filter_height = int(filter_4d.shape[1])

input_3d   = tf.reshape(in_3d, [1, in_height, in_width, in_channels])
kernel_4d = tf.reshape(filter_4d, [filter_height, filter_width, in_channels, out_channels])

#output stacked shape is 3D = 2D x N matrix
output_3d = tf.nn.conv2d(input_3d, kernel_4d, strides=strides_2d, padding='SAME')
print sess.run(output_3d)
</code></pre>
<p><a href="https://i.stack.imgur.com/RghcS.png" rel="noreferrer"><img src="https://i.stack.imgur.com/RghcS.png" alt="enter image description here" /></a>
↑↑↑↑↑ <em><strong>Bonus 1x1 conv in CNN</strong></em>  - GoogLeNet, ..., ↑↑↑↑↑</p>
<ul>
<li>1x1 conv is confusing when you think this as 2D image filter like sobel</li>
<li>for 1x1 conv in CNN, input is 3D shape as above picture.</li>
<li>it calculate depth-wise filtering</li>
<li>input = [W,H,L], filter = <strong>[1,1,L]</strong> output = [W,H]</li>
<li>output stacked shape is <strong>3D = 2D x N</strong> matrix.</li>
</ul>
<h3>tf.nn.conv2d - special case 1x1 conv</h3>
<pre><code>in_channels = 32 # 3 for RGB, 32, 64, 128, ... 
out_channels = 64 # 128, 256, ...
ones_3d = np.ones((1,1,in_channels)) # input is 3d, in_channels = 32
# filter must have 3d-shpae x number of filters = 4D
weight_4d = np.ones((3,3,in_channels, out_channels))
strides_2d = [1, 1, 1, 1]

in_3d = tf.constant(ones_3d, dtype=tf.float32)
filter_4d = tf.constant(weight_4d, dtype=tf.float32)

in_width = int(in_3d.shape[0])
in_height = int(in_3d.shape[1])

filter_width = int(filter_4d.shape[0])
filter_height = int(filter_4d.shape[1])

input_3d   = tf.reshape(in_3d, [1, in_height, in_width, in_channels])
kernel_4d = tf.reshape(filter_4d, [filter_height, filter_width, in_channels, out_channels])

#output stacked shape is 3D = 2D x N matrix
output_3d = tf.nn.conv2d(input_3d, kernel_4d, strides=strides_2d, padding='SAME')
print sess.run(output_3d)
</code></pre>
<h3>Animation (2D Conv with 3D-inputs)</h3>
<p><a href="https://i.stack.imgur.com/FjvuN.gif" rel="noreferrer"><img src="https://i.stack.imgur.com/FjvuN.gif" alt="enter image description here" /></a></p>
<ul>
<li>Original Link : <a href="https://sites.google.com/site/nttrungmtwiki/home/it/data-science---python/tensorflow/tensorflow-and-deep-learning-part-3?tmpl=%2Fsystem%2Fapp%2Ftemplates%2Fprint%2F&amp;showPrintDialog=1" rel="noreferrer">LINK</a></li>
<li>The author: Martin Görner</li>
<li>Twitter: @martin_gorner</li>
<li>Google +: plus.google.com/+MartinGorne</li>
</ul>
<h3>Bonus 1D Convolutions with 2D input</h3>
<p><a href="https://i.stack.imgur.com/woaXM.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/woaXM.jpg" alt="enter image description here" /></a>
↑↑↑↑↑ <em><strong>1D Convolutions with 1D input</strong></em>   ↑↑↑↑↑</p>
<p><a href="https://i.stack.imgur.com/9VBtu.jpg" rel="noreferrer"><img src="https://i.stack.imgur.com/9VBtu.jpg" alt="enter image description here" /></a>
↑↑↑↑↑ <em><strong>1D Convolutions with 2D input</strong></em>   ↑↑↑↑↑</p>
<ul>
<li>Eventhough input is 2D ex) 20x14</li>
<li>output-shape is not <strong>2D</strong> , but <strong>1D</strong> Matrix</li>
<li>because filter height = <strong>L</strong> must be matched with input height = <strong>L</strong></li>
<li><strong>1</strong>-direction (x) to calcuate conv! not 2D</li>
<li>input = [W,<strong>L</strong>], filter = [k,<strong>L</strong>] output = [W]</li>
<li>output-shape is <strong>1D</strong> Matrix</li>
<li>what if we want to train N filters (N is number of filters)</li>
<li>then output shape is (stacked 1D) <strong>2D = 1D x N</strong> matrix.</li>
</ul>
<h3>Bonus C3D</h3>
<pre><code>in_channels = 32 # 3, 32, 64, 128, ... 
out_channels = 64 # 3, 32, 64, 128, ... 
ones_4d = np.ones((5,5,5,in_channels))
weight_5d = np.ones((3,3,3,in_channels,out_channels))
strides_3d = [1, 1, 1, 1, 1]

in_4d = tf.constant(ones_4d, dtype=tf.float32)
filter_5d = tf.constant(weight_5d, dtype=tf.float32)

in_width = int(in_4d.shape[0])
in_height = int(in_4d.shape[1])
in_depth = int(in_4d.shape[2])

filter_width = int(filter_5d.shape[0])
filter_height = int(filter_5d.shape[1])
filter_depth = int(filter_5d.shape[2])

input_4d   = tf.reshape(in_4d, [1, in_depth, in_height, in_width, in_channels])
kernel_5d = tf.reshape(filter_5d, [filter_depth, filter_height, filter_width, in_channels, out_channels])

output_4d = tf.nn.conv3d(input_4d, kernel_5d, strides=strides_3d, padding='SAME')
print sess.run(output_4d)

sess.close()
</code></pre>
<h3>Input &amp; Output in Tensorflow</h3>
<p><a href="https://i.stack.imgur.com/I25ty.png" rel="noreferrer"><img src="https://i.stack.imgur.com/I25ty.png" alt="enter image description here" /></a></p>
<p><a href="https://i.stack.imgur.com/xIdEq.png" rel="noreferrer"><img src="https://i.stack.imgur.com/xIdEq.png" alt="enter image description here" /></a></p>
<h3>Summary</h3>
<p><a href="https://i.stack.imgur.com/HCWgp.png" rel="noreferrer"><img src="https://i.stack.imgur.com/HCWgp.png" alt="enter image description here" /></a></p>
Intuitive understanding of 1D, 2D, and 3D convolutions in convolutional neural networks
<p>Can anyone please clearly explain the difference between 1D, 2D, and 3D convolutions in convolutional neural networks (in deep learning) with the use of examples?</p>