<p>See equation 3 in description of CuDNN <a href="http://arxiv.org/pdf/1410.0759.pdf" rel="noreferrer">here</a></p>

<p>Basically for a single example (<code>n</code>), single row (<code>p</code>) and single column (<code>q</code>), the result of spatial convolution will be a weighted sum of <code>5x5x3</code> values. So each activation will contain information from all 3 colors.</p>
Tensorflow weights for kernels of convolution for colored images?
<p>At the moment I have some networks doing classification stuff with greyscaled images. I want to move on to colored (RGB) images.</p>

<p>In the CIFAR-10 tutorial of Tensorflow I got confused by the weights for the convolution kernels. The first convolution there looks like this:</p>

<pre><code>kernel = _variable_with_weight_decay('weights', shape=[5, 5, 3, 64],
                                         stddev=1e-4, wd=0.0)
conv = tf.nn.conv2d(images, kernel, [1, 1, 1, 1], padding='SAME')
</code></pre>

<p>So it is a <code>5x5</code> convolution with an input of 3 (one for each color channel: the red, green and blue image information) and it is generating 64 feature maps.</p>

<p>However, the second convolution layer takes an input of 64 feature maps:</p>

<pre><code>kernel = _variable_with_weight_decay('weights', shape=[5, 5, 64, 64],
                                         stddev=1e-4, wd=0.0)
conv = tf.nn.conv2d(norm1, kernel, [1, 1, 1, 1], padding='SAME')
</code></pre>

<p>...so, how does this process the color information? Does this mean the different color channels are somehow <em>spread</em> on the 64 feature maps of convolution layer 1?</p>

<p>I thought conv layer 1 produces 64 feature maps for each color channel, therefore ending up in 3 * 64 = 196 feature maps...but obviously I were wrong.</p>

<p>How is the color information <em>mixed</em> there in conv layer 1?</p>