<p>Let's see the sample code in <a href="https://www.tensorflow.org/api_docs/python/tf/nn/" rel="noreferrer">TensorFlow API(r1.7)</a></p>

<p>For <code>depthwise_conv2d</code>,</p>

<pre><code>output[b, i, j, k * channel_multiplier + q] =
    sum_{di, dj} input[b, strides[1] * i + rate[0] * di,
                          strides[2] * j + rate[1] * dj, k] *
                 filter[di, dj, k, q]
</code></pre>

<p>filter is <code>[filter_height, filter_width, in_channels, channel_multiplier]</code></p>

<p>For <code>conv2d</code>,</p>

<pre><code>output[b, i, j, k] =
    sum_{di, dj, q} input[b, strides[1] * i + di,
                             strides[2] * j + dj, q] *
                    filter[di, dj, q, k]
</code></pre>

<p>filter is <code>[filter_height, filter_width, in_channels, out_channels]</code></p>

<p>Focusing on <code>k</code> and <code>q</code>, we can see the difference shown above.</p>

<p>The default format is <code>NHWC</code>, where <code>b</code> is batch size, <code>(i, j)</code> is a coordinate in feature map.</p>

<p>(Note that <code>k</code> and <code>q</code> refer to different things in this two functions.)</p>

<ol>
<li>For <code>depthwise_conv2d</code>, <code>k</code> refers to an input channel and <code>q</code>, <code>0 &lt;= q &lt; channel_multiplier</code>, refers to an output channel. Each input channel <code>k</code> is expanded to <code>k*channel_multiplier</code> with different filters <code>[filter_height, filter_width, channel_multiplier]</code>. It does not conduct cross-channel operation, in some literature, it is referred as <code>channel-wise spatial convolution</code>. Above process can be concluded as applying kernels of each filter separately to each channel and concatenating the outputs.</li>
<li>For <code>conv2d</code>, <code>k</code> refers to an output channel and <code>q</code> refers to an input channel. It sums up among all input channels, meaning that each output channel <code>k</code> is associated with all <code>q</code> input channels by a <code>[filter_height, filter_width, in_channels]</code> filter.</li>
</ol>

<p>For example, </p>

<pre><code>input_size: (_, 14, 14, 32)
filter of conv2d: (3, 3, 32, 64)
params of conv2d filter: 3x3x32x64
filter of depthwise_conv2d: (3, 3, 32, 64)
params of depthwise_conv2d filter: 3x3x32x64
</code></pre>

<p>suppose stride = 1 with padding, then</p>

<pre><code>output of conv2d: (_, 14, 14, 64)
output of depthwise_conv2d: (_, 14, 14, 32*64)
</code></pre>

<p>Some more insights:</p>

<ul>
<li>Standard convolution operation can be split into 2 steps: depthwise convolution and reduction (sum).</li>
<li>Depthwise Convolution is equivalent to setting the number of group to input channel in Group Convolution.</li>
<li>Usually, <code>depthwise_conv2d</code> is followed by <code>pointwise_conv2d</code>(a 1x1 convolution for reduction purpose), making a <code>separable_conv2d</code>. Check <a href="https://arxiv.org/pdf/1610.02357.pdf" rel="noreferrer">Xception</a>, <a href="https://arxiv.org/pdf/1704.04861.pdf" rel="noreferrer">MobileNet</a> for more details.</li>
</ul>
Difference between tf.nn_conv2d and tf.nn.depthwise_conv2d
<p>What is the difference between <code>tf.nn_conv2d</code> and <code>tf.nn.depthwise_conv2d</code> in Tensorflow?</p>