<p>Here's how I implemented it</p>

<pre><code>class LayerCardinalConv(object):
"""Aggregated Residual Transformations for Deep Neural Networks https://arxiv.org/abs/1611.05431"""

def __init__(self, name, w, nin, card, use_bias=True, init='he'):
    self.group = nin // card
    with tf.name_scope(name):
        self.conv = tf.Variable(weight_init(nin, self.group, [*w, nin, self.group], init), name='conv')
        self.bias = tf.Variable(tf.zeros([nin]), name='bias') if use_bias else 0

def __call__(self, vin, train):
    s = tf.shape(vin)
    vout = tf.nn.depthwise_conv2d(vin, self.conv, strides=[1] * 4, padding='SAME')
    vout = tf.reshape(vout, [s[0], s[1], s[2], self.group, s[3]])
    vout = tf.reduce_sum(vout, 3)
    return vout + self.bias
</code></pre>

<p>Notes:</p>

<ul>
<li>w is the kernel shape (3, 3) for example</li>
<li>nin number of input channels</li>
<li>cardinality or number of groups</li>
</ul>

<p>Hope it helps.</p>
Grouped depthwise convolution performance
<p>I'm attempting to improve the performance of my ResNeXt implementation in Tensorflow. David Berthelot mentioned a potential improvement over <a href="https://twitter.com/D_Berthelot_ML/status/908200291274711040" rel="nofollow noreferrer">on twitter</a>. I'd like to apply this to my implementation - how does reshape+sum fit into this?</p>

<pre><code># one resnext block per figure 3c
# see also https://arxiv.org/pdf/1611.05431.pdf
def bottleneck(x, strides, dim):
  x = tf.layers.conv2d(x, filters=64, kernel_size=1, strides=strides)
  x = tf.layers.batch_normalization(x, training=is_training)
  x = tf.nn.relu(x)
  w = tf.get_variable(name='depthwise_filter', shape=[3, 3, 64, cardinality])
  x = tf.nn.depthwise_conv2d_native(x, w, strides=1, padding='same')
  x = tf.layers.batch_normalization(x, training=is_training)
  x = tf.nn.relu(x)
  x = tf.layers.conv2d(x, filters=dim, kernel_size=1, strides=1)
  x = tf.layers.batch_normalization(x, training=is_training)
  return tf.nn.relu(x)
</code></pre>

<p><strong>EDIT:</strong> I thought this implementation was correct, and I just needed to add a couple operations to improve the performance. Taking another look at David's comment, the depthwise+reshape+sum wasn't instead of a single depthwise operation, but instead of some other method; the code above does not compute the equivalent of the bottleneck block version 3d.</p>