title: Implement SeparableConv2D in Pytorch
body: <h2>Main objective</h2>
  <p>PyTorch equivalent for SeparableConv2D with <code>padding = 'same'</code>:</p>
  <pre class="lang-py prettyprint-override"><code>from tensorflow.keras.layers import SeparableConv2D
  x = SeparableConv2D(64, (1, 16), use_bias = False, padding = 'same')(x)
  </code></pre>
  <h2>What is the PyTorch equivalent for SeparableConv2D?</h2>
  <p>This <a href="https://discuss.pytorch.org/t/depthwise-and-separable-convolutions-in-pytorch/7315" rel="nofollow noreferrer">source</a> says:</p>
  <blockquote>
  <p>If groups = nInputPlane, kernel=(K, 1), (and before is a Conv2d layer with groups=1 and kernel=(1, K)), then it is separable.</p>
  </blockquote>
  <p>While this <a href="https://www.programmersought.com/article/1344745736/" rel="nofollow noreferrer">source</a> says:</p>
  <blockquote>
  <p>Its core idea is to break down a complete convolutional acid into a two-step calculation, Depthwise Convolution and Pointwise.</p>
  </blockquote>
  <p>This is my attempt:</p>
  <pre class="lang-py prettyprint-override"><code>class SeparableConv2d(nn.Module):
      def __init__(self, in_channels, out_channels, depth, kernel_size, bias=False):
          super(SeparableConv2d, self).__init__()
          self.depthwise = nn.Conv2d(in_channels, out_channels*depth, kernel_size=kernel_size, groups=in_channels, bias=bias)
          self.pointwise = nn.Conv2d(out_channels*depth, out_channels, kernel_size=1, bias=bias)
  
      def forward(self, x):
          out = self.depthwise(x)
          out = self.pointwise(out)
          return out
  </code></pre>
  <p>Is this correct? Is this equivalent to <code>tensorflow.keras.layers.SeparableConv2D</code>?</p>
  <h2>What about <code>padding = 'same'</code>?</h2>
  <p>How to ensure that my input and output size is the same while doing this?</p>
  <p>My attempt:</p>
  <pre><code>x = F.pad(x, (8, 7, 0, 0), )
  </code></pre>
  <p>Because the kernel size is <code>(1,16)</code>, I added left and right padding, 8 and 7 respectively. Is this the right way (and best way) to achieve <code>padding = 'same'</code>? How can I place this inside my <code>SeparableConv2d</code> class, and calculate on the fly given the input data dimension size?</p>
  <h2>All together</h2>
  <pre class="lang-py prettyprint-override"><code>class SeparableConv2d(nn.Module):
      def __init__(self, in_channels, out_channels, depth, kernel_size, bias=False):
          super(SeparableConv2d, self).__init__()
          self.depthwise = nn.Conv2d(in_channels, out_channels*depth, kernel_size=kernel_size, groups=in_channels, bias=bias)
          self.pointwise = nn.Conv2d(out_channels*depth, out_channels, kernel_size=1, bias=bias)
  
      def forward(self, x):
          out = self.depthwise(x)
          out = self.pointwise(out)
          return out
  
  
  class Net(nn.Module):
      def __init__(self):
          super(Net, self).__init__()
          self.separable_conv = SeparableConv2d(
              in_channels=32, 
              out_channels=64, 
              depth=1, 
              kernel_size=(1,16)
          )
          
      def forward(self, x):
          x = F.pad(x, (8, 7, 0, 0), )
          x = self.separable_conv(x)
          return x
  </code></pre>
  <p>Any problem with these codes?</p>