title: Adding residual connections to simple CNN
body: <p>I implemented the following CNN with <em>residual connections</em> for classification of 10 classes on CIFAR10:</p>
  
  <pre><code>class ConvolutionalNetwork(nn.Module):  
  
    def __init__(self, filters=[32, 32, 32]):
  
      super(ConvolutionalNetwork, self).__init__()
  
      self.conv = []
      self.filters = filters
      in_channels = 3
  
      for out_channels in self.filters:
        self.conv.append(nn.Conv2d(in_channels, out_channels, 5, padding=2))
        in_channels = out_channels
  
      self.conv = nn.ModuleList(self.conv)
      self.fc = nn.Linear(self.filters[-1] * 32 * 32, 10)
  
    def forward(self, X):
      X = X.contiguous()
  
      for i, conv in enumerate(self.conv):
        res = X
        X = F.relu(conv(X))
        X += res
  
      X = X.reshape(-1,self.filters[-1]*self.size*self.size)
  
      X = self.fc(X)
      return F.log_softmax(X, dim=1)
  </code></pre>
  
  <p>Basically: Conv2d(5x5) --> ReLU --> Conv2d(5x5) --> ReLU --> Conv2d(5x5) --> ReLU --> Fully Connected Layer --> SoftMax
  The performence with the <em>residual connections</em> is worse then without. I wonder:</p>
  
  <ol>
  <li>maybe I didn't use it right (should I use nn.Identity() or some other pytorch class?)</li>
  <li>what are the best practices for number of filters (I tried [32,32,32] and [64,64,64]) and for skips (how many hops? i did one but maybe this is the problem?)</li>
  <li>How can I change the dimention of the identity to fit varying dimentions of the convolutional network (i.e. [16, 32, 64] filters)</li>
  </ol>