title: Calculating the number of weights in Convolutional Neural Network using Parameter Sharing
body: <p>While reading a book <a href="https://www.cs.ubc.ca/~murphyk/MLbook/" rel="nofollow noreferrer">Machine Learning: a Probabilistic Perspective by Murphy</a> and <a href="https://www.codeproject.com/Articles/16650/Neural-Network-for-Recognition-of-Handwritten-Digi" rel="nofollow noreferrer">this article</a> by Mike O'Neill I have encountered some calculations about the number of weights in Convolutional Neural Network which I want to understand. The architecture of the network is like this:</p>
  
  <p><a href="https://i.stack.imgur.com/JfqAQ.gif" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JfqAQ.gif" alt="enter image description here"></a></p>
  
  <p>And this is the explanation from the above article:</p>
  
  <blockquote>
    <p>Layer #2 is also a convolutional layer, but with 50 feature maps. Each
    feature map is 5x5, and each unit in the feature maps is a 5x5
    convolutional kernel of corresponding areas of all 6 of the feature
    maps of the previous layers, each of which is a 13x13 feature map.
    There are therefore 5x5x50 = 1250 neurons in Layer #2, (5x5+1)x6x50 =
    7800 weights, and 1250x26 = 32500 connections.</p>
  </blockquote>
  
  <p>The calculation of the number of weights, <strong>(5x5+1)x6x50 = 7800</strong>, seems strange for me. Shouldn't be the actual calculation like this:
  <strong>(5x5x6+1)x50 = 7550</strong> according to the parameter sharing explained <a href="http://cs231n.github.io/convolutional-networks/" rel="nofollow noreferrer">here</a>. </p>
  
  <p>My argument is as follows:
  We have 50 filters of size 5x5x6 and 1 bias for each filter, hence the total number of weights is (5x5x6+1)x50=7550. And this is Pytorch code which verifies this:</p>
  
  <pre><code>import torch
  import torch.nn as nn
  
  model = nn.Conv2d(in_channels=6, out_channels=50, kernel_size=5, stride=2)
  params_count = sum(param.numel() for param in model.parameters() if param.requires_grad)
  print(params_count) # 7550
  </code></pre>
  
  <p>Can anyone explain this and which one is correct?</p>