<p>The depth of the network is not the only thing that affects its number of parameters. The number of parameters per layer has a huge effect. This means that for each convolutional layer, the size of the filter and the number of filters (features learned) would make a huge difference. Have a look at the paper of the VGG group at this link <a href="http://arxiv.org/pdf/1409.1556.pdf" rel="nofollow">http://arxiv.org/pdf/1409.1556.pdf</a></p>

<p>Also please look at the paper "<a href="http://arxiv.org/pdf/1512.03385v1.pdf" rel="nofollow">Deep Residual Learning for image recognition</a>" which represents networks that have up to 152 layers i.e. 8x deeper than VGG nets while still having lower complexity.</p>
Can deeper convolutional neural nets result in lesser parameters?
<p>I trained two convolutional neural nets in Keras. The first is net is as below</p>

<pre><code>def VGG1(weights_path):
    model = Sequential()
    model.add(Convolution2D(nb_filters, nb_conv, nb_conv,
                        border_mode='valid',
                        input_shape=(1, img_rows, img_cols)))
    model.add(Activation('relu'))
    model.add(Convolution2D(nb_filters, nb_conv, nb_conv))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(128))
    model.add(Activation('relu'))
    model.add(Dropout(0.5))
    model.add(Dense(nb_classes))
    model.add(Activation('softmax'))

    if weights_path:
        model.load_weights(weights_path)

    return model
</code></pre>

<p>The second net</p>

<pre><code>def VGG2(weights_path):
    model = Sequential()

    model.add(Convolution2D(nb_filters, nb_conv, nb_conv, border_mode='valid', input_shape=(1, img_rows, img_cols)))
    model.add(Activation('relu'))
    model.add(Convolution2D(nb_filters, nb_conv, nb_conv))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))
    model.add(Dropout(0.25))

    model.add(Convolution2D(64, nb_conv, nb_conv, border_mode='valid'))
    model.add(Activation('relu'))
    model.add(Convolution2D(64, nb_conv, nb_conv))
    model.add(Activation('relu'))
    model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))
    model.add(Dropout(0.25))

    model.add(Flatten())
    model.add(Dense(256))
    model.add(Activation('relu'))
    model.add(Dropout(0.5))

    model.add(Dense(nb_classes))
    model.add(Activation('softmax'))

    if weights_path:
        model.load_weights(weights_path)

    return model
</code></pre>

<p>When I call the <code>model.count_params()</code> method, the first net results in 604035 parameter and the second net results in 336387.</p>

<p>How is this possible? The second net is deeper and should contain more parameters. Is there any mistake?</p>