<blockquote>
  <p>I don't understand the reason behind keeping stride > 1 with window
  size 1. Isn't it just data loss?</p>
</blockquote>

<p>Please refer the section on Deeper Bottleneck Architectures in the resnet paper. Also, Figure 5.
<a href="https://arxiv.org/pdf/1512.03385.pdf" rel="noreferrer">https://arxiv.org/pdf/1512.03385.pdf</a></p>

<p>1 x 1 convolutions are typically used to increase or decrease the dimensionality along the filter dimension. So, in the bottleneck architecture the first 1 x 1 layer reduces the dimensions so that the 3 x 3 layer needs to handle smaller input/output dimensions. Then the final 1 x 1 layer increases the filter dimensions again.</p>

<p>It's done to save on computation/training time.</p>

<p>From the paper,</p>

<p><em>"Because of concerns on the training time that we can afford, we modify the building block as a bottleneck design".</em></p>
Stuck understanding ResNet's Identity block and Convolutional blocks
<p>I'm learning Residual Networks (ResNet50) from Andrew Ng coursera lectures. I understand that one of the main reasons why ResNets work is that they can learn identity function and that's why adding more and more layers in network does not hurt the performance of the network. </p>

<p>Now as described in lectures, there are two type of blocks are used in ResNets: 1) Identity block and Convolutional block. </p>

<p>Identity Block is used when there is no change in input and output dimensions. Convolutional block is almost same as identity block but there is a convolutional layer in <code>short-cut</code> path to just change the dimension such that the dimension of input and output matches. </p>

<p>Here is identity block:</p>

<p><a href="https://i.stack.imgur.com/37qzA.png" rel="noreferrer"><img src="https://i.stack.imgur.com/37qzA.png" alt="enter image description here"></a></p>

<p>and here is convolutional block:</p>

<p><a href="https://i.stack.imgur.com/0mE2p.png" rel="noreferrer"><img src="https://i.stack.imgur.com/0mE2p.png" alt="enter image description here"></a></p>

<p>Now in implementation of convolutional block (2nd image), First block (i.e. <code>conv2d --&gt; BatchNorm --&gt; ReLu</code> is implemented with <code>1x1</code> convolution and stride > 1.</p>

<pre><code># First component of main path 
X = Conv2D(F1, (1, 1), strides = (s,s), name = conv_name_base + '2a', padding = 'valid', kernel_initializer = glorot_uniform(seed=0))(X)
X = BatchNormalization(axis = 3, name = bn_name_base + '2a')(X)
X = Activation('relu')(X)
</code></pre>

<p>I don't understand the reason behind keeping stride > 1 with window size 1. Isn't it just data loss? We are just considering alternate pixels in this case. </p>

<p>What should be the possible reason for such hyperparameter selection? Any intuitive explanation will help! Thanks. </p>