only use a kernel size of $2\times2\times 1$, a stride of $2$, and $1$ group, while Unshared Stagewise O2O DINs has a kernel size of $2\times2\times C$, a stride of $2$, and $C$ groups. One can simply reshape the feature map, i.e., reshaping $H\times W \times C$ to be $C \times H\times W \times 1$, to enable a $2\times2\times 1$ kernel operating on each $H\times W \times 1$ feature slice, respectively. All O2O DINs lead to four downsampled feature maps of size $H/2\times W/2 \times C$. The final index map of size $H\times W \times C$ is composed from the four feature maps by shuffling and rearrangement. Note that the parameters of four columns are not shared.

*Nonlinear O2O DINs.* Nonlinear DINs can be easily modified from linear DINs by inserting four extra convolutional layers. Each of them is followed by a batch normalization (BN) layer and a ReLU unit, as shown in Fig. \[fig:depthwise\_networks\]. The rest remains the same as the linear DINs.

![Depthwise index networks. $M=1,N=1$ for Modelwise O2O DINs and Shared Stagewise O2O DINs; $M=C,N=C$ for Unshared Stagewise O2O DINs; and $M=C,N=1$ for the M2O DINs. The masked modules are invisible to linear networks.[]{data-label="fig:depthwise_networks"}](depthwise_index_networks.pdf){width="3.2in"}

### Many-to-One Depthwise 