1$.

In many situations, a linear relationship is not sufficient. For example, a linear function even cannot approximate the max function. Thus, the second design choice is to introduce nonlinearity into the network.

*Nonlinear HINs.* Fig. \[fig:holistic\_networks\](b) illustrates a nonlinear HIN where the feature map is first projected to a map of size $H/2\times W/2\times 2C$, followed by a batch normalization layer and a ReLU function for nonlinear mappings. We then use point-wise convolution to reduce the channel dimension to an indices-compatible size. The remaining transformations follow its linear counterpart.

![Holistic index networks. (a) a linear index network; (b) a nonlinear index network.[]{data-label="fig:holistic_networks"}](holistic_index_networks.pdf){width="2.4in"}

Depthwise Index Networks {#subsec:depthwise_networks}
------------------------

In DINs, we seek $I(\mathcal{X}):\mathbb{R}^{H\times W\times C}\rightarrow\mathbb{R}^{H\times W \times C}$, i.e., each spatial index corresponds to each spatial activation. As aforementioned, this type of networks further has two different high-level design strategies that correspond to two different assumptions.

### One-to-One Depthwise Index Networks

O2O assumption assumes that each slice of the index map only relates to its corresponding slice of the feature map. It can be denoted by a local index function $l(\mathcal{X}): \mathbb{R}^{k\times k\times1}\rightarrow\mathbb{R}^{k\times k\times1}$, where $k$ denotes the size of the local region. Since the local index function operates on 