is equivalent to $2\times2$ stride-2 convolution with dynamic kernels, but is more efficient than explicit on-the-fly kernel generation.

*Indexed Upsampling* ($\mathcal{IU}$) is the inverse operator of $\mathcal{IP}$. $\mathcal{IU}$ upsamples $d\in\mathbb{R}^{1\times 1}$ that spatially corresponds to $E$ taking the same indices into account. Let $I\in\mathbb{R}^{k\times k}$ be the local index map formed by $I(x)$s, $\mathcal{IU}$ upsamples $d$ as $\mathcal{IU}(d)=I\otimes D$, where $\otimes$ denotes the element-wise multiplication, and $D$ is of the same size as $I$ and is upsampled from $d$ with the nearest neighbor interpolation. $\mathcal{IU}$ also relates to deconvolution, but an important difference between $\mathcal{IU}$ and deconvolution is that, deconvolution applies a fixed kernel to all local regions (even if the kernel is learned), while $\mathcal{IU}$ upsamples different regions with different kernels (indices).

![Conceptual differences between holistic and depthwise index.[]{data-label="fig:holistic_depthwise"}](holistic_depthwise.pdf){width="3in"}

Index Networks
--------------

![A taxonomy of proposed index networks.[]{data-label="fig:indexnet_taxonomy"}](indexnet_taxonomy.pdf){width="2.43in"}

Here we present a taxonomy of proposed index networks. According to the shape of the output index map, index networks can be first categorized into two branches: *holistic index networks* (HINs) and *depthwise (separable) index networks* (DINs). Their conceptual differences are shown in Fig. \[fig:holistic\_depthwise\]. HINs learn an index function $I(\mathcal{X}):\mathbb{R}^{H\times W\times C}\rightarrow\mathbb{R}^{H\times W\times1}$. In this case, all channels of the 