\begin{table}[t]
    \centering
    \renewcommand\arraystretch{1.0}
    \setlength{\tabcolsep}{2.8mm}
    \footnotesize
    
\begin{tabular}{l|c|c|c|c}
	\renewcommand{\arraystretch}{0.1}
	\setlength\tabcolsep{0.1mm}
	Model Name & $C_1$ & $C'$  & $L_{1, 2, 3, 4}$ & \#Params \\
	\hline
	InternImage-T (origin) &64 & 16 & 4, 4, 18, 4 & 30M \\
	InternImage-S &80 & 16 & 4, 4, 21, 4 & 50M \\
	InternImage-B &112 & 16 & 4, 4, 21, 4 & 97M \\
	InternImage-L &160 & 16 & 5, 5, 22, 5 & 223M \\
	InternImage-XL &192 & 16 & 5, 5, 24, 5 & 335M \\
	InternImage-H &320 & 16 & 6, 6, 32, 6 & 1.08B \\
\end{tabular}

    \caption{\textbf{Hyper-parameters for models of different scales}. InternImage-T is the origin model, and -S/B/L/XL/H are scaled up from -T. ``\#Params'' denotes the number of parameters.}
    \label{tab:model_size}
\end{table}
(3) We evaluate the proposed model on representative vision tasks including image classification, object detection, instance and semantic segmentation,
and compared it with state-of-the-art CNNs and large-scale ViTs by scaling the model size ranging from 30 million to 1 billion, the data ranging from 1 million to 400 million.
Specifically, our model with different parameter sizes can consistently outperform prior arts on ImageNet~\cite{deng2009imagenet}. 
InternImage-B achieves 84.9\% top-1 accuracy trained only on the ImageNet-1K dataset, outperforming CNN-based counterparts~\cite{ding2022replknet,liu2022convnet} by at least 1.1 points.
With large-scale parameters (\emph{i.e.}, 1 billion) and training data (\emph{i.e.}, 427 million), the top-1 accuracy of InternImage-H is further boosted to 89.2\%, which is close to well-engineering ViTs~\cite{liu2021swin,zhai2022scaling} and hybrid-ViTs~\cite{dai2021coatnet}.
In addition, on COCO~\cite{caesar2018coco}, a challenging downstream benchmark,
our best model InternImage-H achieves state-of-the-art 65.4\% box mAP with 2.18 billion parameters, 2.3 points higher than SwinV2-G~\cite{liu2021swinv2} (65.4 \vs 63.1) with 27\% fewer parameters.

where $K$ represents the total number of sampling points, and $k$ enumerates the sampling point. 
$\mathbf{w}_k\!\in\!\mathbb{R}^{C\times C}$ denotes the projection weights of the $k$-th sampling point,
and $\mathbf{m}_k\!\in\!\mathbb{R}$ represents the modulation scalar of the $k$-th sampling point, which is normalized by sigmoid function.
$p_k$ denotes the $k$-th location of the pre-defined grid sampling $\{(-1, -1), (-1, 0), ..., (0, +1), ..., (+1, +1)\}$ as in regular convolutions,
and $\Delta p_k$ is the offset corresponding to the $k$-th grid sampling location.
We see from the equation that (1) for long-range dependencies, the sampling offset $\Delta p_k$ is flexible and able to interact with short- or long-range features; and (2) for adaptive spatial aggregation, both the sampling offset $\Delta p_k$ and modulation scalar $\mathbf{m}_k$ are learnable and conditioned by input $\mathbf{x}$. 
So it can be found that \emph{DCNv2 shares similar favorable properties with MHSA}, which motivated us to develop large-scale CNN-based foundation models on the basis of this operator.

where $G$ denotes the total number of aggregation groups.
For the $g$-th group,
$\mathbf{w}_g\!\in\!\mathbb{R}^{C\times C'}$, $\mathbf{m}_{gk}\!\in\!\mathbb{R}$ denote the location-irrelevant projection weights of the group, where $C'\!=\!C/G$ represents the group dimension.
$\mathbf{m}_{gk}\!\in\!\mathbb{R}$ denotes the modulation scalar of the $k$-th sampling point in the $g$-th group, normalized by the softmax function along the dimension $K$.
$\mathbf{x}_g\!\in\!\mathbb{R}^{C'\times H \times W}$ represents the sliced input feature map.
$\Delta p_{gk}$ is the offset corresponding to the grid sampling location $p_k$ in the $g$-th group.

\textbf{Stem \& downsampling layers.}
To obtain hierarchical feature maps, we use 
convolutional stem and downsampling layers to resize the feature maps to different scales.
As shown in Fig. \ref{fig:arch}, the stem layer is placed before the first stage to reduce the input resolution by 4 times.
It consists of two convolutions, two LN layers, and one GELU layer,
where the kernel size of the two convolutions is 3, the stride is 2, the padding is 1,
and the output channel of the first convolution is half of the second one.
Similarly, the downsampling layer is made up of a 3$\times$3 convolution with a stride of 2 and a padding of 1, followed by one LN layer.
It sits between the two stages and is used to downsample the input feature map by 2 times.

\textbf{Stacking rules.}
To clarify the block-stacking process, we first list the integral hyper-parameters of the InternImage\ as follows:\\
\indent $C_i$: the channel number of the $i$-th stage;\\
\indent $G_i$: the group number of the DCNv3 in the $i$-th stage;\\
\indent$L_i$: the number of basic blocks in the $i$-th stage.\\
Since our model has 4 stages, a variant is decided by 12 hyper-parameters,
whose search space is too large to exhaustively enumerate and find the best variant.
To reduce the search space, we summarize the design experiences of prior arts~\cite{he2016deep,liu2021swin,liu2022convnet} into 4 rules as shown in Fig.~\ref{fig:arch},
where the first rule makes the channel numbers of the last three stages determined by the channel number $C_1$ of the first stage,
and the second rule lets the group number corresponding to the channel number of stages.
For the number of stacked blocks in different stages, we simplify the stacking pattern to ``AABA'', which means the block number of stage 1, 2, and 4 are the same, and are not greater than that of the stage 3 as illustrated in the last two rules.
With these rules, a InternImage\ variant can be defined by using only 4 hyper-parameters $(C_1, C', L_1, L_3)$.

Let us choose a model with 30 million parameters as the origin and discretize $C_1$ to $\{16, 32, 64\}$, $L_1$ to $\{1, 2, 3, 4, 5\}$, and $C'$ to $\{16, 32\}$.
In this way, the original huge search space is reduced to 30,
and we can find the best model from the 30 variants by training and evaluating them in ImageNet~\cite{deng2009imagenet}.
In practice, we use the best hyper-parameter setting $(64, 16, 4, 18)$ to define the base model and scale it to different scales.

\textbf{Scaling rules.} Based on the optimal origin model under the aforementioned constraints, we further explore the parameter scaling rules inspired by \cite{tan2019efficientnet}.
Specifically, we consider two scaling dimensions: depth $D$ (\emph{i.e.}, $3L_1\!+\!L_3$) and width $C_1$,
and scale the two dimensions using $\alpha$, $\beta$ and a composite factor $\phi$. The scaling rules can be written as:

where $\alpha\!\geq\!1$, $\beta\!\geq\!1$, and $\alpha\beta^{1.99}\!\approx\!2$. Here, 1.99 is specific for InternImage and calculated by doubling the model width and keeping the depth constant.
We experimentally find out that the best scaling setting is $\alpha\!=\!1.09$ and $\beta\!=\!1.36$, and then we base on it to construct InternImage\ variants with different parameter scales, namely InternImage-T/S/B/L/XL, whose complexity is similar to those of ConvNeXt~\cite{liu2022convnet}. 
To further test the capability, we built a larger InternImage-H with 1 billion parameters.
The detailed configurations are summarized in Table~\ref{tab:model_size}.
{"value": "64", "type": "Hyper-parameter/Architecture", "model": "InternImage-T", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-T", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "30M", "type": "Hyper-parameter/Architecture", "model": "InternImage-T", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "80", "type": "Hyper-parameter/Architecture", "model": "InternImage-S", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-S", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "50M", "type": "Hyper-parameter/Architecture", "model": "InternImage-S", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "112", "type": "Hyper-parameter/Architecture", "model": "InternImage-B", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-B", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "97M", "type": "Hyper-parameter/Architecture", "model": "InternImage-B", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "160", "type": "Hyper-parameter/Architecture", "model": "InternImage-L", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-L", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "223M", "type": "Hyper-parameter/Architecture", "model": "InternImage-L", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "192", "type": "Hyper-parameter/Architecture", "model": "InternImage-XL", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-XL", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "335M", "type": "Hyper-parameter/Architecture", "model": "InternImage-XL", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "320", "type": "Hyper-parameter/Architecture", "model": "InternImage-H", "parameter/architecture name": ["C_1", "channel number of the first stage"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "16", "type": "Hyper-parameter/Architecture", "model": "InternImage-H", "parameter/architecture name": ["C'", "group dimension"], "dataset": ["ImageNet", "COCO", "ADE20K"]} {"value": "1.08B", "type": "Hyper-parameter/Architecture", "model": "InternImage-H", "parameter/architecture name": ["\\#Params", "number of parameters"], "dataset": ["ImageNet", "COCO", "ADE20K"]}