\begin{table}[t]\small
	\centering
	\renewcommand{\arraystretch}{0.95}
	\footnotesize
    \setlength\tabcolsep{0.1mm}{
    \begin{tabular}{l|l|c|cc}
        \multirow{2}{*}{Method} & \multirow{2}{*}{Detector} & \multirow{2}{*}{\#Params} & \multicolumn{2}{c}{$\rm AP^\text{b}$} \\
        &  & & val2017 & test-dev \\
	    \hline
		Swin-L$^\ddagger$~\cite{liu2021swin} 
        & DINO~\cite{zhang2022dino} & 218M & 63.2 & 63.3 \\

        
        Florence-CoSwin-H$^\#$~\cite{yuan2021florence} 
        & DyHead~\cite{dai2021dynamic} & 
        637M
        & 62.0 & 62.4 \\
        
        SwinV2-G$^\#$\cite{liu2021swinv2} 
        & HTC++~\cite{liu2021swin} & 3.00B & 62.5 & 63.1 \\
        
        BEiT-3$^\#$~\cite{wang2022beit3} 
        & ViTDet~\cite{li2022exploring} & 1.90B &$-$ & 63.7 \\
        
        FD-SwinV2-G$^\#$~\cite{wei2022fdswin}
        & HTC++~\cite{liu2021swin} & 3.00B & $-$ & 64.2 \\
        
        FocalNet-H$^\ddagger$~\cite{yang2022focal} 
        & DINO~\cite{zhang2022dino} & 746M & 64.2 & 64.3 \\
        
        ViT-Huge~\cite{chen2022group} 
        & Group-DETRv2~\cite{chen2022group} & 629M & $-$ & 64.5 \\
        \hline
        \rowcolor{gray!10}
        InternImage-XL$^\ddagger$ (ours) 
        & DINO~\cite{zhang2022dino} & 602M & 64.2 & 64.3 \\ 
        \rowcolor{gray!20}
        InternImage-H$^\#$ (ours) 
        & DINO~\cite{zhang2022dino} & 2.18B & 65.0 & 65.4\\
    \end{tabular}
    }
    \vspace{0.05cm}
    \caption{\textbf{Comparison of the state-of-the-art detectors on COCO \texttt{val2017} and test-dev.}
    }
    \label{tab:sota}
\end{table}
\author{
    Wenhai Wang$^{1*}$, 
    Jifeng Dai$^{2,1*}$,
    Zhe Chen$^{3,1*}$,
    Zhenhang Huang$^{1*}$,
    Zhiqi Li$^{3,1*}$,
    Xizhou Zhu$^{4*}$,\\
    Xiaowei Hu$^{1}$,
    Tong Lu$^{3}$,
    Lewei Lu$^{4}$,
    Hongsheng Li$^{5}$,
    Xiaogang Wang$^{3,5}$,
    Yu Qiao$^{1}$\textsuperscript{\Letter}\\
    $^1$ Shanghai AI Laboratory~~~
    $^2$Tsinghua University~~~\\
    $^3$Nanjing University~~~
    $^4$SenseTime Research~~~
    $^5$The Chinese University of Hong Kong\\
    {\small \url{https://github.com/OpenGVLab/InternImage}}
}

(3) We evaluate the proposed model on representative vision tasks including image classification, object detection, instance and semantic segmentation,
and compared it with state-of-the-art CNNs and large-scale ViTs by scaling the model size ranging from 30 million to 1 billion, the data ranging from 1 million to 400 million.
Specifically, our model with different parameter sizes can consistently outperform prior arts on ImageNet~\cite{deng2009imagenet}. 
InternImage-B achieves 84.9\% top-1 accuracy trained only on the ImageNet-1K dataset, outperforming CNN-based counterparts~\cite{ding2022replknet,liu2022convnet} by at least 1.1 points.
With large-scale parameters (\emph{i.e.}, 1 billion) and training data (\emph{i.e.}, 427 million), the top-1 accuracy of InternImage-H is further boosted to 89.2\%, which is close to well-engineering ViTs~\cite{liu2021swin,zhai2022scaling} and hybrid-ViTs~\cite{dai2021coatnet}.
In addition, on COCO~\cite{caesar2018coco}, a challenging downstream benchmark,
our best model InternImage-H achieves state-of-the-art 65.4\% box mAP with 2.18 billion parameters, 2.3 points higher than SwinV2-G~\cite{liu2021swinv2} (65.4 \vs 63.1) with 27\% fewer parameters.

\textbf{Basic block.}
Unlike the widely used bottlenecks in traditional CNNs~\cite{he2016deep}, the design of our basic block is closer to ViTs, which is equipped with more advanced components including LN~\cite{ba2016layernorm}, feed-forward networks (FFN)~\cite{vaswani2017attention}, and GELU~\cite{hendrycks2016gelu}.
This design is proved to be efficient~\cite{wang2021pyramid,wang2021pvtv2,liu2021swin,liu2022convnet,ding2022replknet} in various vision tasks.
The details of our basic block are illustrated in Fig. \ref{fig:arch}, where the core operator is DCNv3,
and the sampling offsets and modulation scales are predicted by passing input feature $\mathbf{x}$ through a separable convolution (a 3$\times$3 depth-wise convolution followed by a linear projection).
For other components, we use the post-normalization setting~\cite{xiong2020layer} by default and follow the same design as that of the plain transformer~\cite{vaswani2017attention,dosovitskiy2020image}.

\textbf{Stem \& downsampling layers.}
To obtain hierarchical feature maps, we use 
convolutional stem and downsampling layers to resize the feature maps to different scales.
As shown in Fig. \ref{fig:arch}, the stem layer is placed before the first stage to reduce the input resolution by 4 times.
It consists of two convolutions, two LN layers, and one GELU layer,
where the kernel size of the two convolutions is 3, the stride is 2, the padding is 1,
and the output channel of the first convolution is half of the second one.
Similarly, the downsampling layer is made up of a 3$\times$3 convolution with a stride of 2 and a padding of 1, followed by one LN layer.
It sits between the two stages and is used to downsample the input feature map by 2 times.

\subsection{Image Classification}
\textbf{Settings.}
We evaluate the classification performance of InternImage\ on ImageNet~\cite{deng2009imagenet}. 
For fair comparisons, following common practices~\cite{touvron2021training,wang2021pyramid,liu2021swin,liu2022convnet}, InternImage-T/S/B are trained on ImageNet-1K ($\sim$1.3 million) for 300 epochs, 
and InternImage-L/XL are first trained on ImageNet-22K ($\sim$14.2 million) for 90 epochs and then fine-tuned on ImageNet-1K for 30 epochs.
To further explore the capability of our model and match the large-scale private data used in previous methods~\cite{dai2021coatnet,liu2021swinv2,yuan2021florence}, we adopt M3I Pre-training~\cite{anonymous2022m2i}, an effective pre-training approach, to pre-train InternImage-H on a 427 million joint dataset of public Laion-400M~\cite{schuhmann2021laion}, YFCC-15M~\cite{thomee2016yfcc100m}, and CC12M~\cite{changpinyo2021conceptual} for 30 epochs,
and then we fine-tune the model on ImageNet-22K and -1K for 30 epochs, respectively.

\textbf{Settings.}
We verify the effectiveness of our InternImage\ on the COCO benchmark~\cite{lin2014microsoft}, on top of two representative object detection frameworks: Mask R-CNN~\cite{he2017mask}, and Cascade Mask R-CNN~\cite{cai2018cascade}.
We follow common practices~\cite{liu2021swin,wang2021pvtv2} to initialize the backbone with pre-trained classification weights, and by default use a 1$\times$ (12 epochs) or 3$\times$ (36 epochs) training schedule to train detection models.

\textbf{Results.}
As shown in Table~\ref{tab:results_detection_mask}, when using Mask R-CNN for object detection, 
we find that under a comparable number of parameters, our models significantly surpass their counterparts. 
For example, with the $1\times$ training schedule, the box AP (AP$^{\rm b}$) of InternImage-T is 4.5 points better than Swin-T~\cite{liu2021swin} (47.2 \vs 42.7),
and 3.0 points higher than ConvNeXt-T~\cite{liu2022convnet} (47.2 \vs 44.2).
As reported in Table~\ref{tab:results_detection_cascade}, with more parameters and more advanced Cascade Mask R-CNN, InternImage-XL achieves AP$^{\rm b}$ of 55.3, surpassing ConvNeXt-XL by 2.7 points (55.3 \vs 53.6).

To further push the performance bound of object detection, we follow the advanced setting used in leading methods~\cite{wang2022beit3, liu2021swinv2, zhang2022dino} to initialize the model with the weights pre-trained on ImageNet-22K or the large-scale joint dataset,
and fine-tune it along with the DINO~\cite{zhang2022dino} detector on the Objects365~\cite{shao2019objects365} and COCO datasets one after another for 26 epochs and 12 epochs, respectively.
As shown in Table~\ref{tab:sota}, our method establishes a new record of 65.0 AP$^\text{b}$ and 65.4 AP$^\text{b}$ on COCO val2017 and test-dev.
Compared to previous state-of-the-art models, we surpass FD-SwinV2-G~\cite{wei2022fdswin} by 1.2 points (65.4 \vs 64.2), using 27\% fewer parameters, 
which shows the effectiveness of our models on the object detection task.
{"value": "218M", "type": "Hyper-parameter/Architecture", "model": "Swin-L", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "63.2", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "Swin-L", "model settings": {"Detector": "DINO"}} {"value": "63.3", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "Swin-L", "model settings": {"Detector": "DINO"}} {"value": "637M", "type": "Hyper-parameter/Architecture", "model": "Florence-CoSwin-H", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "62.0", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "Florence-CoSwin-H", "model settings": {"Detector": "DyHead"}} {"value": "62.4", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "Florence-CoSwin-H", "model settings": {"Detector": "DyHead"}} {"value": "3.00B", "type": "Hyper-parameter/Architecture", "model": "SwinV2-G", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "62.5", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "SwinV2-G", "model settings": {"Detector": "HTC++"}} {"value": "63.1", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "SwinV2-G", "model settings": {"Detector": "HTC++"}} {"value": "1.90B", "type": "Hyper-parameter/Architecture", "model": "BEiT-3", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "63.7", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "BEiT-3", "model settings": {"Detector": "ViTDet"}} {"value": "3.00B", "type": "Hyper-parameter/Architecture", "model": "FD-SwinV2-G", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "64.2", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "FD-SwinV2-G", "model settings": {"Detector": "HTC++"}} {"value": "746M", "type": "Hyper-parameter/Architecture", "model": "FocalNet-H", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "64.2", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "FocalNet-H", "model settings": {"Detector": "DINO"}} {"value": "64.3", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "FocalNet-H", "model settings": {"Detector": "DINO"}} {"value": "629M", "type": "Hyper-parameter/Architecture", "model": "ViT-Huge", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "64.5", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "ViT-Huge", "model settings": {"Detector": "Group-DETRv2"}} {"value": "602M", "type": "Hyper-parameter/Architecture", "model": "InternImage-XL", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "64.2", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "InternImage-XL", "model settings": {"Detector": "DINO"}} {"value": "64.3", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "InternImage-XL", "model settings": {"Detector": "DINO"}} {"value": "2.18B", "type": "Hyper-parameter/Architecture", "model": "InternImage-H", "parameter/architecture name": ["# Params", "number of parameters"], "dataset": "COCO"} {"value": "65.0", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO val2017", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "InternImage-H", "model settings": {"Detector": "DINO"}} {"value": "65.4", "type": "Result", "training data/set": "Objects365 COCO", "test data/set": "COCO test-dev", "task": "object detection", "metric": ["AP^b", "AP"], "experimental settings": {"number of epochs": "26 12"}, "model": "InternImage-H", "model settings": {"Detector": "DINO"}}