DetNAS: Backbone Search for Object Detection
Yukang Chen Tong Yang Xiangyu Zhang Gaofeng Meng Xinyu Xiao Jian Sun


Matrix multiplication and universal scalability of the time on the Intel Scalable processors
    Alexander Russkov^1, Lev Shchur^2,3
    December 30, 2023
============================================================================================




    Object detectors are usually equipped with backbone networks designed for image classification. It might be sub-optimal because of the gap between the tasks of image classification and object detection. In this work, we present DetNAS to use Neural Architecture Search (NAS) for the design of better backbones for object detection. 
   It is non-trivial because detection training typically needs ImageNet pre-training while NAS systems require accuracies on the target detection task as supervisory signals. 
   Based on the technique of one-shot supernet, which contains all possible networks in the search space, we propose a framework for backbone search on object detection.
   We train the supernet under the typical detector training schedule: ImageNet pre-training and detection fine-tuning. Then, the architecture search is performed on the trained supernet, using the detection task as the guidance. This framework makes NAS on backbones very efficient.
   In experiments, we show the effectiveness of DetNAS on various detectors, for instance, one-stage RetinaNet and the two-stage FPN. We empirically find that networks searched on object detection shows consistent superiority compared to those searched on ImageNet classification. The resulting architecture achieves superior performance than hand-crafted networks on COCO with much less FLOPs complexity. Code and models have been made available at: <https://github.com/megvii-model/DetNAS>.
   ^†Equal contribution. ^*Work done during an internship at Megvii Technology. ^ Corresponding author.
   
   





§ INTRODUCTION

Backbones play an important role in object detectors. The performance of object detectors highly relies on features extracted by backbones. For example, a large accuracy increase could be obtained by simply replacing a ResNet-50 <cit.> backbone with stronger networks, e.g., ResNet-101 or ResNet-152. The importance of backbones is also demonstrated in DetNet <cit.>, Deformable ConvNets v2 <cit.>, ThunderNet <cit.> in real-time object detection, and HRNet <cit.> in keypoint detection. 

However, many object detectors directly use networks designed for Image classification as backbones. It might be sub-optimal because image classification focus on what the main object of an image is, while object detection aims at finding where and what each object instance. For instance, the recent hand-crafted network, DetNet <cit.>, has demonstrated this point. ResNet-101 performs better than DetNet-59 <cit.> on ImageNet classification, but is inferior to DetNet-59 <cit.> on object detection. However, the handcrafting process heavily relies on expert knowledge and tedious trials. 

NAS has achieved great progress in recent years. On image classification <cit.>, searched networks reach or even surpass the performance of the hand-crafted networks.
However, NAS for backbones in object detectors is still challenging.
It is infeasible to simply employ previous NAS methods for backbone search in object detectors. The typical detector training schedule requires backbone networks to be pre-trained on ImageNet.
 This results in two difficulties for searching backbones in object detectors: 1) hard to optimize: NAS systems require accuracies on target tasks as reward signals, pre-training accuracy is unqualified for this requirement; 2) inefficiency: In order to obtain the precious performance, each candidate architecture during search has to be first pretrained (e.g. on ImageNet) then finetuned on the detection dataset, which is very costly. 
 Even though training from scratch is an alternative <cit.>, it requires more training iterations to compensate for the lack of pre-training. Moreover, training from scratch breaks down in small datasets, e.g. PASCAL VOC.
 
In this work, we present the first effort on searching for backbones in object detectors.  Recently, a NAS work on object detection, NAS-FPN <cit.>, is proposed. It searches for feature pyramid networks (FPN) <cit.> rather than backbones. It can perform with a pre-trained backbone network and search with the previous NAS algorithm <cit.>. Thus, the difficulty of backbone search is still unsolved. Inspired by one-shot NAS methods <cit.>, we solve this issue by decoupling the weight training and architecture search. Most previous NAS methods optimize weights and architectures in a nested manner. Only if we decouple them into two stages, the pre-training step can be incorporated economically. This framework avoids the inefficiency issue caused by the pre-training and makes backbone search feasible.


The framework of DetNAS consists of three steps: (1) pre-training the one-shot supernet on ImageNet, (2) fine-tuning the one-shot supernet on detection datasets, (3) architecture search on the trained supernet with an evolutionary algorithm (EA). In experiments, the main result backbone network, DetNASNet, with much fewer FLOPs, achieves 2.9% better mmAP than ResNet-50 on COCO with the FPN detector. Its enlarged version, DetNASNet (3.8), is superior to ResNet-101 by 2.0% on COCO with the FPN detector.
In addition, we validate the effectiveness of DetNAS in different detectors (the two-stage FPN <cit.> and the one-stage RetinaNet <cit.>) and various datasets (COCO and VOC). 
DetNAS are consistently better than the network searched on ImageNet classification by more than 3% on VOC and 1% on COCO, no matter on FPN or RetinaNet.


Our main contributions are summarized as below:

    
  * We present DetNAS, a framework that enables backbone search for object detection. To our best knowledge, this is the first work on this challenging task.
    
    
  * We introduce a powerful search space. It helps the searched networks obtain inspiring accuracies with limited FLOPs complexity.
    
    
  * Our result networks, DetNASNet and DetNASNet (3.8), outperforms the hand-crafted networks by a large margin. Without the effect of search space, we show the effectiveness of DetNAS in different detectors (two-stage FPN and one-stage RetinaNet), various datasets (COCO and VOC). The searched networks have consistently better performance and meaningful structural patterns.



 





§ RELATED WORK




 §.§ Object Detection

Object detection aims to locate each object instance and assign a class to it in an image. With the rapid progress of deep convolutional networks, object detectors, such as FPN <cit.> and RetinaNet <cit.>, have achieved great improvements in accuracy. In general, an object detector can be divided into two parts, a backbone network, and a "head". In the past few years, many advances in object detection come from the study of "head", such as architecture <cit.>, loss <cit.>, and anchor <cit.>. FPN <cit.> develops a top-down architecture with lateral connections to integrate features at all scales as an effective feature extractor. The focal loss <cit.> is proposed in RetinaNet to solve the problem of class imbalance, which leads to the instability in early training. MetaAnchor <cit.> proposes a dynamic anchor mechanism to boost the performance for anchor-based object detectors. However, for the backbone network, almost all object detectors adopt networks for image classification, which might be sub-optimal. Because object detection cares about not only "what" object is, which image classification only focuses, but also "where" it is. Similar to our work, DetNet <cit.> also exploits the architecture of the backbone that specially designed for object detection manually. Inspired by NAS, we present DetNAS to find the optimal backbone automatically for object detection in this work. 



 §.§ Neural Architecture Search

NAS on image classification
Techniques to design networks automatically have attracted increasing research interests. NAS <cit.> and NASNet <cit.> use reinforcement learning (RL) to determine neural architectures sequentially. In addition to these RL-based methods, the evolution algorithm (EA) also shows its potential. AmeobaNet <cit.> proves that the basic evolutionary algorithm without any controller can also achieve comparable results and even surpass RL-base methods. 
 To save computational resources, some works propose to use weight sharing or one-shot methods, e.g., ENAS <cit.> and DARTS <cit.>. Many following works, including SNAS <cit.>, Proxyless <cit.> and FBNet <cit.> and others <cit.>, also belong to one-shot NAS to some extent.
 



NAS on other tasks
In addition to NAS works on image classification, some recent works attempt to develop NAS to other tasks, especially semantic segmentation. <cit.> proposes to search auxiliary cells as the segmentation decoders. Auto-DeepLab <cit.> applies the gradient-based method to search backbones for segmentation models. To our best knowledge, no works have attempted to search neural architectures for backbones in object detectors. One main reason might come from the costly ImageNet pre-training for object detectors. Training from scratch scheme <cit.>, as a substitute, proves to bring no computational savings and tends to break down in small datasets. In this work, we overcome this obstacle with a one-shot supernet and the evolutionary search algorithm.





§ DETECTION BACKBONE SEARCH


Our goal is to extend NAS to search for backbones in object detectors. In general, object detector training typically requires ImageNet pre-training. Meanwhile, NAS systems require supervisory signals from target tasks.  For each network candidate, it needs ImageNet pre-training, which is computationally expensive. Additionally, training from scratch is an alternative method while it requires more iterations to optimize and breaks down in small datasets. 
Inspired by the one-shot NAS <cit.>, we decouple the one-shot supernet training and architecture optimization to overcome this obstacle. In this section, we first clarify the motivation of our methodology.



 §.§ Motivation
 

Without loss of generality, the architecture search space 𝒜 can be denoted by a directed acyclic graph (DAG). Any path in the graph corresponds to a specific architecture, a∈𝒜. For the specific architecture, its corresponding network can be represented as 𝒩(a,w) with the network weights w. NAS aims to find the optimal architecture a^*∈𝒜 that minimizes the validation loss ℒ_val(𝒩(a^*,w^*)). w^* denotes the optimal network weights of the architecture a^*. It is obtained by minimizing the training loss. We can formulate NAS process as a nested optimization problem:

    min_a∈𝒜 ℒ_val(𝒩(a,w^*(a))) 
    s.t.    w^*(a)=min_wℒ_train(w,a)


The above formulation can represent NAS on tasks that work without pre-training, e.g., image classification. But for object detection, which needs pre-training and fine-tuning schedule, Eq. (<ref>) needs to be reformulated as follow:


    w^*(a)=min_w w_p(a)^*ℒ_train^det(w,a)
          s.t. w_p(a)^*=min_w_pℒ_train^cls(w_p,a)

where w w_p(a)^* is to optimize w with w_p(a)^* as initialization.
The pre-trained weights w_p(a)^* can not directly serve for the Eq. (<ref>), but it is necessary for w(a)^*. Thus, we can not skip the ImageNet pre-training in DetNAS. However, ImageNet pre-training usually costs several GPU days just for a single network. It is unaffordable to train all candidate networks individually. In one-shot NAS methods <cit.>, the search space is encoded in a supernet which consists of all candidate architectures. They share the weights in their common nodes. In this way, Eq. (<ref>) and Eq. (<ref>) become: 

    min_a∈𝒜 ℒ_val(𝒩(a,W_𝒜^*(a))) 
          s.t. W_𝒜^*=min_Wℒ_train(𝒩(𝒜,W))

where all individual network weights w(a) are inherited from the one-shot supernet W_𝒜. The supernet training, W_𝒜 optimization, is decoupled from the architecture a optimization. Based on this point, we go further step to incorporate pre-training step. This enables NAS on the more complicated task, backbone search in object detecion:

    min_a∈𝒜 ℒ^det_val(𝒩(a,W_𝒜^*(a)))
        
    s.t.     W_𝒜^*=min_W W_p𝒜^*ℒ^det_train(𝒩(𝒜,W))
        
        W_p𝒜^*=min_W_pℒ^cls_train(𝒩(𝒜,W_p))




 §.§ Our NAS Pipeline
 
As in Fig. <ref>, DetNAS consists of 3 steps: supernet pre-training on ImageNet, supernet fine-tuning on detection datasets and architecture search on the trained supernet.

Step 1: Supernet pre-training.   

 ImageNet pre-training is the fundamental step of the fine-tuning schedule.   For some one-shot methods <cit.>, they relax the actually discrete search space into a continuous one, which makes the weights of individual networks deeply coupled. However, in supernet pre-training, we adopt a path-wise <cit.> manner to ensure the trained supernet can reflect the relative performance of candidate networks. Specifically, in each iteration, only one single path is sampled for feedforward and backward propagation. No gradient or weight update acts on other paths or nodes in the supernet graph.
 
Step 2: Supernet fine-tuning.   The supernet fine-tuning is also path-wise but equipped with detection head, metrics and datasets. The other necessary detail to mention is about batch normalization (BN).
BN is a popular normalization method to help optimization. 
Typically, the parameters of BN, during fine-tuning, are fixed as the pre-training batch statistics. However, the freezing BN is infeasible in DetNAS, because the features to normalize are not equal on different paths. On the other hand,
 object detectors are trained with high-resolution images, unlike image classification. This results in small batch sizes as constrained by memory and severely degrades the accuracy of BN. To this end, we replace the conventional BN with Synchronized Batch Normalization (SyncBN) <cit.> during supernet training. It computes batch statistics across multiple GPUs and increases the effective batch size.
 We formulate the supernet training process in Algorithm 1 in the supplementary material.

Step 3: Search on supernet with EA.   The third step is to conduct the architecture search on the trained supernet. Paths in the supernet are picked and evaluated under the direction of the evolutionary controller. For the evolutionary search, please refer to Section <ref> for details.
The necessary detail in this step is also about BN.
During search, different child networks are sampled path-wise in the supernet. The issue is that the batch statistics on one path should be independent of others. Therefore, we need to recompute batch statistics for each single path (child networks) before each evaluation. This detail is indispensable in DetNAS. We extract a small subset of the training set (500 images) to recompute the batch statistics for the single path to be evaluated. This step is to accumulate reasonable running mean and running variance values for BN. It involves no gradient backpropagation.



 §.§ Search Space Design

The details about the search space are described in the Table <ref>. Our search space is based on the ShuffleNetv2 block. It is a kind of efficient and lightweight convolution architectures and involves channel split and shuffle operation <cit.>. We design two search spaces with different sizes, the large one for the main result and the small one for ablation studies.

Large (40 blocks)   This search space is a large one and designed for the main results to compare with hand-crafted backbone networks.
 The channels and blocks in each stage are specified by c_1 and n_1. In each stage, the first block has stride 2 for downsampling.
Except for the first stage, there are 4 stages that contain 8+8+16+8=40 blocks for search. 
For each block to search, there are 4 choices developed from the original ShuffleNetv2 block: changing the kernel size with {3×3, 5×5, 7×7} or replacing the right branch with an Xception block (three repeated separable depthwise 3×3 convolutions). It is easy to count that this search space includes 4^40≈1.2×10^24 candidate architectures.
Most networks in this search space have more than 1G FLOPs. We construct this large search space is for the comparisons with hand-crafted large networks. For example, ResNet-50 and ResNet-101 have 3.8G and 7.6G FLOPs respectively.

Small (20 blocks)   This search space is smaller and designed for ablation studies. The channels and blocks in each stage are specified by c_2 and n_2. The block numbers n_1 are twice as n_2. The channel numbers c_1 are 1.5 times as c_2 in all searched stages. 
This search space includes 4^20≈1.0×10^12 possible architectures. It is still a large number and enough for ablation studies.
Most networks in this search space have around 300M FLOPs. We conduct all situation comparisons in this search space, including various object detectors (FPN or RetinaNet), different datasets (COCO or VOC), and different schemes (training from scratch or with pre-training).

  




 §.§ Search Algorithm

The architecture search step is based on the evolution algorithm.
At first, a population of networks 𝐏 is initialized randomly. Each individual P consists of its architecture P.θ and its fitness P.f. Any architecture against the constraint η would be removed and a substitute would be picked. After initialization, we start to evaluate the individual architecture P.θ to obtain its fitness P.f on the validation set 𝕍_Det. 
Among these evaluated networks, we select the top |𝒫| as parents to generate child networks. The next generation networks are generated by mutation and crossover half by half under the constraint η. By repeating this process in iterations, we can find a single path θ_best with the best validation accuracy or fitness, f_best. We formulate this process as Algorithm 2 in the supplementary material. The hyper-parameters in this step are introduced in Section <ref>.

Compared to RL-based <cit.> and gradient-based NAS methods <cit.>, the evolutionary search can stably meet hard constraints, e.g., FLOPs or inference speed. To optimize FLOPs or inference speed, RL-based methods need a carefully tuned reward function while gradient-based methods require a wise designed loss function. But their outputs are still hard to totally meet the required constraints. To this end, DetNAS chooses the evolutionary search algorithm.




§ EXPERIMENTAL SETTINGS
 
Supernet pre-training.  For ImageNet classification dataset, we use the commonly used 1.28M training images for supernet pre-training. To train the one-shot supernet backbone on ImageNet, we use a batch size of 1024 on 8 GPUs for 300k iterations. We set the initial learning rate to be 0.5 and decrease it linearly to 0.  The momentum is 0.9 and weight decay is 4 × 10^-5.

Supernet fine-tuning.  We validate our method with two different detectors.
The training images are resized such that the shorter size is 800 pixels. We train on 8 GPUs with a total of 16 images per minibatch for 90k iterations on COCO and 22.5k iterations on VOC.  The initial learning rate is 0.02 which is divided by 10 at {60k, 80k} iterations on COCO and {15k, 20k} iterations on VOC. We use weight decay of 1 × 10^-4 and momentum of 0.9. 
For the head of FPN, we replace two fully-connected layers (2fc) with 4 convolutions and 1 fully-connected layer (4conv1fc), which is also used in all baselines in this work, e.g., ResNet-50, ResNet-101, ShuffleNetv2-20 and ShuffleNetv2-40.
For RetinaNet, the training settings is similar to FPN, except that the initial learning rate is 0.01.

Search on the trained supernet.  We split the detection datasets into a training set for supernet fine-tuning, a validation set for architecture search, and a test set for final evaluation. 
For VOC, the validation set contains 5k images randomly selected from qcrtrainval2007 + qcrtrainval2012 and the remains for supernet fine-tuning. For COCO, the validation set contains 5k images randomly selected from qcrtrainval35k <cit.> and the remains for supernet fine-tuning. For evolutionary search, the evolution process is repeated for 20 iterations. The population size |𝐏| is 50 and the parents size |𝒫| is 10. Thus, there are 1000 networks evaluated in one search.

Final architecture evaluation.  The selected architectures are also retrained in the pre-training and fine-tuning schedule. The training configuration is the same as that of the supernet.
For COCO, the test set is qcrminival.
For VOC, the test set is qcrtest2007. 
Results are mainly evaluated with COCO standard metrics (i.e. mmAP) and VOC metric (IOU=.5). 
All networks listed in the paper are trained with the “1x” training setting used in Detectron <cit.> to keep consistency with the supernet fine-tuning. 



§ EXPERIMENTAL RESULTS
 







 §.§ Main Results
 
Our main result architecture, DetNASNet, is searched on FPN in the large search space. 
The architecture of DetNASNet is depicted in the supplementary material.
We search on FPN because it is a mainstream two-stage detector that has been used in other vision tasks, e.g., instance segmentation and skeleton detection <cit.>.
Table <ref> shows the main results. We list three hand-crafted networks for comparisons, including ResNet-50, ResNet-101 and ShuffleNetv2-40. DetNASNet achieves 40.2% mmAP with only 1.3G FLOPs. It is superior to ResNet-50 and equal to ResNet-101. 

To eliminate the effect of search space, we compare with the hand-crafted ShuffleNetv2-40, a baseline in this search space. It has 40 blocks and is scaled to 1.3G FLOPs, which are identical to DetNASNet. ShuffleNetv2-40 is inferior to ResNet-101 and DetNAS by 0.8% mmAP on COCO. This shows the effectiveness of DetNAS without the effect of search space.

After that, we include the effect of search space for consideration and increase the channels of DetNASNet by 1.8 times to 3.8G FLOPs, DetNASNet (3.8).
Its FLOPs are identical to that of ResNet-50. It achieves 42.0% mmAP which surpasses ResNet-50 by 4.7 % and ResNet-101 by 2.0%.






 



 §.§ Ablation Studies
 
The ablation studies are conducted in the small search space introduced in Table <ref>. This search space is much smaller than the large one, but it is efficient and enough for making ablation studies.
As in Table <ref>, we validate the effectiveness of DetNAS with various detectors (FPN and RetinaNet) and datasets (COCO and VOC). All models in Table <ref> and Table <ref> are trained with the same settings described in Section <ref>. Their FLOPs are all similar and under 300M. 

Comparisons to the hand-crafted network.  

ShuffleNetv2-20 is a baseline network constructed with 20 blocks and scaled to 300M FLOPs. It has the same number of blocks to architectures searched in the search space. As in Table <ref>, DetNAS shows a consistent superiority to the hand-crafted ShuffleNetv2-20. DetNAS outperforms ShuffleNetv2-20 by more than 1% mmAP in COCO on both FPN and RetinaNet detectors. This shows that NAS in object detection can also achieve a better performance than the hand-crafted network.

Comparisons to the network for ImageNet classification.  

Some NAS works tend to search on small proxy tasks and then transfer to other tasks or datasets. For example, NASNet is searched from CIFAR-10 image classification and directly applied to object detection <cit.>. We empirically show this manner sub-optimal. ClsNASNet is the best architecture searched on ImageNet classification. The search method and search space follow DetNAS. We use it as the backbone of object detectors. ClsNASNet is the best on ImageNet classification in Table <ref> while its performance on object detection is disappointing. It is the worst in all cases, except a slightly better than ShuffleNetv2-20 on COCO-FPN. This shows that NAS on target tasks can perform better than NAS on proxy tasks.

Comparisons to the from-scratch counterpart.  

DetNAS-scratch is a from-scratch baseline to ablate the effect of pre-training.
In this case, the supernet is trained from scratch on detection datasets without being pre-trained on ImageNet. To compensate for the lack of pretraining, its training iterations on detection datasets are twice as those of DetNAS, that is, 180k on COCO and 45k on VOC. In this way, the computation cost of DetNAS and DetNAS-scratch are similar.
All other settings are the same as DetNAS. Both DetNAS and DetNAS-scratch have consistent improvements on ClsNASNet and ShuffleNetv2-20. This shows that searching directly on object detection is a better choice, no matter from scratch or with ImageNet pre-training. In addition, DetNAS performs also better than DetNAS-scratch in all cases, which reflects the importance of pre-training.

Comparisons to the random baseline.  

As stated in many NAS works <cit.>, the random baseline is also competit
ive. In this work, we also include the random baseline for comparisons as in Table <ref>. In Figure <ref>, the mmAP curve on the supernet search are depicted to compare EA with Random. For each iteration, top 50 models until the current iteration are depicted at each iteration. EA demonstrates a clearly better sampling efficiency than Random.
In addition, we randomly pick 20 networks in the search space and train them with the same settings to other result models. 
On ImageNet classification, the random baseline is comparable to DetNAS. But on the object detection tasks, DetNAS performs better than Random. In Figure <ref>, we depicted the scatter and the average line of random models and the line of DetNAS. DetNAS in the small search space is 36.6% while the Random is 35.6±0.6%. From these points of view, DetNAS performs better than Random not only during search but also in the output models.
  


 



 §.§ DetNAS Architecture and Discussions
 
Our architectures searched for object detectors show meaningful patterns that are distinct from architectures searched for image classification. 
Figure <ref> illustrates three neural architectures searched in the 20 blocks search space. 
The architecture in the top of Figure <ref> is ClsNASNet. The other two are searched with FPN and RetinaNet detectors respectively.
 These architectures are depicted block-wise. The yellow and orange blocks are 5×5 and 7×7 ShuffleNetv2 blocks. The blue blocks have kernel sizes as 3. The larger blue blocks are Xception ShuffleNetv2 blocks which are deeper than the small 3×3 ShuffleNetv2 blocks.  Figure <ref> illustrates the architecture of DetNASNet. It has 40 blocks in total and {8, 8, 16, 8} blocks each stage. 
 
In contrast to ClsNASNet, architectures of DetNAS have large-kernel blocks in low-level layers and deep blocks in high-level layers.
In DetNAS, blocks of large kernels (5×5, 7×7) almost gather in low-level layers, Stage 1 and Stage 2. In contrast, ClsNASNet has all 7×7 blocks in Stage 3 and Stage 4. This pattern also conforms to the architectures of ProxylessNAS <cit.> that is also searched on image classification and ImageNet dataset.
On the other hand, blocks with blue color have 3×3 kernel size. As in the middle of Figure <ref>, blue blocks are almost grouped in Stage 3 and Stage 4. Among these 8 blue blocks, 6 blocks are Xception ShufflNetv2 blocks that are deeper than common 3×3 ShufflNetv2 blocks.
In ClsNASNet, only one Xception ShuffleNetv2 block exists in high-level layers. 
In addition, DetNASNet also shows the meaningful pattern that most high-levels blocks have 3×3 kernel size. Based on these observations, we find that the networks suitable for object detection are visually different from the networks for classification. Therefore, these distinctions further confirm the necessity of directly searching on the target tasks, instead of proxy tasks.








§ CONCLUSION
 
 We present DetNAS, the first attempt to search backbones in object detectors without any proxy. Our method consists of three steps: supernet pre-training on ImageNet, supernet fine-tuning on detection datasets and searching on the trained supernet with EA. Table <ref> shows the computation cost for each steps. The computation cost of DetNAS, 44 GPU days on COCO, is just two times as training a common object detector. In experiments, 
the main result of DetNAS achieves superior performance than ResNet-101 on COCO and the FPN detector with much fewer FLOPs complexity.
We also make comprehensive comparisons to show the effectiveness of DetNAS.
We test DetNAS on various object detectors (FPN and RetinaNet) and different datasets (COCO and VOC).
For further discussions, we spotlight the architecture-level gap between image classification and object detection. ClsNASNet and DetNAS have different and meaningful architecture-level patterns. This might, in return, provide some insights for the hand-crafted architecture design.



§ ACKNOWLEDGEMENT

This work is supported by Major Project for New Generation of AI Grant (No. 2018AAA0100402), National Key R&D Program of China (No. 2017YFA0700800), and the National Natural Science Foundation of China under Grants 61976208, 91646207, 61573352, and 61773377.
This work is also supported by Beijing Academy of Artificial Intelligence (BAAI).





§ SUPPLEMENTARY MATERIAL




 §.§ Algorithms

DetNAS consists of three steps: supernet pre-training on ImageNet, supernet fine-tuning on detection datasets and architecture search on the trained supernet with EA. We formulate them in algorithms here. The first two steps are combined into Algorithm <ref>. The third step is formulated in to Algorithm <ref>.

[t]0.5
  



[t]0.5
  







 §.§ Comparisons in the “2x” Settings



Results in the paper are trained with the “1x” setting in Detectron to keep consistency with the supernet training. Here we report the comparisons in “2x” setting. DetNASNet and DetNASNet (3.8) are still superior to the hand-crafted ResNet-50, ResNet-101 and ShuffleNetv2-40.



 §.§ Inference time comparisons


 

Although inference time is not the target of this work, we measure the FPS to avoid the concern about the speed of DetNASNet.
DetNASNet processes 5 more frames per second than ResNet-101. DetNASNet (3.8) is only 2 FPS slower than ResNet-50 but has a much better mAP.
