Training Quantized Neural Networks with a Full-precision Auxiliary Module
Bohan Zhuang Lingqiao Liu Mingkui Tan Chunhua Shen Ian Reid








Training Quantized Neural Networks with 

a
Full-precision Auxiliary Module
    Bohan Zhuang^1      Lingqiao Liu^1      
Mingkui Tan^2       Chunhua Shen^1

Corresponding author, e-mail: chunhua.shen@adelaide.edu.au       Ian Reid^1


	^1The University of Adelaide     
	^2South China University of Technology
	
	

    December 30, 2023
==============================================================================================================================================================================================================================================





















In this paper, we seek to tackle a challenge in training low-precision networks: the notorious difficulty in propagating gradient through a low-precision network due to the non-differentiable quantization function. We propose a solution by training the low-precision network with a full-precision auxiliary module.  Specifically, 

during training, we construct a mix-precision network by augmenting the original low-precision network with the full precision auxiliary module. Then the augmented mix-precision network and the low-precision network are jointly optimized.

This strategy creates additional full-precision routes to update the parameters of the low-precision model, thus making the gradient back-propagates more easily.



At the inference time, we discard the auxiliary module without introducing any computational complexity to the low-precision network.

We evaluate the proposed method on image classification and object detection over various quantization approaches and show consistent performance increase. In particular, we achieve near lossless performance to the full-precision model by using a 4-bit detector, which is of great practical value.





§ INTRODUCTION

Deep neural networks (DNNs) have made great strides in many computer vision tasks such as image classification <cit.>, segmentation <cit.> and detection <cit.>. Even though deep and/or wide models can achieve promising accuracy, their huge computational complexity makes them incompatible with energy constrained devices which usually have limited memory bandwidth and computational power. This has motivated the community 

to design energy-efficient models, often based on quantized precision, aiming not to sacrifice accuracy relative to the full-precision models. In this paper, we propose to improve the training of the low-precision networks.

The core challenge for quantization is the non-differentiability of the discrete quantizer. As a result, we cannot directly optimize the discretised network with stochastic gradient descent. And the
current solutions can be divided into two categories. The first category is to employ a surrogate of gradient. 
The most commonly used approach is the straight-through estimator (STE) <cit.>. Some recent works have been proposed to relax the discrete quantizer to be continuous for gradient-based optimization <cit.>. Even though the discontinuity of the discretization operation during training can be partly solved by smoothing it appropriately, some important information may still be missed due to the approximation which can lead to an undesirable drop in accuracy.
The second category is to seeking guidance from a full-precision model for discretised network training. For example, new training strategies such as knowledge distillation <cit.> have been proposed to learn a low-precision student network by distilling knowledge from a full-precision teacher network. 


Our method falls into the second category and our approach is based on the idea of sharing parameters between a mixed-precision (partially fully-precision) model and a low-precision model. 


Specifically, our method constructs a full-precision auxiliary module which connects to multiple layers of a low-precision model (see Fig. <ref>).

During training, the low-precision network and the full-precision auxiliary module will be combined to form an augmented mixed-precision network. Then, the mixed-precision network and the low-precision model are jointly optimized. Since the parameters from the low-precision model are shared, they can receive gradient from both full-precision connections and low-precision connections. Consequently, the parameters of low-precision model can be updated via two routes, and this can overcome the gradient propagation difficulty due to the discontinuity of the quantizer. Note that only the low-precision network will be utilized for inference and therefore there is no additional complexity introduced at the test stage. 



In addition to image classification, we further extend the proposed approach to building quantized networks for object detection. Building low-precision networks for object detection is more challenging since detection needs the network outputs richer information, such as locations of bounding boxes. 

There has been several works in literature to address the quantized object detectors <cit.>. However, there still exists a significant performance drop of 4-bit or lower-precision quantized detectors comparing to their full-precision counterpart. We apply our techniques to train a 4-bit RetinaNet <cit.> detector and further propose a modification to RetinaNet to better accommodate the quantization design. Through extensive experiments on the COCO benchmark, we show that our 4-bit models can achieve near lossless performance comparing to the full-precision model,  which has a significant value in practice.











Our contributions can be summarized as follows:


-0.125cm

     
  * We propose a new training method to account for the non-differentiability of the quantization operator in a low-precision network. Our method can lead to more accurate low-precision model without increasing the model complexity at the testing stage. 
     
     
     

	
  * We apply our learning approach and propose a new design modification to build a 4-bit quantized object detection which achieves comparable performance to its full-precision counterpart.
	
	
	

	






 §.§ Related work

	
Network quantization.  
Quantized network represents the weights and activations with very low precision, thus yielding highly compact DNN models compared to their floating-point counterparts. Moreover, the convolution operations can be efficiently computed via bitwise operations.
Quantization can be categorized into fixed-point quantization and binary neural networks (BNNs), in which fixed-point quantization can also be divided into uniform and non-uniform. Uniform approaches <cit.> design quantizers with a constant quantization step. To reduce the quantization error, non-uniform strategies <cit.> propose to learn the quantization intervals by jointly optimizing parameters and quantizers. 
A fundamental problem of quantization is to approximate gradient of the non-differentiable quantizer. To solve this problem, some works have studied relaxed quantization <cit.>. 

Moreover, with the popularity of automatic machine learning, some recent literature employs reinforcement learning to search for the optimal bitwidth for each layer <cit.>.
BNNs <cit.> constrain both weights and activations to binary values (, +1  or  -1), which brings great benefits to specialized hardware devices. The development of BNNs can be classified into two categories: (i) a focus on improving the training of BNNs <cit.>; (ii) multiple binarizations to approximate the full-precision tensor or structure  <cit.>. In this paper, we propose a general auxiliary learning approach that can work on all categories of quantization approaches.


Weight sharing.	
Weight sharing has been attracting increasing attention for efficient, yet accurate computation. In visual recognition, region proposal networks (RPN) in Faster-RCNN <cit.> and Mask-RCNN <cit.> share the same backbone with task-specific networks, which greatly saves testing time.
For neural architecture search, ENAS <cit.> allows parameters to be shared among all architectures in the search space, which saves orders of magnitude GPU hours. 
In the network compression field, weight/activation quantization intends to partition the weight/activation distribution into clusters and use the centers of clusters as the possible discrete values. This strategy can be interpreted as a special case of weight sharing.
Different from these approaches, we propose to utilize weight sharing 
for jointly optimizing the full-precision auxiliary module and the original low-precision network to improve the accuracy of the latter quantized model.


Auxiliary supervision.
One straightforward way of adding auxiliary supervision is introducing additional losses into intermediate layers, which serves to combat the vanishing gradient problem while providing regularization. The effectiveness of additional losses has been demonstrated in some literature, like GoogLeNet <cit.>, DSN <cit.>, semantic segmentation <cit.>, etc.

However, these methods are usually sensitive to the positions and scales of the guidance signals. Knowledge distillation (KD) is initially proposed for model compression, where a powerful wide/deep teacher distills knowledge to a narrow/shallow student to improve its performance <cit.>, which can also be treated as adding auxiliary supervisions. 
In terms of the definition of knowledge to be distilled from the teacher, existing models typically use teacher's class probabilities <cit.> and/or intermediate features <cit.>.
It is worth noting that our proposed auxiliary learning strategy uses weight sharing to assist optimization, where the motivation is very different from the KD methods.
We does not need to pre-train a teacher network which is usually much deeper and may be the upper bound of the performance. Moreover, on network quantization, we show consistent superior performance over KD methods in Sec. <ref>.


Object detection.
Object detection can be divided into two categories. As one of the dominant detection framework, two-stage detection methods <cit.> first generate region proposals and then refine them by subsequent networks.
Another main category is the one-stage methods which are represented by YOLO <cit.>, SSD <cit.> and RetinaNet <cit.>. The objective is to improve the detection efficiency by directly classifying and regressing the pre-defined anchors without the proposal generation step.
The recent developing trends in object detection is designing light-weight frameworks for mobile applications <cit.>, which usually requires real-time, low-power and fully embedded.

In this paper, we explore to compress and accelerate detectors from the quantization perspective.
Note that, 
we are the first to achieve near lossless 4-bit detectors in the literature.







§ METHOD


In this section, we describe the proposed learning strategy for training a low-precision network. We first give an overview of the proposed approach in Sec. <ref>. Then we introduce the auxiliary module design in Sec. <ref> and optimization in Sec. <ref>, respectively. After that, we provide discussions in Sec. <ref> and explain how to extend it to quantized object detection in Sec. <ref>. Through the following part, we adopt the following terminologies: A layer is a standard parameterized layer in a network such as a dense or convolutional layer. A block is a collection of layers in which the output of its last layer is connected to the input of the next block (, a residual block). 




 §.§ Overview and motivation
 
The overview of the framework is shown in Fig. <ref>. The blue part, denoted as F, shows the low-precision network we aim to learn. The pink part, denoted as H, is a full-precision sub-network, which we call the auxiliary module. It connects to intermediate outputs from F. The input image is fed into F but generates two outputs, one is from the last layer in F and the other is from the last layer in H. In other words, the combination of F and H forms an augmented mix-precision network F∘ H and the parameters of the low-precision network is shared by this augmented network. At the training time, two loss functions are applied to both outputs, and the mixed-precision network and low-precision network are trained jointly. After training, the auxiliary module H will be discarded and only F will be used at the test time.

The motivation of such a design is to create full-precision routes to update parameters of the low-precision model and thus alleviating the difficulty of propagating gradient in a quantized model. Specifically, the intermediate output of each block in F can directly influence the output of the mixed-precision network F∘ H through the full-precision connections in H. Consequently, the gradient from the loss of the second output will back propagate to the parameters of each block in the low-precision model.     










 §.§ Module design
 






We now elaborate the design for the auxiliary module H, which is made up of a sequential of adaptors and aggregators as shown in Fig. <ref> (a).
In particular, the auxiliary module H receives P output feature maps {O_p} _p = 1^P of the corresponding blocks in F.
Let {B_1,...,B_P} be the block indexes where we generate the feature maps.
For the p-th input of H, we adopt a trainable adaptor ϕ_p ( · ), which receives the output feature map O_p of the B_p-th block from F and outputs an adapted feature representation
ϕ _p(O_p).
The motivation of using the adapter is to compensate the distribution discrepancy between the low-precision model and full-precision model. It ensures the quantized activations {O_p} _p = 1^P to be compatible to the full-precision calculation in H. We implement those adapters by a simple 1×1 convolutional layer followed by a batch normalization layer in this paper.












In the auxiliary module H, the outputs of the adaptor are then sequentially aggregated. Formally, let _p denotes the p-th aggregated feature. It is achieved by adding the adapted feature ϕ _p(O_p) from F and the (p-1)-th aggregated feature from H followed by a ReLU(·) nonlinearity:

    _p = ReLU(ϕ_p (O_p) + _p - 1).

At the last layer in H, a classifier layer is applied to _P to make the class prediction. Then an auxiliary loss is employed. Note that the auxiliary module H is akin the skip connections in ResNet <cit.>.






 §.§ Optimization
 

Let {x_i,y_i} _i = 1^N be the training samples. The proposed method jointly optimize the main network F and the mixed-precision network which is the combination of F and H, denoting as F ∘ H. The training objective is:


    min_{ ^F, ^H}∑_i = 1^N     L(F(_i; ^F),_i ) 
       +  L_aux((F∘ H)(_i; ^H, ^F),_i ),

where ^F and ^H represent the parameters for the backbone F and the auxiliary module H, respectively.  is the task objective and _aux is the auxiliary loss. 

In the classification task, both terms are set to the cross-entropy loss.


From Eq. (<ref>), we can note that ^F is shared among F and F ∘ H.

Following the chain rule, the gradient of ^F will have an additional term comes from L_aux. As a result, the approximated gradient is averaged from both the mixed-precision network and the original low-precision network to achieve more accurate updating direction.
In other words, the full-precision module H provides direct gradient for F using weight sharing during back-propagation. 
We summarize the proposed learning process for a quantized neural network in Algorithm <ref>. 

	



















 §.§ Relationship to other methods
 

In this section, we will elaborate the relationship between the proposed auxiliary learning and other related approaches. 




Differences to applying auxiliary classification losses to intermediate activations.
An alternatively way to create auxiliary full-precision routes for learning low-precision network is to apply classification losses to intermediate activations. The schematic  illustration of this idea is shown in Fig. <ref> (a), where each intermediate output is attached to a classifier to perform classification.  In such a case, the final training objective becomes:

    _obj = + ∑_i = 1^M α _iℓ_i,
where  is the classification loss of the original low-precision network, ℓ_i is the classification loss applies to the i-th intermediate output, and α_i is the weight associated to the i-th loss function. 

This scheme can directly propagate gradient to each block during training through the full-precision classifiers. However, its supervision is very restrictive since it essentially assumes that the intermediate output can be directly used for classification. In practice, we often find that choosing the positions of adding the additional supervisions or the weight α_i can be challenging. An inappropriate setting of those factors may lead to inferior performance than that achieved by directly training the low-precision model. 










Differences with knowledge distillation.
Knowledge distillation (KD) has been explored to assist the quantized model training <cit.>. In particular, a low-precision student network learns to generate similar posterior probabilities and/or feature representations of a full-precision teacher network (see Fig. <ref> (b)). The training objective can be formulated as  

    _obj = _1 + _2 + ∑_i = 1^M β _iℓ̂_i ,

where _1 and _2 are task-specific objectives for student and teacher networks respectively. ℓ̂_i represents the i-th distillation loss. 

Although both the proposed method and KD use a full-precision network to guide the training of a low-precision network, the ways of exerting this guidance are significantly different in KD and the proposed method. Specifically, in KD, the guidance from the full-precision model to the low-precision model is from the distillation losses while in our method this is achieved by making the parameters of a low-precision model shared with the mixed-precision model. There are many advantages of our method comparing to KD: (1) Our method only needs an extra memory to store the auxiliary module rather than a full-precision network. Comparing to KD, our method is more memory-efficient. (2) Our method only uses a single auxiliary loss but can create guidance signals to various blocks of the low-precision model. In contrast, KD needs multiple distillation losses to achieve this. Thus it usually involves more hyper-parameters, i.e., the weight of each loss term β_i. Moreover, we empirically find that the proposed learning strategy performs consistently superior than KD for learning quantized networks in Sec. <ref> and Sec. <ref>.












§ EXTENSION TO OBJECT DETECTION
 
 

Most existing methods evaluate low-precision network with the classification task. Building a low-precision network for more difficult object detection task remains a challenge.
To fill this gap, we further extend our method to build a quantized object detector. 

Following the work in <cit.>, we consider the object detection framework consisting of a backbone, a feature pyramid and prediction heads. We directly use the quantized network pretrained on the ImageNet classification task to initialize the detection backbone. We adopt the uniform quantization approach QIL <cit.> to quantize both weights and activations, where the quantization intervals are explicitly parameterized and jointly optimized with the network parameters.
We add an individual auxiliary module for each prediction head while sharing a single module for the backbone. 


Beside of applying the proposed auxiliary module and learning strategy to help training the quantized detector, we also propose a modification which we find beneficial. Specifically, unlike <cit.> which freezes the batch normalization (BN) statistics during training to stabilize optimization, we instead propose an alternative strategy where the BN statistics still keeps updating: we propose that except the last layers for classification and regression, parameters of the prediction heads are not shared across all feature pyramid levels. This is different from the common full-precision setup. The motivation of this design is that the multi-scale semantic information can not be encoded effectively due to the quantization process at different pyramid levels. For a full-precision network, using a shared head is sufficient to represent rich semantic information for classification and regression with the continuous activations.  However, 
in the low-precision setting, the representational capability of activations is highly degraded due to its discrete values. For the same reason, the batch statistics of quantized activations may differ drastically across different levels.
Therefore, each head should learn independent parameters to capture the corresponding multi-scale information.


Remark: (1) The proposed modification does not share prediction heads and thus uses more parameters in the low-precision model. However, we should note that without sharing does not increase any additional computational complexity.  Even though the number of parameters is increased, the memory consumption is still significantly reduced comparing to the full-precision model due to the low-bit storage. 



(2) We empirically find that not sharing heads may not improve (but reduce) the performance in the full-precision setting. So the proposed modification is only for low-precision networks. Please check the experiments in Sec. <ref> for more discussions. 















§ EXPERIMENTS
 
In this section, we evaluate our proposed methods on image classification in Sec. <ref> and object detection in Sec. <ref>, respectively.
To investigate the effectiveness of the proposed method, we define several methods for comparison: 
Auxi: We optimize the network with the auxiliary module. 
KD: We employ the joint knowledge distillation in <cit.> to improve the quantized network.
Additional loss: We evenly insert classification losses at intermediate layers to assist training. Note that we will detail the settings in specific sections.




 §.§ Experiments on image classification
  

We perform experiments on two standard image classification datasets: CIFAR-100 <cit.> and ImageNet <cit.>. The CIFAR-100 dataset consists of 60,000 color images of size 32 × 32 belonging to 100 classes. There are 50,000 training and 10,000 test images. ImageNet contains about 1.2 million training and 50K validation images of 1,000 object categories. 
To verify the effectiveness of the proposed auxiliary learning strategy, we experiment on various representative quantization approaches, including uniform fixed-point approach DoReFa-Net <cit.>, non-uniform fixed-point method LQ-Net <cit.>, as well as binary neural network approaches BiReal-Net <cit.> and Group-Net <cit.>. 





  §.§.§ Implementation details

Following previous approaches <cit.>, we quantize all the convolutional layers to ultra-low precision except the first and last layers. However, to further improve the efficiency, we quantize the first convolutional layer and the last fully-connected layer to 8-bit.
We first pre-train the full-precision counterpart as initialization and then fine-tune the quantized model. For all ImageNet experiments, training images are resized to 256 × 256, and 224 × 224 patches are randomly cropped from an image or its horizontal flip, with the per-pixel mean subtracted. We use the single-crop setting for testing. No bias terms are used. We use SGD optimizer for the pre-training stage. For the fine-tuning stage, we adopt the Adam optimizer <cit.>. The mini-batch size is set to 256. We train a maximum 35 epochs and decay the learning rate by 10 at the 25-th and 30-th epochs. For fine-tuning the fixed-point methods <cit.>, the learning rate is initialized to 1e-3. For fine-tuning binary neural networks <cit.>, the initial learning rate is set to 5e-4. In practice, we take the output of each residual block  <cit.> as the input of the auxiliary module. 
Our implementation is based on PyTorch.  




  §.§.§ Effect of the auxiliary module
  








In this section, we explore the effect of auxiliary module on assisting the low-precision network optimization. 
The results are reported in Table <ref>. By combining the baseline with Auxi, we can observe a steady performance increase compared with the original baseline. This strongly supports that the learned auxiliary gradient can facilitate the convergence of the low-precision model. In particular, the gradient of the shared weights is averaged from both the auxiliary module and the original low-precision network to achieve more accurate updating direction. Moreover, increasing gradient paths is important to solve the non-differentiability of the discrete quantization process has been proved by <cit.>. 
To make it more clear, we plot the convergence curves of DoReFa-Net with ResNet-50 in Fig. <ref>. From the figure, we can observe that baseline + Auxi converges much faster and better than the baseline. After the first epoch, the baseline + Auxi outperforms baseline by ∼20% on the Top-1 accuracy. This result strongly justifies that the auxiliary module efficiently solves the non-differentiable problem by providing accurate hierarchical gradient for updating parameters during back-propagation.

It is worth noting that when the network becomes deeper (e.g., ResNet-50, 101), the improvement turns out to be more obvious. For instance, Auxi brings 3.8% Top-1 accuracy increase over the 2-bit baseline on ResNet-101. It can be attributed to that when the quantized network goes deeper, the optimization becomes more difficult due to the non-differentiable discretization process. However, the full-precision auxiliary module can provide direct hierarchical gradient to effectively solve this problem.
Note that with the basic uniform DoReFa-Net baseline, we currently achieve the comparing results with state-of-the-arts on ResNet-50 and ResNet-101 <cit.> without advanced non-uniform or relaxation strategies. 








  §.§.§ Comparison with other related approaches
 


In this section, we compare the auxiliary module with the related approaches discussed in Sec. <ref> and report the performance in Table <ref>.
The experiments are based on the 2-bit DoReFa-Net with ResNet-18, ResNet-34 and ResNet-50 on ImageNet. For KD experiments, the results are directly cited from <cit.>.
We can observe that introducing additional losses into intermediate layers does not show obvious improvement to the final performance.
Compared to KD, we don't need to pre-train a complex teacher network whose quality is sensitive to the final performance. In contrast, we propose a simpler yet effective weight sharing strategy to jointly optimize the low-precision network and the floating-point auxiliary module. More detailed analysis on the difference between two approaches can be referred to Sec. <ref>.
From the results, we can observe that Auxi consistently outperforms KD. For instance, on ResNet-50, Auxi  
exceeds KD by 2.4% on the Top-1 accuracy.
These results justify that the auxiliary module can effectively solve the non-differentiable problem during back-propagation in the low-precision network training. With a very different learning strategy, Auxi shows consistently superior empirical results than KD. We therefore argue that the proposed auxiliary learning may be a substitute to KD methods  <cit.> in network quantization. 











  §.§.§ Experiments on plain networks
   




 

We further explore an interesting by-product of the auxiliary module for network quantization. We assume that the auxiliary module mimics the effect of skip connections and can partially share its effect.
We therefore analyze training a plain low-precision network without skip connections. The results can be referred in Table <ref> and Table <ref>. 
plain represents we directly optimize a low-precision plain network without skip connections. 
By comparing plain and plain + Auxi, we observe apparent accuracy increase by incorporating Auxi.
For example, in LQ-Net ResNet-34 based experiments, introducing Auxi can boost the Top-1 accuracy by 5.1%. On tiny CIFAR-100 dataset, plain + Auxi even outperforms the Top-1 baseline. Moreover, same as the observation in Sec. <ref>,  Auxi still performs consistently better than KD. From the plain network setting, we can strongly justify that the auxiliary module can provide hierarchical gradient to promote convergence of the quantized network.

However, we still observe performance gap between plain + Auxi and the baseline on large-scale ImageNet. This can be attributed to two assumptions of skip connections. First, the skip connections may improve the convergence of training, as indicated by the improvement observed when using Auxi.

Second, the skip connection and the feature map after one convolution are added through a tensor addition. Then the representational capability (i.e., the value range) of each entry in the added activations is significantly enhanced. In other words, the plain network has less representational capability than its residual counterpart. 
 






  §.§.§ Effect of different auxiliary architectures
    


We further explore the influence of different auxiliary module architectures in Table <ref>.
From the table, we observe that increasing the complexity of the auxiliary module can further boost the performance. For example, by replacing the 1×1 convolution in the adaptor with a larger kernel of 3×3, we further get slightly performance gain. This can be attributed to that the gradient of shared parameters is averaged from F and F ∘ H, where better representational capability of H can result in more accurate gradient update.






 §.§ Experiments on quantized object detection
 

In this section, we evaluate the proposed approach on the general object detection task. Our experiments are conducted on the large-scale detection benchmark COCO <cit.>. Following <cit.>, we use the COCO trainval35k split (115K images) for training and minival split (5K images) for validation. We conduct experiments based on RetinaNet <cit.> and compare with the state-of-the-art FQN <cit.>. 




  §.§.§ Training details

Training is divided into two stages. In the first stage, the detection framework is kept to full-precision. The backbone is initialized with the classification model pre-trained on the ImageNet dataset. Unless specified, we use the same hyper-parameters with RetinaNet. Specifically, all training and evaluation images are resized so that their shorter edges are 800 pixels. We augment training images by random horizontal flipping while no evaluation augmentations are performed. 
Our network is trained with stochastic gradient descent (SGD) for 90K iterations with the initial learning rate being 0.01 and the batch size of 16. The learning rate is decayed by a factor of 10 at iterations 60K and 80K, respectively. 
In the second stage, we use the converged model in the first stage as initialization and fine-tune with quantization. This stage uses identical settings as the full-precision training, except that we use Adam optimizer and the initial learning rate is set to 1e-3. Our implementation is based on Detectron2 <cit.>.





  §.§.§ Performance evaluation


We report the performance of the proposed quantized detection framework in Table <ref>. From the results, we can observe that our 4-bit detector can achieve near lossless results over the full-precision counterparts, which meets the requirement for practical
deployment. Moreover, we can achieve significant performance boost over FQN on all comparing architectures. For example, on ResNet-50, the improvement reaches to 3.6 on AP.







  §.§.§ Ablation studies



We now perform ablation studies to give comprehensive analysis and insights of the quantized detection framework. The results are reported in Table <ref>. “Backbone only” indicates we only quantize the backbone to 4-bit while other parts are kept to full-precision. “Baseline” represents we directly quantize the RetinaNet using QIL <cit.>.  

When we only quantize the backbone network, we don't observe AP drop at least on ResNet-18. This justifies that 4-bit backbone can encode accurate features for further decoding. However, when quantizing all the components including feature pyramid and prediction heads, we can find obvious precision drop. The baseline result indicates that quantizing the continuous features to a fixed range of integers can cause great multi-scale information loss. 

We now explore the effect of the two strategies described in Sec. <ref>.
First, we incorporate the proposed auxiliary learning strategy to assist the convergence of quantized detector. Specifically, Baseline + Auxi can boost the AP of Baseline by 1.4.
Second, not sharing heads is particularly designed for the low-precision detector. We should note that this strategy deteriorates the performance in the full-precision setting. The reason is that a separate head can only see a certain range size of objects.
In contrast, not sharing heads can boost the AP by 1.3 in the quantization setting compared with the sharing heads counterpart. 
It can be attributed that when quantizing both weights and activations to 4-bit, the representational capability of each head is very limited and the statistics of activations differ a lot.
Therefore, each head should learn independent parameters to transform the corresponding level feature for classification and regression. 





§ CONCLUSION

In this paper, we have proposed an auxiliary learning strategy to tackle the 
non-differentiable quantization process in training low-bitwise convolutional neural networks. 

Specifically, we have explicitly utilized weight sharing to construct a full-precision auxiliary module. During training, the auxiliary module is combined with the low-precision network to form a mix-precision network, which is jointly optimized with the low-precision model.
In this way, the full-precision auxiliary module can provide direct hierarchical gradient during back-propagation to assist the optimization of the low-precision network. In the testing phase, the auxiliary module is removed without introducing any additional computational complexity.
Moreover, we have also worked on quantized object detection and proposed several practical solutions.
We have conducted extensive experiments based on various quantization approaches and observed consistent performance increase on the image classification and object detection. To be emphasized, we have achieved near lossless results using 4-bit detectors.






Acknowledgement

M. Tan was in part supported by Guangdong Provincial Scientific and Technological Funds under Grants 2018B010107001.
This work was in part supported by ARC DP Project `Deep learning that scales'.

ieee_fullname



