layer, several processes are done (sorting, applying softmax to classification cells, removing low confidence boxes, etc.) to produce bounding boxes and then applying NMS to these boxes. These processes are computationally slow due to the multiple steps of processing, and also running over CPU by the implementation. Consequently, directly adopting these loss functions will be also computationally expensive during the inference stage.

With this observation, we adopt the mean square error (MSE) between the tensors generated by the student decoder and the oracle decoder, which should be the fastest method. However, the side effects are also notorious. The oracle model generates noises over some parts of frame which have no object existences; hence directly forcing the student model to retrain will hurt its performance.

[0.6]{}

[0.4]{}

Another approach could be calculating the MSE between the tensor cells which have high confidence of object existence. But, the approach will hurt the student’s recognition accuracy too. By applying this loss function, the student model tends to generate redundant detection boxes which yield a larger number of false positives.

To alleviate the downsides of both loss designs and still to preserve their advantages, we introduce a novel loss by a combination of them 