function. They generate boxes and labels using both the student and the teacher model, then calculate two loss values comparing the teacher’s output and the ground truth. In the end, they sum up the weighted loss values. If the student model outperforms the teacher model, they continue training only using ground-truth supervision. More recently, Mehta et al. [@mehta2018object] applied the similar procedure to the one-stage object detection models (Tiny-Yolo v2 with some modification). They generate bounding boxes and labels, and apply Non-Maximum Suppression (NMS) to these boxes and then follow the loss function to optimize the student model. The loss is defined in the following equation: $$\small
\setlength{\abovedisplayskip}{5pt}
\setlength{\belowdisplayskip}{5pt}
\begin{aligned}
  L_{final}= L_{bb}^{C}(b_i^{gt}, \hat{b_i}, b_i^{T}, o_i^{T}) + L_{cl}^{C}(p_i^{gt}, \hat{p_i}, p_i^{T}, o_i^{T})\\
  + L_{obj}^{C}(o_i^{gt}, \hat{o_i}, o_i^{T}) ,
  \end{aligned}
  \label{eq:c}$$ where $L_{bb}^{C},L_{cl}^{C},L_{obj}^{C}$ are objectness loss, classification loss and regression loss which are calculated using both ground truth and the teacher output. Also, $\hat{b_i},\hat{p_i},\hat{o_i}$ are bounding box coordinates, class probability and objectness of the the student model. $b_i^{gt},p_i^{gt},o_i^{gt}$ and $b_i^{T},p_i^{T},o_i^{T}$ are values derived from ground truth and the teacher model output.

In our study of the Yolo-v3 and Tiny-Yolo models, we noticed that the detection layer is the most computationally expensive part. In this 