using the teacher knowledge. However, the naive usage of a fixed period may not be efficient in their approach. Moreover, their shallow model struggles to handle emerging new objects in the scene when these objects are observed in the middle of the fixed period. Here, ours is able to select the period length based on the incoming frames, by which TKD could trigger re-training and thus detecting the emerging new objects, as demonstrated experimentally in Sec. \[sec:exp\].

Temporal Knowledge Distillation {#distillation}
===============================

The conventional use of knowledge distillation has been proposed for training CNNs based classification models. In these models, we have a dataset $(x_{i},y_{i}), i=1,2,...,n$ where $x_{i}$ and $y_{i}$ are input images and the class labels. The student model is trained to optimize the following general loss function (with $\beta$ is a modulation factor): $$\small
\setlength{\abovedisplayskip}{5pt}
\setlength{\belowdisplayskip}{5pt}
\centering
\begin{aligned}
& O_s = Student(x); O_t= Teacher(x), \\
& L(O_s,(y,O_t))  = \beta L_{gt}(O_s,y) + (1-\beta)L_t(O_s,O_t), \label{eq:a}
\end{aligned}$$ where $L_t$ is the loss using teacher output ($O_t$) and $L_{gt}$ is the loss using ground truth $y$ [@mehta2018object; @chen2017learning; @hinton2015distilling].

In addition to the classification task, object detection also could benefit from the knowledge distillation procedure. However, it’s not as straightforward as the 