start our decision process which we formulate in Equations \[eq:d\]: $$\small
\setlength{\abovedisplayskip}{5pt}
\setlength{\belowdisplayskip}{5pt}
\begin{aligned}
& I \in  \{0,1\} \left\{\begin{matrix}
 &0  &Do \ not \ distill \ knowledge, \\ 
 &1  &Distill \ knowledge, 
\end{matrix}\right.
\\
& I = LSTM(F_s) \ \vee \ I_R, \quad  I_R\sim B(2,P_t),
\\
& P_t=\left\{\begin{matrix}
 &max((P_{t-1}-0.05),0.05) \ \ \ \ \ \ \  \Delta L <\sigma,  &   \\ 
 &min(2P_{t-1},1.0) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \  \Delta L >\sigma,   & 
\end{matrix}\right. 
\end{aligned}
\label{eq:d}$$ where $I$ is the indicator that denotes our final decision. It takes the disjunction of the LSTM’s output and the random module’s output. We pass the features extracted from the student model $F_s$ (the last layer before the decoder) to the LSTM module (with one LSTM layer & one fully connected layer) which outputs a signal indicating to train the student model or not. Here, it is worth to note that we introduce another binary random module $I_R$ (with binomial distribution $B(2,p_t)$) which decides in a random fashion to train the student model or not. The random procedure is added as a safeguard in case the LSTM model outputs a 