the fact that LSTM is a model for the short-term memory which can last for a long period of time. An LSTM is well-suited to classify, process and predict time series given time lags of unknown size and duration between important events. Besides, LSTM shows the promising result in sentence encoding in many NLP applications  [@sutskever2014sequence; @gers2002learning]. The LSTM archticture are shown in figure \[fig:traditional LSTM neural network\], the computations of LSTM cells are:

[1.0]{} $$\begin{aligned}
    f_t = \sigma(W_f[h_{t-1},x_t] + b_f)\end{aligned}$$ $$\begin{aligned}
     i_t = \sigma(W_i[h_{t-1},x_t] + b_i)\end{aligned}$$ $$\begin{aligned}
    \tilde{C}_t = tanh(W_C[h_{t-1}, x_t] + b_C)\end{aligned}$$ $$\begin{aligned}
    C_t = f_t \otimes C_{t-1} + i_t \otimes \tilde{C}_t\end{aligned}$$ $$\begin{aligned}
     o_t = \sigma(W_o[h_{t-1},x_t] + b_o)\end{aligned}$$ $$\begin{aligned}
    h_t = o_t \otimes tanh(C_t)\end{aligned}$$

The forget gate is described in Equation (1) and is used to decide whether a value should be reserved. The Input gate in Equation (2) is used to control which values should be updated through a certain time step. Here, a new value $\tilde{C}_t$ is created as per Equation (3) using the $tanh$ function. Next, with Equation (4), the cell state will be updated from $C_{t-1}$ to $C_t$, $f_t$ and $i_t$ are used here to decide whether an information should be discarded. in Equation 