also initialized with one-hot encode and updated during training as ${X^{\mathcal T}=[Y^{\mathcal S}, D^{\mathcal T'}]}$.

And then self-attention mechanism is also adopted to model temporal dependencies. The input to the temporal transformer is a temporal sequence $X^{\mathcal T}\in {\mathbb R}^{M\times d^{\mathcal T}}$ with a slide window of length $M$ and $d^{\mathcal T}$ channels. Similar to spatial transformer, temporal dependencies are dynamically computed in high-dimensional subspaces. The three subspaces, the queries $Q^ {\mathcal T}\in {\mathbb R}^{M\times d^{\mathcal T}_q}$, the keys $K^{\mathcal T}\in {\mathbb R}^{M\times d^{\mathcal T}_k}$, and the values $V^{\mathcal T}\in {\mathbb R}^{M\times d^{\mathcal T}_v}$, can be computed as $$\begin{array}{lcr}
       Q^ {\mathcal T}=X^{{\mathcal T}}W^{\mathcal T}_q\\
       K^{\mathcal T}=X^{\mathcal T}W^{\mathcal T}_k\\
       V^{\mathcal T}=X^{\mathcal T}W^{\mathcal T}_v
       \end{array}$$ where $W_q\in {\mathbb R}^{d^{\mathcal T}\times d^{\mathcal T}_q}$,$W_k\in {\mathbb R}^{d^{\mathcal T}\times d^{\mathcal T}_k}$ and $W_v\in {\mathbb R}^{d^{\mathcal T}\times d^{\mathcal T}_v}$ are the learned liner mappings.

Unlike RNNs-based models which temporal dependencies are limited to previous time steps, we learn bidirectional temporal dependencies with scaled dot-product function as $$S^{\mathcal T}=\text{softmax}(\frac{Q^{\mathcal T}(K^{\mathcal T})^T}{\sqrt{d^{\mathcal T}_k}})$$ Then, the temporal features are obtained by aggregating the values $V^{\mathcal T}$ weighted by the bidirectional temporal dependencies. $$M^{\mathcal T}=S^{\mathcal T}V^{\mathcal T}$$ The temporal dependencies modeling process is illustrated in Fig.4. To explore the interactions among 