is also adopted to calculate the weights among nodes, however, their graph topology are predefined, thus they only model the edges weights dynamically. In traffic networks, there are some hidden spatial dependencies that are not fully reflected by the road topology, thus the graph should also be dynamical constructed as well as the edge weights. The self-attention mechanism we adopted in this paper is shown in Fig.4.

Formally, the input features (e.g traffic condition together with spatial-temporal position information) are first projected into latent high dimensional subspaces with learnable mappings which are realized with feed-forward neural networks. Basically, for single-head attention model which model one relationship pattern, three subspaces are obtained for each node, namely, query subspace $Q^{\mathcal S}\in {\mathbb R}^{N\times d^{\mathcal S}_q}$, key subspace $K^{\mathcal S}\in {\mathbb R}^{N\times d^{\mathcal S}_k}$ and value subspace $V^{\mathcal S}\in {\mathbb R}^{N\times d^{\mathcal S}_v}$. And the latent subspace learning process can be formulated as $$\begin{array}{lcr}
       Q^{\mathcal S}=X^{{\mathcal S}}W^{\mathcal S}_q\\
       K^{\mathcal S}=X^{{\mathcal S}}W^{\mathcal S}_k\\
       V^{\mathcal S}=X^{{\mathcal S}}W^{\mathcal S}_v
       \end{array}$$ where $W^{\mathcal S}_q, W^{\mathcal S}_k, W^{\mathcal S}_v$ are the weight matrices for $Q^{\mathcal S}, K^{\mathcal S}, V^{\mathcal S}$, respectively.

After obtaining the three latent high dimensional subspaces, dynamical spatial dependencies $S^{\mathcal S}\in {\mathbb R^{N\times N}}$ are 