the quick propagation of prediction errors. In this paper, we proposed to address this problem with an effective modeling of long-range temporal dependencies as well as predicting multi-step results at the same time, as shown in Fig.2(b). One one hand, more information is utilized to make predictions. On the other hand, past temporal context instead of error-prone predictions can be directly leveraged to make multi-step predictions, bypassing the auto-regressive manner, the error of earlier predictions will not be propagated to further predictions.

Inspired by the newly proposed transformer [@NIPS2017_7181] for efficiently and efficiently modeling long-range dependencies. We develop a temporal transformer to capture dynamical long-range temporal dependencies in traffic flows to directly forecast multi-steps traffic conditions, bypassing the auto-regressive process. Specifically, on one hand, in each temporal transformer layer, each step in a sequence can attend to the context of all other steps to customize its long-range dependencies in a time-varying fashion. Thus, we can model long-range temporal dependencies in each layer. In contrast to the RNNs-based or convolution-based models, it also enables more parallelization of long-range dependencies, facilitating a more efficient model training. One the other hand, we can directly scaled to long-sequence without increasing model depth and complexity.

