natural way for temporal action detection is to further classify the proposals into known action classes, making the process in two-stage manner [@Chao:CVPR18; @Heilbron:CVPR17; @Shou:CVPR17; @Shou:CVPR16; @Xu:ICCV17; @Xiong:ICCV17]. However, the separate of proposal generation and classification may result in sub-optimal solutions. To further facilitate temporal action detection, there have been several one-stage techniques [@Buch:BMVC17; @Lin:MM17; @Yeung:CVPR16] being proposed recently. For example, Single Stream Temporal Action Detection (SS-TAD) [@Buch:BMVC17] utilizes the Recurrent Neural Network (RNN) based architecture to jointly learn action proposal and classification. Inspired by SSD [@Liu:ECCV16], Lin *et al.* [@Lin:MM17] devise 1D temporal convolution to generate multiple temporal action anchors for action proposal and detection. Moreover, with the development of reinforcement learning, Yeung *et al.* [@Yeung:CVPR16] explore RNN to learn a glimpse policy for predicting the starting and ending points of actions in an end-to-end manner. Nevertheless, most of one-stage methods are still facing the challenge in localizing all the action proposals due to the predetermined temporal scales.

In short, our approach belongs to one-stage temporal action detection techniques. Different from the aforementioned one-stage methods which often predetermine temporal scales of action proposals, our GTAN in this paper contributes by studying not only learning temporal structure through Gaussian kernels, 