each anchor layer can capture action proposals whose temporal intervals are well aligned with the size of receptive fields, however, posts difficulty to the detection of proposals with various length. The design limits the utility on localizing actions with complex variations.

To address this issue, we introduce temporal Gaussian kernel to dynamically control the temporal scales of proposals in each feature map. In the literature, there has been evidences on the use of Gaussian kernels for event detection in videos [@Piergiovanni:AAAI17; @Piergiovanni:CVPR18]. In particular, as shown in Figure \[fig2:1\], eight 1D temporal convolutional layers (anchor layers) are first cascaded for action proposal generation in different temporal resolution. For each cell in the feature map of the anchor layer, a Gaussian kernel is learnt to predict a particular interval of an action proposal corresponding to that cell. Formally, we denote the feature map of $j$-th convolutional layer as , $1\leq j\leq 8$, where $T^j$ and $D^j$ are the temporal length and feature dimension of the feature map. For a proposal $P^j_{t}$ whose center location is $t$, we leverage its temporal scale by a Gaussian kernel $G^j_{t}$. The standard deviation $\sigma^j_{t}$ of $G^j_{t}$ is learnt via a 1D convolutional layer on a 