boundary adjustment and overlap confidence score for each action proposal, respectively.

Base Feature Network
--------------------

The ultimate target of action localization is to detect action instances in temporal dimension. Given an input video, we first extract clip-level features from continuous clips via a 3D ConvNet which could capture both appearance and motion information of the video. Specifically, a sequence of features are extracted from 3D ConvNet, where $T$ is the temporal length. We concatenate all the features into one feature map and then feed the map into two 1D convolutional layers (“conv1” and “conv2” with temporal kernel size 3, stride 1) plus one max-pooling layer (“pool1” with temporal kernel size 3, stride 2) to increase the temporal size of receptive fields. The base feature network is composed of 3D ConvNet, two 1D convolutional layers and max-pooling layer. The outputs of the base feature network are further exploited for action proposal generation.

Gaussian Kernel Learning {#sec:3.2}
------------------------

Given the feature map output from the base feature network, a natural way for one-stage action localization is to stack 1D temporal convolutional layers (anchor layers) to generate proposals (anchors) for classification and boundary regression. This kind of structure with predetermined temporal scale in 