zero extra cost. (4) Our discovered attention cells outperform non-local blocks, on both the Kinetics-600 and MiT dataset. We achieve state-of-the-art performance on both datasets by inserting our discovered attention cells into I3D-R50. Our attention cells also demonstrate strong generalization capability when being applied to different modalities, backbones, or datasets.

![Illustration of the operation-level search space (left) and cell-level search space (right). The example attention operations use temporal as the attention dimension and the tuple under each feature map denotes its shape.[]{data-label="fig-search-space"}](SearchSpace.pdf){width="0.85\columnwidth"}

Related Work
============

**Video Classification.** Early work on video classification extends image classification CNNs with recurrent networks [@donahue2015long; @yue2015beyond] or two-stream architectures [@simonyan2014two; @feichtenhofer2016convolutional] that take both RGB frames and optical flow frames as inputs. Recent work on video classification are mainly based on 3D convolution [@tran2015learning] or its variants to directly learn video representations from RGB frames. I3D [@carreira2017quo] proposes to inflate the filters and pooling kernels of a 2D CNN into 3D to leverage successful 2D CNN architecture designs and their ImageNet pretrained weights. S3D [@xie2018rethinking] improves upon I3D by decomposing a 3D convolution into a 2D spatial convolution and a 1D temporal convolution. A similar idea is also explored in P3D [@qiu2017learning]. CPNet [@liu2019learningvideo] learns 