to apply an attention operation to videos? Should an operation be applied to the temporal, spatial, or spatiotemporal dimension? How to compose multiple attention operations applied to different dimensions?*

Towards a principled way of applying attention to videos, we address the task of spatiotemporal attention cell search, i.e., the automatic discovery of cells that use attention operations as the primary building block. The discovered attention cells can be seamlessly inserted into a wide range of backbone networks, e.g., I3D [@carreira2017quo] or S3D [@xie2018rethinking], to improve the performance on video understanding tasks.

Specifically, we propose a search space for spatiotemporal attention cells, which allows the search algorithm to flexibly explore all of the aforementioned design choices in the cell. The attention cell is constrcuted by composing several primitive attention operations. Importantly, we consider two types of primitive attention operations: (1) map-based attention [@Park2018BAMBA; @woo2018cbam] and (2) dot-product attention (a.k.a., self-attention) [@vaswani2017attention; @wang2018non; @bello2019attention]. Map-based attention explicitly models where to focus in videos, compensating for the fact that convolutional operations apply the same filter to all the positions in videos. Dot-product attention enables the explicit modeling of long-range dependencies between distant positions in videos, accommodating the fact that convolutional operations only operate 