'egbib.bib'
title:
- 'AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification'
- |
    AttentionNAS: Spatiotemporal Attention Cell Search for Video Classification\
    Supplementary Materials
---

Introduction
============

One major contributing factor to the success of neural networks in computer vision is the novel design of network architectures. In early work, most network architectures [@krizhevsky2012imagenet; @szegedy2015going; @he2016deep] were manually designed by human experts based on their knowledge and intuition of specific tasks. Recent work on neural architecture search (NAS) [@zoph2016neural; @zoph2018learning; @liu2018darts; @liu2018progressive; @real2019regularized] proposes to directly learn the architecture for a specific task from data and discovered architectures have been shown to outperform human-designed ones.

Convolutional Neural Networks (CNNs) have been the *de facto* architecture choice. Most work in computer vision uses convolutional operations as the primary building block to construct the network. However, convolutional operations still have their limitations. It has been shown that attention is complementary to convolutional operations, and they can be combined to further improve performance on vision tasks [@woo2018cbam; @wang2018non; @bello2019attention].

While being complementary to convolution, many design choices remain to be determined to use attention. The design becomes more complex when applying attention to videos, where the following questions arise: *What is the right dimension 