Inception [@szegedy2015going] network. When examining the generalization of the found cells, we also consider the backbone I3D-R50 [@wang2018non], which is constructed based on ResNet-50 [@he2016deep].

**Baselines.** Non-local blocks [@wang2018non] are the state-of-the-art manually designed attention cell for video classification and are the most direct competitor of our automatically searched attention cells. We mainly focus on the relative improvement brought by our attention cells after being inserted into backbones. Besides non-local blocks, we also compare with other state-of-the-art methods for video classification, such as TSN [@wang2016temporal], TRN [@zhou2018temporal], and SlowFast [@feichtenhofer2019slowfast].

  ----- ------------------------------- -- -- ----------- ----------- --------------- -- -- ----------- ----------- --------------- --
                                                                                                                                    
        Model                                    Top-1       Top-5     $\Delta$Top-1           Top-1       Top-5     $\Delta$Top-1  
  I3D   Backbone [@carreira2017quo]              75.58       92.93           -                 27.38       54.29           -        
        Non-local [@wang2018non]                 76.87       93.44         1.29              **28.54**     55.35       **1.16**     
        Ours - GPB                             **77.39**   **93.63**     **1.81**              28.41     **55.49**       1.03       
  S3D   Backbone [@xie2018rethinking]            76.15       93.22           -                 27.69       54.68           -        
        Non-local [@wang2018non]                 77.56       93.68         1.41              **29.52**   **56.91**     **1.83**     
        Ours - GPB                             **78.28**   **94.04**     **2.13**              29.23       56.22         1.54       
  ----- ------------------------------- -- -- ----------- ----------- --------------- -- -- ----------- ----------- --------------- --

  : Search results on Kinetics-600 and MiT using the differentiable method. Our attention cells consistently outperform non-local blocks on all the combinations of backbones and 