Directly substituting the non-local module on the $64 \times 64$ feature map in SAGAN [@sagan] yields a 17-time saving of memory and 32-time saving of computation. The gap widens rapidly with the increase of the input size. For a $256 \times 256$ feature map, a non-local module would require impractical amounts of memory (17.2 GB) and computation (412 GMACC). With the same input size, an efficient attention module uses 1/260 the amount of memory and 1/515 the amount of computation. The difference is more prominent for videos. Replacing the non-local module on the tiny $28 \times 28 \times 4$ feature volume in res3 of the non-local I3D-ResNet-50 network [@nonlocal] results in 2-time memory and computational saving. On a larger $64 \times 64 \times 32$ feature volume, an efficient attention module requires 1/32 the amount of memory and 1/1025 the amount of computation.

Experimental Results {#sec:experiment}
====================

Comparison with the Non-Local Module and the State-of-the-Art {#sec:vs}
-------------------------------------------------------------

### MS-COCO Task Suite {#sec:vs-coco}

This section presents comparison experiments on the MS-COCO 2017 dataset for object detection and instance segmentation. The section first compares the efficient attention module with the non-local module, which uses dot-product attention. Then, it compares models using efficient attention 