three of the corresponding global attention maps from the efficient attention module at FPN level 1 for each respective example.[]{data-label="fig:coco"}](figures/vis.png){width="\linewidth"}

Figure \[fig:coco\] shows visualization of the global attention maps for various examples from the efficient attention module at fpn1 in the model corresponding to the last row in Table \[tab:layer\]. The figure illustrates 3 sets of global attention maps each with a distinct, semantic focus. Column 2 tends to capture the foreground, column 3 tends to capture the core parts of objects, and column 4 tends to capture the peripheral of objects. The semantic distinctiveness of each set of global attention maps supports the analysis in Section \[sec:intro\] that the attention maps are linear combinations of a set of basis attention maps each focusing on a semantically significant area.

Conclusion
==========

This paper has presented the efficient attention mechanism, an attention mechanism that is quadratically more memory- and computationally-efficient than the widely adopted dot-product attention mechanism. By dramatically reducing the resource usage, efficient attention enables a large number of new use cases of attention, particularly in domains with tight resource constraints or large inputs.

The experiments verified its effectiveness on four distinct tasks, object detection, instance segmentation, stereo depth estimation, 