motivates the exploration for a resource-efficient attention algorithm. An investigation into the non-local module revealed an intriguing phenomenon. The attention maps at each position, despite generated independently, are correlated. As [@nonlocal] and [@sagan] analyzed, the attention map of a position mainly focuses on semantically related regions. Figure \[fig:factorizing\] shows the learned attention maps in a non-local module. When generating an image of a bird before a bush, pixels on the legs tend to attend to other leg pixels for structural consistency. Similarly, body pixels mainly attend to the body, and background pixels focus on the bush.

This observation inspired the design of the efficient attention mechanism that this paper proposes. Similar to dot-product attention, the mechanism first generates a key feature map, a query feature map, and a value feature map from an input. It interprets each channel of the key feature map as a global attention map. Using each global attention map as the weights, efficient attention aggregates the value feature map to produce a global context vector that summarizes an aspect of the global features. Then, at each position, the module regards the query feature as a set of coefficients over the global context vectors. Finally, the module 