inserted at different layers. When adding a single module, the reader can infer a general trend that among the same type of layers (, among FPN levels or among ResNet levels), the higher the resolution at a layer, the more performance gain the insertion of an efficient attention module brings. This result reiterates the significance of efficient attention. Since attention is more beneficial on higher-resolution inputs, the efficiency advantage of efficient attention is highly impactful. The layer res4 is the only one where insertion lead to degraded performance. A hypothesis is that the low resolution and large gap between the dimensionality of the input (2048) and the keys (64) might be the cause. A defect in the implementation caused the experiments for res1 and res2 to fail, so the results are not available for report in this paper.

Subsequent experiments explored adding multiple efficient attention modules to a network. Adding an efficient attention module to res3, res4, and every FPN level resulted in an improvement of 1.8 box AP and 1.6 mask AP. The results back the hypothesis in Section \[sec:intro\] that a core advantage of attention is the ability to model multi-hop long-range dependencies.

Interestingly, although a single insertion 