classification. The baseline is ResNet-50 [@resnet]. As in Table \[tab:vs-ca-class\], insertions at res1 and res2 both lead to significant improvements. Inserting simultaneously at both layers resulted in the largest gain of performance. In comparison, experiments with the same settings but with the non-local module all encountered OOM errors.

  Layer(s)                                  EA module          Nonlocal
  ----------------------- --------------------------- -----------------
  Baseline                            $76.052/92.952$   $76.052/92.952$
  $\mathrm{res_1}$                    $76.932/93.252$               OOM
  $\mathrm{res_2}$                    $76.532/93.274$               OOM
  $\mathrm{res_{1, 2}}$     $\bm{77.312}/\bm{93.650}$               OOM

  : **Experiments on ImageNet.** All results are in the form top-1/top-5. All experiments used single-crop testing.[]{data-label="tab:vs-ca-class"}

Ablation Studies {#sec:ablation}
----------------

### Attention Normalization

These experiments empirically validated the necessity of attention normalization and compared the two methods Section \[sec:ea\] specified, namely scaling and softmax normalization. Table \[tab:norm\] reports the experimental outcomes. The results demonstrate that while attention normalization is critical for the training of efficient attention-augmented models, the effectiveness does not depend on the specific normalization technique. Following [@nonlocal], all other experiments used softmax normalization.

  Method           Box AP    Mask AP
  --------- ------------- ----------
  None                NaN        NaN
  Scaling     $\bm{40.2}$     $35.9$
  Softmax     $\bm{40.2}$   **36.0**

  : **Experiments for attention normalization methods**. All experiments inserted a single efficient attention module at res3.[]{data-label="tab:norm"}

### Layer of Insertion {#sec:layer}

Table \[tab:layer\] shows experiments contrasting models with EA modules 