# Boosting Adversarial Transferability by Block Shuffle and Rotation

Kunyu Wang1, Xuanran He2, Wenxuan Wang1, Xiaosen Wang3
1Chinese University of Hong Kong, 2Nanyang Technological University,
3Huawei Singularity Security Lab
Corresponding author. Email<EMAIL_ADDRESS>
###### Abstract

Adversarial examples mislead deep neural networks with imperceptible
perturbations and have brought significant threats to deep learning. An
important aspect is their transferability, which refers to their ability to
deceive other models, thus enabling attacks in the black-box setting. Though
various methods have been proposed to boost transferability, the performance
still falls short compared with white-box attacks. In this work, we observe
that existing input transformation based attacks, one of the mainstream
transfer-based attacks, result in different attention heatmaps on various
models, which might limit the transferability. We also find that breaking the
intrinsic relation of the image can disrupt the attention heatmap of the
original image. Based on this finding, we propose a novel input transformation
based attack called block shuffle and rotation (BSR). Specifically, BSR splits
the input image into several blocks, then randomly shuffles and rotates these
blocks to construct a set of new images for gradient calculation. Empirical
evaluations on the ImageNet dataset demonstrate that BSR could achieve
significantly better transferability than the existing input transformation
based methods under single-model and ensemble-model settings. Combining BSR
with the current input transformation method can further improve the
transferability, which significantly outperforms the state-of-the-art methods.
Code is available at https://github.com/Trustworthy-AI-Group/BSR.

##  1 Introduction

Deep neural networks (DNNs) have established superior performance in many
tasks, such as image classification [12, 13], segmentation [21], object
detection [27, 26], face recognition [39], among others. Despite their
achievements, DNNs have been observed to exhibit significant vulnerability to
adversarial examples [34, 8, 41], which closely resemble legitimate examples,
yet can deliberately misguide deep learning models to produce unreasonable
predictions. The existence of such vulnerabilities gives rise to serious
concerns, particularly in security-sensitive applications, such as autonomous
driving [5], face verification [30].

Raw Image

Shuffled Image

Reshuffled Image

Figure 1: The attention heatmaps of the raw images, shuffled images, and
reshuffled heatmaps on the shuffled image generated on Inception-v3 model
using Grad-CAM.

Adversarial attacks can generally be categorized into two types: white-box
attacks and black-box attacks. White-box attacks involve having complete
access to the target model’s architecture and parameters. In contrast, black-
box attacks only have limited information about the target model, which makes
them more applicable for real-world applications. In the white-box setting,
some studies employ the gradient with respect to the input sample to generate
adversarial examples [8]. Those crafted adversarial examples exhibit
transferability across neural models [25], which is the ability of adversarial
examples generated on one model to deceive not only the victim model but also
other models, making them suitable for black-box attacks. However, existing
attack methods [14, 22] demonstrate outstanding white-box attack performance
but relatively poorer transferability, limiting their efficacy in attacking
real-world applications.

Recently, several approaches have emerged to enhance adversarial
transferability, including incorporating momentum into gradient-based attacks
[3, 18], attacking multiple models simultaneously [19], transforming the image
before gradient calculation [51, 46], leveraging victim model features [47],
and modifying the forward or backward process [48, 45]. Among these, input
transformation based methods, which modify the input image for gradient
calculation, have demonstrated significant effectiveness in improving
transferability. However, we find that all the existing input transformation
based attacks result in different attention heatmaps [29] on different models.
This discrepancy in attention heatmaps could potentially limit the extent of
adversarial transferability.

The attention heatmaps highlight the crucial regions for classification.
Motivated by this, we aim to maintain consistency in the attention heatmaps of
adversarial examples across different models. Since we only have access to a
single white-box model for the attack, we initially explore methods to disrupt
the attention heatmaps. As shown in Fig. 1, we can disrupt the intrinsic
relation within the image by randomly shuffling the divided blocks of the
image, leading to different attention heatmaps compared with the raw image.
Based on this finding, we propose a novel input transformation based attack,
called block shuffle and rotation (BSR), which optimizes the adversarial
perturbation on several transformed images to eliminate the variance among the
attention heatmaps on various models. In particular, BSR randomly divides the
image into several blocks, which are subsequently shuffled and rotated to
create new images for gradient calculation. To eliminate the variance of
random transformation and stabilize the optimization, BSR adopts the average
gradient on several transformed images.

In summary, we highlight our contributions as follows:

  * •

We show that breaking the intrinsic relation of the image can disrupt the
attention heatmaps of the deep model.

  * •

We propose a new attack called block shuffle and rotation (BSR), which is the
first input transformation based attack to disrupt attention heatmaps for
better transferability.

  * •

Empirical evaluations on the ImageNet dataset demonstrate BSR achieves much
better transferability than the state-of-the-art input transformation based
attacks.

  * •

BSR is compatible with other transfer-based attacks and can be integrated with
each other to boost the adversarial transferability further.

##  2 Related Work

Here we briefly introduce adversarial attacks and defenses and summarize the
visualization of attention heatmaps.

###  2.1 Adversarial Attacks

Szegedy et al. [34] first identified adversarial examples, which bring a great
threat to DNN applications. Recently, numerous attacks have been proposed,
which mainly fall into two categories: 1) white-box attacks can access all the
information of the target model, such as gradient, weight, architecture,
_etc_. Gradient-based attacks [22, 1] that maximize the loss function using
the gradient w.r.t. input are the predominant white-box attacks. 2) black-box
attacks are only allowed limited access to the target model, which can be
further categorized into Score-based attacks [38, 11], Decision-based attacks
[15, 44] and Transfer-based attacks [3, 51, 7]. Among these, transfer-based
attacks cannot access the target model, in which the attacker adopts the
adversarial examples generated by the surrogate model to attack the target
model directly. Hence, transfer-based attacks can be effectively deployed in
the real world and have attracted wide interest.

Fast Gradient Sign Method (FGSM) [8] adds the perturbation in the gradient
direction of the input. Iterative FGSM (I-FGSM) [14] extends FGSM into an
iterative version, showing better white-box attack performance but poor
transferability. Recently, numerous works have been proposed to improve
adversarial transferability.

MI-FGSM [3] introduces momentum into I-FGSM to stabilize the optimization
procedure and escape the local optima. Later, more advanced momentum based
attacks are proposed to further boost transferability, such as NI-FGSM [18],
VMI-FGSM [40], EMI-FGSM [43], PGN [6] and so on. Ensemble attacks [19, 52, 16]
generate more transferable adversarial examples by attacking multiple models
simultaneously. Several works [57, 49] disrupt the feature space to generate
adversarial examples.

On the other hand, input transformation has become one of the most effective
ways to improve transferability. Diverse input method (DIM) [51] first resizes
the image into random size and adds the padding to fixed size before the
gradient calculation. Translation invariant method (TIM) [4] translates the
image into a set of images for gradient calculation, which is approximated by
convolving the gradient with a Gaussian kernel. Scale invariant method (SIM)
[18] calculates the gradient of several scaled images. Admix [42] mixes a
small portion of images from other categories to the input image to craft a
set of admixed images. PAM [55] augments the input images from several
augmentation paths.

In this work, we propose a new input transformation, called BSR, which breaks
the intrinsic semantic relation of the input image for more diverse
transformed images and results in much better transferability than the
baselines.

###  2.2 Adversarial Defenses

With the increasing interest of adversarial attacks, researchers have been
struggling to mitigate such threats. Adversarial training [8, 22, 37] injects
adversarial examples into the training process to boost the model robustness.
Among them, Tramèr et al. [37] propose ensemble adversarial training using
adversarial perturbation generated on several models, showing great
effectiveness against transfer-based attacks. Denoising filter is a data
preprocessing method that filters out the adversarial perturbation before
feeding them into the target model. For instance, Liao et al. [17] design a
high-level representation guided denoiser (HGD) based on U-Net to eliminate
adversarial perturbation. Naseer et al. [23] train a neural representation
purifier (NRP) using a self-supervised adversarial training mechanism to
purify the input sample. Researchers also introduce several input
transformation based defenses that transform the image before prediction to
eliminate the effect of adversarial perturbation, such as random resizing and
padding [51], feature squeezing using bit reduction [53], feature distillation
[20]. Different from the above empirical defenses, several certified defense
methods provide a provable defense in a given radius, such as interval bound
propagation (IBP) [9], CROWN-IBP [54], randomized smoothing (RS) [2], _etc_.

###  2.3 Attention Heatmaps

As the inner mechanism of deep models remains elusive to researchers, several
techniques [31, 33, 29] have been developed to interpret these models. Among
them, attention heatmap is a widely adopted way to interpret deep models. For
instance, Zhou et al. [56] adopts global average pooling to highlight the
discriminative object parts. Selvaraju et al. [29] proposed Grad-CAM using the
gradient information to generate more accurate attention heatmaps. In this
work, we adopt the attention heatmap to investigate how to boost adversarial
transferability.

Source: Inc-v3

Target: Inc-v4

Raw Image

DIM

TIM

SIM

Admix

PAM

BSR

Figure 2: Attention heatmaps of adversarial examples generated by various
input transformations using Grad-CAM.

##  3 Methodology

In this section, we first introduce the preliminaries and motivation. Then we
provide a detailed description of our BSR, and summarize the difference
between RLFAT [32] and BSR.

###  3.1 Preliminaries

Given a victim model $f$ with parameters $\theta$ and a clean image
$\mathbf{x}$ with ground-truth label $\mathbf{y}$, the attacker aims to
generate an adversarial example $\mathbf{x}^{adv}$ that is indistinguishable
from original image $\mathbf{x}$ (_i.e_.,
$\|\mathbf{x}^{adv}-\mathbf{x}\|_{p}\leq\epsilon$) but can fool the victim
model $f(\mathbf{x}^{adv};\theta)\neq f(\mathbf{x};\theta)=\mathbf{y}$. Here
$\epsilon$ is the perturbation budget, and $\|\cdot\|_{p}$ is the $\ell_{p}$
norm distance. In this paper, we adopt $\ell_{\infty}$ distance to align with
existing works. To generate such an adversarial example, the attacker often
maximizes the objective function, which can be formalized as:

$\mathbf{x}^{adv}=\operatorname*{argmax}_{\|\mathbf{x}^{adv}-\mathbf{x}\|_{p}\leq\epsilon}J(\mathbf{x}^{adv},\mathbf{y};\theta),$
(1)

where $J(\cdot)$ is the corresponding loss function (_e.g_., cross-entropy
loss). For instance, FGSM [8] updates the benign sample by adding a small
perturbation in the direction of the gradient sign:

$\mathbf{x}^{adv}=\mathbf{x}+\epsilon\cdot\operatorname{sign}(\nabla_{x}J(\mathbf{x},\mathbf{y};\theta)).$
(2)

FGSM can efficiently craft adversarial examples but showing poor attack
performance. Thus, I-FGSM [14] extends FGSM into an iterative version, which
iteratively updates the the adversarial example by adding small perturbation
with a step size $\alpha$:

$\displaystyle\mathbf{x}^{adv}_{t}=\mathbf{x}^{adv}_{t-1}+\alpha\cdot\operatorname{sign}(\nabla_{\mathbf{x}^{adv}_{t-1}}J(\mathbf{x}^{adv}_{t-1},\mathbf{y};\theta)),$
(3)

where $\mathbf{x}^{adv}_{0}=\mathbf{x}$. Considering the poor transferability
of I-FGSM, MI-FGSM [3] integrates momentum into the gradient for more
transferable adversarial examples:

$\displaystyle g_{t}=\mu\cdot
g_{t-1}+\frac{\nabla_{\mathbf{x}^{adv}_{t-1}}J(\mathbf{x}^{adv}_{t-1},\mathbf{y};\theta)}{\|\nabla_{\mathbf{x}^{adv}_{t-1}}J(\mathbf{x}^{adv}_{t-1},\mathbf{y};\theta)\|_{1}},$
(4)
$\displaystyle\mathbf{x}^{adv}_{t}=\mathbf{x}^{adv}_{t-1}+\alpha\cdot\operatorname{sign}(g_{t}),\
g_{0}=0,$

where $\mu$ is the decay factor. Suppose $\mathcal{T}$ is a transformation
operator, existing input transformation based attacks are often integrated
into MI-FGSM to boost adversarial transferability, _i.e_., adopting
$\nabla_{\mathbf{x}^{adv}_{t-1}}J(\mathcal{T}(\mathbf{x}^{adv}_{t-1}),\mathbf{y};\theta)$
for Eq. (4).

###  3.2 Motivation

While different models may have distinct parameters and architectures, there
are often shared characteristics in their learned features for image
recognition tasks [56, 31]. In this work, we hypothesize that the adversarial
perturbations which target these salient features have a greater impact on
adversarial transferability. Wu et al. [49] find disrupting the attention
heatmaps can enhance transferability, which also supports our hypothesis.
Intuitively, when the attention heatmaps of adversarial examples exhibit
consistency across various models, it is expected to yield better adversarial
transferability. To explore this idea, we initially assess the consistency of
attention heatmaps generated by Grad-CAM [28] for several input transformation
based attacks. Unfortunately, as shown in Fig. 2, the attention heatmaps of
adversarial examples on the white-box model are different from that on the
target black-box model, resulting in limited adversarial transferability. This
finding motivates us to investigate a new problem:

How can we generate adversarial examples with consistent attention heatmaps
across different models?

To maintain the consistency of attention heatmaps across multiple models, one
direct approach is to optimize the adversarial perturbation by utilizing
gradients w.r.t. the input image obtained from different models, _a.k.a_.
ensemble attack [19]. By incorporating gradients from various models, each
associated with its own attention heatmap for the same input image, ensemble
attack helps eliminate the variance among attention heatmaps, resulting in
improved transferability. In practice, however, it is often challenging and
costly to access multiple models, making it more feasible to work with a
single surrogate model. In this work, we explore how to obtain the gradients
with different attention heatmaps on a single model using input
transformation. With such transformation, we can optimize the perturbation to
eliminate the variance among attention heatmaps of various transformed images,
thus enhancing the consistency of attention heatmaps and adversarial
transferability.

###  3.3 Block Shuffle and Rotation

With a single source model, we have to transform the input image to obtain
diverse attention heatmaps when calculating the gradient. Thus, we should
address the problem:

How to transform the image to disrupt the attention heatmap on a single source
model?

Attention heatmap highlights the significant features that contribute to the
deep model’s accurate prediction. For human perception, we are capable of
recognizing objects based on partial visual cues, even when they are partially
obstructed by other objects. For instance, we can identify a cat by observing
only a portion of its body (_e.g_., head). This observation motivates us to
employ image transformations that draw attention to specific regions of the
main object, thereby varying the attention heatmap on a single model.
Intuitively, randomly masking the object partially can force the deep model to
focus on the remaining object, leading to various attention heatmaps. However,
masking the object leads to a loss of information in the image, rendering the
gradient meaningless for the masked block. Consequently, it can slow down the
attack efficiency and effectiveness.

Algorithm 1 Block Shuffle and Rotation

1:A classifier $f$ with parameters $\theta$, loss function $J$; a raw example
$x$ with ground-truth label $y$; the magnitude of perturbation $\epsilon$;
number of iteration $T$; decay factor $\mu$; the number of transformed images
$N$; the number of blocks $n$; the maximum angle $\tau$ for rotation

2:An adversarial example $x^{adv}$

3:$\alpha=\epsilon/T$, $g_{0}=0$

4:for $t=1\rightarrow T$ do

5: Generate several transformed images: $\mathcal{T}(\mathbf{x}^{adv},n,\tau)$

6: Calculate the average gradient $\bar{g}_{t}$ by Eq. (6):

7: Update the momentum $g_{t}$ by:  $g_{t}=\mu\cdot
g_{t-1}+\frac{\bar{g}_{t}}{\|\bar{g}_{t}\|_{1}}$

8: Update the adversarial example:
$x^{adv}_{t}=x^{adv}_{t-1}+\alpha\cdot\operatorname{sign}(g_{t})$ (5)

9:end for

10:return $x^{adv}_{T}$

On the other hand, humans exhibit a remarkable ability to not only recognize
the visible parts of the objects but also mentally reconstruct the occluded
portions when the objects are partially obstructed by other elements. This
cognitive process is attributed to our perception of intrinsic relationships
inherent within the object, such as the understanding that a horse’s legs are
positioned beneath its body. Recognizing the significance of intrinsic
relationships for human perception, we try to disrupt these relationships to
affect attention heatmaps. In particular, we split the image into several
blocks and shuffle the blocks to construct new images that appear visually
distinct from the original one. As expected, the attention heatmaps are also
disrupted on the transformed image, even when recovering the attention
heatmaps to match the original image, as depicted in Fig. 1. Thus, we can
break the intrinsic relationships for more diverse attention heatmaps to boost
adversarial transferability.

To achieve this goal, we propose a new input transformation
$\mathcal{T}(\mathbf{x},n,\tau)$, which randomly splits the image into
$n\times n$ blocks followed by the random shuffling of these blocks. To
further disrupt the intrinsic relationship, each block is independently
rotated by an angle within the range of $-\tau\leq\beta\leq\tau$ degrees.
During the rotation of each block, any portions that extend beyond the image
boundaries are removed while the resulting gaps are filled with zero.

With the transformation $\mathcal{T}(\mathbf{x},n,\tau)$, we can obtain more
diverse attention heatmaps compared to those obtained from various models.
Hence, strengthening the consistency of heatmaps (harder) helps result in
consistent attention across models (easier). This motivates us to adopt
transformation $\mathcal{T}(\mathbf{x},n,\tau)$ for gradient calculation to
achieve more consistent heatmaps. Note that transformation
$\mathcal{T}(\mathbf{x},n,\tau)$ cannot guarantee that all transformed images
are correctly classified ($\sim 86.8\%$ on Inc-v3). To eliminate the variance
introduced by the intrinsic relation corruption, we calculate the average
gradient on $N$ transformed images as follows:

$\bar{g}=\frac{1}{N}\sum_{i=0}^{N}\nabla_{\mathbf{x}^{adv}}J(\mathcal{T}(\mathbf{x}^{adv},n,\tau),\mathbf{y};\theta)).$
(6)

The input transformation $\mathcal{T}(\mathbf{x},n,\tau)$ is general to any
existing attacks. Here we integrate it into MI-FGSM, denoted as Block Shuffle
and Rotation (BSR), and summarize the algorithm in Algorithm 1.

###  3.4 BSR Vs. RLFAT

Song et al. [32] propose Robust Local Features for Adversarial Training
(RLFAT) by minimizing the distance between the high-level feature of the
original image and block shuffled image for better generalization. Since RLFAT
also shuffles the image blocks, we highlight the difference between BSR and
RLFAT as follows:

  * •

Goal. BSR aims to generate more transferable adversarial examples, while RLFAT
boosts the generalization of adversarial training.

  * •

Strategy. BSR shuffles and rotates the image blocks, while RLFAT only shuffles
the blocks.

  * •

Usage. BSR directly adopts the transformed images for gradient calculation to
craft adversarial examples, while RLFAT treats it as a regularizer for the
original image.

With the different goals, strategies, and usages, BSR should definitely be a
new and novel input transformation based attack to effectively boost the
adversarial transferability.

##  4 Experiments

In this section, we conduct empirical evaluations on ImageNet dataset to
evaluate the effectiveness of BSR.

Model | Attack | Inc-v3 | Inc-v4 | IncRes-v2 | Res-101 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---|---|---
Inc-v3 | DIM |  98.6* | 64.4 | 60.2 | 53.5 | 18.8 | 18.4 | 9.5
TIM |  100.0* | 49.3 | 43.9 | 40.2 | 24.6 | 21.7 | 13.4
SIM |  100.0* | 69.5 | 68.5 | 63.5 | 32.3 | 31.0 | 17.4
Admix |  100.0* | 82.2 | 81.1 | 73.8 | 38.7 | 37.9 | 19.8
PAM |  100.0* | 76.4 | 75.5 | 69.6 | 39.0 | 38.8 | 20.0
BSR | 100.0* | 96.2 | 94.7 | 90.5 | 55.0 | 51.6 | 29.3
Inc-v4 | DIM | 72.0 |  97.6* | 63.8 | 57.2 | 22.6 | 21.1 | 11.7
TIM | 59.1 |  99.7* | 49.0 | 41.9 | 26.8 | 22.9 | 16.6
SIM | 80.4 |  99.7* | 73.4 | 69.4 | 48.6 | 45.2 | 29.6
Admix | 89.0 |  99.9* | 85.3 | 79.0 | 55.5 | 51.7 | 32.3
PAM | 86.7 |  99.9* | 81.6 | 75.9 | 55.4 | 50.5 | 33.2
BSR | 96.1 |  99.9* | 93.4 | 88.4 | 57.6 | 52.1 | 34.3
IncRes-v2 | DIM | 70.3 | 64.7 |  93.1* | 58.0 | 30.4 | 23.5 | 16.9
TIM | 62.2 | 55.6 |  97.4* | 50.3 | 32.4 | 27.5 | 22.6
SIM | 85.9 | 80.0 |  98.7* | 76.1 | 56.2 | 49.1 | 42.5
Admix | 90.8 | 86.3 |  99.2* | 82.2 | 63.6 | 56.6 | 49.4
PAM | 88.6 | 86.3 |  99.4* | 81.6 | 66.0 | 58.3 | 51.0
BSR | 94.6 | 93.8 |  98.5* | 90.7 | 71.4 | 63.1 | 51.0
Res-101 | DIM | 76.0 | 68.4 | 70.3 |  98.0* | 34.7 | 31.8 | 19.6
TIM | 59.9 | 52.2 | 51.9 |  99.2* | 34.4 | 31.2 | 23.7
SIM | 74.1 | 69.6 | 69.1 |  99.7* | 42.8 | 39.6 | 25.7
Admix | 84.5 | 80.2 | 80.7 |  99.9* | 51.6 | 44.7 | 29.9
PAM | 77.4 | 73.9 | 75.7 |  99.9* | 51.2 | 46.3 | 32.2
BSR | 97.1 | 96.6 | 96.6 |  99.7* | 78.7 | 74.7 | 55.6
|  |  |  |  |  |  |  |
Table 1: Attack success rates (%) on seven models under single model setting with various single input transformations. The adversaries are crafted on Inc-v3, Inc-v4, IncRes-v2 and Res-101 respectively. * indicates white-box attacks. Attack | Inc-v4 | IncRes-v2 | Res-101 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---
BSR-DIM | 98.3$\uparrow$31.9 | 94.8$\uparrow$34.6 | 90.1$\uparrow$36.6 | 57.8$\uparrow$39.0 | 54.5$\uparrow$36.1 | 32.3$\uparrow$22.8
BSR-TIM | 94.7$\uparrow$45.4 | 92.5$\uparrow$48.6 | 87.0$\uparrow$47.0 | 71.3$\uparrow$46.7 | 68.4$\uparrow$46.7 | 47.9$\uparrow$34.5
BSR-SIM | 99.4$\uparrow$29.9 | 98.4$\uparrow$29.9 | 97.8$\uparrow$34.3 | 84.3$\uparrow$52.0 | 81.4$\uparrow$50.4 | 59.2$\uparrow$41.8
BSR-Admix | 98.9$\uparrow$16.7 | 98.8$\uparrow$17.7 | 98.2$\uparrow$24.4 | 89.1$\uparrow$50.4 | 86.9$\uparrow$49.5 | 68.0$\uparrow$48.2
BSR-PAM | 98.5$\uparrow$22.1 | 97.3$\uparrow$21.8 | 96.9$\uparrow$27.3 | 79.4$\uparrow$40.4 | 75.3$\uparrow$36.5 | 50.5$\uparrow$30.5
Admix-TI-DIM | 90.4 | 87.3 | 83.7 | 72.4 | 68.4 | 53.4
PAM-TI-DIM | 89.3 | 85.5 | 80.7 | 73.6 | 69.1 | 52.1
BSR-TI-DIM | 95.2 | 92.9 | 87.9 | 74.2 | 70.7 | 50.0
BSR-SI-TI-DIM | 98.5 | 97.1 | 95.4 | 90.6 | 90.0 | 75.1
|  |  |  |  |  |
Table 2: Attack success rates (%) on seven models under single model setting
with various input transformations combined with BSR. The adversaries are
crafted on Inc-v3. $\uparrow$ indicates the increase of attack success rate
when combined with BSR.

###  4.1 Experimental Setup

Dataset. We evaluate our proposed BSR on 1000 images belonging to 1000
categories from the validation set of ImageNet dataset [24].

Models. We adopt four popular models, _i.e_., Inception-v3 (Inc-v3) [35],
Inception-v4 (Inc-v4), Inception-Resnet-v3 (IncRes-v3) [36], Resnet-v2-101
(Res-101) [12], and three ensemble adversarially trained models, _i.e_.,
$\text{Inc-v3}_{ens3}$, $\text{Inc-v3}_{ens4}$, $\text{IncRes-v2}_{ens2}$ [37]
as victim models to evaluate the transferability. To further verify the
effectiveness of BSR, we utilize several advanced defense methods, including
HGD [17], R&P [50],
NIPS-r3111https://github.com/anlthms/nips-2017/tree/master/mmd, Bit-RD [53],
JPEG [10], FD [20], RS [2] and NRP [23].

Baselines. To verify the effectiveness of BSR, we choose five competitive
input transformation based attacks as our baselines, _i.e_. DIM [51], TIM [4],
SIM [18], Admix [42] and PAM[55]. For fairness, all the input transformations
are integrated into MI-FGSM [3].

Parameters Settings. We set the maximum perturbation $\epsilon=16$, number of
iteration $T=10$, step size $\alpha=\epsilon/T$ and the decay factor $\mu=1$
for MI-FGSM [3]. DIM [51] adopts the transformation probability of $0.5$. TIM
[4] utilizes a kernel size of $7\times 7$. The number of copies of SIM [18]
and Admix [42] is $5$. Admix admixes $3$ images with the strength of $0.2$.
The number of scale for PAM [55] is 4, and the number of augmented path is 3 .
Our BSR splits the image into $2\times 2$ blocks with the maximum rotation
angle $\tau=24^{\circ}$ and calculates the gradients on $N=20$ transformed
images.

###  4.2 Evaluation on Single Model

We first evaluate the attack performance on various input transformation based
attacks, _i.e_., DIM TIM, SIM, Admix, PAM, and our proposed BSR. We craft the
adversaries on the four standard trained models and test them on seven models.
The attack success rates, _i.e_., the misclassification rates of the victim
model on the adversarial examples, are summarized in Tab. 1. Each column
denotes the model to be attacked and each row indicates that the attacker
generates the adversarial examples on the corresponding models.

Attack | Inc-v3 | Inc-v4 | IncRes-v2 | Res-101 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---|---
DIM |  99.0* | 97.1* |  93.4* |  99.7* | 57.6 | 51.5 | 35.9
TIM |  99.8* | 97.4* |  94.7* |  99.8* | 61.6 | 55.5 | 45.6
SIM |  99.9* | 99.1* |  98.5* |  100.0* | 78.4 | 75.2 | 60.6
Admix |  100.0* | 99.6* |  99.0* |  100.0* | 85.1 | 80.9 | 67.8
PAM |  99.9* | 99.7* |  99.4* |  100.0* | 86.1 | 81.6 | 69.1
BSR |  100.0* |  99.9* |  99.9* |  99.9* | 92.4 | 89.0 | 77.2
Admix-TI-DIM |  99.6* | 98.8* |  98.2* |  99.8* | 93.1 | 92.4 | 89.4
PAM-TI-DIM |  99.8* | 99.8* |  99.2* |  99.8* | 95.8 | 95.2 | 93.0
BSR–TI-DIM |  99.8* | 99.8* |  99.7* |  99.8* | 96.1 | 95.1 | 90.8
BSR-SI-TI-DIM |  99.9* |  99.9* |  99.9* |  99.8* | 99.1 | 99.1 | 97.0
|  |  |  |  |  |  |
Table 3: Attack success rates (%) on seven models under ensemble model setting with various input transformations. The adversaries are crafted on Inc-v3, Inc-v4, IncRes-v2 and Res-101 model. * indicates white-box attacks. Method | HGD | R&P | NIPS-r3 | Bit-RD | JPEG | FD | RS | NRP | Average
---|---|---|---|---|---|---|---|---|---
Admix-TI-DIM | 92.8 | 93.5 | 94.5 | 82.4 | 97.6 | 90.9 | 72.6 | 80.4 | 88.1
PAM-TI-DIM | 95.4 | 95.3 | 96.4 | 85.9 | 98.4 | 93.4 | 74.0 | 83.8 | 91.6
BSR-TI-DIM | 97.1 | 98.0 | 97.9 | 84.9 | 98.8 | 93.2 | 69.1 | 73.6 | 89.1
BSR-SI-TI-DIM | 98.5 | 99.1 | 99.4 | 91.4 | 99.2 | 97.1 | 83.9 | 84.2 | 94.1
|  |  |  |  |  |  |  |  |
Table 4: Attack success rates (%) of eight defense methods by Admix, SSA and
BSR input transformations. The adversaries are crafted on Inc-v3, Inc-v4,
IncRes-v2 and Res-101 synchronously.

It can be observed that for DIM and TIM, DIM exhibits superior performance on
standard trained models while TIM exhibits better transferability on
adversarially trained models. SIM, as a special case of Admix, can achieve
better performance than DIM and TIM, while Admix shows the best performance on
standardly trained model among the four baselines and PAM exhibits better
transferability on adversarially trained model. In contrast, our proposed BSR,
surpasses existing input transformation based attacks while maintaining
comparable performance in white-box attacks. Remarkably, on standard trained
models, BSR attains an average attack success rate of $93.8\%$, exhibiting a
substantial improvement over Admix by a clear margin of at least $6.5\%$.
Similarly, on adversarially trained models, BSR achieves an average attack
success rate of $56.2\%$, outperforming PAM by a significant margin of
$11.0\%$. These exceptional findings substantiate the superiority of BSR in
generating transferable adversarial examples, thereby highlighting the
importance of maintaining attention heatmap consistency across different
models as a means to enhance transferability.

###  4.3 Evaluation on Combined Input Transformation

Previous works [42] have shown that a good input transformation based attack
should not only exhibit better transferability, but also be compatible with
other input transformations to generate more transferable adversarial
examples. Following the evaluation setting of Admix, we combine our BSR with
various input transformations, denoted BSR-DIM, BSR-TIM, BSR-SIM, BSR-Admix
and BSR-PAM. Here we also combine BSR with multiple input transformations,
denoted as BSR-TI-DIM and BSR-SI-TI-DIM to compare Admix-TI-DIM and PAM-TI-
DIM.

We report the attack success rates of the adversarial examples generated on
Inc-v3 in Tab. 2 and the results for other models in Appendix. Our BSR
significantly improves the transferability of these input transformation based
attacks. In general, BSR can improve the attack success rate with a range from
$16.7\%$ to $52.0\%$. In particular, when combining these input transformation
with BSR, the attack performance on adversarially trained models are
significantly improved by a margin of $22.8\%\thicksim 52.0\%$. Although
Admix-TI-DIM demonstrates the best performance among combined methods, our
proposed BSR-TI-DIM surpasses Admix-TI-DIM with a clear margin of $2.6\%$ on
average attack success rates. Notably, when BSR is combined with SI-DI-TIM, it
further enhances transferability by a margin ranging from $8.1\%$ to $31.6\%$.
This further supports the high effectiveness of BSR and shows its excellent
compatibility with other input transformation based attacks.

Figure 3: Attack success rates (%) of various models on the adversarial
examples generated by MI-FGSM, BS, BR and BSR, respectively.

Figure 4: Attack successful rates (%) of various models on the adversarial
examples generated by BSR with various numbers of blocks, number of
transformed images and range of the rotation angles. The adversarial examples
are crafted on Inc-v3 model and tested on the other six models under the
black-box setting.

###  4.4 Evaluation on Ensemble Model

Liu  _et al_. [19] first propose ensemble attack to boost the transferability
by synchronously attacking several models. To evaluate the compatibility of
the proposed BSR with ensemble attack, we generate the adversarial examples on
four standard trained models and test them on adversarially trained models
following the setting in MI-FGSM [3]. We evaluate the attack performance of
single input transformation as well as BSR combined with the existing input
transformations, respectively.

As shown in Tab. 3, under ensemble model setting, PAM showcases superior
white-box attack capabilities, surpassing other baselines significantly with a
margin of at least $0.7\%$ for adversarially trained models. In contrast, BSR
consistently outperforms PAM on these models by a margin of $6.3\%$ to
$8.1\%$, and maintains comparable white-box attack performance with Admix.
Although PAM-TI-DIM can achieve at least $93.0\%$ attack success rate on
adversarially trained models. Notably, BSR-SI-TI-DIM achieves average attack
success rate of $98.4\%$ on adversarially trained models, surpassing PAM-TI-
DIM with a margin of at least $3.1\%$. Such remarkable attack performance
validates the remarkable effectiveness of BSR for improving the
transferability and poses a huge threat to security-critical applications once
again.

###  4.5 Evaluation on Defense Method

To thoroughly evaluate the effectiveness of our proposed method, we assess the
attack performance of BSR against several defense mechanisms, including HGD,
R&P, NIPS-r3, Bit-RD, JPEG, FD, RS and NRP. From previous experiments,
combined input transformations with ensemble attack exhibit the best attack
performance. Here we adopt the adversarial examples generated by PAM-TI-DIM,
BSR-TI-DIM and BSR-SI-TI-DIM under the ensemble setting to attack these
defense approaches.

As shown in Tab. 4, BSR-SI-TI-DIM achieves the average attack success rate of
$94.1\%$, which outperforms PAM-TI-DIM by an average margin of $2.5\%$.
Notably, even on the certified defense RS and powerful denoiser NRP, BSR-SI-
TI-DIM achieve the attack success rate of $83.9\%$ and $84.2\%$, which
outperforms PAM-TI-DIM with a clear margin of $9.9\%$ and $0.4\%$,
respectively. Such excellent attack performance further shows the superiority
of BSR and reveals the inefficiency of existing defenses.

###  4.6 Ablation Study

To further gain insight into the performance improvement of BSR, we conduct
ablation and hyper-parameter studies by generating the adversarial examples on
Inc-v3 and evaluating them on the other six models.

On the effectiveness of shuffle and rotation. After splitting the image into
several blocks, we shuffle the image blocks and rotate each block. To explore
the effectiveness of shuffle and rotation, we conduct two additional attacks,
_i.e_., block shuffle (BS) and block rotation (BR). As shown in Fig. 4, BS and
BR can achieve better transferability than MI-FGSM, supporting our proposition
that optimizing the adversarial perturbation on the input image with different
attention heatmaps can eliminate the variance among attention heatmaps to
boost the transferability. By combining BS and BR, BSR exhibits the best
transferability, showing its rationality and high effectiveness in crafting
transferable adversarial examples.

On the number of blocks $n$. As shown in Fig. 4, when $n=1$, BSR only rotates
the raw image, which cannot bring disruption on the attention heatmaps,
showing the poorest transferability. When $n>3$, increasing $n$ results in too
much variance that cannot be effectively eliminated, decreasing the
transferability. Hence, a suitable magnitude of disruption on the raw image is
significant to improve the transferability and we set $n=2$ in our
experiments.

On the number of transformed images $N$. Since BSR introduces variance when
breaking the intrinsic relation, we adopt the average gradient on $N$
transformed images to eliminate such variance. As shown in Fig. 4, when $N=5$,
BSR can already achieve better transferability than MI-FGSM, showing its high
efficiency and effectiveness. When we increase $N$, the attack performance
would be further improved and be stable when $N>20$. Hence, we set $N=20$ in
our experiments.

On the range of rotation angles $\tau$. We randomly rotate the image blocks
with the angle $-\tau\leq\beta\leq\tau$, which also affects the magnitude of
disruption on the image. As shown in Fig. 4, we conduct the experiments from
$\tau=6^{\circ}$ to $\tau=180^{\circ}$. When $\tau$ is smaller than
$24^{\circ}$, increasing $\tau$ results in more disruption on the image so as
to achieve better transferability. If we continue to increase $\tau$, the
rotation will introduce too much disruption, which decays the performance.
Hence, we set $\tau=24^{\circ}$ in our experiments.

##  5 Conclusion

Intuitively, the consistent attention heatmaps of adversaries on different
models will have better transferability. However, we find that the existing
input transformation based attacks often result in inconsistent attention
heatmaps on various models, limiting the transferability. To this end, we
propose a novel input transformation based attack called block shuffle and
rotation (BSR), which optimizes the perturbation on several transformed images
with different attention heatmaps to eliminate the variance among the
attention heatmaps on various models. Empirical evaluations on ImageNet
dataset show that BSR achieves better transferability than the SOTA attacks
under various attack settings. We hope our approach can provide new insight to
improve the transferability by generating adversarial examples with more
stable attention heatmaps on different models.

## References

  * Carlini and Wagner [2017] Nicholas Carlini and David Wagner.  Towards Evaluating the Robustness of Neural Networks.  In _IEEE Symposium on Security and Privacy_ , pages 39–57, 2017.
  * Cohen et al. [2019] Jeremy M Cohen, Elan Rosenfeld, and J Zico Kolter.  Certified adversarial robustness via randomized smoothing.  In _International Conference on Machine Learning_ , 2019.
  * Dong et al. [2018] Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li.  Boosting Adversarial Attacks with Momentum.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 9185–9193, 2018.
  * Dong et al. [2019] Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu.  Evading Defenses to Transferable Adversarial Examples by Translation-invariant Attacks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4312–4321, 2019.
  * Eykholt et al. [2018] Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Tadayoshi Kohno, and Dawn Song.  Robust Physical-World Attacks on Deep Learning Visual Classification.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1625–1634, 2018.
  * Ge et al. [2023a] Zhijin Ge, Hongying Liu, Xiaosen Wang, Fanhua Shang, and Yuanyuan Liu.  Boosting Adversarial Transferability by Achieving Flat Local Maxima.  In _Proceedings of the Advances in Neural Information Processing Systems_ , 2023a.
  * Ge et al. [2023b] Zhijin Ge, Fanhua Shang, Hongying Liu, Yuanyuan Liu, Liang Wan, Wei Feng, and Xiaosen Wang.  Improving the Transferability of Adversarial Examples with Arbitrary Style Transfer.  In _Proceedings of the ACM International Conference on Multimedia_ , page 4440–4449, 2023b.
  * Goodfellow et al. [2015] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy.  Explaining and harnessing adversarial examples.  In _International Conference on Learning Representations_ , 2015.
  * Gowal et al. [2019] Sven Gowal, Krishnamurthy Dj Dvijotham, Robert Stanforth, Rudy Bunel, Chongli Qin, Jonathan Uesato, Relja Arandjelovic, Timothy Mann, and Pushmeet Kohli.  Scalable Verified Training for Provably Robust Image Classification.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4842–4851, 2019.
  * Guo et al. [2018] Chuan Guo, Mayank Rana, Moustapha Cisse, and Laurens Van Der Maaten.  Countering adversarial images using input transformations.  _International Conference on Learning Representations_ , 2018.
  * Guo et al. [2019] Chuan Guo, Jacob Gardner, Yurong You, Andrew Gordon Wilson, and Kilian Weinberger.  Simple Black-box Adversarial Attacks.  In _International Conference on Machine Learning_ , pages 2484–2493, 2019.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep Residual Learning for Image Recognition.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 770–778, 2016.
  * Huang et al. [2017] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 4700–4708, 2017.
  * Kurakin et al. [2017] Alexey Kurakin, Ian Goodfellow, and Samy Bengio.  Adversarial Examples in the Physical World.  In _International Conference on Learning Representations (Workshop)_ , 2017.
  * Li et al. [2020a] Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and Bo Li.  QEBA: Query-Efficient Boundary-Based Blackbox Attack.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1218–1227, 2020a.
  * Li et al. [2020b] Yingwei Li, Song Bai, Yuyin Zhou, Cihang Xie, Zhishuai Zhang, and Alan L. Yuille.  Learning Transferable Adversarial Examples via Ghost Networks.  In _AAAI Conference on Artificial Intelligence_ , 2020b.
  * Liao et al. [2018] Fangzhou Liao, Ming Liang, Yinpeng Dong, Tianyu Pang, Xiaolin Hu, and Jun Zhu.  Defense against Adversarial Attacks using High-level Representation Guided Denoiser.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1778–1787, 2018.
  * Lin et al. [2020] Jiadong Lin, Chuanbiao Song, Kun He, Liwei Wang, and John E. Hopcroft.  Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks.  In _International Conference on Learning Representations_ , 2020.
  * Liu et al. [2017] Yanpei Liu, Xinyun Chen, Chang Liu, and Dawn Song.  Delving into transferable adversarial examples and black-box attacks.  In _International Conference on Learning Representations_ , 2017.
  * Liu et al. [2019] Zihao Liu, Qi Liu, Tao Liu, Nuo Xu, Xue Liu, Yanzhi Wang, and Wujie Wen.  Feature distillation: Dnn-oriented jpeg compresstion against adversarial examples.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 860–868, 2019.
  * Long et al. [2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell.  Fully Convolutional Networks for Semantic Segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 3431–3440, 2015.
  * Madry et al. [2018] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards Deep Learning Models Resistant to Adversarial Attacks.  In _International Conference on Learning Representations_ , 2018.
  * Naseer et al. [2020] Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli.  A Self-supervised Approach for Adversarial Robustness.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 262–271, 2020.
  * Olga et al. [2015] Russakovsky Olga, Deng Jia, Su Hao, Krause Jonathan, Satheesh Sanjeev, Ma Sean, hUANG Zhiheng, Karpathy Andrej, Khosla Adiya, and Bernstein Michael et al.  Imagenet large scale visual recognition challenge.  In _International Journal of Computer Vision_ , pages 211–252, 2015.
  * Papernot et al. [2017] Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z Berkay Celik, and Ananthram Swami.  Practical black-box attacks against machine learning.  In _Proceedings of the 2017 ACM on Asia conference on computer and communications security_ , pages 506–519, 2017.
  * Redmon et al. [2016] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You Only Look Once: Unified, Real-time Object Detection.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 779–788, 2016.
  * Ren et al. [2015] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In _Advances in Neural Information Processing Systems_ , 2015.
  * Selvaraju et al. [2016] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2921–2929, 2016.
  * Selvaraju et al. [2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In _International Conference on Computer Vision_ , pages 618–626, 2017.
  * Sharif et al. [2016] Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter.  Accessorize to a Crime: Real and Stealthy Attacks on State-of-the-Art Face Recognition.  In _Proceedings of the ACM SIGSAC Conference on Computer and Communications Security_ , pages 1528–1540, 2016.
  * Smilkov et al. [2017] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg.  Smoothgrad: Removing Noise by Adding Noise.  _arXiv preprint arXiv:1706.03825_ , 2017.
  * Song et al. [2020] Chuanbiao Song, Kun He, Lin Jiadong, Liwei Wang, and E. John Hopcroft.  Robust Local Features for Improving the generalization of adversarial training.  In _International Conference on Learning Representaion_ , 2020.
  * Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan.  Axiomatic attribution for deep networks.  In _International conference on machine learning_ , pages 3319–3328, 2017.
  * Szegedy et al. [2014] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In _International Conference on Learning Representations_ , 2014.
  * Szegedy et al. [2016] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Wojna Zbjgniew.  Rethinking the inception architecture for computer vision.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2818–2826, 2016.
  * Szegedy et al. [2017] Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi.  Inception-v4, inception-resnet and the impact of residual connections on learning.  In _AAAI Conference on Artificial Intelligence_ , 2017.
  * Tramèr et al. [2018] Florian Tramèr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel.  Ensemble Adversarial Training: Attacks and Defenses.  _International Conference on Learning Representations_ , 2018.
  * Uesato et al. [2018] Jonathan Uesato, Brendan O’donoghue, Pushmeet Kohli, and Aaron Oord.  Adversarial Risk and the Dangers of Evaluating against Weak Attacks.  In _International Conference on Machine Learning_ , pages 5025–5034, 2018.
  * Wang et al. [2018] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu.  CosFace: Large Margin Cosine Loss for Deep Face Recognition.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 5265–5274, 2018.
  * Wang and He [2021] Xiaosen Wang and Kun He.  Enhancing the transferability of adversarial attacks through variance tuning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2021.
  * Wang et al. [2019] Xiaosen Wang, Kun He, Chuanbiao Song, Liwei Wang, and John E. Hopcroft.  AT-GAN: A Generative Attack Model for Adversarial Transferring on Generative Adversarial Nets.  _arXiv preprint arXiv:1904.07793_ , 2019.
  * Wang et al. [2021a] Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He.  Admix: Enhancing the transferability of adversarial attacks.  In _International Conference on Computer Vision_ , pages 16138–16147, 2021a.
  * Wang et al. [2021b] Xiaosen Wang, Jiadong Lin, Han Hu, Jingdong Wang, and Kun He.  Boosting Adversarial Transferability through Enhanced Momentum.  In _British Machine Vision Conference_ , page 272, 2021b.
  * Wang et al. [2022] Xiaosen Wang, Zeliang Zhang, Kangheng Tong, Dihong Gong, Kun He, Zhifeng Li, and Wei Liu.  Triangle Attack: A Query-efficient Decision-based Adversarial Attack.  In _European conference on computer vision_ , 2022.
  * Wang et al. [2023a] Xiaosen Wang, Kangheng Tong, and Kun He.  Rethinking the Backward Propagation for Adversarial Transferability.  In _Proceedings of the Advances in Neural Information Processing Systems_ , 2023a.
  * Wang et al. [2023b] Xiaosen Wang, Zeliang Zhang, and Jianping Zhang.  Structure Invariant Transformation for better Adversarial Transferability.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 4607–4619, 2023b.
  * Wang et al. [2021c] Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Qin Zhan, and Kui Ren.  Feature importance-aware transferable adversarial attacks.  In _International Conference on Computer Vision_ , pages 7639–7648, 2021c.
  * Wang et al. [2023c] Zhiyuan Wang, Zeliang Zhang, Siyuan Liang, and Xiaosen Wang.  Diversifying the High-level Features for better Adversarial Transferability.  In _Proceedings of the British Machine Vision Conference_ , 2023c.
  * Wu et al. [2020] Weibin Wu, Yuxin Su, Xixian Chen, Shenglin Zhao, Irwin King, Michael R Lyu, and Yu-Wing Tai.  Boosting the Transferability of Adversarial Samples via Attention.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 1161–1170, 2020.
  * Xie et al. [2018] Cihang Xie, Jianyu Wang, Zhishuai Zhang, Zhou Ren, and Alan Yuille.  Mitigating adversarial effects through randomization.  In _International Conference on Learning Representations_ , 2018.
  * Xie et al. [2019] Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L Yuille.  Improving Transferability of Adversarial Examples with Input Diversity.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2730–2739, 2019.
  * Xiong et al. [2022] Yifeng Xiong, Jiadong Lin, Min Zhang, John E Hopcroft, and Kun He.  Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 14983–14992, 2022.
  * Xu et al. [2018] Weilin Xu, David Evans, and Yanjun Qi.  Feature squeezing: Detecting adversarial examples in deep neural networks.  _Network and Distributed System Security Symposium_ , 2018.
  * Zhang et al. [2019] Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, and Cho-Jui Hsieh.  Towards Stable and Efficient Training of Verifiably Robust Neural Networks.  In _International Conference on Learning Representations_ , 2019.
  * Zhang et al. [2023] Jianping Zhang, Jen-tse Huang, Wenxuan Wang, Yichen Li, Weibin Wu, Xiaosen Wang, Yuxin Su, and Michael R. Lyu.  Improving the Transferability of Adversarial Samples by Path-Augmented Method.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 8173–8182, 2023.
  * Zhou et al. [2016] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba.  Learning deep features for discriminative localization.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2921–2929, 2016.
  * Zhou et al. [2018] Wen Zhou, Xin Hou, Yongjun Chen, Mengyun Tang, Xiangqi Huang, Xiang Gan, and Yong Yang.  Transferable adversarial perturbations.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 452–467, 2018.

##  6 Appendix

In this appendix, we first provide more results of integrating BSR with DIM,
TIM, Admix and PAM, the adversaries are crafted on three other models _i.e_.
Inc-v4, IncRes-v2 and Res-101. Then we provide more comparison on the heatmap
of various images using different methods.

###  6.1 More Evaluations on Combined Input Transformation

We provide the attack results of adversarial examples generated by integrating
BSR with DIM, TIM, Admix, and PAM on the other three models, namely Inc-v4,
IncRes-v2 and Res-101. As shown in Tab. 5-7, BSR can remarkably boost the
transferability of various input transformation based attacks when generating
the adversarial examples on the three models. Zhang et al. [55] have
demonstrated that the combination of PAM-TI-DIM achieves state-of-the-art
transferability. However, its performance on adversarially trained models
remains relatively weak. Our proposed BSR-SI-DI-TIM consistently outperforms
PAM-TI-DIM with a margin of at least $6.4\%$. Such excellent performance
improvement further validates the high teffectiveness of the proposed BSR for
boosting the adversarial transferability.

###  6.2 Visualization on Attention Heatmaps

To further support our motivation, we visualize more attention heatmaps of
sampled benign images on both source model Inc-v4 and target model Inc-v3 in
Fig. 5. We can observe that the attention heatmaps of adversarial examples
crafted by BSR on source model are more similar with heatmap on target model
than the results of other attacks. These results validate our motivation that
our proposed method can eliminate the variance of attention heatmaps among
various models, which is of benefit to generate more transferable adversarial
examples.

Attack | Inc-v3 | IncRes-v2 | Res-101 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---
BSR-DIM | 96.8$\uparrow$24.8 | 94.1$\uparrow$30.3 | 87.8$\uparrow$30.4 | 59.7$\uparrow$37.1 | 53.6$\uparrow$32.5 | 37.6$\uparrow$11.7
BSR-TIM | 93.8$\uparrow$34.7 | 91.7$\uparrow$42.7 | 84.6$\uparrow$42.7 | 69.4$\uparrow$42.6 | 63.4$\uparrow$40.5 | 51.1$\uparrow$34.5
BSR-SIM | 99.0$\uparrow$18.6 | 97.9$\uparrow$24.5 | 95.2$\uparrow$25.8 | 86.7$\uparrow$38.1 | 85.2$\uparrow$40.0 | 70.2$\uparrow$40.6
BSR-Admix | 99.1$\uparrow$10.1 | 98.5$\uparrow$13.2 | 97.0$\uparrow$18.0 | 88.0$\uparrow$32.5 | 86.0$\uparrow$34.3 | 74.0$\uparrow$41.7
BSR-PAM | 99.2$\uparrow$12.5 | 98.2$\uparrow$16.6 | 96.2$\uparrow$20.3 | 87.2$\uparrow$31.8 | 81.4$\uparrow$30.9 | 62.8$\uparrow$29.6
Admix-TI-DIM | 91.0 | 88.6 | 83.2 | 76.0 | 74.7 | 64.3
PAM-TI-DIM | 91.5 | 88.5 | 83.5 | 77.1 | 73.2 | 62.1
BSR-TI-DIM | 91.2 | 88.5 | 83.5 | 77.1 | 73.2 | 62.1
BSR-SI-TI-DIM | 98.1 | 96.2 | 94.7 | 90.1 | 89.0 | 80.0
|  |  |  |  |  |
Table 5: Attack success rates (%) on seven models under single model setting with various input transformations combined with BSR. The adversaries are crafted on Inc-v4. $\uparrow$ indicates the increase of attack success rate when combined with BSR. Attack | Inc-v3 | Inc-v4 | Res-101 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---
BSR-DIM | 95.1$\uparrow$24.8 | 95.0$\uparrow$30.3 | 91.5$\uparrow$33.5 | 71.7$\uparrow$41.3 | 65.4$\uparrow$41.9 | 54.8$\uparrow$37.9
BSR-TIM | 93.4$\uparrow$31.2 | 93.2$\uparrow$37.6 | 89.1$\uparrow$38.8 | 80.3$\uparrow$47.9 | 75.9$\uparrow$48.4 | 70.0$\uparrow$47.4
BSR-SIM | 98.5$\uparrow$12.6 | 98.4$\uparrow$18.4 | 97.4$\uparrow$21.3 | 92.5$\uparrow$36.3 | 89.5$\uparrow$40.4 | 81.9$\uparrow$39.4
BSR-Admix | 99.1$\uparrow$ 8.3 | 99.3$\uparrow$13.0 | 98.1$\uparrow$15.9 | 93.7$\uparrow$30.1 | 92.1$\uparrow$35.5 | 87.3$\uparrow$37.9
BSR-PAM | 99.3$\uparrow$10.7 | 99.2$\uparrow$12.9 | 98.5$\uparrow$16.9 | 94.2$\uparrow$28.2 | 91.2$\uparrow$32.9 | 83.3$\uparrow$32.3
Admix-TI-DIM | 90.9 | 89.5 | 86.5 | 81.7 | 77.7 | 76.4
PAM-TI-DIM | 92.2 | 90.5 | 86.9 | 84.2 | 80.9 | 78.4
BSR-TI-DIM | 94.5 | 93.6 | 90.6 | 81.0 | 77.5 | 72.9
BSR-SI-TI-DIM | 98.6 | 98.1 | 96.3 | 95.4 | 94.0 | 91.7
|  |  |  |  |  |
Table 6: Attack success rates (%) on seven models under single model setting with various input transformations combined with BSR. The adversaries are crafted on IncRes-v2. $\uparrow$ indicates the increase of attack success rate when combined with BSR. Attack | Inc-v3 | Inc-v4 | IncRes-v2 | Inc-v3ens3 | Inc-v3ens4 | IncRes-v2ens
---|---|---|---|---|---|---
BSR-DIM | 97.7$\uparrow$21.7 | 96.6$\uparrow$28.2 | 97.1$\uparrow$26.8 | 82.7$\uparrow$48.0 | 75.8$\uparrow$44.0 | 59.7$\uparrow$40.1
BSR-TIM | 96.7$\uparrow$36.8 | 95.6$\uparrow$43.4 | 94.9$\uparrow$43.0 | 86.6$\uparrow$52.2 | 83.4$\uparrow$52.2 | 75.3$\uparrow$51.6
BSR-SIM | 98.8 $\uparrow$20.3 | 98.3$\uparrow$25.8 | 98.1$\uparrow$26.9 | 91.5$\uparrow$46.7 | 89.1$\uparrow$49.0 | 75.8$\uparrow$48.5
BSR-Admix | 99.6$\uparrow$15.1 | 99.2$\uparrow$19.0 | 99.4$\uparrow$18.7 | 94.2$\uparrow$42.6 | 92.5$\uparrow$47.8 | 82.4$\uparrow$52.5
BSR-PAM | 98.7$\uparrow$21.3 | 97.2$\uparrow$23.3 | 97.7$\uparrow$ 22.0 | 90.0$\uparrow$38.8 | 86.8$\uparrow$40.3 | 70.3$\uparrow$38.1
Admix-TI-DIM | 89.5 | 85.6 | 87.5 | 80.4 | 75.2 | 67.5
PAM-TI-DIM | 85.8 | 83.2 | 84.9 | 81.1 | 76.1 | 66.9
BSR-TI-DIM | 97.3 | 95.1 | 95.6 | 88.3 | 85.4 | 76.1
BSR-SI-TI-DIM | 97.4 | 97.0 | 96.6 | 94.8 | 93.2 | 89.1
|  |  |  |  |  |
Table 7: Attack success rates (%) on seven models under single model setting
with various input transformations combined with BSR. The adversaries are
crafted on Res-101. $\uparrow$ indicates the increase of attack success rate
when combined with BSR.

Source: Inc-v3

Target: Inc-v4

Source: Inc-v3

Target: Inc-v4

Source: Inc-v3

Target: Inc-v4

Source: Inc-v3

Target: Inc-v4

Raw Image

DIM

TIM

SIM

Admix

PAM

BSR

Figure 5: The visualization of attention heatmaps [28] of adversaries crafted
by various input transformation methods on source model Inc-v3 [36], and
target model Inc-v4 [35]


GR;L}(k_{\sigma_{l}},\cdots,k_{\sigma_{k-1}},(\ell+P)^{h^{\prime}_{2}},-\ell^{h^{\prime}_{1}})=0\,,$
(172)

thus it is safe to replace ${\cal I}^{\epsilon}_{P_{+}\sigma_{k-1}\sigma_{k}}$
by ${\cal I}^{\epsilon}_{P_{+}\sigma_{k-1}-}$. Consequently, we get another
combinatory operator

$\displaystyle\Big{(}\prod_{i=l}^{k-2}\,{\cal
I}^{\epsilon}_{P_{+}\sigma_{i}\sigma_{i+1}}\Big{)}\,{\cal
I}^{\epsilon}_{P_{+}\sigma_{k-1}-}\,{\cal
T}^{\epsilon}[\overline{P_{+},-}]={\cal
T}^{\epsilon}[\overline{P_{+},\sigma_{l},\cdots,\sigma_{k-1},-}]\,,$ (173)

which transmutes ${\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm GR;L}$ to
${\cal A}^{\widetilde{\epsilon}}_{{\rm
YM};L}\big{(}\overline{(P_{+})^{h_{2}},\sigma_{l},\cdots,\sigma_{k-1},(-)^{h_{1}}}\big{)}$,
and annihilates ${\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm GR;R}$. Based
on the above discussion, we conclude that under the on-shell condition
$(l+P)^{2}=0$, the operator ${\cal
T}^{\epsilon}[\overline{+,\sigma_{k},\cdots,\sigma_{k-1},-}]$ factorizes into

$\displaystyle{\cal O}_{L}={\cal
T}^{\epsilon}[\overline{P_{+},\sigma_{l},\cdots,\sigma_{k-1},-}]\,,~{}~{}~{}~{}~{}~{}~{}~{}{\cal
O}_{R}={\cal
T}^{\epsilon}[\overline{+,\sigma_{k},\cdots,\sigma_{l-1},P_{-}}]\,,$ (174)

and transmutes the factorization of GR amplitude to the factorization of color
ordered YM amplitude.

###  5.3 Operator ${\cal L}^{\epsilon}_{\circ}\,{\cal D}$

The next example is the factorization of operator ${\cal
L}^{\epsilon}_{\circ}\,{\cal D}$. We will demonstrate it by considering the
relation ${\bf I}^{\widetilde{\epsilon}}_{\rm BI}={\cal
L}^{\epsilon}_{\circ}\,{\cal D}\,{\bf I}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}$. In the case under consideration, (158) becomes

$\displaystyle\lim_{\ell^{2}\to 0}\,\ell^{2}\,{\cal L}^{\epsilon}_{\circ
C}\,{\cal D}\,{\bf I}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}({\boldsymbol{H}}_{n})$ $\displaystyle=$ $\displaystyle\sum_{h_{1}}\,{\cal
L}^{\epsilon}\,{\cal T}^{\epsilon}[\overline{+,-}]\,{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}({\boldsymbol{H}}_{n}+\\{-\ell^{h^{\prime}_{1}},\ell^{\bar{h}^{\prime}_{1}}\\})$
(175) $\displaystyle=$ $\displaystyle\sum_{h_{1}}\,{\cal
A}^{\widetilde{\epsilon}}_{\rm
BI}({\boldsymbol{P}}_{n}+\\{-\ell^{h_{1}},\ell^{\bar{h}_{1}}\\})\,.$

Doing the second step in (156), then we need to show

$\displaystyle\lim_{(\ell+P)^{2}\to 0}\,(\ell+P)^{2}\,{\cal
L}^{\epsilon}\,{\cal T}^{\epsilon}[\overline{+,-}]\,{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}({\boldsymbol{H}}_{n}+\\{-\ell^{h^{\prime}_{1}},\ell^{\bar{h}^{\prime}_{1}}\\})$
(176) $\displaystyle=$ $\displaystyle\sum_{h_{2}}\,\Big{(}{\cal O}_{L}\,{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{{\rm
GR};L}(k_{\sigma_{l}},\cdots,k_{\sigma_{k-1}},(\ell+P)^{h^{\prime}_{2}},-\ell^{h^{\prime}_{1}})\Big{)}\,\Big{(}{\cal
O}_{R}\,{\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm
GR};R}(k_{\sigma_{k}},\cdots,k_{\sigma_{l-1}},-(\ell+P)^{\bar{h}^{\prime}_{2}},\ell^{\bar{h}^{\prime}_{1}})\Big{)}\,,~{}~{}~{}~{}$

which is just the tree level factorization

$\displaystyle\lim_{(\ell+P)^{2}\to 0}\,(\ell+P)^{2}\,{\cal
A}^{\widetilde{\epsilon}}_{\rm
BI}({\boldsymbol{P}}_{n}+\\{-\ell^{h_{1}},\ell^{\bar{h}_{1}}\\})$ (177)
$\displaystyle=$ $\displaystyle\sum_{h_{2}}\,{\cal
A}^{\widetilde{\epsilon}}_{{\rm
BI};L}(k_{\sigma_{l}},\cdots,k_{\sigma_{k-1}},(\ell+P)^{h_{2}},-\ell^{h_{1}})\,{\cal
A}^{\widetilde{\epsilon}}_{{\rm
BI};R}(k_{\sigma_{k}},\cdots,k_{\sigma_{l-1}},-(\ell+P)^{\bar{h}_{2}},\ell^{\bar{h}_{1}})\,.$

Again, the factorization of the operator ${\cal L}^{\epsilon}\,{\cal
T}^{\epsilon}[\overline{+,-}]$ in (176) is ensured by the factorization of
tree amplitude.

It is straightforward to understand the factorization of the operator ${\cal
L}^{\epsilon}\,{\cal T}^{\epsilon}[\overline{+,-}]$ at tree level. In the
previous subsection, we proved that the operator ${\cal
T}^{\epsilon}[\overline{+,-}]$ factorizes into ${\cal
T}^{\epsilon}[\overline{+,P_{-}}]$ and ${\cal
T}^{\epsilon}[\overline{P_{+},-}]$ under the on-shell condition
$(\ell+P)^{2}=0$. One can regroup ${\cal L}^{\epsilon}$ to be ${\cal
L}^{\epsilon}={\cal L}^{\epsilon}_{L}\,{\cal L}^{\epsilon}_{R}$ directly, by
considering whether ${\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm GR;L}$ or
${\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm GR;R}$ contains the
polarization vector $\epsilon_{\sigma_{i}}$. We emphasize that two definitions
${\cal L}^{\epsilon}$ and $\bar{{\cal L}}^{\epsilon}$ in (39) leads to the
same factorization. Notice that when considering $\bar{{\cal L}}^{\epsilon}$,
for the operator $(k_{i}\cdot k_{j})\partial_{\epsilon_{i}\cdot\epsilon_{j}}$
with $\epsilon_{i}\in{\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};L}$
and $\epsilon_{j}\in{\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};R}$,
one can not use the technic of separating ${\cal
T}^{\epsilon}[\overline{+,-}]$ into ${\cal T}^{\epsilon}[\overline{P_{+},-}]$
and ${\cal T}^{\epsilon}[\overline{+,P_{-}}]$ to separate
$\partial_{\epsilon_{i}\cdot\epsilon_{j}}$. The reason is, the summation
$\sum_{r}\,(\epsilon_{P_{+}}^{r})^{\mu}(\epsilon_{P_{-}}^{r})^{\nu}$ has been
removed when separating ${\cal T}^{\epsilon}[\overline{+,-}]$. Thus, the
operator $(k_{i}\cdot k_{j})\partial_{\epsilon_{i}\cdot\epsilon_{j}}$ across
${\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};L}$ and ${\cal
A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};R}$ annihilates both two
amplitudes. After regrouping ${\cal L}^{\epsilon}$, we find that the operator
${\cal L}^{\epsilon}\,{\cal T}^{\epsilon}[\overline{+,-}]$ factorizes into

$\displaystyle{\cal O}_{L}={\cal L}^{\epsilon}_{L}\,{\cal
T}^{\epsilon}[\overline{P_{+},-}]\,,~{}~{}~{}~{}~{}~{}~{}~{}{\cal O}_{R}={\cal
L}^{\epsilon}_{R}\,{\cal T}^{\epsilon}[\overline{+,P_{-}}]\,.$ (178)

###  5.4 Operator ${\cal T}^{\epsilon}_{X_{2m}}\,({\cal D}+1)$

The final example is the operator ${\cal T}^{\epsilon}_{X_{2m}}\,({\cal
D}+1)$, which links the $1$-loop GR and EM Feynman integrands together as
${\bf I}^{\epsilon,\widetilde{\epsilon}}_{\rm
EM}({\boldsymbol{P}}_{2m};{\boldsymbol{H}}_{n-2m})={\cal
T}^{\epsilon}_{X_{2m}}\,({\cal D}+1)\,{\bf
I}^{\epsilon,\widetilde{\epsilon}}_{\rm GR}({\boldsymbol{H}}_{n})$. The
factorization of the operator ${\cal T}^{\epsilon}_{{\cal X}_{2m}}\,(N\,{\cal
D}+1)$ can be discussed similarly. For latter convenience, let us assume that
the external gravitons in the set $\\{1,\cdots,2m\\}$ are turned to photons by
the operator ${\cal T}^{\epsilon}_{X_{2m}}\,({\cal D}+1)$. For the current
example, the new situation occurs. A special cut should be recognized not only
by the momenta channel, but also what kind of cut virtual particle. We need to
discuss the following three cases, which are cutting two gravitons, cutting
one graviton and one photon, as well as cutting two photons.

As before, we start by doing the first step in (156). Using (129) and (143),
we know that

$\displaystyle\lim_{\ell^{2}\to 0}\,\ell^{2}\,{\cal
T}^{\epsilon}_{X_{2m}}\,({\cal D}+1)\,{\bf
I}^{\epsilon,\widetilde{\epsilon}}_{\rm GR}({\boldsymbol{H}}_{n})$
$\displaystyle=$ $\displaystyle\sum_{h^{\prime}_{1}}\,{\cal
T}^{\epsilon}_{X_{2m}}\,{\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}({\boldsymbol{H}}_{n}+\\{-\ell^{h^{\prime}_{1}},\ell^{\bar{h}^{\prime}_{1}}\\})$
(179) $\displaystyle=$ $\displaystyle\sum_{h^{\prime}_{1}}\,{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{\rm
EM}({\boldsymbol{P}}_{2m};{\boldsymbol{H}}_{n-2m}+\\{-\ell^{h^{\prime}_{1}},\ell^{\bar{h}^{\prime}_{1}}\\})\,,~{}~{}~{}~{}$

if we cut two gravitons. We also have

$\displaystyle\lim_{\ell^{2}\to 0}\,\ell^{2}\,{\cal
T}^{\epsilon}_{X_{2m}}\,({\cal D}+1)\,{\bf
I}^{\epsilon,\widetilde{\epsilon}}_{\rm GR}({\boldsymbol{H}}_{n})$
$\displaystyle=$ $\displaystyle\sum_{h_{1}}\,{\cal
T}^{\epsilon}_{X_{2m+2}}\,{\cal A}^{\epsilon,\widetilde{\epsilon}}_{\rm
GR}({\boldsymbol{H}}_{n}+\\{-\ell^{h^{\prime}_{1}},\ell^{\bar{h}^{\prime}_{1}}\\})$
(180) $\displaystyle=$ $\displaystyle\sum_{h_{1}}\,{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{\rm
EM}({\boldsymbol{P}}_{2m}+\\{-\ell^{h_{1}},\ell^{\bar{h}_{1}}\\};{\boldsymbol{H}}_{n-2m})\,,~{}~{}~{}~{}$

if we cut two photons.

Now we do the second step in (156). For the first case (179), if we cut two
gravitons in the second step, all ${\cal T}^{\epsilon}[\overline{a,b}]$ with
$\epsilon_{a}\in{\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};L}$ and
$\epsilon_{b}\in{\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};R}$
annihilates both two amplitudes, thus the effective operator becomes

$\displaystyle{\cal T}^{\epsilon}_{X_{2m}}|_{\rm effective}$ $\displaystyle=$
$\displaystyle\Big{(}\sum_{\rho_{L}}\,\prod_{i_{k},j_{k}\in\rho_{L}}\,{\cal
T}^{\epsilon}[\overline{i_{k},j_{k}}]\Big{)}\,\Big{(}\sum_{\rho_{R}}\,\prod_{i_{l},j_{l}\in\rho_{R}}\,{\cal
T}^{\epsilon}[\overline{i_{l},j_{l}}]\Big{)}$ (181) $\displaystyle=$
$\displaystyle{\cal T}^{\epsilon}_{X_{m_{L}}}\,{\cal
T}^{\epsilon}_{X_{m_{R}}}\,,$

where $m_{L}$ and $m_{R}$ denote the number of photons in ${\cal A}_{{\rm
EM};L}$ and ${\cal A}_{{\rm EM};R}$ respectively, and satisfy
$m_{L}+m_{R}=2m$. More explicitly, we have assumed that the operator ${\cal
T}^{\epsilon}_{X_{2m}}$ turns external gravitons belong to $\\{1,\cdots,2m\\}$
to photons, and now we divide $\\{1,\cdots,2m\\}$ into
$\\{a_{1},\cdots,a_{m_{L}}\\}$ and $\\{b_{1},\cdots,b_{m_{R}}\\}$, where the
external graviton labeled by $a_{i}$ is included in ${\cal
A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};L}$, while that labeled by
$b_{i}$ is included in ${\cal A}^{\epsilon,\widetilde{\epsilon}}_{{\rm
GR};R}$. Partitions $\rho_{L}$ and $\rho_{R}$ are understood as partitions for
$\\{a_{1},\cdots,a_{m_{L}}\\}$ and $\\{b_{1},\cdots,b_{m_{R}}\\}$,
respectively. Thus, for this case, the operator ${\cal T}^{\epsilon}_{X_{2m}}$
factorizes into

$\displaystyle{\cal O}_{L}={\cal
T}^{\epsilon}_{X_{m_{L}}}\,,~{}~{}~{}~{}~{}~{}~{}~{}{\cal O}_{R}={\cal
T}^{\epsilon}_{X_{m_{R}}}\,.~{}~{}~{}~{}$ (182)

If we cut two photons in the second step, the effective part of the operator
${\cal T}^{\epsilon}_{X_{2m}}$ should contain one and only one ${\cal
T}^{\epsilon}[\overline{a,b}]$ with $\epsilon_{a}\in{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};L}$ and $\epsilon_{b}\in{\cal
A}^{\epsilon,\widetilde{\epsilon}}_{{\rm GR};R}$, and we can use (167) to see
that ${\cal T}^{\epsilon}[\overline{a,b}]$ factorizes into ${\cal
T}^{\epsilon}[\overline{a,P_{+}}]$ and ${\cal
T}^{\epsilon}[\overline{P_{-},b}]$. Notice that such decomposition can be used
only once, as discussed in the previous subsection. Then we find

$\displaystyle{\cal T}^{\epsilon}_{X_{2m}}|_{\rm effective}$ $\displaystyle=$
$\displaystyle\sum_{\begin{subarray}{c}a\in\\{a_{1},\cdots,a_{2m_{L}}\\}\\\
b\in\\{b_{1},\cdots,b_{2m_{R}}\\}\end{subarray}}\,\Big{(}{\cal
T}^{\epsilon}[\overline{a,P_{+}}]\,\sum_{\rho^{a}_{L}}\,\prod_{i_{k},j_{k}\in\rho^{a}_{L}}\,{\cal
T}^{\epsilon}[\overline{i_{k},j_{k}}]\Big{)}\,\Big{(}{\cal
T}^{\epsilon}[\overline{P_{-},b}]\,\sum_{\rho^{b}_{R}}\,\prod_{i_{l},j_{l}\in\rho^{b}_{R}}\,{\cal
T}^{\epsilon}[\overline{i_{l},j_{l}}]\Big{)}$ (183) $\displaystyle=$
$\displaystyle{\cal T}^{\epsilon}_{X_{m_{L}+1}}\,{\cal
T}^{\epsilon}_{X_{m_{R}+1}}\,.$

Here $\rho^{a}_{L}$ and $\rho^{b}_{R}$ are understood as partitions for
$\\{a_{1},\cdots,a_{m_{L}}\\}\setminus a$ and
$\\{b_{1},\cdots,b_{m_{R}}\\}\setminus b$, respectively. Thus the operator
${\cal T}^{\epsilon}_{X_{2m}}$ factorizes into

$\displaystyle{\cal O}_{L}={\cal
T}^{\epsilon}_{X_{m_{L}+1}}\,,~{}~{}~{}~{}~{}~{}~{}~{}{\cal O}_{R}={\cal
T}^{\epsilon}_{X_{m_{R}+1}}\,.~{}~{}~{}~{}$ (184)

Similar discussion holds for the second case (180). For the second case, if we
cut two gravitons at the second step, we find

$\displaystyle{\cal T}^{\epsilon}_{X_{2m+2}}|_{\rm effective}$
$\displaystyle=$
$\displaystyle\Big{(}\sum_{\rho^{\prime}_{L}}\,\prod_{i_{k},j_{k}\in\rho^{\prime}_{L}}\,{\cal
T}^{\epsilon}[\overline{i_{k},j_{k}}]\Big{)}\,\Big{(}\sum_{\rho^{\prime}_{R}}\,\prod_{i_{l},j_{l}\in\rho^{\prime}_{R}}\,{\cal
T}^{\epsilon}[\overline{i_{l},j_{l}}]\Big{)}$ (185) $\displaystyle=$
$\displaystyle{\cal T}^{\epsilon}_{X^{\prime}_{m_{L}+1}}\,{\cal
T}^{\epsilon}_{X^{\prime}_{m_{R}+1}}\,.~{}~{}~{}~{}$

Here we use the notations ${\cal T}^{\epsilon}_{X^{\prime}_{m_{L}+1}}$ and
${\cal T}^{\epsilon}_{X^{\prime}_{m_{R}+1}}$ to distinguish them from ${\cal
T}^{\epsilon}_{X_{m_{L}+1}}$ and ${\cal T}^{\epsilon}_{X_{m_{R}+1}}$ in (184).
This formula should be understood as follows. The operator ${\cal
T}^{\epsilon}_{X_{2m+2}}$ turns external gravitons in $\\{1,\cdots,2m,+,-\\}$
to photons, and we divide $\\{1,\cdots,2m\\}$ into
$\\{a_{1},\cdots,a_{m_{L}}\\}$ and $\\{b_{1},\cdots,b_{m_{R}}\\}$ as before.
Then $\rho^{\prime}_{L}$ and $\rho^{\prime}_{R}$ are partitions for
$\\{a_{1},\cdots,a_{m_{L}},-\\}$ and $\\{b_{1},\cdots,b_{m_{R}},+\\}$,
respectively. The factorized formula in (185) is the similar as that in (184).
This similarity is quite natural, since from the unitarity cut point of view,
both two cases are cutting one graviton and one photon. Cutting two photons at
the second step gives

$\displaystyle{\cal T}^{\epsilon}_{X_{2m+2}}|_{\rm effective}$
$\displaystyle=$
$\displaystyle\sum_{\begin{subarray}{c}a\in\\{a_{1},\cdots,a_{m_{L}},-\\}\\\
b\in\\{b_{1},\cdots,b_{m_{R}},+\\}\end{subarray}}\,\Big{(}{\cal
T}^{\epsilon}[\overline{a,P_{+}}]\,\sum_{\rho^{\prime
a}_{L}}\,\prod_{i_{k},j_{k}\in\rho^{\prime a}_{L}}\,{\cal
T}^{\epsilon}[\overline{i_{k},j_{k}}]\Big{)}\,\Big{(}{\cal
T}^{\epsilon}[\overline{P_{-},b}]\,\sum_{\rho^{\prime
b}_{R}}\,\prod_{i_{l},j_{l}\in\rho^{\prime b}_{R}}\,{\cal
T}^{\epsilon}[\overline{i_{l},j_{l}}]\Big{)}$ (186) $\displaystyle=$
$\displaystyle{\cal T}^{\epsilon}_{X_{m_{L}+2}}\,{\cal
T}^{\epsilon}_{X_{m_{R}+2}}\,,$

thus the operator ${\cal T}^{\epsilon}_{X_{2m+2}}$ factorizes into ${\cal
T}^{\epsilon}_{X_{m_{L}+2}}$ and ${\cal T}^{\epsilon}_{X_{m_{R}+2}}$. Here
$\rho^{\prime a}_{L}$ and $\rho^{\prime b}_{R}$ are partitions of
$\\{a_{1},\cdots,a_{m_{L}},-\\}\setminus a$ and
$\\{b_{1},\cdots,b_{m_{R}},+\\}\setminus b$, respectively.

##  6 Summary and discussions

Feynman integrand | ${\cal O}^{\epsilon}_{\circ}$ | ${\cal O}^{\widetilde{\epsilon}}_{\circ}$
---|---|---
${\bf I}_{{\rm GR}}^{\epsilon,\widetilde{\epsilon}}(\boldsymbol{H}_{n})$ | $\mathbb{I}$ | $\mathbb{I}$
${\bf I}_{{\rm ssEYM}}^{\epsilon,\widetilde{\epsilon}}(\overline{\boldsymbol{\sigma}}_{m};\boldsymbol{H}_{n-m})$ | $\mathbb{I}$ | ${\cal T}^{\widetilde{\epsilon}}_{\circ}[\overline{\boldsymbol{\sigma}}_{m}]$
${\bf I}_{{\rm EMf}}^{\epsilon,\widetilde{\epsilon}}(\boldsymbol{P}_{2m};\boldsymbol{H}_{n-2m})$ | $\mathbb{I}$ | ${\cal T}^{\widetilde{\epsilon}}_{{\cal X}_{2m}}(N\,\widetilde{\cal D}+1)$
${\bf I}_{{\rm EM}}^{\epsilon,\widetilde{\epsilon}}(\boldsymbol{P}_{2m};\boldsymbol{H}_{n-2m})$ | $\mathbb{I}$ | ${\cal T}^{\widetilde{\epsilon}}_{X_{2m}}(\widetilde{\cal D}+1)$
${\bf I}_{{\rm BI}}^{\epsilon}(\boldsymbol{P}_{n})$ | $\mathbb{I}$ | ${\cal L}^{\widetilde{\epsilon}}_{\circ}\,\widetilde{\cal D}$
${\bf I}_{{\rm YM}}^{\epsilon}(\overline{\boldsymbol{\sigma}}_{n})$ | $\mathbb{I}$ | ${\cal T}^{\widetilde{\epsilon}}_{\circ}[\overline{\boldsymbol{\sigma}}_{n}]$
${\bf I}_{{\rm ssYMS}}^{\widetilde{\epsilon}}(\overline{\boldsymbol{\sigma}}_{m};\boldsymbol{G}_{n-m}|\overline{\boldsymbol{\sigma}^{\prime}}_{n})$ | ${\cal T}^{\epsilon}_{\circ}[\overline{\boldsymbol{\sigma}^{\prime}}_{n}]$ | ${\cal T}^{\widetilde{\epsilon}}_{\circ}[\overline{\boldsymbol{\sigma}}_{m}]$
${\bf I}_{{\rm NLSM}}(\overline{\boldsymbol{\sigma}^{\prime}}_{n})$ | ${\cal T}^{\epsilon}_{\circ}[\overline{\boldsymbol{\sigma}^{\prime}}_{n}]$ | ${\cal L}^{\widetilde{\epsilon}}_{\circ}\,\widetilde{\cal D}$
${\bf I}_{{\rm BAS}}(\overline{\boldsymbol{\sigma}}_{n}|\overline{\boldsymbol{\sigma}^{\prime}}_{n})$ | ${\cal T}^{\epsilon}_{\circ}[\overline{\boldsymbol{\sigma}^{\prime}}_{n}]$ | ${\cal T}^{\widetilde{\epsilon}}_{\circ}[\overline{\boldsymbol{\sigma}}_{n}]$
${\bf I}_{{\rm DBI}}^{\widetilde{\epsilon}}(\boldsymbol{S}_{2m};\boldsymbol{P}_{n-2m})$ | ${\cal L}^{\epsilon}_{\circ}\,{\cal D}$ | ${\cal T}^{\widetilde{\epsilon}}_{{\cal X}_{2m}}(N\,\widetilde{\cal D}+1)$
${\bf I}_{{\rm SG}}(\boldsymbol{S}_{n})$ | ${\cal L}^{\epsilon}_{\circ}\,{\cal D}$ | ${\cal L}^{\widetilde{\epsilon}}_{\circ}\,\widetilde{\cal D}$
Table 4: Unifying relations at $1$-loop level.

In this paper, we have constructed the $1$-loop level differential operators
which transmute the $1$-loop GR Feynman integrand to the Feynman integrands of
various theories, include Einstein-Yang-Mills theory, Einstein-Maxwell theory,
pure Yang-Mills theory, Yang-Mills-scalar theory, Born-Infeld theory, Dirac-
Born-Infeld theory, bi-adjoint scalar theory, non-linear sigma model, as well
as special Galileon theory. Similar as the tree level formula (43), the
$1$-loop level relations can be summarized as

$\displaystyle{\bf I}={\cal O}^{\epsilon}_{\circ}\,{\cal
O}^{\widetilde{\epsilon}}_{\circ}\,{\bf
I}^{\epsilon,\widetilde{\epsilon}}_{\rm GR}\,.$ (187)

Operators ${\cal O}^{\epsilon}_{\circ}$ and ${\cal
O}^{\widetilde{\epsilon}}_{\circ}$ for different theories are listed in
Table.4. Comparing Table.4 with Table.3, we see that the $1$-loop level
relations are paralleled to the tree level relations. Based on the relations
in Table.4, one can construct the unified web for these theories, as shown in
Fig.3. In this web, different kinds of lines represents different operators.
Along a line from theory $A$ to theory $B$, one can generate the $1$-loop
Feynman integrand ${\bf I}_{B}$, by applying the corresponding operator to
${\bf I}_{A}$. For example, there is a double straight line from EMf to DBI,
thus we have the relation

$\displaystyle{\bf I}^{\widetilde{\epsilon}}_{\rm
DBI}({\boldsymbol{S}}_{2m};{\boldsymbol{P}}_{n-2m})={\cal
L}^{\epsilon}_{\circ}\,{\cal D}\,{\bf I}^{\epsilon,\widetilde{\epsilon}}_{\rm
EMf}({\boldsymbol{P}}_{2m};{\boldsymbol{H}}_{n-2m})\,.$ (188)

We also discussed the factorization of differential operators under the well
known unitarity cut. Our result shows that the $1$-loop level operator
factorizes into two tree level pieces, paralleled to the factorization of
amplitudes. As a by product, we also found the mechanism of the factorization
of tree level operators. These properties propose an alternative way to verify
the operators: applying the tree level operator to the lowest points tree
level amplitudes. A natural conjecture is, maybe one can construct a general
operator recursively, from the lowest points operator, similar as the well
known on-shell recursion relation for constructing amplitudes.

For the EYM theory, in this paper we only consider the single trace Feynman
integrands with a virtual gluon running in the loop, and similar does the YMS
theory. We have not discussed the more general case, due to some technical
difficulties those we do not know how to overcome. The general multiple traces
case with arbitrary virtual particle in the loop is left to be the future
direction.

The basic idea used in this paper is to find the operator ${\cal O}_{\circ}$
satisfying ${\cal O}_{\circ}\,{\cal F}\,\bullet={\cal F}\,{\cal O}\,\bullet$,
where ${\cal O}$ is the already known tree level operator. This idea based on
the validity of the forward limit method. However, as mentioned in section 1,
the forward limit method is not the only candidate for generating the $1$-loop
CHY formula. Thus, how to reveal relations in Table.4 and Fig.3 in the
ambitwistor string framework is an interesting question. The Feynman
integrands provided by the current $1$-loop CHY formula in the sense of ${\cal
Q}$-cut Baadsgaard:2015twa ; Huang:2015cwh . They include propagators linear
in $\ell$ rather than quadratic, and are related to the standard ones via the
partial fraction identity. Until now we do not have a systematical way to
construct the CHY formula which leads to Feynman integrands with propagators
quadratic in $\ell$. Searching the differential operators link Feynman
integrands with quadratic propagators together, via a path which is
independent of the CHY formula, is another interesting challenge.

Figure 3: Unified web for $1$-loop Feynman integrands. The bold straight line
represents the operator ${\cal
T}_{\circ}[\overline{\boldsymbol{\sigma}}_{n}]$, the straight line represents
the operator ${\cal T}_{\circ}[\overline{\boldsymbol{\sigma}}_{m}]$, the
double straight line represents the operator ${\cal L}\,{\cal D}$, the dashed
line represents the operator ${\cal T}_{{\cal X}_{2m}}(N\,{\cal D}+1)$, the
double dashed line represents the operator ${\cal T}_{X_{2m}}({\cal D}+1)$,
the thin dashed line represents the insertion operators.

## Acknowledgments

The author would thank Prof. Bo Feng, Xiaodi-Li, Chang Hu, Yaobo Zhang,
Tingfei Li, especially Prof. Bo Feng and Xiaod- Li, for helpful discussions
and valuable suggestions. This work is supported by Chinese NSF funding under
contracts No.11805163, as well as NSF of Jiangsu Province under Grant
No.BK20180897.

## References

  * (1) H. Kawai, D. C. Lewellen and S. H. H. Tye, “A Relation Between Tree Amplitudes of Closed and Open Strings,” Nucl. Phys. B269 (1986) 1.
  * (2) Z. Bern, J. J. M. Carrasco and H. Johansson, “New Relations for Gauge-Theory Amplitudes,” Phys. Rev. D 78, 085011 (2008) doi:10.1103/PhysRevD.78.085011 [arXiv:0805.3993 [hep-ph]].
  * (3) Z. Bern, J. J. M. Carrasco and H. Johansson, “Perturbative Quantum Gravity as a Double Copy of Gauge Theory,” Phys. Rev. Lett. 105, 061602 (2010) doi:10.1103/PhysRevLett.105.061602 [arXiv:1004.0476 [hep-th]].
  * (4) Z. Bern, T. Dennen, Y. t. Huang and M. Kiermaier, “Gravity as the Square of Gauge Theory,” Phys. Rev. D 82, 065003 (2010) doi:10.1103/PhysRevD.82.065003 [arXiv:1004.0693 [hep-th]].
  * (5) F. Cachazo, S. He and E. Y. Yuan, “Scattering equations and Kawai-Lewellen-Tye orthogonality,” Phys. Rev. D 90, no. 6, 065001 (2014) doi:10.1103/PhysRevD.90.065001 [arXiv:1306.6575 [hep-th]].
  * (6) F. Cachazo, S. He and E. Y. Yuan, “Scattering of Massless Particles in Arbitrary Dimensions,” Phys. Rev. Lett. 113, no. 17, 171601 (2014) doi:10.1103/PhysRevLett.113.171601 [arXiv:1307.2199 [hep-th]].
  * (7) F. Cachazo, S. He and E. Y. Yuan, “Scattering of Massless Particles: Scalars, Gluons and Gravitons,” JHEP 1407, 033 (2014) doi:10.1007/JHEP07(2014)033 [arXiv:1309.0885 [hep-th]].
  * (8) F. Cachazo, S. He and E. Y. Yuan, “Einstein-Yang-Mills Scattering Amplitudes From Scattering Equations,” JHEP 1501, 121 (2015) doi:10.1007/JHEP01(2015)121 [arXiv:1409.8256 [hep-th]].
  * (9) F. Cachazo, S. He and E. Y. Yuan, “Scattering Equations and Matrices: From Einstein To Yang-Mills, DBI and NLSM,” JHEP 1507, 149 (2015) doi:10.1007/JHEP07(2015)149 [arXiv:1412.3479 [hep-th]].
  * (10) C. Cheung, C. H. Shen and C. Wen, “Unifying Relations for Scattering Amplitudes,” JHEP 1802, 095 (2018) doi:10.1007/JHEP02(2018)095 [arXiv:1705.03025 [hep-th]].
  * (11) K. Zhou and B. Feng, “Note on differential operators, CHY integrands, and unifying relations for amplitudes,” JHEP 1809, 160 (2018) [arXiv:1808.06835 [hep-th]].
  * (12) M. Bollmann and L. Ferro, “Transmuting CHY formulae,” JHEP 1901, 180 (2019) [arXiv:1808.07451 [hep-th]].
  * (13) K. Zhou and G. J. Zhou, “Transmuting off-shell CHY integrals in the double-cover framework,” Eur. Phys. J. C 80, no.11, 1068 (2020) doi:10.1140/epjc/s10052-020-08624-1 [arXiv:2006.12188 [hep-th]].
  * (14) T. Adamo, “Worldsheet factorization for twistor-strings,” JHEP 04, 080 (2014) doi:10.1007/JHEP04(2014)080 [arXiv:1310.8602 [hep-th]].
  * (15) L. Mason and D. Skinner, “Ambitwistor strings and the scattering equations,” JHEP 07, 048 (2014) doi:10.1007/JHEP07(2014)048 [arXiv:1311.2564 [hep-th]].
  * (16) T. Adamo, E. Casali and D. Skinner, “Ambitwistor strings and the scattering equations at one loop,” JHEP 04, 104 (2014) doi:10.1007/JHEP04(2014)104 [arXiv:1312.3828 [hep-th]].
  * (17) E. Casali and P. Tourkine, “Infrared behaviour of the one-loop scattering equations and supergravity integrands,” JHEP 04, 013 (2015) doi:10.1007/JHEP04(2015)013 [arXiv:1412.3787 [hep-th]].
  * (18) Y. Geyer, L. Mason, R. Monteiro and P. Tourkine, “Loop Integrands for Scattering Amplitudes from the Riemann Sphere,” Phys. Rev. Lett. 115, no.12, 121603 (2015) doi:10.1103/PhysRevLett.115.121603 [arXiv:1507.00321 [hep-th]].
  * (19) Y. Geyer, L. Mason, R. Monteiro and P. Tourkine, “One-loop amplitudes on the Riemann sphere,” JHEP 03, 114 (2016) doi:10.1007/JHEP03(2016)114 [arXiv:1511.06315 [hep-th]].
  * (20) Y. Geyer and R. Monteiro, “Gluons and gravitons at one loop from ambitwistor strings,” JHEP 03, 068 (2018) doi:10.1007/JHEP03(2018)068 [arXiv:1711.09923 [hep-th]].
  * (21) T. Adamo and E. Casali, “Scattering equations, supergravity integrands, and pure spinors,” JHEP 05, 120 (2015) doi:10.1007/JHEP05(2015)120 [arXiv:1502.06826 [hep-th]].
  * (22) S. He and E. Y. Yuan, “One-loop Scattering Equations and Amplitudes from Forward Limit,” Phys. Rev. D 92, no.10, 105004 (2015) doi:10.1103/PhysRevD.92.105004 [arXiv:1508.06027 [hep-th]].
  * (23) F. Cachazo, S. He and E. Y. Yuan, “One-Loop Corrections from Higher Dimensional Tree Amplitudes,” JHEP 08, 008 (2016) doi:10.1007/JHEP08(2016)008 [arXiv:1512.05001 [hep-th]].
  * (24) B. Feng, “CHY-construction of Planar Loop Integrands of Cubic Scalar Theory,” JHEP 05, 061 (2016) doi:10.1007/JHEP05(2016)061 [arXiv:1601.05864 [hep-th]].
  * (25) B. Feng and C. Hu, “One-loop CHY-Integrand of Bi-adjoint Scalar Theory,” JHEP 02, 187 (2020) doi:10.1007/JHEP02(2020)187 [arXiv:1912.12960 [hep-th]].
  * (26) C. S. Lam and Y. P. Yao, “Evaluation of the Cachazo-He-Yuan gauge amplitude,” Phys. Rev. D 93, no.10, 105008 (2016) doi:10.1103/PhysRevD.93.105008 [arXiv:1602.06419 [hep-th]].
  * (27) K. A. Roehrig and D. Skinner, JHEP 01, 069 (2018) doi:10.1007/JHEP01(2018)069 [arXiv:1709.03262 [hep-th]].
  * (28) K. Zhou, “Factorizations for tree amplitudes in the double-cover framework: from gravity to other theories,” JHEP 07, 008 (2020) doi:10.1007/JHEP07(2020)008 [arXiv:2003.12528 [hep-th]].
  * (29) C. Baadsgaard, N. E. J. Bjerrum-Bohr, J. L. Bourjaily, S. Caron-Huot, P. H. Damgaard and B. Feng, “New Representations of the Perturbative S-Matrix,” Phys. Rev. Lett. 116, no.6, 061601 (2016) doi:10.1103/PhysRevLett.116.061601 [arXiv:1509.02169 [hep-th]].
  * (30) R. Huang, Q. Jin, J. Rao, K. Zhou and B. Feng, “The Q-cut Representation of One-loop Integrands and Unitarity Cut Method,” JHEP 03, 057 (2016) doi:10.1007/JHEP03(2016)057 [arXiv:1512.02860 [hep-th]].