# EVD4UAV: An Altitude-Sensitive Benchmark to Evade Vehicle Detection in UAV

Huiming Sun, Jiacheng Guo, Zibo Meng, Tianyun Zhang, Jianwu Fang, Yuewei Lin,
Hongkai Yu∗ Huiming Sun, Jiacheng Guo, Tianyun Zhang and Hongkai Yu are with
Cleveland State University, Cleveland, OH, 44115, USA. Zibo Meng are with OPPO
US Research Center, Palo Alto, CA, 94303, USA. Jianwu Fang is with Xi’an
Jiaotong University, Xi’an, 710049, China. Yuewei Lin is with Brookhaven
National Laboratory, Upton, NY, 11973, USA. ∗Corresponding author: Hongkai Yu
<EMAIL_ADDRESS>
###### Abstract

Vehicle detection in Unmanned Aerial Vehicle (UAV) captured images has wide
applications in aerial photography and remote sensing. There are many public
benchmark datasets proposed for the vehicle detection and tracking in UAV
images. Recent studies show that adding an adversarial patch on objects can
fool the well-trained deep neural networks based object detectors, posing
security concerns to the downstream tasks. However, the current public UAV
datasets might ignore the diverse altitudes, vehicle attributes, fine-grained
instance-level annotation in mostly side view with blurred vehicle roof, so
none of them is good to study the adversarial patch based vehicle detection
attack problem. In this paper, we propose a new dataset named EVD4UAV as an
altitude-sensitive benchmark to evade vehicle detection in UAV with 6,284
images and 90,886 fine-grained annotated vehicles. The EVD4UAV dataset has
diverse altitudes (50m, 70m, 90m), vehicle attributes (color, type), fine-
grained annotation (horizontal and rotated bounding boxes, instance-level
mask) in top view with clear vehicle roof. One white-box and two black-box
patch based attack methods are implemented to attack three classic deep neural
networks based object detectors on EVD4UAV. The experimental results show that
these representative attack methods could not achieve the robust altitude-
insensitive attack performance.

##  I Introduction

Unmanned Aerial Vehicles (UAVs) have been widely used in aerial photography,
intelligent transportation, and remote sensing [1] [2]. UAV captured images
include rich visualized information on the road surfaces. Many public
benchmark datasets are proposed for the vehicle detection and tracking in UAV
images, such as UAVDT [3], VeRi [4], UAVid [5] and UAVW [6].

Based on the recent research of adversarial machine learning [7], the deep
neural networks based object detectors are found not robust to some
adversarial perturbations. Among them, the physical adversarial patch based
detection attack arises severe security concerns to the downstream tasks. The
physical adversarial patches can be printed [7] to put on the object to
evade/fool the well-trained deep neural networks based object detectors.

Figure 1: Example vehicle detection before and after the adversarial attack in
the altitudes of 50m, 90m in the proposed altitude-sensitive EVD4UAV dataset.
Due to the space limit, the altitude of 70m is skipped. TABLE I: Comparison of
our EVD4UAV dataset and existing representative UAV datasets. HBB: object
annotation with Horizontal Bounding Boxes. RBB: object annotation with Rotated
Bounding Boxes. iMask: object annotation with instance-level mask.

Dataset | Purpose |  | Images
---
/ FPS
Altitude (m) |  | Altitude
---
Label
| Vehicle
---
Attributes
| Resolution
---
| HBB
---
| RBB
---
| iMask
---
View |  | Vehicle
---
Roof
UAVDT [3] |  | Detect & Track
---
80,000 / 30 | Low/Medium/High | × | Type | $1080\times 540$ | ✓ | × | × | Mostly side | Blurred
VeRi [4] |  | Detect & Track
---
1,685 / 30 | N/A | × | × | $2720\times 1530$ | ✓ | × | × | Mostly side | Blurred
UAVid [5] |  | Semantic Segment
---
300 / 0.2 | 50 | × | × |  | $4096\times 2160$
---
× | × | × | Mostly side | Blurred
UAVW [6] | Detect | 46,037 / 30 |  | 50/60/70/80/90/100
---
✓ | × | $3840\times 2160$ | ✓ | × | × | Side | Blurred
EVD4UAV | Detect Attack | 6,284 / 1 | 50/70/90 | ✓ | Color/Type | $1920\times 1080$ | ✓ | ✓ | ✓ | Top | Clear

However, all the existing public UAV datasets are not suitable to investigate
the problem of evading vehicle detection in UAV images, because they are for
vehicle detection and tracking rather than patch-based detection attack. For
the task of patch-based vehicle detection attack, altitude label is important
because UAV flying height affects the object scale/size significantly, and
keeping top-view images with clear vehicle roof is better for this study
because the printed patch is more convenient to be put on the top of vehicles.
The existing public UAV datasets might ignore the diverse altitudes, in mostly
side view with the blurred vehicle roof. In addition, their annotations miss
many details, _i.e_ let@tokeneonedot, vehicle attributes, and fine-grained
instance-level annotation.

In this paper, we propose a new dataset named EVD4UAV as an altitude-sensitive
benchmark to Evade Vehicle Detection in UAV with 6,284 images and 90,886 fine-
grained annotated vehicles. The comparison between the proposed EVD4UAV and
existing representative UAV datasets is shown in Table I. Our EVD4UAV dataset
has diverse altitudes (50m, 70m, 90m), vehicle attributes (color, type), fine-
grained annotation (horizontal and rotated bounding boxes, instance-level
mask) in top view with clear vehicle roof. One white-box and two black-box
patch-based attack methods are implemented to attack three classic DNN based
object detectors on EVD4UAV. As shown in Fig. 1, the vehicles wearing the
learned adversarial patch cannot be detected by the classic object detectors,
like YOLOv8 [8], Faster R-CNN [9]. Our extensive experimental results show
that the representative attack methods could not achieve robust altitude-
insensitive attack performance, so it is necessary to continue the research
direction of altitude-insensitive attack using our EVD4UAV. The contribution
of this paper can be summarized as follows.

  * •

A new dataset with fine-grained annotations named EVD4UAV is proposed as an
altitude-sensitive benchmark to evade vehicle detection in UAV.

  * •

Using the proposed EVD4UAV, this paper investigates representative attacks
(one white-box and two black-box methods) to test the detection evasion
performance in UAV.

  * •

Using the proposed EVD4UAV for extensive experiments, this paper points out
that the altitude-insensitive detection attack in UAV is worth further
investigation.

##  II Related Work

###  II-A UAV Object Detection

Aiming at improving clarity with reasonable accuracy and higher surveying
efficiency, Unmanned Aerial Vehicles (UAV) has become an advanced way to deal
with object detection tasks. UAVs can be categorized into two classes: fixed-
wing UAVs and rotary-wing UAVs [10]. A typical usage of UAV is UAV-aided
information dissemination and data collection by wireless communication
technologies [11]. Those UAVs have sustainable speed when flying across large
certain areas in a short time, which could collect and process data in high
efficiency [11] [10]. However, it is pointed out that UAV object detection may
suffer from hostile natural environments, camera issues, and illumination
issue challenges in actual application operation [12] [13] [14] [15] [16] [17]
[18]. In the last decade, many researchers have provided insights into these
challenges. In the first stage, object detection is correlated with Deep
Neural Networks (DNNs) in [19], which obtains high-resolution object detection
images at a low cost. Later, a benchmark was established in [8], where the
proposed “You Only Look Once” (YOLO) structure is sustainable for large-scale
single-shot object detectors. The bounding box, class score and object score
could be directly predicted by only a single pass going through the structure
[20] [8]. Moreover, an end-to-end object relation detector is built in [21]
which further improves the detecting accuracy by duplicating removal steps.
Recently, a novel object detection framework has been proposed by [22]. The
framework is designed based on autonomous driving under foggy weather, which
builds an adversarial gradient reversal layer to perform better among hard
examples together with data augmentation and a new domain-level metric
regularization.

Overall, the accuracy of object detection has been improving in recent years
using DNN models. However, most existing UAV image datasets are designed for
object detection and constructing datasets for adversarial attacks remains
rare. This paper aims to propose a new benchmark to attack the object
detectors using images taken by UAV.

###  II-B Adversarial Attack in Object Detection

Adversarial attacks could be divided into two categories: digital attack and
physical attack [23]. Digital attack means the attack input is generated by
digital attack images; physical attack means the attack images which are taken
as the attack input are retaken by a camera [23]. The first application is
proposed by [24] named Dense Adversary Generation (DAG) algorithm which is
based on optimization methods. DAG established adversarial labels in each
proposal region and then misclassified the proposals using iterative gradient
back-propagation. However, DAG is time-consuming and needs many iterations to
go through an adversarial image. Later, the adversarial attack was applied on
face detectors in [25], but the model still relies on proposal-based
detectors. Besides adding noise among the whole image, a number of methods
only modify part of regions of the image to attack object detectors [26] [27].
However, most of these methods give little improvement. A more enhanced
generated model is proposed by [28] whose detectors with adversarial examples
could be used in multiple scenes in the real world with higher robustness and
transferability. The first adversarial patches generated for attacking intra-
class variety was established in [7], taking persons as the specific class. By
generating patches, they propose adversarial attacks with much higher
accuracy. Moreover, [23] generated adversarial patches with are more
naturalistic looking and less doubtful. While ensuring higher attack accuracy,
the adversarial patches are more hidden and less noticeable.

In general, the current public UAV datasets may successfully ignore the
diverse altitudes, vehicle attributes, and fine-grained instance-level
annotation. However, these datasets might not be effectively used for
adversarial attack tasks with mostly side view and blurred vehicle roofs.
Therefore, none of them is good enough to study the adversarial patch-based
vehicle detection attack problem. In this paper, we propose a new dataset
named EVD4UAV as an altitude-sensitive benchmark with top view and clear roof
to evade vehicle detection in UAV.

##  III Benchmark

###  III-A Data Collection

All the images in our EVD4UAV dataset are taken from UAV cameras. The images
are all from a top-view overhead perspective taken from three different
altitudes: 50m, 70m, and 90m above the ground. Each image contains several
moving and stable vehicles and each vehicle is labeled with fine-grained
annotation. The UAV images were taken in California and Ohio, USA. We use DJI
Mini3 Pro to capture sparsely sampled videos (FPS=1) and some discrete single
images with a camera degree of nearly vertical view angle. We focus on the
object detection task and categorize the images into four different scenes
based on the location of vehicles: Urban Road, Highway, Parking Lot, and
Residential Area, reflecting the diversity of vehicle activities.

Specifically, we have a total of 6,284 images of $1,920\times 1,080$
resolution with a total number of 90,886 fine-grained annotated vehicles. The
total number of images and annotated vehicles from different altitudes is
shown in Fig. 2. 1,789 images are taken from altitude 50m above the ground,
and 2,227 and 2,268 images are taken from 70m and 90m respectively. The number
of fine-grained annotated vehicles of 50m and 70m are around 20,000 each, and
the images taken from altitude 90m has more than 50,000 due to a larger
perception field.

Figure 2: Total number of (a) images and (b) fine-grained annotated vehicles
of our EVD4UAV in different UAV altitudes (_i.e_ let@tokeneonedot, 50m, 70m,
90m). Figure 3: Example vehicle images in EVD4UAV with labeled horizontal
bounding boxes, rotated bounding boxes, and instance-level segmentation masks
from top to bottom. Figure 4: Proportions of (a) images in four scenes, (b)
annotated vehicle color attribute, (c) annotated vehicle type attribute in
EVD4UAV.

We labeled horizontal and rotated bounding boxes for each vehicle in the
images in the proposed EVD4UAV. Besides, we accomplish an instance-level
segmentation task for each vehicle as shown in Fig. 3. The first row shows the
vehicles labeled with horizontal bounding boxes. The second and third row
shows the vehicles labeled with rotated bounding boxes and instance-level
segmentation masks respectively. Vehicles under various scenes can be seen in
different columns.

###  III-B Data Analysis

Figure 5: Sensitive vehicle scale/size in various UAV altitudes: Gaussian
distributions of the vehicle areas (pixels) of EVD4UAV in three different
altitudes.

We categorized the images into four scenes based on where the vehicles are: on
the Urban Roads, on the Highways, in the Parking Lots, and in the Residential
Areas, with the proportion of each scene shown in Fig. 4(a). About half of the
vehicles are mainly on urban roads. The proportions of the remaining three
categories are relatively even, at around 13% to 17%.

Our EVD4UAV dataset also has a diversity of vehicles under various weather
conditions and brightness. The proportions of images taken from various
weather conditions. Most of the images are taken in cloudy weather, mostly
cloudy days in Ohio. One-fifth of the images are taken on sunny days and about
10% of the images are taken at dusk. Leveraging images under different weather
conditions and brightness conditions helps build more robust adversarial
attack models in various situations.

The Fig. 4(b) and Fig. 4(c) illustrate the annotated vehicle attributes in
colors and types in our EVD4UAV dataset respectively. As shown in Fig. 4(b),
there are seven common colors of vehicles, and the largest two proportions are
Black and White colors. As the types illustrated in Fig. 4(c), most of the
vehicles contained in the images belong to Cars, with nearly a quarter of the
vehicles belonging to Buses and a small percentage of Trucks.

Based on the instance-level segmentation masks of all vehicles in EVD4UAV, we
learned the Gaussian distribution of the vehicle areas captured in the images
taken from three different altitudes, as shown in Fig. 5. The $x$-axis
represents the number of pixels of the vehicles occupied in the images. The
images taken from the altitude of 90m has the smallest mean value $\mu$ (about
3,134 pixels) and smallest standard deviation (about 102 pixels) as the curve
shown on the left. It means that the vehicles occupy the smallest area at the
altitude of 90m and the area variance is small. The images taken at the
altitude of 50m have the largest mean value (about 8,142 pixels) as the curve
shown on the right, which means the vehicles occupied much larger area in the
images since the camera height is closer to the ground. The image with a
height of 70m is tilted to the left in the middle. The average number of
pixels occupied by vehicles in each image with a height of 70m is 4,512. The
standard deviations at the heights of 50m and 70m are also not much, which are
136 and 141 pixels respectively. The learned Gaussian distributions shown in
Fig. 5 illustrate that the vehicle scale/size is sensitive in various UAV
flying altitudes.

Figure 6: Schematic representation of adversarial patch generation and
application processes for white-box and black-box attacks on UAV vehicle
detectors, highlighting the path from patch creation to detector evasion and
classification discrepancy.

##  IV Methodology

This paper aims to assess the effectiveness of adversarial patches in evading
vehicle detection from various altitudes using UAVs. We focus on both white-
box and black-box attack scenarios to understand the robustness of current UAV
detection models under adversarial conditions. The pipeline of our designed
representative white-box and black-box attack methods are shown in Fig. 6.

###  IV-A Adversarial Patch Generation

The adversarial patch is initialized as a tensor $P$ with the dimension
$\mathbb{R}^{H\times W\times 3}$. Our optimization goal is to learn a unified
adversarial patch adding to any vehicle tops so as to evade the vehicle
detection in UAV, _i.e_ let@tokeneonedot, one learned adversarial patch for
the whole dataset. We constrain that the patch size is less than 40% of the
mean value of the Gaussian distributions of the vehicle areas in corresponding
altitudes. During training, the unified patch is added to the center of each
vehicle top on the original UAV image, and then the manipulated UAV image is
sent to the object detection model. The patch $P$ is then learned through a
training procedure using Stochastic Gradient Descent (SGD) as the optimizer.
The patch updating process during optimization is defined as:

$P^{i+1}=P^{i}-\eta\cdot\nabla_{P}\mathcal{L}_{a},$ (1)

where $P^{i}$ is the learned adversarial patch with the batch identity $i$,
$\eta$ denotes the learning rate, and $\nabla_{P}$ denotes the gradient of the
corresponding attack loss function $\mathcal{L}_{a}$ with respect to the $P$.
The details of the attack loss functions $\mathcal{L}_{a}$ in different
settings will be described in the following.

###  IV-B White-Box Patch (WBP) Attack

This part focuses on developing white-box patch based attacks targeting at
classic object detectors in UAV images. Similar to [7], the goal of optimizing
the adversarial patch is to minimize the inverse of the corresponding object
class loss by directly feeding the original UAV image plus the added patches
on each vehicle into the corresponding detectors. In this paper, we studied
three classic object detectors, _i.e_ let@tokeneonedot, Faster R-CNN [9], DETR
[29],YOLOv8 [8]. Because of their different architectures, we define different
attack loss functions $\mathcal{L}_{a}$ respectively.

$\text{WBP}_{\text}{RCNN}$: If the target network is Faster R-CNN [9], we
define the attack loss function as:

$\mathcal{L}_{a}=\frac{1}{N}\sum_{i=1}^{N}s_{i},$ (2)

where $s_{i}$ is the score/confidence of the $i$-th region proposal, and $N$
represents the number of region proposals generated by the Region Proposal
Network (RPN) in Faster R-CNN prediction. Minimizing this loss function will
reduce the score/confidence of all the generated object proposals by RPN in
Faster R-CNN.

$\text{WBP}_{\text}{DETR}$: If the target network is DETR [29], we define the
attack loss function as:

$\mathcal{L}_{a}=-1\cdot\sum_{i=1}^{m}\sum_{j=1}^{n}-y_{j}\log(s_{ij}),$ (3)

where the right part is the normal cross-entropy loss for the correct
classification of the predicted bounding boxes of multiple feature stages, $m$
represents the number of feature stages, $n$ represents the number of
predicted bounding boxes, $y$ represents the object class label, $s$
represents the predicted score/confidence. Minimizing this loss function will
confuse the correct classification of the predicted bounding boxes in DETR.

$\text{WBP}_{\text}{YOLOv8}$: If the target network is YOLOv8 [8], we define
the attack loss function as:

$\mathcal{L}_{a}=-1\cdot\frac{-\sum_{i=1}^{n}\left[y_{i}\cdot\log(s_{i})+(1-y_{i})\cdot\log(1-s_{i})\right]}{N_{t}}$
(4)

where the right part is the normal binary cross-entropy loss for the correct
classification of predicted bounding boxes, $n$ represents the number of
predicted bounding boxes, $y$ represents the object class label, $s$
represents the predicted object score/confidence, and $N_{t}$ represents the
normalization term defined in YOLOv8. Minimizing this loss function will
improve the misclassification of the predicted bounding boxes in YOLOv8.

In summary, the optimization procedure of WBP methods to discover the
adversarial patch is denoted as below: arg min_PL_a(δ(ϕ(I_clean,P)), y), where
$P$ represents the adversarial patch to be learned, $\mathcal{L}_{a}$
represents the attack loss function, $\delta$ represents the targeting object
detector, $\phi$ represents the patch adding process, $I_{clean}$ represents
the original clean image, and $y$ represents the detection ground truth. Our
$\text{WBP}$ method requires the targeting object detection model to compute
gradients for optimization, so it is a white-box attack.

Figure 7:  The detect bounding boxes and confidence results predicted by
different models. The $\text{BBP}_{\text}{CLIP}$ and $\text{BBP}_{\text}{D}$
are detected by YOLOv8 detector. Other attack methods are targeting their own
detectors.

###  IV-C Black-Box Patch (BBP) Attack

In this section, we introduce two BBP attack methods, _i.e_ let@tokeneonedot,
Language-Image Model based BBP called $\text{BBP}_{\text}{CLIP}$ and
Distribution based BBP called $\text{BBP}_{\text}{D}$.

####  IV-C1 Language-Image Model based BBP

We employ a Language-Image contrastive learning framework, _i.e_
let@tokeneonedot, the state-of-the-art CLIP model [30]. CLIP (Contrastive
Language-Image Pretraining) utilizes a multimodal approach to understand
images in the context of natural language descriptions. In our framework,
CLIP’s visual image encoder, $\text{CLIP}_{\text{VE}}$, processes the cropped
vehicle image Region of Interests (RoIs), while its text encoder,
$\text{CLIP}_{\text{TE}}$, encodes the textual descriptions corresponding to
object classes. Specifically, the cropped vehicle image RoIs added with the
adversarial patches, denoted as $\mathbf{R_{adv}}$, are passed through the
CLIP image encoder. Let us define a text [‘vehicle’, ‘confusion class’] as
$\mathbf{T}$, then we use the CLIP encoders to extract the multimodal
features:

$F_{img}^{adv}=\text{CLIP}_{\text{VE}}(\mathbf{R_{adv}}),$ (5)
$F_{text}=\text{CLIP}_{\text{TE}}(\mathbf{T}).$ (6)

Score/confidence Calculation: The score/confidence of each vehicle image RoI
with the adversarial patch belonging to the respective text classes in
$\mathbf{T}$ can be calculated via contrastive learning as:

$s_{\mathbf{T}}=\mathbf{softmax}(\frac{F_{img}^{adv}}{\|F_{img}^{adv}\|_{2}}\cdot\frac{F_{text}^{T}}{\|F_{text}\|_{2}}).$
(7)

Attack Loss Function: For the attack purpose, our optimization aims to reduce
the score/confidence of the ‘vehicle’ class (making it close to 0) and
increase the score of the ‘confusion class’ class (making it close to 1). This
goal could be achieved by minimizing the following Mean Square Error (MSE)
loss: L_a = 1K ∑_k=1^K (s_T \- [0,1])^2, where $K$ is the number of total
cropped vehicle image RoIs in the training set. The optimization process is
denoted: arg min_PL_a(s_T).

Minimizing this loss function serves to reduce the likelihood of correct class
(‘vehicle’) and increase its probability to be the ‘confusion class’. In our
experiment, we use ‘boat’ as the ‘confusion class’.

TABLE II: The mAP50 results (before attack) for different Object Detection models on EVD4UAV dataset. Model | 50m | 70m | 90m | Mean
---|---|---|---|---
Faster R-CNN [9] | 0.9503 | 0.9473 | 0.9573 | 0.9552
DETR [29] | 0.9524 | 0.9314 | 0.9145 | 0.9263
YOLOv8 [8] | 0.9824 | 0.9893 | 0.9912 | 0.9880
TABLE III: Detection results (mAP50) of EVD4UAV across various altitudes, using different adversarial patches by different detectors. Attack |  Faster R-CNN [9] |  DETR [29] |  YOLOv8 [8]
---|---|---|---
Methods | 50m | 70m | 90m | Mean | 50m | 70m | 90m | Mean | 50m | 70m | 90m | Mean
Naive Cover | 0.9119 | 0.9132 | 0.9044 | 0.9098 | 0.9443 | 0.9226 | 0.9098 | 0.9256 | 0.9700 | 0.9812 | 0.9878 | 0.9797
Random Noise | 0.9296 | 0.9366 | 0.9374 | 0.9345 | 0.9382 | 0.9169 | 0.9007 | 0.9186 | 0.9526 | 0.9578 | 0.9799 | 0.9634
$\text{WBP}_{\text}{RCNN}$ | 0.1746 | 0.2895 | 0.5110 | 0.3250 | 0.8873 | 0.8541 | 0.8662 | 0.8692 | 0.9557 | 0.9422 | 0.9759 | 0.9579
$\text{WBP}_{\text}{DETR}$ | 0.7368 | 0.8560 | 0.8638 | 0.8189 | 0.5069 | 0.7394 | 0.8749 | 0.7071 | 0.9174 | 0.8844 | 0.9705 | 0.9241
$\text{WBP}_{\text}{YOLOv8}$ | 0.5114 | 0.6846 | 0.9239 | 0.7066 | 0.3127 | 0.5386 | 0.8623 | 0.5712 | 0.3367 | 0.1004 | 0.7453 | 0.3941
$\text{BBP}_{\text}{CLIP}$ | 0.9296 | 0.9135 | 0.9164 | 0.9198 | 0.9111 | 0.8112 | 0.8820 | 0.8681 | 0.9401 | 0.8300 | 0.9718 | 0.9139
$\text{BBP}_{\text}{D}$ | 0.9289 | 0.9458 | 0.9473 | 0.9407 | 0.9111 | 0.9349 | 0.8966 | 0.9142 | 0.9539 | 0.9643 | 0.9796 | 0.9659
TABLE IV: Comparison of object detection performance (mAP50) with different patch sizes with $\text{WBP}_{\text}{YOLOv8}$ attack. Patch |  Faster R-CNN [9] |  DETR [29] |  YOLOv8 [8]
---|---|---|---
Size (pixel) | 50m | 70m | 90m | Mean | 50m | 70m | 90m | Mean | 50m | 70m | 90m | Mean
50/40/30 | 0.5114 | 0.6846 | 0.9239 | 0.7066 | 0.3127 | 0.5386 | 0.8623 | 0.5712 | 0.3367 | 0.1004 | 0.7453 | 0.3941
40/30/20 | 0.8486 | 0.9426 | 0.9481 | 0.9131 | 0.9349 | 0.9133 | 0.8966 | 0.9149 | 0.3466 | 0.3994 | 0.9856 | 0.5772
30/20/10 | 0.9409 | 0.9472 | 0.9278 | 0.9386 | 0.9446 | 0.9290 | 0.9129 | 0.9288 | 0.8420 | 0.9900 | 0.9842 | 0.9387

####  IV-C2 Distribution based BBP

In this section, we detail our approach of a feature distribution-based
adversarial attack targeting UAV-based object detection systems.

Feature Extraction: The cropped vehicle image RoIs are fed into a pre-trained
encoder model $\mathbf{E}$ for feature extraction. The encoder $\mathbf{E}$ is
a neural network pre-trained on a large dataset and is capable of extracting
complex features from images. We use the ResNet50 model pre-trained on
ImageNet as $\mathbf{E}$ in our experiment. The features from both attacked
and clean RoIs are extracted separately via:

$\split\mathbf{F_{adv}}&=\mathbf{E}(\mathbf{R_{adv}}),\\\
\mathbf{F_{clean}}=\mathbf{E}(\mathbf{R_{clean}}),$ (8)

where $\mathbf{F_{adv}}$ and $\mathbf{F_{clean}}$ represent the features
extracted from attacked vehicle image RoI set $\mathbf{R_{adv}}$ and the
original clean vehicle image RoI set $\mathbf{R_{clean}}$, respectively.

Feature Distribution Discrepancy: We use the Maximum Mean Discrepancy (MMD)
[31] metric to quantify the difference in the feature distributions between
the attacked and clean vehicle image RoIs. The MMD metric is particularly
effective in measuring the distance between high-dimensional feature
distributions [31]. A higher MMD value indicates a greater divergence between
the attacked and clean image RoIs, suggesting a more effective adversarial
attack, which is computed as:

$\mathcal{L}_{a}=-1\cdot\mathbf{MMD}(\mathbf{F_{adv}},\mathbf{F_{clean}}),$
(9)

where $\mathbf{MMD}(\cdot)$ denotes the function computing the MMD metric
between two feature distributions. Then the attack optimization is defined as:
arg min_PL_a(F_adv, F_clean), whose goal is to enlarge the feature
distribution difference between the cropped vehicle image RoIs added with the
adversarial patch and the cropped original vehicle image RoIs.

In summary, our $\text{BBP}_{\text}{CLIP}$ and $\text{BBP}_{\text}{D}$ methods
do not require object detection models, so they are black-box attacks.

##  V Experiment

###  V-A Object Detection Result

The experimental results of the Object Detection are summarized in Table II.
Faster R-CNN [9], DETR [29], and YOLOv8 [8] are trained to evaluate the
performance at different altitudes (50m, 70m, 90m). We keep all parameters
default and set total epochs as 100. The training set and testing are set to
1:1 in EVD4UAV dataset. We treat all classes as one class “Car”.

###  V-B Implementation Details

Each of the input images has a resolution of $1920\times 1080$. For altitudes
of 50m, 70m, and 90m, the size of $P$ is set to $50\times 50$, $40\times 40$,
and $30\times 30$ pixels respectively. We initialize $P$ with the value of 0.5
in size of $50\times 50$. We use it as the starting point of backpropagation
to update, then directly resize it to $40\times 40$ and $30\times 30$ at the
corresponding altitudes. For the training process, we employed the SGD
optimizer. The training was configured with specific hyperparameters: a batch
size of 4 and 100 epochs. Different learning rates were applied for various
attack methods: 0.1 for $\text{WBP}_{\text}{YOLOv8}$
$\text{BBP}_{\text}{CLIP}$, and $\text{BBP}_{\text}{D}$, and 0.4 for
$\text{WBP}_{\text}{RCNN}$ and $\text{WBP}_{\text}{DETR}$. We also used a
learning rate scheduler which reduces the rate by $50\%$ for every 20 epochs
to optimize the performance. In the context of white box attack patch
training, we resized the images to match the default size of each pre-trained
detector to calculate the corresponding loss. All experiments were using 8
NVIDIA RTX A6000 GPU cards.

###  V-C Experiment Evaluation

The experiments conducted for the EVD4UAV dataset offer insights into the
robustness of object detection models against adversarial attacks at varying
altitudes. Table III presents the mean Average Precision (mAP50) scores for
different models when exposed to various adversarial patches, including those
attack methods $\text{WBP}_{\text}{RCNN}$, $\text{WBP}_{\text}{DETR}$,
$\text{WBP}_{\text}{YOLOv8}$, $\text{BBP}_{\text}{CLIP}$,
$\text{BBP}_{\text}{D}$, Naive Cover, and Random Noise across altitudes of
50m, 70m, and 90m. The visualization on different $P$ examples is shown in
Fig. 8. As Fig. 8 shows, both black box attacks and white box attacks can
serve to reduce performance, but the effect of the white box is far superior
to the black box. Besides, there is no existing method that could achieve a
good enough attack ability across three altitudes, so it is necessary to
conduct more in-depth future research of the altitude-insensitive attack to
UAV detection.

The results are indicative of the models’ susceptibility to $P$ shown in Fig.
7, with a noticeable decline in mAP50 scores as adversarial patches are
applied. $\text{WBP}_{\text}{RCNN}$, when attacked itself, exhibits a
significant increase in mAP50 from 0.1746 to 0.511 as the altitude changes
from 50m to 90m, suggesting that altitude plays a role in the effectiveness of
the adversarial patches. However, when exposed to adversarial patches
generated by $\text{WBP}_{\text}{YOLOv8}$, its performance drops markedly at
lower altitudes (0.5114 at 50m), but less so at higher altitudes (0.9239 at
90m).

Figure 8: Example images of attack patches and confidence score on different
altitudes. The detection results of $\text{BBP}_{\text}{CLIP}$,
$\text{BBP}_{\text}{D}$, Naiver Cover, and Random Noise are detected by YOLOv8
detector. Other attack methods are targeting their own detectors.

###  V-D Ablation Study

To assess the influence of patch size on the performance of object detection
algorithms at different altitudes, we conducted an extensive ablation study.
The experiments were performed with adversarial patches of different sizes
($50\times 50$, $40\times 40$, $30\times 30$ pixels, decreasing the size of
the patch with each experiment). We directly use the $P$ obtained from the
YOLOv8 attack. Table IV presents the mAP50 detection results. The performance
of all models generally decreases with smaller patch sizes, illustrating the
challenge of using smaller adversarial patches, especially at higher
altitudes. Notably, Faster R-CNN showed a significant drop in attack
performance at 90m when the patch size was reduced to $30\times 30$,
indicating sensitivity of patch size to higher altitudes.

##  VI Conclusion

This study presents the EVD4UAV dataset, a novel approach for evading vehicle
detection in UAV imagery, highlighting the critical influence of altitude on
the effectiveness of adversarial attacks. The experiments demonstrate that
current object detection models are susceptible to such attacks, particularly
at varying altitudes. This research not only exposes a significant
vulnerability in UAV-based object detection systems but also lays the
groundwork for developing more advanced defense mechanisms.

## References

  * [1] A. Bansal, N. Agrawal, and K. Singh, “Rate-splitting multiple access for uav-based ris-enabled interference-limited vehicular communication system,” _IEEE Transactions on Intelligent Vehicles_ , vol. 8, no. 1, pp. 936–948, 2022.
  * [2] S. P. Bharati, Y. Wu, Y. Sui, C. Padgett, and G. Wang, “Real-time obstacle detection and tracking for sense-and-avoid mechanism in uavs,” _IEEE Transactions on Intelligent Vehicles_ , vol. 3, no. 2, pp. 185–197, 2018.
  * [3] D. Du, Y. Qi, H. Yu, Y. Yang, K. Duan, G. Li, W. Zhang, Q. Huang, and Q. Tian, “The unmanned aerial vehicle benchmark: Object detection and tracking,” in _European Conference on Computer Vision_ , 2018, pp. 370–386.
  * [4] J. Wang, S. Simeonova, and M. Shahbazi, “Orientation-and scale-invariant multi-vehicle detection and tracking from unmanned aerial videos,” _Remote Sensing_ , vol. 11, no. 18, p. 2155, 2019.
  * [5] Y. Lyu, G. Vosselman, G.-S. Xia, A. Yilmaz, and M. Y. Yang, “Uavid: A semantic segmentation dataset for uav imagery,” _ISPRS Journal of Photogrammetry and Remote Sensing_ , vol. 165, pp. 108–119, 2020.
  * [6] W. Wang, H. Chen, X. Zhang, W. Zhou, and W. Shi, “Aprus: An airborne altitude-adaptive purpose-related uav system for object detection,” in _IEEE/CVF Conference on High Performance Computing_. IEEE, 2022, pp. 402–411.
  * [7] S. Thys, W. Van Ranst, and T. Goedemé, “Fooling automated surveillance cameras: adversarial patches to attack person detection,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 0–0.
  * [8] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 779–788.
  * [9] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” _Advances in Neural Information Processing Systems_ , vol. 28, 2015.
  * [10] Z. Zuo, C. Liu, Q.-L. Han, and J. Song, “Unmanned aerial vehicles: Control methods and future challenges,” _IEEE/CAA Journal of Automatica Sinica_ , vol. 9, no. 4, pp. 601–614, 2022.
  * [11] Y. Zeng, R. Zhang, and T. J. Lim, “Wireless communications with unmanned aerial vehicles: Opportunities and challenges,” _IEEE Communications Magazine_ , vol. 54, no. 5, pp. 36–42, 2016.
  * [12] H. Shakhatreh, A. H. Sawalmeh, A. Al-Fuqaha, Z. Dou, E. Almaita, I. Khalil, N. S. Othman, A. Khreishah, and M. Guizani, “Unmanned aerial vehicles (uavs): A survey on civil applications and key research challenges,” _IEEE Access_ , vol. 7, pp. 48 572–48 634, 2019.
  * [13] R. Xu, X. Xia, J. Li, H. Li, S. Zhang, Z. Tu, Z. Meng, H. Xiang, X. Dong, R. Song _et al._ , “V2v4real: A real-world large-scale dataset for vehicle-to-vehicle cooperative perception,” in _IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2023, pp. 13 712–13 722.
  * [14] H. Sun, Y. Lin, Q. Zou, S. Song, J. Fang, and H. Yu, “Convolutional neural networks based remote sensing scene classification under clear and cloudy environments,” in _IEEE/CVF International Conference on Computer Vision Workshop_ , 2021, pp. 713–720.
  * [15] H. Sun, J. Ma, Q. Guo, Q. Zou, S. Song, Y. Lin, and H. Yu, “Coarse-to-fine task-driven inpainting for geoscience images,” _IEEE Transactions on Circuits and Systems for Video Technology_ , 2023.
  * [16] H. Sun, L. Fu, J. Li, Q. Guo, Z. Meng, T. Zhang, Y. Lin, and H. Yu, “Defense against adversarial cloud attack on remote sensing salient object detection,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2024, pp. 8345–8354.
  * [17] X. Liu, J. Li, J. Ma, H. Sun, Z. Xu, T. Zhang, and H. Yu, “Deep transfer learning for intelligent vehicle perception: a survey,” _Green Energy and Intelligent Transportation_ , p. 100125, 2023.
  * [18] J. Li, Z. Xu, L. Fu, X. Zhou, and H. Yu, “Domain adaptation from daytime to nighttime: A situation-sensitive vehicle detection and traffic flow parameter estimation framework,” _Transportation Research Part C: Emerging Technologies_ , vol. 124, p. 102946, 2021.
  * [19] C. Szegedy, A. Toshev, and D. Erhan, “Deep neural networks for object detection,” _Advances in Neural Information Processing Systems_ , vol. 26, 2013.
  * [20] G. Jocher, A. Chaurasia, and J. Qiu, “Ultralytics yolov8,” 2023. [Online]. Available: https://github.com/ultralytics/ultralytics
  * [21] H. Hu, J. Gu, Z. Zhang, J. Dai, and Y. Wei, “Relation networks for object detection,” in _IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 3588–3597.
  * [22] J. Li, R. Xu, J. Ma, Q. Zou, J. Ma, and H. Yu, “Domain adaptive object detection for autonomous driving under foggy weather,” in _IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2023, pp. 612–622.
  * [23] Y.-C.-T. Hu, B.-H. Kung, D. S. Tan, J.-C. Chen, K.-L. Hua, and W.-H. Cheng, “Naturalistic physical adversarial patch for object detectors,” in _IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 7848–7857.
  * [24] C. Xie, J. Wang, Z. Zhang, Y. Zhou, L. Xie, and A. Yuille, “Adversarial examples for semantic segmentation and object detection,” in _IEEE International Conference on Computer Vision_ , 2017, pp. 1369–1378.
  * [25] A. J. Bose and P. Aarabi, “Adversarial attacks on face detectors using neural net based constrained optimization,” in _International Workshop on Multimedia Signal Processing_. IEEE, 2018, pp. 1–6.
  * [26] Y. Li, X. Bian, M.-C. Chang, and S. Lyu, “Exploring the vulnerability of single shot module in object detectors via imperceptible background patches,” _arXiv preprint arXiv:1809.05966_ , 2018.
  * [27] X. Liu, H. Yang, Z. Liu, L. Song, H. Li, and Y. Chen, “Dpatch: An adversarial patch attack on object detectors,” _arXiv preprint arXiv:1806.02299_ , 2018.
  * [28] Y. Zhao, H. Zhu, R. Liang, Q. Shen, S. Zhang, and K. Chen, “Seeing isn’t believing: Towards more robust adversarial attack against real world object detectors,” in _2019 ACM SIGSAC Conference on Computer and Communications Security_ , 2019, pp. 1989–2004.
  * [29] N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in _European Conference on Computer Vision_. Springer, 2020, pp. 213–229.
  * [30] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark _et al._ , “Learning transferable visual models from natural language supervision,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8748–8763.
  * [31] A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. Smola, “A kernel method for the two-sample-problem,” _Advances in Neural Information Processing Systems_ , vol. 19, 2006.