Bkd-FedGNN: A Benchmark for Classification Backdoor Attacks on Federated
  Graph Neural Network
Federated Graph Neural Network (FedGNN) has recently emerged as a rapidly
growing research topic, as it integrates the strengths of graph neural networks
and federated learning to enable advanced machine learning applications without
direct access to sensitive data. Despite its advantages, the distributed nature
of FedGNN introduces additional vulnerabilities, particularly backdoor attacks
stemming from malicious participants. Although graph backdoor attacks have been
explored, the compounded complexity introduced by the combination of GNNs and
federated learning has hindered a comprehensive understanding of these attacks,
as existing research lacks extensive benchmark coverage and in-depth analysis
of critical factors. To address these limitations, we propose Bkd-FedGNN, a
benchmark for backdoor attacks on FedGNN. Specifically, Bkd-FedGNN decomposes
the graph backdoor attack into trigger generation and injection steps, and
extending the attack to the node-level federated setting, resulting in a
unified framework that covers both node-level and graph-level classification
tasks. Moreover, we thoroughly investigate the impact of multiple critical
factors in backdoor attacks on FedGNN. These factors are categorized into
global-level and local-level factors, including data distribution, the number
of malicious attackers, attack time, overlapping rate, trigger size, trigger
type, trigger position, and poisoning rate. Finally, we conduct comprehensive
evaluations on 13 benchmark datasets and 13 critical factors, comprising 1,725
experimental configurations for node-level and graph-level tasks from six
domains. These experiments encompass over 8,000 individual tests, allowing us
to provide a thorough evaluation and insightful observations that advance our
understanding of backdoor attacks on FedGNN.The Bkd-FedGNN benchmark is
publicly available at https://github.com/usail-hkust/BkdFedGCN.
Fan Liu, Siqi Lai, Yansong Ning, Hao Liu
# Bkd-FedGNN: A Benchmark for Classification Backdoor Attacks on Federated Graph Neural Network

###### Abstract

Federated Graph Neural Network (FedGNN) has recently emerged as a rapidly growing research topic, as it integrates the strengths of graph neural networks and federated learning to enable advanced machine learning applications without direct access to sensitive data. Despite its advantages, the distributed nature of FedGNN introduces additional vulnerabilities, particularly backdoor attacks stemming from malicious participants. Although graph backdoor attacks have been explored, the compounded complexity introduced by the combination of GNNs and federated learning has hindered a comprehensive understanding of these attacks, as existing research lacks extensive benchmark coverage and in-depth analysis of critical factors. To address these limitations, we propose Bkd-FedGNN, a benchmark for backdoor attacks on FedGNN. Specifically, Bkd-FedGNN decomposes the graph backdoor attack into trigger generation and injection steps, and extending the attack to the node-level federated setting, resulting in a unified framework that covers both node-level and graph-level classification tasks. Moreover, we thoroughly investigate the impact of multiple critical factors in backdoor attacks on FedGNN. These factors are categorized into global-level and local-level factors, including data distribution, the number of malicious attackers, attack time, overlapping rate, trigger size, trigger type, trigger position, and poisoning rate. Finally, we conduct comprehensive evaluations on 13 benchmark datasets and 13 critical factors, comprising 1,725 experimental configurations for node-level and graph-level tasks from six domains. These experiments encompass over 8,000 individual tests, allowing us to provide a thorough evaluation and insightful observations that advance our understanding of backdoor attacks on FedGNN. The Bkd-FedGNN benchmark is publicly available at [https://github.com/usail-hkust/BkdFedGCN](https://github.com/usail-hkust/BkdFedGCN).

## 1 Introduction

The Federated Graph Neural Network (FedGNN) has emerged as a fast-evolving research area that combines the capabilities of graph neural networks and federated learning. Such integration allows for advanced machine learning applications without requiring direct access to sensitive data [1, 2, 3, 4, 5, 6, 7, 8, 9]. However, despite its numerous advantages, the distributed nature of FedGNN introduces additional vulnerabilities, particularly related to backdoor attacks originating from malicious participants. In particular, these adversaries have the ability to inject graph backdoor triggers into their training data, thereby undermining the overall trustworthiness of the system [10, 11, 12, 13, 14].
Although considerable research efforts have explored graph backdoor attacks on FedGNN [15, 16, 17, 18], a comprehensive understanding of these attacks is hindered by the compounded complexity introduced by the combination of Graph Neural Networks (GNNs) and Federated Learning (FL). Existing studies suffer from a lack of extensive benchmark coverage and in-depth analysis of critical factors. **(1) Lack of Extensive Benchmark Coverage**. Specifically, the lack of extensive benchmark coverage poses challenges in fairly and comprehensively comparing graph backdoor attacks on FedGNN across different settings. These settings can be categorized into two levels: the graph backdoor attack level and the FedGNN task level. At the graph backdoor attack level, trigger generation and injection steps are involved. Additionally, the classification tasks in FedGNN encompass both node and graph classification tasks. However, there is still a dearth of comprehensive exploration of graph backdoor attacks on FedGNN under these various settings. **(2) Insufficient Exploration of Multiple Factors.** Furthermore, there has been the insufficient exploration of multiple factors that impact FedGNN. The combination of GNN with FL introduces various factors that affect backdoor attacks, such as trigger type, trigger size, and data distribution. The insufficient exploration and analysis of these multiple factors make it difficult to understand the influence of key factors on the behavior of FedGNN.

To address these limitations, we propose a benchmark for graph backdoor attacks on FedGNN, called Bkd-FedGNN. As far as we are aware, our work is the first comprehensive investigation of graph backdoor attacks on FedGNN. Our contributions can be summarized as follows.

* **Unified Framework**: We propose a unified framework for classification backdoor attacks on FedGNN. Bkd-FedGNN decomposes the graph backdoor attack into trigger generation and injection steps and extends the attack to the node-level federated setting, resulting in a unified framework that covers both node-level and graph-level classification tasks.
* **Exploration of Multiple Critical Factors**: We thoroughly investigate the impact of multiple critical factors on graph backdoor attacks in FedGNN. We systematically categorize these factors into two levels: global level and local level. At the global level, factors such as data distribution, the number of malicious attackers, the start time of backdoor attacks, and the overlapping rate play significant roles. In addition, the local level factors involve factors such as trigger size, trigger type, trigger position, and poisoning rate.
* **Comprehensive Experiments and Analysis**: We conduct comprehensive experiments on both benchmark experiments and critical factor analysis. For the benchmark experiments, we consider combinations of trigger types, trigger positions, datasets, and models, resulting in 315 configurations for the node level and 270 configurations for the graph-level tasks. Regarding the critical factors, we consider combinations of factors, datasets, and models, resulting in 672 configurations for the node-level tasks and 468 configurations for the graph-level tasks. Each configuration is tested five times, resulting in approximately 8,000 individual experiments in total. Based on these experiments, we thoroughly evaluate the presented comprehensive analysis and provide insightful observations that advance the field.

## 2 Federated Graph Neural Network

In this section, we provide an introduction to the preliminary aspects of FedGNN. Currently, FedGNN primarily focuses on exploring common classification tasks, which involve both node-level and graph-level classification. The FedGNN consists of two levels: client-level local training and server-level federated optimization. We will begin by providing an overview of the notations used, followed by a detailed explanation of the client-level local training, which encompasses message passing and readout techniques. Lastly, we will introduce server-level federated optimization.

### Notations

Assume that there exist \(K\) clients denoted as \(\mathcal{C}=\{c_{k}\}_{k=1}^{K}\). Each client, \(c_{i}\), possesses a private dataset denoted as \(\mathcal{D}^{i}=\{(\mathcal{G}_{j}^{i},\mathcal{Y}_{j}^{i})\}_{j=1}^{N_{i}}\), wherein \(\mathcal{G}_{j}^{i}=(\mathcal{V}_{j}^{i},\mathcal{E}_{j}^{i})\) is the graph, where \(\mathcal{V}^{i}=\{v_{t}\}_{t=1}^{n_{i}}\) (\(n_{i}\) denotes the number of nodes) is the set of nodes, and \(\mathcal{E}^{i}=\{e_{tk}\}_{t,k}\) is the set of edges (for simplicity, we exclude the subscript \(j\) that indicates the index of the \(j\)-th dataset in the dataset \(\mathcal{D}^{i}\)). \(N_{i}=\left|\mathcal{D}^{i}\right|\) denotes the total number of data samples in the private dataset of client \(c_{i}\). We employ the
notation \(\mathbf{A}_{j}^{i}\) to denote the adjacency matrix of graph \(\mathcal{G}_{j}^{i}\) belonging to client \(c_{i}\) within the set of clients \(\mathcal{C}\). \(\mathbf{X}_{j}^{i}\) represents the node feature set, and \(\mathbf{Y}_{j}^{i}\) corresponds to the label sets.

### Client-level Local Training

To ensure versatility and inclusiveness, we employ the message passing neural network (MPNN) framework [19, 20], which encompasses a diverse range of spectral-based GNNs, such as GCN [21], as well as spatial-based GNNs including GAT [22] and GraphSage [23], _etc._ Each client possesses a GNN model that collaboratively trains a global model. The local graph learning process can be divided into two stages: message passing and readout.

**Message Passing.** For each client \(c_{i}\), the \(l\)-th layer in MPNN can be formulated as follows,

\[\mathbf{h}_{j}^{l,i}=\sigma(w^{l,i}\cdot(\mathbf{h}_{j}^{l-1,i},\textit{Agg}( \{\mathbf{h}_{k}^{l-1,i}|v_{k}\in\mathcal{N}(v_{j})\}))), \tag{1}\]

where \(\mathbf{h}_{j}^{l,i}\) (\(l=0,\cdots,L-1\)) represents the hidden feature of node \(v_{j}\) in client \(c_{i}\) and \(\mathbf{h}_{j}^{0,i}=\mathbf{x}_{j}\) denotes the node \(v_{j}\)'s raw feature. The \(\sigma\) represents the activation function (e.g., ReLU, sigmoid). The parameter \(w^{l,i}\) corresponds to the \(l\)-th learnable parameter. The aggregation operation _Agg_ (e.g., mean pooling) combines the hidden features \(\mathbf{h}_{k}^{l-1,i}\) of neighboring nodes \(v_{k}\in\mathcal{N}(v_{j})\) for node \(v_{j}\), where \(\mathcal{N}(v_{j})\) represents the set of neighbors of node \(v_{j}\). Assume that the \(\mathbf{w}^{i}=\{w^{l,i}\}_{l=0}^{L-1}\) is the set of learn able parameters for client \(c_{i}\).

**Readout.** Following the propagation of information through \(L\) layers of MPNN, the final hidden feature is computed using a readout function for subsequent tasks.

\[\hat{y}_{I}^{i}=R_{\theta^{i}}(\{\mathbf{h}_{j}^{L,i}|v_{j}\in\mathcal{V}_{I}^ {i}\}), \tag{2}\]

where \(\hat{y}_{I}^{i}\) represents the prediction for a node or graph. Specifically, \(I\) serves as an indicator, where \(I=v_{j}\) denotes the prediction for node \(v_{j}\), and \(I=\mathcal{G}^{i}\) denotes the prediction for the graph \(\mathcal{G}^{i}\). The readout function \(R_{\theta^{i}}(\cdot)\) encompasses methods such as mean pooling or sum pooling _etc._, where \(\theta^{i}\) is the parameter for readout function.

### Server-level Federated Optimization

Let us consider that \(\mathbf{w}^{i}=\{w^{l,i}\}_{l=0}^{L-1}\) represents the set of trainable parameters within the MPNN framework associated with client \(c_{i}\). Consequently, we define the overall model parameters as \(\mathbf{W}^{i}=\{\mathbf{w}^{i},\theta^{i}\}\) for each client \(c_{i}\in\mathcal{C}\). The GNNs, which constitute a part of this framework, can be represented as \(f_{i}(\mathbf{X}_{j}^{i},\mathbf{A}_{j}^{i};\mathbf{W}^{i})\). The objective of FL is to optimize the global objective function while preserving the privacy of local data on each individual local model. The overall objective function can be formulated as follows,

\[\min_{\{\mathbf{W}^{i}\}}\sum_{i\in\mathcal{C}}\frac{N_{i}}{N}F_{i}(\mathbf{ W}^{i}),\quad F_{i}(\mathbf{W}^{i})=\frac{1}{N_{i}}\sum_{j\in\mathcal{D}^{i}} \mathcal{L}((f_{i}(\mathbf{X}_{j}^{i},\mathbf{A}_{j}^{i};\mathbf{W}^{i}), \mathbf{Y}_{j}^{i}), \tag{3}\]

where \(F_{i}(\cdot)\) denotes the local objective function, and \(\mathcal{L}(\cdot)\) denote the loss function (_e.g._, cross-entropy _etc._), and \(N=\sum_{i=1}^{K}N_{i}\) represent the total number of data samples encompassing all clients.

We illustrate the process of federated optimization, aimed at achieving a generalized model while ensuring privacy preservation, by utilizing a representative federated algorithm, FedAvg [24]. Specifically, in each round denoted by \(t\), the central server transmits the global model parameter \(\mathbf{W}_{t}\) to a subset of clients that have been selected for local training. Subsequently, each chosen client \(c_{i}\) refines the received parameter \(\mathbf{W}_{t}\) using an optimizer operating on its private dataset \(\mathcal{D}^{i}\). Following this, the selected clients upload the updated model parameter \(\mathbf{W}_{t}^{i}\), and the central server aggregates the local model parameters to obtain the enhanced global model parameter \(\mathbf{W}_{t+1}\).

In FedGNN setting, there exist diverse scenarios involving distributed graphs that are motivated by real-world applications. In these scenarios, classification tasks can be classified into two distinct settings based on how graphs are distributed across clients. **Node-level FedGNN**. Each client is equipped with a subgraph, and the prevalent tasks involve node classification. Real-world applications,
such as social networks, demonstrate situations where relationships between nodes can span across different clients, and each node possesses a unique label. **Graph-level FedGNN**. Each client possesses a set of graphs, and the primary focus lies on graph classification tasks. Real-world applications, such as protein discovery, exemplify instances where each institution holds a limited graph along with associated labels.

## 3 A Unified Framework for Classification Backdoor Attack on FedGNN

This section presents a unified framework for classification backdoor attacks on federated GNNs. Our primary focus is on graph-based backdoor attacks, where malicious entities strategically insert triggers into graphs or subgraphs to compromise the trustworthiness of FedGNN. A comprehensive illustration of our unified framework for classification backdoor attacks on FedGNN can be found in Figure 1. In detail, we first introduce the dataset and models and then give the evaluation metric, then introduce the threat model. Next, we introduce the federated graph backdoor attack, which involves the formulation of the attack goal and a two-step attack process: trigger generation and trigger injection. Finally, we explore various critical factors at both global and local levels.

### Datasets and Models

In this study, we have considered six distinct domains comprising a total of thirteen datasets, along with three widely used GNNs. _Node-level Datasets:_ For node-level analysis, we have included three extensively studied citation graphs, such as Cora, CiteSeer, and PubMed. Additionally, we have incorporated the Co-authorship graphs (CS and Physics), along with the Amazon Co-purchase graphs (Photo and Computers). _Graph-level Datasets:_ For graph-level analysis, we have utilized molecular graphs such as AIDS and NCI1. Furthermore, bioinformatics graphs, including PROTEINS-full, DD, and ENZYMES, have been incorporated. Lastly, a synthetic graph, COLORS-3, has also been employed. _Models:_ We have employed three widely adopted GNNs: GCN, GAT, and

Figure 1: A unified framework for classification backdoor attack on FedGNN.
GraphSage, which have been demonstrated effective in various graph-based tasks. For detailed statistical information about the graphs used, please refer to Appendix A.1.

### Evaluation Metrics

To assess the effectiveness of the graph backdoor attack on FedGNN, three metrics are employed: the average clean accuracy (ACC) across all clients, the average attack success rate (ASR) on malicious clients, and the transferred attack success rate (TAST) on normal clients. The ACC metric evaluates the performance of federated GNNs when exposed to clean examples from all clients. The ASR metric measures the performance of the graph backdoor attack specifically on the malicious clients. Lastly, the TAST metric gauges the vulnerability of normal clients to the graph backdoor attack. For the detailed equations corresponding to these metrics, please refer to Appendix A.2.

### Threat Model

**Attack Objective.** Assuming there are a total of \(K\) clients, with \(M\) (\(M\leq K\)) of them being malicious, each malicious attacker independently conducts the backdoor attack on their own models. The primary goal of a backdoor attack is to manipulate the model in such a way that it misclassifies specific pre-defined labels (known as target labels) only within the poisoned data samples. It is important to ensure that the model's accuracy remains unaffected when processing clean data. **Attack Knowledge.** In this setting, we assume that the malicious attacker has complete knowledge of their own training data. They have the capability to generate triggers. It should be noted that this scenario is quite practical since the clients have full control over their own data. **Attacker Capability.** The malicious client has the ability to inject triggers into the training datasets, but this capability is limited within predetermined constraints such as trigger size and poisoned data rate. The intention is to contaminate the training datasets. However, the malicious client lacks the ability to manipulate the server-side aggregation process or interfere with other clients' training processes and models.

### Federated Graph Backdoor Attack

Mathematically, the formal attack objective for each malicious client \(c_{i}\) during round \(t\) can be defined as follows,

\[\begin{split}&\mathbf{W}_{t}^{i*}=\arg\min_{\mathbf{W}_{t}} \frac{1}{N_{i}}\left[\sum_{j\in\mathcal{D}_{p}^{i}}\mathcal{L}((f_{i}( \mathbf{X}_{j}^{i},g_{\tau}\circ\mathbf{A}_{j}^{i};\mathbf{W}_{t-1}^{i}),\tau )+\sum_{j\in\mathcal{D}_{c}^{i}}\mathcal{L}((f_{i}(\mathbf{X}_{j}^{i},\mathbf{ A}_{j}^{i};\mathbf{W}_{t-1}^{i}),\mathbf{Y}_{j}^{i})\right],\\ &\forall j\in\mathcal{D}_{p}^{i},N_{\tau}=|g_{\tau}|\leq\bigtriangle _{g}\quad\text{and}\quad\rho=\frac{|\mathcal{D}_{p}^{i}|}{|\mathcal{D}^{i}|} \leq\bigtriangleup_{p},\end{split} \tag{4}\]

where \(\mathcal{D}_{p}^{i}\) refers to the set of poisoned data and \(\mathcal{D}_{c}^{i}\) corresponds to the clean dataset. Noted that \(\mathcal{D}_{p}^{i}\sqcup\mathcal{D}_{c}^{i}=\mathcal{D}^{i}\) and \(\mathcal{D}_{p}^{i}\sqcap\mathcal{D}_{c}^{i}=\phi\), indicating the union and intersection of the poisoned and clean data sets, respectively. \(g_{\tau}\circ\mathbf{A}_{j}^{i}\) represents the poisoned graph resulting from an attack. \(g_{\tau}\) represents the trigger generated by the attacker, which is then embedded into the clean graph, thereby contaminating the datasets. Additionally, \(\tau\) denotes the target label. \(N_{\tau}=|g_{\tau}|\) denotes the trigger size and \(\bigtriangleup_{g}\) represents the constrain to ensures that the trigger size remains within the specified limit. \(\rho=\frac{|\mathcal{D}_{p}^{i}|}{|\mathcal{D}^{i}|}\) represents the poisoned rate, and \(\bigtriangleup p\) denotes the budget allocated for poisoned data.

In the federated graph backdoor attack, to generate the trigger and poisoned data sets, the graph backdoor attack can be divided into two steps: trigger generation and trigger injection. The term "trigger" (a specific pattern) has been formally defined as a subgraph in the work by Zhang _et al._ (2021), providing a clear and established framework for its characterization [25].

**Trigger Generation.** The process of trigger generation can be defined as the function \(\varphi(\mathbf{X}_{j}^{i},\mathbf{A}_{j}^{i})\), which yields the generated trigger \(g_{\tau}\) through \(\varphi(\mathbf{X}_{j}^{i},\mathbf{A}_{j}^{i})=g_{\tau}\).
**Trigger Injection.** The process of trigger injection can be defined as the function \(a(g_{\tau},\mathbf{A}_{j}^{i})\), which generates the final poisoned graph \(g_{\tau}\circ\mathbf{A}_{j}^{i}\) by incorporating the trigger \(g_{\tau}\) into the pristine graph \(\mathbf{A}_{j}^{i}\).

### Factors in Federated Graph Backdoor

The graph backdoor attack framework in FedGNN encompasses various critical factors that warrant exploration. These factors can be categorized into two levels: the global level and the local level. At the global level, factors such as data distribution, the number of malicious attackers, the start time of backdoor attacks, and overlapping rate play significant roles. On the other hand, the local level involves parameters like trigger size, trigger type, trigger position, and poisoning rate. Notably, the overlapping rate holds particular importance in node-level FedGNN, as it involves cross-nodes across multiple clients.

_Global Level Factors:_ **Data Distribution.** The data distribution encompasses two distinct types: independent and identically distributed (IID) and non-independent and identically distributed (Non-IID). In detail, IID refers to data distribution among clients remaining constant, while Non-IID (L-Non-IID [26, 27], PD-Non-IID [28], N-Non-IID [29]) refers that the data distribution among clients exhibiting variations. **Number of Malicious Attackers.** The concept of the number of malicious attackers, denoted as \(M\), can be defined in the following manner. Let us assume that the set of malicious clients is denoted as \(\mathcal{C}_{m}\), and the set of normal clients is denoted as \(\mathcal{C}_{n}\). It can be inferred that \(\mathcal{C}_{m}\sqcup\mathcal{C}_{m}=\mathcal{C}\) and \(\mathcal{C}_{m}\sqcap\mathcal{C}_{c}=\phi\). **Attack Time.** In the context of FL, the attack time denotes the precise moment when a malicious attack is launched. The attack time can be denoted by \(t^{*}\). **Overlapping Rate (specific to Node-level FedGNN).** The overlapping rate, represented by the variable \(\alpha\), pertains to the proportion of additional samples of overlapping data that across clients. This phenomenon arises in node-level FedGNN, where cross-client nodes exist, resulting in the sharing of common data samples between different clients.

_Local Level Factors:_ **Trigger Size.** The size of the trigger can be quantified by counting the number of nodes within the corresponding graph. The trigger size is denoted by \(N_{\tau}\). **Trigger Type.** Based on the methods used to generate triggers(_e.g._, Renyi [25], WS [30], BA [31], RR [32], GTA [33], and UGBA [34]_etc._), the categorization of trigger types can be refined into two categories: universal triggers and adaptive triggers. Universal triggers are pre-generated through graph generation techniques, such as the Erdos-Renyi (ER) model [35], which are agnostic to the underlying graph datasets. On the other hand, adaptive triggers are specifically designed for individual graphs using optimization methods. **Trigger Position.** The trigger position refers to the specific location within a graph or sub-graph where the trigger is injected. Typically, the trigger position can be categorized into two types: random position and important indicator position. In the case of the random position, the trigger is injected into the graph in a random manner without any specific consideration. Conversely, the important indicator position entails injecting the trigger based on certain crucial centrality values, such as the degree or cluster-based scores, that indicate the significance of specific nodes within the graph. **Poisoning Rate.** The concept of poisoning rate, denoted as \(\rho\), can be defined as the ratio of the cardinality of the set of poisoned data samples, \(\mathcal{C}_{p}^{i}\), to the total number of data samples, denoted as \(\mathcal{D}^{i}\). Mathematically, this can be expressed as \(\rho=\frac{|\mathcal{D}_{p}^{i}|}{|\mathcal{D}^{i}|}\), where \(\forall c_{i}\in\mathcal{C}\) signifies that the cardinality calculations are performed for every client \(c_{i}\) belonging to the set \(\mathcal{C}\).

## 4 Experimental Studies

In this section, we present the experimental studies conducted to investigate classification backdoor attacks on FedGNN. Our main objective is to evaluate the impact of graph backdoor attacks on FedGNN covering both the node and graph level tasks. Additionally, we aim to explore the critical

\begin{table}
\begin{tabular}{c c|c|c} \hline \hline  & Factors & Symbol & Node Level & Graph Level \\ \hline \multirow{4}{*}{Global Level} & Data Distribution & - & \{\(\text{IID}^{*},\text{L-Non-IID}\) \} & \{\(\text{IID}^{*},\text{PD-Non-IID},\text{N-Non-IID}\) \\ \cline{2-4}  & \# of Malicious Attack & \(M\) & \(\{1^{*},2,3,4,5\}\) \\ \cline{2-4}  & Attack Time & \(t^{*}\) & \(T*\{0.0^{*},0.1,0.2,0.3,0.4,0.5\}\) \\ \cline{2-4}  & Overlapping Rate & \(\alpha\) & \(\{0.1^{*},0.2,0.3,0.4,0.5\}\) \\ \hline \multirow{4}{*}{Local Level} & Trigger Size & \(N_{\tau}\) & \(\{3^{*},4,5,6,7,8,9,10\}\) & \(N_{\tau}*\{0.1^{*},0.2,0.3,0.4,0.5\}\) \\ \cline{2-4}  & Trigger Type & \(g_{\tau}\) & \(\{\text{Rensyi}^{*},\text{WS},\text{BA},\text{GTA},\text{UGBA}\) \} & \{ Renyi\({}^{*},\text{WS},\text{BA},\text{RR},\text{GTA}\) \} \\ \cline{1-1}  & Trigger Position & - & \(\{\text{Random}^{*},\text{Degree},\text{Cluster}\) \} \\ \cline{1-1}  & Poisoning Rate & \(\rho\) & \(\{0.1^{*},0.2,0.3,0.4,0.5\}\) \\ \hline \hline \end{tabular}

* marks the default value and # represents the Number.

\(T\) represents the total training round time and \(N_{\tau}\) represents the average number of graph nodes.

\end{table}
Table 1: Critical factors in federated graph backdoor.
factors that influence the effectiveness of graph backdoor attacks on FedGNN, considering aspects from both the global and local levels.

### Experimental Settings

**Factors Settings.** We present the detailed factors setup considered in our study. It is important to note that the first value presented represents the default setting. To assess the individual impact of each factor, we keep the remaining factors fixed while systematically varying the corresponding values in our experiments. The factors range is shown in Table 1. For the detailed setting for factor, please refer to Appendix A.3.

**Federated Graph Backdoor Attack.** The federated graph backdoor attack can be characterized by the combination of trigger generation techniques (Renyi [25], WS [30], BA [31], RR [32], GTA [33], and UGBA [34]) and trigger position strategies (Random, Degree, and Cluster). For instance, the attack method Renyi-Random refers to the utilization of the ER model to generate the trigger, which is then randomly injected into the graph.

**Implementation Details.** Our implementation of the backdoor attack on FedGNN is based on the PyTorch framework. The experiments were carried out on two server configurations: three Linux Centos Servers, each with 4 RTX 3090 GPUs, and two Linux Ubuntu Servers, each with 2 V100 GPUs. In both node-level and graph-level tasks, we adopt the inductive learning settings as outlined in [16, 34]. For each dataset, we ensure consistent experimental conditions by employing the same training and attack settings. We set the total number of clients to \(5\), and all clients participate in the training process at each round. Each experiment is repeated five times. For a detailed description of the training and attack settings, please refer to Appendix A.4.

### Benchmark Results of Graph Backdoor Attack on FedGNN

The results of the benchmark for the graph backdoor attack on FedGNN are presented in Figure 2. The observations are summarized as follows. (1) The node-level task exhibits higher vulnerability to attacks compared to the graph-level task at a relatively small trigger size. Specifically, a significant majority of graph backdoor attacks achieve an ASR (Attack Success Rate) exceeding \(90\%\), while the highest ASR recorded at the graph level is \(82.24\%\). (2) Despite not being intentionally poisoned by malicious attackers, the normal clients are still susceptible to graph backdoor attacks. For instance, in the node-level task, there is a TASR (Transferred Attack Success Rate) of \(24.52\%\), while the graph-level task exhibits even higher vulnerability with a TASR of \(61.86\%\). This observation suggests that the weights uploaded by the malicious clients can inadvertently influence the normal clients when they download the global model's weights. 3). The combination of trigger size and trigger position significantly influences the attack performance on the graph-level task compared to the node-level

Figure 2: Graph backdoor attack on both node and graph level tasks for GCN. (Color intensity corresponds to value magnitude)
task. For instance, the attack WS-Cluster achieves an ASR of approximately \(82.24\%\), while the GTA-Random achieves only about \(13.87\%\). Due to the page limit, the benchmark results on other datasets and models please refer to Appendix A.5.1.

### Factors in Federated GNN

The overall results of factors can be shown in Figures 3-4. _Global Level Factors:_**Data Distribution (DD).** For node-level tasks, there models trained on IID data are more vulnerable than models trained on Non-IID data. For graph-level tasks, the GCN trained on IID data are more vulnerable than models trained on Non-IID data (PD-Non-IID and N-Non-IID), while GAT and GraphSagr trained on Non-IID data are more vulnerable than models trained on IID data. **Number of Malicious Attackers (NMA).** For node-level tasks, an increase in NMA leads to an increase in ASR for both GCN and GAT models. Conversely, an increase in NMA results in a decrease in ASR for both GraphSage. Concerning graph-level tasks, the ASR demonstrates an increase with the increase of NMA in the case of GAT and GraphSage. However, in the scenario of GCN, the ASR shows a decrease with the increase of NMA. **Attack Time (AT).** For both node-level and graph-level tasks, an increase in AT results in a decrease in ASR for three models. **Overlapping Rate (OR).** The ASR demonstrates an upward trend as the overlapping rate increases. This correlation can be attributed to the possibility that overlapping nodes facilitate the backdooring of normal clients, primarily through the presence of cross-edges.

_Local Level Factors:_**Trigger Size (TS).** For node-level tasks, an increase in TS leads to an increase in ASR for GCN. However, in the case of GAT and GraphSage, the ASR demonstrates a decrease with the increase of TS. Concerning the graph-level task, the ASR shows an increase with the increase of TS across all three GNNs. **Trigger Types (TT).** In the node-level task, the adaptive trigger demonstrates a higher ASR on most models. Conversely, in the graph-level task, the universal trigger exhibits higher ASR. **Trigger Position (TP).** In node-level tasks, we observed a significantly large ASR when using importance-based positions (Degree and Cluster) compared to random positions.

Figure 4: Graph-level task factors.

Figure 3: Node-level task factors.
However, for the graph-level task, while importance-based positions showed higher ASR for GCN, random positions yielded higher ASR for GAT and GraphSage. **Poisoning Rate (PR).** On node classification, an increase in PR results in a slight decrease in ASR. However, graph classification exhibits an upward trend in ASR. Due to the page limit, the results on other datasets and metrics, please refer to Appendix A.5.2.

## 5 Related Works

**FedGNN.** FedGNN present a distributed machine learning paradigm that facilitates collaborative training of GNNs among multiple parties, ensuring the privacy of their sensitive data. In recent years, extensive research has been conducted on FedGNN, with a particular focus on addressing security concerns [15, 16, 17, 18]. Among these concerns, poisoning attacks have garnered significant attention, encompassing both data poisoning attacks and model poisoning attacks. Data poisoning attacks occur when an adversary employs tainted data to train the local model, while model poisoning attacks involve manipulation of either the training process or the local model itself. Currently, the majority of attacks on FedGNN primarily concentrate on data poisoning attacks. Chen _et al._[15] proposed adversarial attacks on vertical federated learning, utilizing adversarial perturbations on global node embeddings based on gradient leakage from pairwise nodes. Additionally, Xu _et al._[16] investigated centralized and distributed backdoor attacks on FedGNN.

**Graph Backdoor Attacks.** Backdoor attacks on GNNs have received significant attention in recent years [25, 36, 37, 38, 33, 39, 34]. Regarding graph backdoor attacks, they can be classified into two types based on the employed trigger: universal graph backdoor attacks and adaptive backdoor attacks. In universal graph backdoor attacks, Zhang _et al._[25] generated sub-graphs using the Erdos-Renyi (ER) model as triggers and injected them into the training data. Additionally, Xu _et al._[33] observed that the position of the trigger injection into the graph can also affect the attack's performance. As for adaptive trigger backdoor attacks, Xi _et al._[33] developed an adaptive trigger generator that optimizes the attack's effectiveness for both transductive and inductive tasks. In our benchmark, we focus primarily on data poisoning attacks. While model poisoning attacks can be effective, data poisoning attacks may be more convenient because they do not require tampering with the model learning process, and they allow non-expert actors to participate [40].

## 6 Conclusions and Open Problems

**Conclusions.** In this paper, we proposed a unified framework for classification backdoor attacks on FedGNN. We then introduced the critical factors involved in graph backdoor attacks on FedGNN, including both global and local level factors. Along this line, we performed approximately 8,000 experiments on the graph backdoor attacks benchmark and conduct critical factor experiments to provide a comprehensive analysis.

**Open Problems.** (1) Enhancing the success rate of transferred attacks: Our findings reveal that malicious attackers can also backdoor normal clients through the FL mechanism. However, there is a need to explore methods that can identify and exploit the worst vulnerabilities under these circumstances. (2) Evaluating the defense method under backdoor attack: We demonstrate that FedGNN can be compromised by malicious attackers. However, assessing the effectiveness of defense mechanisms against such attacks still requires further exploration. (3) Cooperative malicious attackers: Currently, the majority of malicious attackers operate independently during the attack process, neglecting the potential benefits of collaboration. An intriguing research direction lies in investigating the utilization of collaboration to enhance attack performance.

## Acknowledgments and Disclosure of Funding

This research was supported in part by the National Natural Science Foundation of China under Grant No.62102110, Guangzhou Science and Technology Plan Guangzhou-HKUST(GZ) Joint Project No. 2023A03J0144, and Foshan HKUST Projects (FSUST21-FYTRI01A, FSUST21-FYTRI02A).