# FedRel: An Adaptive Federated Relevance Framework for Spatial Temporal Graph
Learning

Tiehua Zhang∗ Member, IEEE, Yuze Liu∗ Zhishu Shen  Member, IEEE, Rui Xu  Xin
Chen  Xiaowei Huang  Xi Zheng  Member, IEEE Tiehua Zhang, Yuze Liu, Rui Xu,
Xin Chen and Xiaowei Huang are with Ant Group, Shanghai, China
(e-mail:{zhangtiehua.zth, liuyuze.liuyuze, furui.xr, jinming.cx,
wei.huangxw}@antgroup.com).Zhishu Shen is with the School of Computer Science
and Artificial Intelligence, Wuhan University of Technology, Wuhan, China
(e-mail: z_shen@ieee.org).Xi Zheng is with the Department of Computing,
Macquarie University, Sydney, Australia (e-mail: james.zheng@mq.edu.au).∗ co-
first authorship. (Corresponding author: Tiehua Zhang)Manuscript received XXX,
2022;

###### Abstract

Spatial-temporal data contains rich information and has been widely studied in
recent years due to the rapid development of relevant applications in many
fields. For instance, medical institutions often use electrodes attached to
different parts of a patient to analyse the electorencephal data rich with
spatial and temporal features for health assessment and disease diagnosis.
Existing research has mainly used deep learning techniques such as
convolutional neural network (CNN) or recurrent neural network (RNN) to
extract hidden spatial-temporal features. Yet, it is challenging to
incorporate both inter-dependencies spatial information and dynamic temporal
changes simultaneously. In reality, for a model that leverages these spatial-
temporal features to fulfil complex prediction tasks, it often requires a
colossal amount of training data in order to obtain satisfactory model
performance. Considering the above-mentioned challenges, we propose an
adaptive federated relevance framework, namely FedRel, for spatial-temporal
graph learning in this paper. After transforming the raw spatial-temporal data
into high quality features, the core Dynamic Inter-Intra Graph (DIIG) module
in the framework is able to use these features to generate the spatial-
temporal graphs capable of capturing the hidden topological and long-term
temporal correlation information in these graphs. To improve the model
generalization ability and performance while preserving the local data
privacy, we also design a relevance-driven federated learning module in our
framework to leverage diverse data distributions from different participants
with attentive aggregations of their models. In addition, we conduct extensive
experiments on two real-world spatial-temporal datasets. The results
demonstrate the effectiveness of our proposed framework in spatial-temporal
interpretation, collaborative model training, and divergent data distribution
handling in most settings of comparison.

Spatial-temporal Data, Collaborative Graph Learning, Distribution-based
Relevance, Graph Neural Network.

##  1 Introduction

In recent years, spatial-temporal data has drawn increasing attention owing to
its great potential in various fields and significant effects on daily
activities such as traffic management [1], health monitoring [2], and action
recognition [3]. The advancement of deep learning has shed light on processing
by extracting valuable features from spatial-temporal data to facilitate the
downstream prediction/regression tasks. For instance, traditional deep
learning techniques such as convoluational neural networks (CNNs) [2] and
recurrent neural networks (RNNs) [4, 5] have been heavily investigated to
uncover the hidden correlations and interdependencies in spatial-temporal data
sequence.

Even though the existing models are able to capture and utilise the extracted
correlation to some extent, many limitations start surfacing: CNN-based
methods face challenges in discovering long-term temporal correlations while
RNN-based ones lack the capability to comprehend the global spatial structure.
Additionally, one common issue is that both model types require the grid-like
data input, which is considered intractable when decoding the topological
information from the spatial dimension. It is argued that the non-Euclidean
graph data structure shows great potential in terms of representing inter-
connectivity and dynamic trends in both spatial and temporal aspects [6].
Recently, modelling the spatial-temporal data into graph representation has
risen to the spotlight owing to the development of graph neural network (GNN),
reporting some encouraging improvements on both classification and regression
tasks compared with CNN and RNN based models [7, 8]. It is also proven in
prior work [9] that even for data in which explicit graph structure does not
naturally exist, extracting and modelling the hidden graph structure from the
original data could lead to a significant performance improvement. Following
that, most existing methods [7, 8, 10, 11] first transform spatial-temporal
data into the static graphs by modelling both nodes and edges, and then use
GNNs to learn the embedding representations from the generated static graphs.
Taking the traffic flow graph as an example in which different districts are
treated as nodes, the flow of traffic between nodes will experience the
changes at different hours of the day [1]. Ignoring dynamic correlations
between nodes in the temporal dimension will undoubtedly lead to compromised
and less desirable results. It is thus crucial to capture the inter-
dependencies and dynamic temporal changes over time when turning the spatial-
temporal data into the graph structure.

Apart from the quest to utilise spatial-temporal data more effectively,
another pressing problem regards the data scarcity issue in real-world
applications. For all learning-based tasks, insufficient data essentially
causes an over-fitting problem of the model and make it hard to generalise
when unseen data appears during inference time. One common way to solve this
problem is to collect a colossal amount of training data from multiple parties
and train the model in a centralised manner. However, it has drawn increasing
concerns over data privacy and security in many data-sensitive institutions.
For instance, hospitals and drug research institutions rarely publish patient
and clinical data due to their sensitivity, and trainable models thus fail to
infer results that are not present in the training set. To this end, federated
learning (FL) [12] is introduced as a collaborative training scheme, which
involves multiple parties without exposing local data to others. By
integrating all participants’ model weights or gradients, the model trained by
FL will thus have a higher generalisation ability.

In the commonly used FL framework [13], all participants’ data is by default
in the same distribution, while in real-world cases, the divergence of data
distribution from different participants is nontrivial since it is determined
by many uncontrollable factors like separate geographic locations. The most
prevalent FL algorithm is called FedAvg [13], in which the local model weights
are sent to the server and aggregated in an iterative manner. However, it
ignores the non-IID data distribution fact and assumes equal contribution from
all participants. It is thus both intuitive and critical to incorporate the
difference in data distribution from participants and make it part of FL
process to facilitate the training process.

In this work, we propose a novel adaptive federated relevance framework called
FedRel for spatial-temporal graph learning. This framework is composed of
three key modules, including a feature transformation net to extract the
important spatial-temporal features from raw data, a dynamic inter-intra graph
(DIIG) module to generate the spatial-temporal graphs while capturing inter-
dependencies and dynamic temporal changes across these graphs, and a
relevance-based federated learning module to expedite the collaborative graph
learning. It is worth mentioning that the intra-graph concept in our work
refers to the spatial node correlations inside one generated spatial graph at
any timestamp, while inter-graphs indicate temporal node correlations across
multiple spatial graphs. The main advantages of our proposed framework are two
folds: 1) DIIG uncovers the hidden topological graph structures of the
transformed spatial-temporal features, which effectively encodes both spatial
and temporal information into the embedding space when updating the node
embedding through the GNN model; 2) The DIIGs from different participants are
an integral part of the federated relevance training process. Thus the models
are trained collaboratively to enable a better generalisation capability.

The contributions of this work are summarised as follows:

  1. 1.

We propose a novel adaptive federated relevance framework, namely FedRel, for
spatial-temporal graph learning. The relevance of each participant is
determined by adaptively attending the distance scores between local data
distribution and the approximated global distributions, enabling the framework
to put more attention on the participants with divergent data distributions
and improve model’s generalisation ability.

  2. 2.

We implement a feature transformation net using a variety of model structures
with different design philosophies, aiming to find the most practical model
that excels in processing spatial-temporal raw data and is capable of
generating high-quality spatial-temporal features.

  3. 3.

We conceptualise the inter-intra graphs learning from spatial-temporal
features. We also design and implement a dynamic inter-intra graph (DIIG)
module to generate the spatial-temporal graphs so as to capture inter-
dependencies and dynamic temporal changes across these graphs.

  4. 4.

We conducted a comprehensive experiment of our framework on real-world dataset
ISRUC_S3 [14] and SHL_Small [15], along with detailed ablation studies on each
module for the interpretability purpose. The experimental results show that
the proposed model achieves promising results in task-specific
classifications, model convergence performance, and scalability.

Figure 1: The overview of the FedRel (better viewed in color): (a) an example
of exacting spatial-temporal features from the raw input data. In dynamic
inter-intra graph (DIIG) module, The intra-spatial graph block first takes the
initial node features at each timestamp to update embedding for each spatial
graph. The inter-temporal graph block then merge across different spatial
graphs in temporal dimension, producing the embedding with both spatial and
temporal information; (b) the process of collaborative training on DIIGs from
different participants. It enables an attentive model aggregation based on the
relevance-guided divergence between local data distributions and approximated
global data distribution.

##  2 Related Work

###  2.1 Deep Learning for Spatial-Temporal Data

The study of spatial-temporal data has attracted increasing attention in
recent years, especially after the emergence of many advanced deep learning
models. It is pointed out that spatial-temporal data could benefit
applications in many disciplines such as emotion detection, traffic flow
prediction, and sleep stage/quality classification [6]. However, It has been a
long-standing challenge to better uncover and understand the hidden attributes
in these numeric data.

Recent research tries to interpret spatial and temporal dependencies, of which
the deep learning model could take advantage to generate better
prediction/classification results. For instance, a fast discriminative
complex-valued convolutional neural network (CNN) [2], namely FDCCNN, is
designed to apprehend the hidden correlations in the electroencephalogram
(EEG) signals to expedite the sleep stage classification task. Alternatively,
MLP-Mixer [16] is also used in the multi-channel temporal signal data,
resulting in promising results on the regression task. It is proven effective
to use recurrent neural network (RNN) based models for capturing non-linear
interdependencies, including ConvLSTM [4], Bi-LSTM [5]. Following that, some
studies start to use the attention mechanisms for capturing the long-term
correlations [17]. Specifically, relevant spatial and temporal attention
modules are designed to serve this purpose, reporting promising results in
clustering multivariate time series data with varying lengths.

###  2.2 Graph Neural Networks (GNNs)

Although applying CNN and RNN to spatial-temporal data has yielded promising
results, the limitations are also clear. First, both of these techniques still
require processing data in the Euclidean space, essentially ignoring the
potential connectivity information in the spatial dimension. Also, the inter-
dependent relations uncovered using both methods are difficult to interpret
from human’s perspective. The rise of GNN has shed light on this direction as
some researchers have started investigating how to use graphs to form better
topological representations in both spatial and temporal dimension [18].

However, constructing graph structure from the spatial-temporal data is the
prerequisite before using GNN models, and there are two main hurdles when
transforming the spatial-temporal sequence to GNN-required graph input: 1. to
uncover the spatial correlations of the data streams to generate the adjacency
matrix; 2. to extract the node features from the temporal values. A line of
research has focused on solving this issue [7, 8, 10, 11]. Specifically, graph
convolutional recurrent network (GCRN) [10] combines the LSTM network with
ChebNet [19] to handle spatial-temporal data. Structural-RNN [7] uses node-
level and edge-level RNN to uncover spatial correlations in the data.
Alternatively, CNN could be used to embed temporal relationships to solve the
exploding/vanishing gradient problems. For example, ST-GCN [11] uses the
partition graph convolution layer to extract spatial information, while a one-
dimension convolution layer is designed to extract temporal dependencies.
Similarly, CGCN [8] combines a one-dimension convolution layer with a ChebNet
or GCN layer, to handle spatial-temporal data. Regardless of using the CNN-,
RNN-based models, or the recent advancements on GNN-based ones, the existing
research is not able to utilise the underlying topological graph structures
and temporal transition information in the spatial-temporal data
simultaneously, and we intend to deal with this issue in our work.

###  2.3 Federated Learning (FL)

Federated Learning is a privacy-preserving collaborative learning paradigm
designed to protect participants’ local data privacy while enjoying a superior
model performance. It enables the communication of participants’ local
gradients/weights to avoid exposing the raw data to others. FL is considered
helpful in many fields, where data privacy is a major concern [20]. One of the
widely used FL algorithms is FedAvg [13], in which the local model weights are
sent to the server and aggregated in an iterative manner. However, due to the
diversity and divergence of local data distribution, the performance of FL is
greatly compromised. It is pointed out in [21] that the non-IID data
distribution from different participants lead to severe weight divergence
throughout the training phase. To cope with that, [21, 22, 23] propose to
generate part of the global dataset by sharing some local data with the server
so that the negative impact of non-IID could be mitigated. But the problem is
obvious: it violates the design philosophy of FL by exposing partial data to
others. Following that, Shin et al. [24] proposes to upload XOR encoded seeds
from participants’ local data to the server for the global data approximation.
The global data is decoded using these seeds to re-train the model at server
side. Instead of simply averaging the local model weights that causes the
weight divergence, the contributions from different participants should be
considered to facilitate the convergence of training. By calculating the
distance between global and participant models at every communication round,
attention scores are calculated as weighting coefficients on each local model
when being aggregated [25]. Instead of considering the weight distance or
violating the local data privacy in other works, we believe the divergent
local data distribution from different participants could contribute greatly
when it comes to producing a well-performing model. In the FL process of our
work, we intend to explore how the local data distribution and approximated
global distribution could facilitate the training. The relevance of each
participant is quantified and incorporated during training to generalise the
model well.

Another line of research explores applying the GNN-based approach to
understand distributed spatial-temporal data in FL. To achieve an accurate
traffic forecasting performance, Zhang et al. [26] introduces GNN-based models
that exploit the spatial correlations of the traffic graph in the FL-based
system. Zhang et al. [27] follows this work in traffic prediction and proposes
an attention-based spatial-temporal GNN model under FL settings, which reports
a good result in the traffic speed prediction task. Meng et al. [28] proposes
a federated spatial-temporal model which explicitly encodes the underlying
graph structure using GNN under the constraint of cross-node FL. This model
can ensure the data generated locally remains decentralised without extra
computation cost at the distributed edge devices. Compared with these, we
propose to conceptualise the spatial-temporal correlation into an inter-intra
graph learning module, from which both inter-dependencies among spatial
dimensions and dynamic temporal changes can be better encoded into the
embedding space.

##  3 Preliminaries

Table 1: Table of important notations

Notation |  Description
---|---
$\oplus$ | Concatenation operation
$\left|\cdot\right|$ | The set size
$\left\|\cdot\right\|_{2}$ | L2 norm
$\emph{f}_{mn}(\cdot)$ | Row-wise mean function
$\mathcal{N}(\cdot)$ | Multivariate Gaussian function
$\sigma(\cdot)$ | Sigmoid activation function
$\emph{softmax}(\cdot)$ | Softmax function
$\exp(\cdot)$ | Exponential function
$\emph{MSE}(\cdot)$ | Mean square error function
$\emph{LN}(\cdot)$ | Layer normalisation
$\emph{msg}(\cdot)$ | Message passing function
$\emph{readout}(\cdot)$ | Graph readout function
$\emph{g}_{\theta}(\cdot)$ | Global distribution estimator
$\mathcal{L}$ | Loss function
w | Window size
N | The number of channels/nodes
K | The number of participants
T | Time steps
D | Raw data signal dimension
d | Feature dimension
L | Largest message passing layers
$\mathcal{S}$ | A set of raw spatial-temporal raw data
$\boldsymbol{S}$ | Spatial-temporal feature matrix
$\boldsymbol{s}_{n}$ | A row of raw signal frequency from $\boldsymbol{S}$
$\mathcal{X}$ | A set of initial node features
$\boldsymbol{X}$ | Initial node feature matrix of a graph
$\boldsymbol{x}_{i}$ | Initial feature vector of node $\emph{v}_{i}$
$\mathcal{G}$ | A graph
$\mathcal{V}$ | The set of nodes in the graph
$\mathcal{E}$ | The set of edges in the graph
v | A node $\emph{v}\in\mathcal{V}$
$\emph{e}_{i,j}$ | Edge between $\emph{v}_{i}$ and $\emph{v}_{j}$
$\boldsymbol{A}$ | Adjacency matrix of a graph
$A_{\emph{i},\emph{j}}$ | Correlation value between nodes $\emph{v}_{i}$ and $\emph{v}_{j}$
$\mathcal{G}_{spa}\left(t\right)$ | Spatial graph at $t$th time step
$\mathcal{G}_{tem}(t)$ | A stack of spatial graphs within a time window
$\tilde{\boldsymbol{S}}^{(k)}$ | Reshaped local dataset at participants $k$
$\tilde{\boldsymbol{s}}_{i}^{(k)}$ | A local data sample from $\tilde{\boldsymbol{S}}^{(k)}$ at participant $k$
$\boldsymbol{H}$ | node embedding matrix at a graph
$\boldsymbol{h}_{i}$ | node $\emph{v}_{i}$’s embedding vector
$\tilde{\boldsymbol{z}}$ | Predictor output
$\boldsymbol{y}$ | True label
$\boldsymbol{z}$ | Latent data representation of one local data point
$\boldsymbol{d}$ | Latent representation of local data
$\tilde{\boldsymbol{d}}$ | Synthesised global data representation
$\boldsymbol{I}$ | Identity matrix
$\boldsymbol{W}$,$\Theta$,$\theta$,$\theta_{S}$ | Learnable model parameters

###  3.1 Spatial-Temporal Graph

A spatial-temporal data can be represented as
$\mathcal{S}=\left\\{\boldsymbol{S}\left(1\right),\boldsymbol{S}\left(2\right),...,\boldsymbol{S}\left(T\right)\right\\}\in\mathbb{R}^{T\times
N\times D}$, where
$\boldsymbol{S}\left(t\right)=\left[\boldsymbol{s}_{1}\left(t\right),\boldsymbol{s}_{2}\left(t\right),...,\boldsymbol{s}_{N}\left(t\right)\right]\in\mathbb{R}^{N\times
D}$, $t\in\left[1,2,...,T\right]$, is the time series, and
$\boldsymbol{s}_{n}\left(t\right)\in\mathbb{R}^{D}$,
$n\in\left[1,2,...,N\right]$, represents the original signal feature dimension
$D$ at $n$th channel in temporal context. $N$ indicates the number of channels
in the spatial dimension (e.g., devices and sensors). For each
$\boldsymbol{S}\left(t\right)$, the features generated by the feature
transformation net is
$\boldsymbol{X}\left(t\right)=\left[\boldsymbol{x}_{1}\left(t\right),\boldsymbol{x}_{2}\left(t\right),...,\boldsymbol{x}_{N}\left(t\right)\right]\in\mathbb{R}^{N\times
d}$, where $d$ is the size of the extracted features (As shown in the left
part of Fig. 1.a).

To model the inter-dependencies over time steps, we define a time window $w$
to capture the historical sequence in temporal context, i.e.,
$\left\\{\boldsymbol{X}\left(t-w\right),...,\boldsymbol{X}\left(t\right)\right\\},w\in\left[0,1,...,t-1\right]$.
We define a graph as
$\mathcal{G}=\left(\mathcal{V},\mathcal{E},\boldsymbol{A}\right)$, where
$\mathcal{V}=\left\\{\emph{v}_{1},...,\emph{v}_{N}\right\\}$ is the set of
nodes, and
$\mathcal{E}=\left\\{\left(\emph{v}_{i},\emph{v}_{j}\right)|v_{i},v_{j}\in\mathcal{V}\right\\}$
denotes the set of edges in the graph, which can be quantified by an adjacency
matrix
$\boldsymbol{A}\in\mathbb{R}^{\left|\mathcal{V}\right|\times\left|\mathcal{V}\right|}$.
The $A_{i,j}>0$ means there exists an edge $e_{i,j}$ between $v_{i}$ and
$v_{j}$, and $A_{i,j}=0$ otherwise.

Since there is no explicit graph structure in the extracted feature
$\boldsymbol{X}\left(t\right)$, we use spatial channels to represent nodes in
our problem setting, meaning $\left|\mathcal{V}\right|$ = $N$, which then is
referred as a spatial graph $\mathcal{G}_{spa}$. The temporal graph, on the
other hand, is composed of a stack of spatial graphs in time window $w$, which
is denoted as Given
$\mathcal{G}_{tem}\left(t\right)=\left[\mathcal{G}_{spa}\left(t-w\right),\mathcal{G}_{spa}\left(t-w+1\right),...,\mathcal{G}_{spa}\left(t\right)\right]$.

###  3.2 Graph Neural Networks (GNNs)

GNN works on the graph structure data and presents an effective way of
learning both node and graph embeddings. It requires both initial node
features and adjacency matrix as the input, and conducts the layer-based
message passing to aggregate neighboring nodes information and update
embeddings [29]. Given any
$\mathcal{G}=\left(\mathcal{V},\mathcal{E},\boldsymbol{A}\right)$, the message
passing function can be formalised as a function
$\emph{msg}\left(\cdot\right)$ with trainable weights:

$\boldsymbol{H}^{l}=\emph{msg}\left(\boldsymbol{A},\boldsymbol{H}^{l-1}\right)$
(1)

$\emph{msg}\left(\boldsymbol{A},\boldsymbol{H}^{l-1}\right)=\sigma\left(\boldsymbol{W}_{agg}^{l}\cdot\left(\boldsymbol{H}^{l-1}\oplus\frac{1}{N}\cdot\boldsymbol{A}\cdot\boldsymbol{H}^{l-1}\right)\right)$
(2)

where $\emph{l}=1,..,L$ denotes the message passing layer.
$\boldsymbol{h}^{l}_{i}\in\boldsymbol{H}^{l}$ is updated embedding of node
$v_{i}$, and $\boldsymbol{W}_{agg}^{l}$ is the trainable weight matrix at that
layer. Note it exists $\boldsymbol{H}^{0}=\boldsymbol{X}$ at the initial
stage, using the initial node features as the embedding information. $\oplus$
is the vector concatenation operation, and $\sigma\left(\cdot\right)$ is the
activation function.

To derive the graph representation after the iterative updates of node
embeddings, a graph readout function can be defined as
$\emph{readout}\left(\cdot\right)$:

$\boldsymbol{h}^{\mathcal{G}}=\emph{readout}\left(\boldsymbol{H}^{L},\boldsymbol{X}\right)$
(3)

$\emph{readout}\left(\boldsymbol{H}^{L},\boldsymbol{X}\right)=\sigma\left(\boldsymbol{W}_{pool}\cdot\left(\frac{1}{N}\sum_{i=1}^{N}\boldsymbol{h}_{i}^{L}\oplus\boldsymbol{x}_{i}\right)\right)$
(4)

where $\boldsymbol{H}^{L}$ denotes the up-to-date node embeddings after $L$th
message passing, and $\boldsymbol{h}^{\mathcal{G}}$ denotes the embedding
representation of the graph derived from $\boldsymbol{H}^{L}$. We use a
trainable weight $\boldsymbol{W}_{pool}$ as the linear transformation pooling,
which can also be replaced by alternative operations such as min-pooling, max-
pooling, or attentive-pooling [30].

###  3.3 Federated Learning (FL)

Federated learning is a collaborative training protocol to learn a well-
generalised model without exposing the local participants’ data to others. It
enables the trainable weight aggregation on the server side. FL encompasses
several rounds of training, in which each participant uploads the locally
trained model to the server for model aggregation (e.g., FedAvg algorithm
[13]). The updated model is then distributed back to participants. FL repeats
this training protocol until the model converges. The objective function to
minimise at the server side is:

$\min_{\Theta}F\left(\Theta\right):\mbox{where}\;F^{\left(k\right)}\left(\Theta\right)=\sum_{k=1}^{K}r^{(k)}F\left(\Theta\right)^{\left(k\right)}$
(5)

where $K$ is the number of participants. $r^{(k)}\geq 0$ is the weighting
coefficient of participant $k$ for attentively aggregating the uploaded
weights, where $\sum_{k=1}^{K}r^{(k)}=1$. $F(\Theta)^{(k)}$ is the local
objective function of participant $k$ where $\Theta$ is the trainable weight
sets at that participant. Note that in FedAvg the $r^{(k)}$ is set to be
$1$/$K$, meaning this algorithm considers equal contribution of each
participant.

##  4 Details of FedRel

The overview of the architecture of the FedRel is shown in Fig. 1. We
elaborate on the proposed framework in this section and formalise the four key
components of the framework: 1) transforming the raw spatial-temporal data to
high-quality initial node features; 2) intra-spatial graph block to uncover
the attentive spatial correlations of different nodes and update node
embedding per spatial graph; 3) inter-temporal graph block to incorporate the
inter-dependencies into node embedding among neighbouring spatial graphs in
temporal dimension; 4) FL relevance module for attentive weight aggregation
based on divergent data distribution. The important notations appeared in this
paper are summarised in Table 1.

###  4.1 Feature Transformation Net

Since GNN requires both node features and adjacency matrix, the framework
first takes the spatial-temporal sequence data as the raw input, followed by
the designated feature transformation net to generate the initial node
features matrix $\boldsymbol{X}\left(t\right)$ at each time step $t$ (Fig.
1.a). The feature transformation net can be implemented using variants of
prevalent deep learning models, and the commonly used ones are CNN and RNN-
based models [31, 4, 5]. It is already clarified in Section III-A that the raw
sequence data
$\mathcal{S}=\left\\{\boldsymbol{S}\left(1\right),\boldsymbol{S}\left(2\right),...,\boldsymbol{S}\left(T\right)\right\\}\in\mathbb{R}^{T\times
N\times D}$, the feature transformation process can be defined as:

$\left\\{\boldsymbol{X}\left(1\right),...,\boldsymbol{X}\left(t\right)\right\\}=\emph{transform}\left(\left\\{\boldsymbol{S}\left(1\right),...,\boldsymbol{S}\left(t\right)\right\\};\theta_{S}\right)$
(6)

where $\theta_{S}$ is the trainable parameters in the feature transformation
net, and
$\mathcal{X}=\left\\{\boldsymbol{X}\left(1\right),\boldsymbol{X}\left(2\right),...,\boldsymbol{X}\left(t\right)\right\\}\in\mathbb{R}^{T\times
N\times d}$ is the high-quality output, i.e. the generated initial node
features. Note the feature transformation is performed as the local process at
each participant, so it serves as the local pre-trained model and differs the
$\theta_{S}$ in different participants.

###  4.2 Dynamic Inter-Intra Graph (DIIG) Module

The dynamic inter-intra graph encompasses two primary goals: 1) to enable the
intra-graph embedding update through the intra-spatial graph block, which
uncovers the hidden correlations between spatial nodes and update node
embedding in spatial dimension; 2) to further encode temporal changes of these
spatial graphs and update the node embedding with these inter-dependencies in
the time axis, in which the inter-node correlations between two adjacent
spatial graphs are also quantified in the inter-temporal graph block.

####  4.2.1 Intra-Spatial Graph Block

The intra-spatial graph block defines a spatial graph at any time $t$ based on
the generated initial node features
$\boldsymbol{X}\left(t\right)\in\mathbb{R}^{N\times d}$ from the feature
transformation net. As explained in Section III-B, we deploy GNN for intra-
graph learning, while updating the embedding of each node based on the
correlations with spatial neighbour. We have reported using different static
correlation functions in our preliminary work [31], including PCC, PLV, and
$K$-NN. Simply, these static correlation functions take a node feature matrix
as the input, and use different ways of quantifying a numeric value based on a
pair of node features (details of each explained in Section V-A). However, the
shortcomings of using the static functions are obvious: 1) the connectivity
between nodes will not change along with dynamic changes of node embedding; 2)
it is unable to form the correlations between nodes residing in different
graphs. To solve that, we design a dynamic intra correlation layer
$\emph{c}\left(\cdot,\>\cdot\right)$ to generate the correlation of two nodes,
which is calculated as:

$A_{spa}^{i,j}\\!\left(t\right)\\!=\\!\emph{c}\\!\left(\boldsymbol{x}_{i}\\!\left(t\right)\\!,\\!\boldsymbol{x}_{j}\\!\left(t\right)\\!\right)\\!=\\!\frac{\exp\left(\left(\boldsymbol{x}_{i}\left(t\right)\right)^{T}\\!\cdot\\!\boldsymbol{W}_{spa}\\!\cdot\\!\boldsymbol{x}_{j}\left(t\right)\right)}{\sum_{n=1}^{N}\\!\exp\\!\left(\left(\boldsymbol{x}_{i}\left(t\right)\right)^{T}\\!\cdot\\!\boldsymbol{W}_{spa}\\!\cdot\\!\boldsymbol{x}_{n}\left(t\right)\right)}$
(7)

where $\boldsymbol{x}_{i},\>\boldsymbol{x}_{j}\in\mathbb{R}^{1\times d}$.
$\boldsymbol{W}_{spa}\in\mathbb{R}^{d\times d}$ is a learnable weight matrix
and used to project the feature vectors into the correlation space, from which
a correlation scalar between node $\emph{v}_{i}$ and $\emph{v}_{j}$ can be
learnt dynamically. The softmax operation serves two purposes: 1) to ensure
the non-negative correlation value; 2) to normalise the value between 0 and 1.
The iterative calculation of each pair in the feature matrix
$\boldsymbol{X}\left(t\right)$ leads to the full adjacency matrix
$\boldsymbol{A}_{spa}\left(t\right)$ of this spatial graph
$\mathcal{G}_{spa}\left(t\right)$.

As mentioned in Section III-B, the embedding update of
$\mathcal{G}_{spa}\left(t\right)$ using GNN can be formalised as:

$\boldsymbol{H}^{l}_{spa}\left(t\right)=\emph{msg}\left(\boldsymbol{A}_{spa}\left(t\right),\boldsymbol{H}_{spa}^{l-1}\left(t\right)\right)$
(8)

where $\emph{l}\\!=\\!1,..,L$ denotes the message passing layer, and
$\boldsymbol{H}^{0}_{spa}\left(t\right)=\boldsymbol{X}\left(t\right)$. We have
$\boldsymbol{H}_{spa}^{L}\left(t\right)=\left[\boldsymbol{h}^{spa}_{1}\left(t\right),\boldsymbol{h}^{spa}_{2}\left(t\right),...,\boldsymbol{h}^{spa}_{N}\left(t\right)\right]$
as the up-to-date node embeddings in $\mathcal{G}_{spa}\left(t\right)$, and
the graph embedding of which can be calculated as:

$\boldsymbol{h}^{\mathcal{G}}\left(t\right)=\emph{readout}\left(\boldsymbol{H}^{L}_{spa}\left(t\right),\boldsymbol{X}\left(t\right)\right)$
(9)

Eq.8 and Eq.9 essentially provide the embeddings of nodes and graph at each
timestamp, which are then taken as the input to Inter-temporal graph block for
embeddings update in temporal dimension.

####  4.2.2 Inter-Temporal Graph Block

After obtaining the embeddings of nodes and graph at the spatial graph level,
it is critical to incorporate the inter-dependencies between different spatial
graphs in the temporal dimension.

As indicated by the Fig. 1a, the inter-temporal graph block takes the
embeddings of nodes and graph as input of the embedding fusion layer. The
process can be defined as:

$\boldsymbol{h}^{fuse}_{i}\left(t\right)=\emph{fusion}\left(\boldsymbol{h}^{spa}_{i}\left(t\right),\boldsymbol{h}^{\mathcal{G}}\left(t\right)\right)$
(10)

where $\boldsymbol{h}^{spa}_{i}\left(t\right)$ denotes the $\emph{v}_{i}$’s
embedding at spatial graph $\mathcal{G}_{spa}\left(t\right)$, and
$\boldsymbol{h}^{\mathcal{G}}\left(t\right)$ is the embedding of this graph.
The detailed fusion operations can be expanded as:

$\emph{fusion}\\!\left(\boldsymbol{h}^{spa}_{i}\\!\\!\left(t\right)\\!,\boldsymbol{h}^{\mathcal{G}}\\!\\!\left(t\right)\\!\right)\\!\\!=\\!\\!\emph{LN}\left(\sigma\\!\left(\boldsymbol{W}_{fuse}\\!\cdot\\!\left(\boldsymbol{h}_{i}^{spa}\\!\\!\left(t\right)\oplus\boldsymbol{h}^{\mathcal{G}}\left(t\right)\\!\right)\\!\right)\\!\right)$
(11)

where $\oplus$ iteratively concatenates the graph embedding with nodes
embeddings within that graph, making sure the updated node embedding contains
not only local neighborhood but also global graph-level structure information.
$\emph{LN}\left(\cdot\right)$ is the layer normalisation operation used to
improve the training speed and mitigate the overfitting [32]. The matrix form
of the generated fusion embedding of one graph is
$\boldsymbol{H}_{fuse}\left(t\right)=\left[\boldsymbol{h}^{fuse}_{1}\left(t\right),\boldsymbol{h}^{fuse}_{2}\left(t\right),...,\boldsymbol{h}^{fuse}_{N}\left(t\right)\right]$.

Similar to the intra correlation layer, the inter correlation layer between
nodes in two consecutive spatial graphs can be formalised as:

$\begin{split}A_{tem}^{i,j}\left(t\right)&=\emph{c}\left(\boldsymbol{h}^{fuse}_{i}\left(t\right),\boldsymbol{h}^{fuse}_{i}\left(t-1\right)\right)\\\
&=\frac{\exp\left(\left(\boldsymbol{h}^{fuse}_{i}\left(t\right)\right)^{T}\cdot\boldsymbol{W}_{tem}\cdot\boldsymbol{h}^{fuse}_{i}\left(t-1\right)\right)}{\sum_{n=1}^{N}\exp\left(\left(\boldsymbol{h}^{fuse}_{i}\left(t\right)\right)^{T}\cdot\boldsymbol{W}_{tem}\cdot\boldsymbol{h}^{fuse}_{n}\left(t-1\right)\right)}\end{split}$
(12)

It can be observed from Eq.12 that the inter correlation layer dynamically
quantify the latent correlations using fused node embeddings in temporal
dimension. $A_{tem}^{i,j}$ represents the normalised correlation value of any
$\emph{v}_{i}$ and $\emph{v}_{j}$ node pairs at adjacent time steps, and
$\boldsymbol{A}_{tem}$ records the values of learned temporal adjacency
matrix.

Inter-temporal graph block employs a recursive approach to capture inter-
dependencies across different time steps, which can be formulated as:

$\begin{split}\boldsymbol{H}\\!_{tem}\\!\left(\\!t\\!\right)\\!\\!=&\emph{msg}\\!\bigl{(}...\\\
&\emph{msg}\\!\left(\\!\boldsymbol{A}\\!\left(t-\\!w\\!+\\!2\right)\\!,\\!\emph{msg}\\!\left(\boldsymbol{A}_{tem}\\!\left(t-\\!w\\!+1\\!\right)\\!,\\!\boldsymbol{H}\\!_{fuse}\\!\left(t\\!-w\\!\right)\\!\right)\\!\right)\\!\\!\bigl{)}\end{split}$
(13)

where $\boldsymbol{H}_{tem}\left(t\right)$ represents the updated temporal
node embeddings at time step $t$. The depth of recursion indicates whether a
long-term temporal dependencies is needed, which is controlled by an
adjustable time window $w$. Note that $w=0$ means the node embeddings will not
be updated by inter-graph message passing, and there exists
$\boldsymbol{H}_{tem}\left(t\right)=\boldsymbol{H}_{fuse}\left(t\right)$.

The final node embeddings are expected to comprise both topological spatial
and inter-dependent temporal information. It can thus be derived as:

$\hat{\boldsymbol{H}}_{emb}\left(t\right)=\sigma\left(\boldsymbol{W}_{o}\cdot\left(\boldsymbol{H}_{spa}^{L}\left(t\right)\oplus\boldsymbol{H}_{tem}\left(t\right)\right)\right)$
(14)

where $\boldsymbol{W}_{o}$ is a linear output layer.
$\hat{\boldsymbol{H}}_{emb}\left(t\right)$ is input for the row-wise mean
function $\emph{f}_{mn}\left(\cdot\right)$ to generate the logits of this
graph $\tilde{\boldsymbol{z}}\left(t\right)$, which can be used in downstream
classification tasks.

$\tilde{\boldsymbol{z}}\left(t\right)=\emph{softmax}\left(\emph{f}_{mn}\left(\hat{\boldsymbol{H}}_{emb}\left(t\right)\right)\right)$
(15)

Specifically, we adopt binary cross-entropy loss function to train the graph
classification tasks in this work.

$\begin{split}\mathcal{L}_{DIIG}=&\sum_{i}\bigl{[}\tilde{z_{i}}\left(t\right)\cdot\log
y_{i}\left(t\right)\\\
&+\left(1-\tilde{z_{i}}\left(t\right)\right)\cdot\left(1-\log
y_{i}\left(t\right)\right)\bigr{]}\end{split}$ (16)

###  4.3 Federated Relevance Module

We also design the federated relevance module, in which different participants
can train a well-generalised model by considering the divergent local data
distributions.

Given $K$ participants, the local data at participant $k$ is denoted as
$\mathcal{S}^{\left(k\right)}\in\mathbb{R}^{T^{(k)}\times N\times D}$ (as
explained in Section III-A), which can be reshaped to
$\tilde{\mathcal{S}}^{\left(k\right)}=\left\\{\tilde{\boldsymbol{s}}_{t}^{\left(k\right)}\right\\}_{t=1}^{T^{(k)}}\in\mathbb{R}^{T^{(k)}\times
ND}$, and $T^{(k)}$ is the number of datapoints at participant $k$. As pointed
out in related research [33, 34], it is intractable to compute local data
distribution $p_{data}\left(\tilde{\boldsymbol{s}}^{\left(k\right)}\right)$
due to the undifferentiable marginal likelihood. To circumvent this problem,
we intend to discover a latent space where encoded representations
$\boldsymbol{z}^{\left(k\right)}$ can be learned to characterise the local
data distribution
$p_{data}\left(\tilde{\boldsymbol{s}}^{\left(k\right)}\right)$ instead.
Variational autoencoder (VAE) is considered as an ideal fit for this scenario
as it uses a multivariate Gaussian to model the distribution of the latent
space, which is defined as
$q_{\phi}\left(\boldsymbol{z}^{\left(k\right)}_{i}\mid\tilde{\boldsymbol{s}}_{i}^{\left(k\right)}\right)=\mathcal{N}\left(\boldsymbol{z}^{\left(k\right)}_{i};\mu^{\left(k\right)}_{i},\sigma^{2\left(k\right)}_{i}\boldsymbol{I}\right)$[33][34].
The variational parameter $\phi$ in the probabilistic encoder can be learnt
beforehand. The mean $\mu_{i}$ and s.d. $\sigma_{i}$ are outputs of the
probabilistic encoder.

We collect the latent representations $\boldsymbol{z}_{i}^{\left(k\right)}$ of
each data point $\tilde{\boldsymbol{s}_{i}}^{\left(k\right)}$. The
distribution representation $\boldsymbol{d}^{\left(k\right)}$ of
$\mathcal{S}^{\left(k\right)}$ can be derived from:

$\boldsymbol{d}^{\left(k\right)}=\frac{1}{T^{(k)}}\sum_{i=1}^{T^{(k)}}\boldsymbol{z}_{i}^{\left(k\right)}$
(17)

We define a learnable global distribution estimator
$g_{\theta}^{(k)}\left(\cdot\right)$ at each participant to approximate the
global data representation, the approximate global representation from
participant $k$ is calculated as:

$\hat{\boldsymbol{d}}^{\left(k\right)}=g_{\theta}^{(k)}\left(\boldsymbol{d}^{\left(k\right)}\right)$
(18)

where $\theta$ is parameterised by an MLP. The update of $\theta$ is an
integral part of the federated learning process and will be explained in the
following part.

Each participant then uploads the latent local distribution vector,
approximated global distribution vector and trainable weight sets
$\Theta^{\left(k\right)}$ of DIIG to the server side for global model update.

$\tilde{\boldsymbol{d}}=\frac{1}{K}\sum_{k=1}^{K}\hat{\boldsymbol{d}}^{\left(k\right)}$
(19)
$r^{\left(k\right)}=\frac{\exp\left(\left\|\hat{\boldsymbol{d}}^{\left(k\right)}-\tilde{\boldsymbol{d}}\right\|_{2}\right)}{\sum_{k=1}^{K}\exp\left(\left\|\hat{\boldsymbol{d}}^{\left(k\right)}-\tilde{\boldsymbol{d}}\right\|_{2}\right)}$
(20)
$\Theta=\sum_{k=1}^{K}\left(r^{\left(k\right)}\cdot\Theta^{\left(k\right)}\right)$
(21)

The server receives approximated global distribution vectors from different
participants and uses a vector aggregator to synthesise the true global data
representation $\tilde{\boldsymbol{d}}$ (Eq.19). Afterwards, the relevance
score of each participant can be quantified using any distance-based measure
like Euclidean distance in our work. $r^{(k)}$ is the relevance score of
participant $k$, it measures how far its local distribution representation is
away from the synthesised global data distribution $\tilde{\boldsymbol{d}}$ at
the server side (Eq.20). Therefore, a larger score indicates this local data
deviates from the global one, which needs to assign more attention to
calibrate. Eq.21 refers to the attentive weight aggregation to produce the
global model. The advantage of this process has two folds: 1) it reflects how
relevant each participant is from the perspective of the data distributions,
which follows the less relevant, less obtained weight strategy; 2) the
relevance score is adjusted adaptively at different communication rounds to
lead to improve the generalisation ability of the global model, in which non-
IID data distributions exist in different participants.

The server then delivers the global model $\Theta$ and synthesised global data
distribution representation $\tilde{\boldsymbol{d}}$ to each participant for a
local model update. The next communication round leverages both information
from the server. We propose a distribution-aware loss function, which combines
the loss of the updated local model and the local loss of the approximated
global distribution vectors simultaneously.

$\mathcal{L}^{\left(k\right)}=\mathcal{L}^{\left(k\right)}_{DIIG}+\emph{MSE}\left(\hat{\boldsymbol{d}}^{\left(k\right)},\tilde{\boldsymbol{d}}\right)$
(22)

The gradients w.r.t DIIG and global distribution estimator at participant $k$
can be derived from $\mathcal{L}^{\left(k\right)}$ via
$\mathbf{g}_{d}^{(k)}=\frac{\partial\mathcal{L}^{(k)}}{\partial\Theta^{(k)}}$
and
$\mathbf{g}_{e}^{(k)}=\frac{\partial\mathcal{L}^{(k)}}{\partial\theta^{(k)}}$,
where $\Theta$ and $\theta$ are trainable weight sets of DIIG and global
estimator function $g_{\theta}\left(\cdot\right)$, respectively. Afterwards,
$\theta$ and $\Theta$ can be updated at each participant:

$\theta^{(k)}=\theta^{(k)}-\gamma\mathbf{g}^{(k)}_{e}$ (23)
$\Theta^{(k)}=\Theta^{(k)}-\gamma\mathbf{g}^{(k)}_{d}$ (24)

where $\gamma$ is the learning rate. The full process of federated relevance
training algorithm is described in Algorithm 1.

Algorithm 1 Federated Relevance Training

1:the set of participant $\mathcal{K}$, $\tilde{\mathcal{S}}^{\left(k\right)}$
is the reshaped spatial-temporal data at paticipant $k$, learning rate
$\gamma$.

2:Initialise $\Theta^{\left(k\right)}$ and $\theta^{\left(k\right)}$ at each
participant

3:Compute local data latent representation $\boldsymbol{d}^{\left(k\right)}$
at each participant

4:for each round t = 1,2,…,n do

5: for each $k\in\mathcal{K}$ (in parallel) do

6:
$\hat{\boldsymbol{d}}^{\left(k\right)}=g_{\theta}^{(k)}\left(\boldsymbol{d}^{\left(k\right)}\right)$
$\triangleright$ approximate global data vector Eq.18

7: Upload $\Theta^{\left(k\right)}$, $\theta^{\left(k\right)}$ and
$\hat{\boldsymbol{d}}^{\left(k\right)}$ to the server

8: end for

9:
$\tilde{\boldsymbol{d}}=\frac{1}{K}\sum_{k=1}^{K}\hat{\boldsymbol{d}}^{\left(k\right)}$
$\triangleright$ server synthesise global data representation

10: Compute relevance scores $\boldsymbol{r}$={$r^{(1)}$, $r^{(2)}$, $...$,
$r^{(K)}$} by Eq.20

11: $\Theta=\sum_{k=1}^{K}r^{\left(k\right)}\cdot\Theta^{\left(k\right)}$
$\triangleright$ attentive aggregation

12: Server distributes $\Theta$ and $\tilde{\boldsymbol{d}}$ to each
participant

13: for each $k\in\mathcal{K}$ (in parallel) do

14: Update local model $\Theta^{\left(k\right)}$ = $\Theta$

15:
$\mathcal{L}^{\left(k\right)}=\mathcal{L}^{\left(k\right)}_{DIIG}+\emph{MSE}\left(\hat{\boldsymbol{d}}^{\left(k\right)},\tilde{\boldsymbol{d}}\right)$
$\triangleright$ compute loss for next round

16: Compute estimator’s gradient
$\mathbf{g}_{e}^{(k)}=\frac{\partial\mathcal{L}^{(k)}}{\partial\theta^{(k)}}$

17: Compute DIIG’s gradient
$\mathbf{g}_{d}^{(k)}=\frac{\partial\mathcal{L}^{(k)}}{\partial\Theta^{(k)}}$

18: $\theta^{(k)}=\theta^{(k)}-\gamma\mathbf{g}^{(k)}_{e}$

19: $\Theta^{(k)}=\Theta^{(k)}-\gamma\mathbf{g}^{(k)}_{d}$ $\triangleright$
local model update

20: end for

21:end for

##  5 Experiment

We conduct extensive experiments on real-world datasets to evaluate our
proposed framework from different perspectives. We aim to answer the following
research questions:

  * •

RQ1: How does our proposed FedRel perform compared with the baselines
(centralized implementation and FedAvg) on the basis of the same feature
transformation net and correlation module?

  * •

RQ2: How does the DIIG contribute to the performance improvement compared with
other static connectivity methods?

  * •

RQ3: Which commonly used models perform well in terms of extracting the
feature information from the spatial-temporal data?

  * •

RQ4: What is the best GNN option when being incorporated in the DIIG?

  * •

RQ5: How does the time window $w$ in DIIG affect the performance of the
framework?

Table 2: Shared parameter setup in DIIG

Name |  Setup
---|---
Readout func (GNN) | 2 linear layers, 1 output layer
Layer depth (GNN) | 2
Loss function | Cross entropy loss
Optimiser | Adam [35]
Learning rate | 1.5e-3
Dropout rate | 0.3
Weight initialiser | Xavier [36]
Communication round | 150

###  5.1 Experiment Setup

Table 3: Summary of comparison results under FedRel for ISRUC_S3 and SHL_Small
w.r.t F1 score. The best results are marked in bold and underlined.

| ISRUC_S3 | SHL_Small
---|---|---
CNN | MLP-Mixer | ConvLSTM | Bi-LSTM | Transformer | CNN | MLP-Mixer | ConvLSTM | Bi-LSTM | Transformer
GCN |  Intra Only |  $K$-NN | 0.627 | 0.702 | 0.504 | 0.591 | 0.376 | 0.659 | 0.943 | 0.960 | 0.627 | 0.596
PCC | 0.775 | 0.713 | 0.632 | 0.574 | 0.496 | 0.881 | 0.947 | 0.958 | 0.621 | 0.591
PLV | 0.723 | 0.701 | 0.579 | 0.579 | 0.487 | 0.881 | 0.958 | 0.947 | 0.631 | 0.604
Inter-Intra | DIIG$\left(w=2\right)$ | 0.804 | 0.794 | 0.703 | 0.679 | 0.612 | 0.891 | 0.972 | 0.967 | 0.804 | 0.625
DIIG$\left(w=3\right)$ | 0.814 | 0.792 | 0.681 | 0.655 | 0.618 | 0.893 | 0.967 | 0.957 | 0.788 | 0.638
DIIG$\left(w=4\right)$ | 0.785 | 0.768 | 0.625 | 0.634 | 0.609 | 0.887 | 0.952 | 0.952 | 0.773 | 0.609
GAT |  Intra Only |  $K$-NN | 0.812 | 0.770 | 0.585 | 0.571 | 0.402 | 0.902 | 0.957 | 0.967 | 0.625 | 0.599
PCC | 0.806 | 0.818 | 0.552 | 0.591 | 0.426 | 0.909 | 0.962 | 0.952 | 0.631 | 0.620
PLV | 0.816 | 0.817 | 0.563 | 0.628 | 0.420 | 0.911 | 0.962 | 0.964 | 0.651 | 0.623
Inter-Intra | DIIG$\left(w=2\right)$ | 0.837 | 0.830 | 0.682 | 0.718 | 0.630 | 0.931 | 0.981 | 0.974 | 0.894 | 0.635
DIIG$\left(w=3\right)$ | 0.855 | 0.831 | 0.712 | 0.719 | 0.628 | 0.928 | 0.973 | 0.966 | 0.888 | 0.614
DIIG$\left(w=4\right)$ | 0.827 | 0.825 | 0.693 | 0.704 | 0.615 | 0.921 | 0.958 | 0.957 | 0.886 | 0.621
GPS |  Intra Only |  $K$-NN | 0.764 | 0.679 | 0.579 | 0.541 | 0.426 | 0.866 | 0.950 | 0.958 | 0.880 | 0.596
PCC | 0.731 | 0.743 | 0.565 | 0.592 | 0.388 | 0.914 | 0.946 | 0.957 | 0.892 | 0.612
PLV | 0.804 | 0.710 | 0.622 | 0.547 | 0.424 | 0.902 | 0.952 | 0.961 | 0.886 | 0.590
Inter-Intra | DIIG$\left(w=2\right)$ | 0.808 | 0.748 | 0.632 | 0.685 | 0.602 | 0.918 | 0.957 | 0.969 | 0.902 | 0.761
DIIG$\left(w=3\right)$ | 0.792 | 0.742 | 0.641 | 0.624 | 0.552 | 0.893 | 0.948 | 0.952 | 0.895 | 0.747
DIIG$\left(w=4\right)$ | 0.787 | 0.768 | 0.647 | 0.675 | 0.583 | 0.902 | 0.945 | 0.952 | 0.891 | 0.742
GraphSAGE |  Intra Only |  $K$-NN | 0.714 | 0.721 | 0.545 | 0.570 | 0.433 | 0.887 | 0.958 | 0.961 | 0.628 | 0.642
PCC | 0.741 | 0.720 | 0.564 | 0.555 | 0.391 | 0.790 | 0.956 | 0.948 | 0.628 | 0.614
PLV | 0.793 | 0.729 | 0.563 | 0.581 | 0.453 | 0.887 | 0.958 | 0.954 | 0.625 | 0.579
Inter-Intra | DIIG$\left(w=2\right)$ | 0.829 | 0.821 | 0.727 | 0.701 | 0.596 | 0.922 | 0.968 | 0.958 | 0.683 | 0.669
DIIG$\left(w=3\right)$ | 0.820 | 0.826 | 0.715 | 0.664 | 0.618 | 0.901 | 0.957 | 0.961 | 0.639 | 0.595
DIIG$\left(w=4\right)$ | 0.798 | 0.828 | 0.683 | 0.696 | 0.603 | 0.914 | 0.955 | 0.962 | 0.639 | 0.613

####  5.1.1 Dataset

In our experiments, ISRUC_S3 [14] and SHL [15] are used as the benchmark
dataset. ISRUC_S3111https://sleeptight.isr.uc.pt/?page_id=48 collects
polysomnography (PSG) recordings in 10 channels from 10 healthy subjects
(i.e., sleep experiment participants). These PSG recordings are labeled with
five different sleep stages according to the American academy of sleep
medicine (AASM) standard [37], including Wake, N1, N2, N3 and REM. The SHL
dataset contains multi-modal signal data from a body-worn camera and from 4
smartphones being equipped at various body locations. The
SHL_Small222https://www.dropbox.com/s/gqlqugj6rpojq64/SHL-
HUAWEI_10000s.npz.zip?dl=0 is a concise version of SHL, which contains
labelled locomotion signals with different motions, including Stay Still,
Walk, Run, and Train.

####  5.1.2 Data Partition

In the experiment, we randomly select 80% as the global training data and 20%
as the global test set. For the training data of each participant, we emulate
the partial non-IID data setup in real-world and draw the respective training
data for each participant from this 80% data pool [38]. To verify the
scalability of the proposed framework, we set the number of participants to be
{2, 3, 5}, respectively. Note that the global test set is used to verify the
model performance at each communication round.

####  5.1.3 Parameter Settings

The shared parameter setup across different modules of the framework can be
found in Table 2. Specifically, we use Adam [35] as the optimiser with the
learning rate of 1.5e-3 for all trainable models in the framework. The dropout
rate [39] is set to be 0.3. We apply the same readout function for all GNN
models with the architecture of a shallow MLP with [32, 64, 64] neurons at
each layer. Note the input node embedding size is 32 and output graph
embedding size is 64. Models are collaboratively trained for 150 communication
rounds and the batch size is set to be 8 in both training and test data. All
trainable parameters are initialised through Xavier [36].

####  5.1.4 Comparison Methods

As our framework is composed of three key components, we thus examine on using
different models in feature transformation part, spatial-temporal graph
generation part, and federated learning part.

We first consider different candidates in the feature transformation net,
aiming to find the best match that is capable of deriving the high-quality
features from the raw spatial-temporal data, which could be used as the
initial node embedding for the spatial-temporal graph generation part. We
choose CNN [31] as the default feature transformation net as it has already
proven effective in our preliminary work [31]. However, we re-implement other
models, which are used to process the spatial-temporal related data in other
works, to verify their capability of extracting features under our framework:

  * •

MLP-Mixer [16] is a simple yet powerful model initially designed to capture
spatial location features for image data, and also found useful when
processing temporal signal data.

  * •

ConvLSTM [4] reports an excellent and consistent performance in capturing
spatio-temporal correlations. It replaces the fully-connected layers commonly
used in the traditional LSTM with convolutinal layers.

  * •

Bi-LSTM [5] has a structure that contains two stacked unidirectional LSTMs for
both forward and backward pass. It is heavily used in processing spatial-
temporal features.

  * •

Transformer [1] is one of the most prevalent models in recent years. Apart
from its great success in the natural language processing (NLP) field, it also
reports a promising results in uncovering highly nonlinear and dynamical
spatial-temporal dependencies.

We also explore different strategies to quantify the node correlations, which
serves as the adjacency matrix of the spatial-temporal graph. As explained in
Section IV-B, DIIG enables a trainable correlation uncovering and embedding
update simultaneously, and other three static methods are $K$-Nearest Neighbor
($K$-NN), Pearson Correlation Coefficient (PCC) and Phase Locking Value (PLV).
The different node correlation functions are described as below:

  * •

$K$-NN [40] generates the adjacency matrix that only selects the $k$-th
nearest neighbor of each node to represent the node correlation of the graph.

  * •

PCC [41] is known as the Pearson correlation function to measure similarity
between each pairs of nodes.

  * •

PLV [42] is a time-dependent node correlation function to measure signals of
each pairs of nodes.

Regarding the GNN used in the DIIG for each participant (shown in Fig. 1.a),
we compare the following models:

  * •

GCN [43] is the first order approximation to spectral GNN model.

  * •

GraphSAGE [44] is the first type of GNN that introduces sampling technique,
enabling a good model generalisation on unseen nodes.

  * •

GPS [45] introduces an adaptive sampling technique, reporting good results on
graph representation learning.

  * •

GAT [30] is the first work that incorporates the attention mechanism in GNN,
computing attention scores for all connected node pairs.

When it comes to the FL setup, we compare our proposed FedRel with different
baselines listed below. Note that only the trainable weights in DIIG unit and
the global distribution estimator at participant side contribute to the
federated learning process (as indicated in Fig. 1b), and the feature
transformation net only serves as the pre-trained model, which shares the same
parameters across different participants.

  * •

Centralized, which presents the training of DIIG without FL settings.

  * •

FedAvg [13], which introduces the unweighted weights aggregating algorithm
with full participants.

  * •

FedP [26], which selects a fraction of participants (0.6 of full participants
as default) at each communication round.

  * •

FedAtt [25], which proposes a layer-based attentive weight aggregating
algorithm. The regularisation norm is set to be 2 (default setting in the
released code), and the learning rate $\epsilon$ is set to be 1.5e-3 for a
fair comparison.

Figure 2: Comparison of the F1 of our framework with baselines FL models.
Table 4: Performance comparison of different participants (best Accuracy (ACC)
and F1 score at different communication rounds).

| # of Participant
---|---
2 | 3 | 5
ISRUC_S3 | ACC | 0.8736 | 0.8722 | 0.8698
Round | 28 | 55 | 72
F1 | 0.8558 | 0.8547 | 0.8431
Round | 33 | 52 | 73
SHL_Small | ACC | 0.9889 | 0.9883 | 0.9886
Round | 47 | 72 | 128
F1 | 0.9865 | 0.9859 | 0.9816
Round | 44 | 77 | 128

###  5.2 Experiment Results and Analysis

In this part, we evaluate the performance of our proposed framework by
conducting comprehensive experiments and analysis. It essentially helps us to
understand the significance of each each modules under different settings
separately.

####  5.2.1 Performance Analysis

The performance of FedRel and DIIG are analysed separately.
FedRel Performance (RQ1): We first compare the overall performance of our
proposed framework with several baseline FL models, as well as the centralized
setting. Note that all other modules, including the feature transformation net
and GNN model, are cherry-picked based on the best results reported in Table 3
(detailed analysis unveiled in ablation study). Herein, the framework is
composed of CNN_GAT_DIIG_FedRel ($w=3$) for ISRUC_S3 and MLP-
Mixer_GAT_DIIG_FedRel ($w=2$) for SHL_Small, both using GAT in DIIG module.
For a fair comparison, other baselines are under the same module setup.

Fig. 2 reports the performance of the proposed framework and baseline methods
in terms of the best F1 score and its changes over communication rounds. Fig.
2a and Fig. 2b show the F1 test score curve over communication rounds. It can
be observed that FedAvg and FedP experience a slightly stronger oscillation on
both datasets, while FedRel maintains a better performance throughout the
training process. Comparing with FedAtt on ISRUC_S3, FedRel demonstrates a
faster convergence speed while maintaining a higher score. When it comes to
the smaller dataset SHL_Small, it can be seen that the FedRel can converge
even faster and demonstrate a more stable performance than other baselines
(around 40 communication rounds for FedRel). In general, the numerical results
in Fig. 2 demonstrate that, under the same models setup, FedRel achieves the
best performance compared with baselines.

Regarding the best F1 score metrics, FedRel is better than the Centralized by
around 1.8% while having around 3% improvement compared with FedAvg on
ISRUC_S3 (Fig. 2c). Comparing with FedP and FedAtt, FedRel also demonstrates a
clear increase. It shows similar results on SHL_Small dataset (Fig. 2d),
whereas FedRel still achieves the best results compared with the other
baselines and even enjoys a more considerable increase compared with FedP
(around 3%). To sum up, the promising performance of FedRel is credited to the
design of distribution-driven relevance calculation, which helps facilitate
the convergence speed by adaptively calibrating the data dispersion between
each participant and the estimated global. The integrated attentive weight
aggregations on the server side leads to the correct convergence direction and
making the global model a more representative one.

We also explore the effect of the number of participants on the performance of
FedRel. As shown in Table 4, we set the number of participants to be {2, 3, 5}
in our framework and record data based on two metrics, performance
(represented by the best Accuracy and F1 Scores) and communication rounds.
Note that the training data being assigned to each participant follows the
partial non-IID data distribution. It can be clearly observed that the
performance of the framework decreases slightly with the increase in the
number of participants on both datasets, and the degradation is acceptable
(around 1% in ISRUC_S3 and negligible in SHL_Small). We also notice that the
increasing number of participants leads to more required communication rounds
when achieving the best results. The rationale is that the increase of
participants essentially results in a more strict non-IID data distribution,
thus requiring more weight updates to converge. This part of experiments
indicates the stable scalability of our proposed framework and its potential
to be deployed in a real-world environment.
DIIG Performance (RQ2): It is worth mentioning that the comparison between
DIIG and other static correlation functions is intractable as DIIG considers
the concept of inter-intra and takes at least two temporal graph snippets as
input. Static correlation functions can only draw the connectivity of nodes on
each spatial graph at one time step. As depicted in Table 3, the DIIG,
regardless of combined feature transformation nets and GNNs, and the
selections of time window $w$, achieves performance improvements on graph-
level classification tasks compared with most static correlation methods.
Intuitively, this result verifies that the inter-dependencies in the temporal
dimension contain valuable topological information from the GNN’s perspective,
which is critical and helpful for learning the respective embedding of nodes
and graphs.

####  5.2.2 Comprehensive Ablation Study

We conduct comprehensive ablation studies in this part, while taking a close
look at each module in our framework.
Spatial-temporal Data Processing (RQ3): As shown in Fig 1, the performance of
different models adopted to generate initial node features is first studied.
We implement different feature transformation models whiling having other
modules fixed for fair comparison. To be more specifically, the results of
CNN_GNN_FedRel, MLP-Mixer_GNN_FedRel, ConvLSTM_GNN_FedRel, Bi-LSTM_GNN_FedRel
and Transformer_GNN_FedRel are gathered for comparison.

It can be found in Table 3 that the initial node features generated by CNN
helps our framework achieve the best results in ISRUC_S3 dataset.
Specifically, CNN improves by 12.39% on average across different GNN models.
On the other hand, MLP-Mixer generates a better quality of initial node
features than CNN in the SHL_Small dataset, achieving an average 9.8%
improvement. It is found that for smaller spatial-temporal dataset (SHL_Small
with hours of temporal sequence with only four labels), the MLP-Mixer is able
to extract the critical features much more effective compared with other
complex models.
GNN Models for Graph Learning (RQ4): When it comes to discovering the best
match of the GNN model in DIIG, we first fix the choices of feature
transformation net for each dataset and examine the results when deploying
different GNN models. It can be observed that, in most cases, GAT achieves the
best results compared with other GNN models. It proves that the attention-
based weighted message passing in GAT also excels in the graphs containing
only a small number of nodes (10 nodes per graph in ISRUC_S3 and four nodes
per graph in SHL_Small).
Impact of Time Window $w$ (RQ5): To answer RQ5, the effect of time window $w$
in DIIG is analysed. As observed in Table 3, the DIIG generally achieves the
best results when $w=2$ or $w=3$, and it varies slightly based on the
combinations of feature transformation nets and GNN models. We take a close
look at the CNN_GAT and MLP-Mixer_GAT combinations, both of which achieve the
best results in their respective datasets. For CNN_GAT in ISRUC_S3, DIIG
reaches the best at $w=3$, while MLP-Mixer_GAT, on the other hand, attains the
most prominent result when $w=2$. This result attributes to the length of time
sequence in each dataset. In other words, ISRUC_S3 collects days of spatial-
temporal signal sequences, it is thus less sensitive to the slight increase of
the $w$ (from 2 to 3). SHL_Small, on the other hand, composes only a fraction
of the ISRUC_S3’s sequence length and is much easier to cause the overfitting
problem with the increase of $w$. Therefore, we use CNN_GAT_DIIG($w=3$) and
MLP-Mixer_GAT_DIIG($w=2$) as the ideal setup of each dataset in our
experiment.

##  6 Conclusion and Future Work

In this paper, we propose an adaptive federated relevance framework for
collaborative spatial-temporal graph learning. The feature transformation net
module first extracts the important spatial-temporal feature information from
the raw input data, followed by a dynamic inter-intral graph (DIIG) module to
generate the spatial-temporal graphs while capturing inter-dependencies and
dynamic temporal changes across these graphs. Most importantly, the federated
relevance module facilitates the collaborative training of DIIGs from
different participants, enabling an attentive weight aggregation on the basis
of diverse data distributions. The effectiveness and superiority of the
proposed framework are verified through extensive experiments with a variety
of variants and baseline methods. Detailed ablation studies also demonstrate
the design rationale of all key components. Finally, we believe that our
proposed approach provides a general framework for better exploiting the
spatial-temporal data in graph forms from different participants while
respecting data privacy at the same time. In future work, we will explore new
strategies that could take advantage of both feature and label space when it
comes to calculating the local data distribution representations.

## References

  * [1] M. Xu, W. Dai, C. Liu, X. Gao, W. Lin, G.-J. Qi, and H. Xiong, “Spatial-Temporal Transformer Networks for Traffic Flow Forecasting,” _arXiv preprint arXiv:2001.02908_ , 2020.
  * [2] J. Zhang and Y. Wu, “A New Method for Automatic Sleep Stage Classification,” _IEEE Transactions on Biomedical Circuits and Systems_ , vol. 11, pp. 1097–1110, 2017.
  * [3] X. Wang, Y. Ma, Y. Wang, W. Jin, X. Wang, J. Tang, C. Jia, and J. Yu, “Traffic Flow Prediction via Spatial Temporal Graph Neural Network,” in _Proceedings of the Web Conference_ , 2020, pp. 1082–1092.
  * [4] X. Shi, Z. Chen, H. Wang, D. Y. Yeung, W. K. Wong, and W. c. Woo, “Convolutional LSTM Network: A machine Learning Approach for Precipitation nowcasting,” in _Advances in Neural Information Processing Systems_ , 2015, pp. 802–810.
  * [5] H. Zheng, F. Lin, X. Feng, and Y. Chen, “A Hybrid Deep Learning Model With Attention-Based Conv-LSTM Networks for Short-Term Traffic Flow Prediction,” _IEEE Transactions on Intelligent Transportation Systems_ , vol. 22, pp. 6910–6920, 2020.
  * [6] Z. Jia, Y. Lin, J. Wang, X. Ning, Y. He, R. Zhou, Y. Zhou, and L.-w. H. Lehman, “Multi-View Spatial-Temporal Graph Convolutional Networks With Domain Generalization for Sleep Stage Classification,” _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , vol. 29, pp. 1977–1986, 2021\.
  * [7] A. Jain, A. R. Zamir, S. Savarese, and A. Saxena, “Structural-RNN: Deep Learning on Spatio-Temporal Graphs,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 5308–5317.
  * [8] B. Yu, H. Yin, and Z. Zhu, “Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting,” in _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence_ , 2018, pp. 3634–3640.
  * [9] Z. Wu, S. Pan, G. Long, J. Jiang, X. Chang, and C. Zhang, “Connecting The Dots: Multivariate Time Series Forecasting with Graph Neural Networks,” in _Association for Computing Machinery’s Special Interest Group on Knowledge Discovery and Data Mining_ , 2020, pp. 753–763.
  * [10] Y. Seo, M. Defferrard, P. Vandergheynst, and X. Bresson, “Structured Sequence Modeling with Graph Convolutional Recurrent Networks,” in _Proceedings of the International Conference on Neural Information Processing_ , 2018, pp. 362–373.
  * [11] S. Yan, Y. Xiong, and D. Lin, “Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition,” in _The Association for the Advance of Artificial Intelligence_ , 2018, pp. 7444–7452.
  * [12] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings _et al._ , “Advances and Open Problems in Federated Learning,” _Foundations and Trends in Machine Learning_ , pp. 1–210, 2021.
  * [13] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks From Decentralized Data,” in _Proceedings of the International Conference on Artificial Intelligence and Statistics_ , 2017, pp. 1273–1282.
  * [14] S. Khalighi, T. Sousa, J. M. Santos, and U. Nunes, “ISRUC-Sleep: A Comprehensive Public Dataset for Sleep Researchers,” _Computer Methods and Programs in Biomedicine_ , vol. 124, pp. 180–192, 2016.
  * [15] L. Wang, H. Gjoreski, M. Ciliberto, S. Mekki, S. Valentin, and D. Roggen, “Enabling Reproducible Research in Sensor-Based Transportation Mode Recognition With the Sussex-Huawei Dataset,” _IEEE Access_ , vol. 7, pp. 10 870–10 891, 2019.
  * [16] B. Huang, W. Chen, C.-L. Lin, C.-F. Juang, and J. Wang, “MLP-BP: A Novel Framework for Cuffless Blood Pressure Measurement with PPG and ECG Signals Based on MLP-Mixer Neural Networks,” _Biomedical Signal Processing and Control_ , vol. 73, p. 103404, 2022.
  * [17] D. Ienco and R. Interdonato, “Deep Multivariate Time Series Embedding Clustering via Attentive-Gated Autoencoder,” in _Pacific-Asia Conference on Knowledge Discovery and Data Mining_ , 2020, pp. 318–329.
  * [18] Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y. Philip, “A Comprehensive Survey on Graph Neural Networks,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 32, pp. 4–24, 2020.
  * [19] M. Defferrard, X. Bresson, and P. Vandergheynst, “Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering,” in _Proceedings of the International Conference on Neural Information Processing Systems_ , 2016, pp. 3837–3845.
  * [20] H. Zhang, T. Shen, F. Wu, M. Yin, H. Yang, and C. Wu, “Federated Graph Learning – A Position Paper,” _arXiv preprint arXiv:2105.11099_ , 2021\.
  * [21] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, “Federated Learning with Non-IID Data,” _arXiv preprint arXiv:1806.00582_ , 2018.
  * [22] T. Tuor, S. Wang, B. J. Ko, C. Liu, and K. K. Leung, “Overcoming Noisy and Irrelevant Data in Federated Learning,” in _International Conference on Pattern Recognition_ , 2021, pp. 5020–5027.
  * [23] N. Yoshida, T. Nishio, M. Morikura, K. Yamamoto, and R. Yonetani, “Hybrid-FL: Cooperative Learning Mechanism Using Non-IID Data in Wireless Networks,” in _IEEE International Conference on Communications_ , 2020, pp. 1–7.
  * [24] M. Shin, C. Hwang, J. Kim, J. Park, M. Bennis, and S.-L. Kim, “XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning,” _arXiv preprint arXiv:2006.05148_ , 2020.
  * [25] S. Ji, S. Pan, G. Long, X. Li, J. Jiang, and Z. Huang, “Learning Private Neural Language Modeling with Attentive Aggregation,” in _International Joint Conference on Neural Networks_ , 2019, pp. 1–8.
  * [26] C. Zhang, L. Cui, S. Yu, and J. J. Q. Yu, “A communication-efficient federated learning scheme for iot-based traffic forecasting,” _IEEE Internet of Things Journal_ , vol. 9, no. 14, pp. 11 918–11 931, 2022.
  * [27] C. Zhang, S. Zhang, J. J. Q. Yu, and S. Yu, “FASTGNN: A topological information protected federated learning approach for traffic speed forecasting,” _IEEE Transactions on Industrial Informatics_ , vol. 17, no. 12, pp. 8464–8474, 2021.
  * [28] C. Meng, S. Rambhatla, and Y. Liu, “Cross-node federated graph neural network for spatio-temporal data modeling,” in _Proceedings of the ACM SIGKDD Conference on Knowledge Discovery & Data Mining_, 2021, p. 1202–1211.
  * [29] K. Xu, W. Hu, J. Leskovec, and S. Jegelka, “How Powerful Are Graph Neural Networks?” in _International Conference on Learning Representations_ , 2019, pp. 1–17.
  * [30] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio, “Graph Attention Networks,” in _International Conference on Learning Representations_ , 2017, pp. 1–12.
  * [31] G. Lou, Y. Liu, T. Zhang, and X. Zheng, “STFL: A Temporal-Spatial Federated Learning Framework for Graph Neural Networks,” _arXiv preprint arXiv:2111.06750_ , 2021.
  * [32] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” _arXiv preprint arXiv:1607.06450_ , 2016.
  * [33] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in _International Conference on Learning Representations_ , 2014, pp. 1–14.
  * [34] S. Zhao, J. Song, and S. Ermon, “Towards deeper understanding of variational autoencoding models,” in _International Conference on Machine Learning_ , 2017, pp. 1–9.
  * [35] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” in _International Conference on Learning Representations_ , 2015, pp. 1269–1272.
  * [36] X. Glorot and Y. Bengio, “Understanding the Difficulty of Training Deep Feedforward Neural Networks,” in _International Conference on Artificial Intelligence and Statistics_ , 2010, pp. 249–256.
  * [37] Z. Jia, Y. Lin, J. Wang, R. Zhou, X. Ning, Y. He, and Y. Zhao, “GraphSleepNet: Adaptive Spatial-Temporal Graph Convolutional Networks for Sleep Stage Classification,” in _Proceedings of the International Joint Conference on Artificial Intelligence_ , 2020, pp. 1324–1330.
  * [38] T. Zhang, Z. Shen, J. Jin, X. Zheng, A. Tagami, and X. Cao, “Achieving Democracy in Edge Intelligence: A Fog-Based Collaborative Learning Scheme,” _IEEE Internet of Things Journal_ , vol. 8, pp. 2751–2761, 2020.
  * [39] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A Simple Way to Prevent Neural Networks from Overfitting,” _The Journal of Machine Learning Research_ , vol. 15, pp. 1929–1958, 2014\.
  * [40] B. Jiang, C. Ding, B. Luo, and J. Tang, “Graph-Laplacian PCA: Closed-Form Solution and Robustness,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2013, pp. 3492–3498.
  * [41] K. Pearson and A. Lee, “On the Laws of Inheritance In Man: I. Inheritance of Physical Characters,” _Biometrika_ , vol. 2, pp. 357–462, 1903.
  * [42] S. Aydore, D. Pantazis, and R. M. Leahy, “A Note on The Phase Locking Value and its Properties,” _Neuroimage_ , vol. 74, pp. 231–244, 2013.
  * [43] T. N. Kipf and M. Welling, “Semi-Supervised Classification with Graph Convolutional Networks,” in _International Conference on Learning Representations_ , 2017, pp. 1–14.
  * [44] W. Hamilton, Z. Ying, and J. Leskovec, “Inductive Representation Learning on Large Graphs,” in _Advances in Neural Information Processing Systems_ , 2017, pp. 1024–1034.
  * [45] T. Zhang, Y. Liu, X. Chen, X. Huang, F. Zhu, and X. Zheng, “GPS: A Policy-driven Sampling Approach for Graph Representation Learning,” _arXiv preprint arXiv:2112.14482_ , 2021.


# Adversarial Examples for Good: Adversarial Examples Guided Imbalanced
Learning

###### Abstract

Adversarial examples are inputs for machine learning models that have been
designed by attackers to cause the model to make mistakes. In this paper, we
demonstrate that adversarial examples can also be utilized for good to improve
the performance of imbalanced learning. We provide a new perspective on how to
deal with imbalanced data: adjust the biased decision boundary by training
with Guiding Adversarial Examples (GAEs). Our method can effectively increase
the accuracy of minority classes while sacrificing little accuracy on majority
classes. We empirically show, on several benchmark datasets, our proposed
method is comparable to the state-of-the-art method. To our best knowledge, we
are the first to deal with imbalanced learning with adversarial examples.

Index Terms—  adversarial examples, long-tail data, imbalanced learning

##  1 Introduction

In practical, most of real-world datasets have long-tailed label distributions
[1]. As a result, deep learning algorithms usually perform poorly or even
collapse when the training data suffers from heavy class-imbalance, especially
for highly skewed data [2]. Due to the imbalanced data, networks can be over-
fitting to the minority classes, which leads to the deviation of the learned
classification boundary [3].

Recent research aims to modify the data sampler to balance class frequency
during optimization (re-sampling), and modify the weights of the
classification loss to increase the importance of tail classes (re-weighting).
In general, re-weighting and re-sampling are two main approaches that are
commonly used to learn long-tailed data [3]. In contrast to previous works, we
provide a new perspective on how to deal with imbalanced data: adjust the
biased decision boundary by training with Guiding Adversarial Examples (GAEs).

Fig. 1: An illustration of Guiding Adversarial Examples.

Fig. 2: Top-1 accuracy for each class on CIFAR10.

Adversarial examples are inputs to neural networks that are deliberately
designed to cause the networks to misclassify [4, 5, 6, 7]. Prior studies of
adversarial examples have primarily focused on robustness, which indicates the
vulnerability of models against adversarial attacks [8, 9, 10]. Rather than
focusing on the robustness of models, we present an interesting use of
adversarial examples to increase the performance of models on long-tailed
data. Interestingly, in addition to attacking and reducing the accuracy of a
model, adversarial samples can also be employed to enhance its accuracy. To
begin with, we show a definition of Guiding Adversarial Examples in Figure 1,
which refers to adversarial examples in minority classes that can be
transferred to majority classes within a few steps. These examples can further
be utilized to guide the learning of the biased decision boundary.

To get examples beneficial to guiding the learning, we modify an attack scheme
to search examples closed to the biased decision boundary. For the boundary-
oriented adjustment, we generate GAEs that have just crossed the decision
boundary with a few steps to guide the learning of the model. As illustrated
in Figure 2, the accuracy on plain models (train with Cross-Entropy) drops
from head to tail, which is exactly what traditional long-tailed recognition
aims to solve. It is worth noting that our method can effectively increase the
accuracy of minority classes while sacrificing little accuracy on majority
classes.

Our main contributions are summarized as follows:

  * $\bullet$

To our best knowledge, we are the first to deal with imbalanced learning with
adversarial examples. We demonstrate that adversarial examples can also be
utilized for good to improve the performance of imbalanced learning.

  * $\bullet$

Numerous experiments demonstrate the superiority of our approach which is as
effective as the state-of-the-art algorithms on various datasets.

##  2 Related Works

###  2.1 Learning from Imbalanced Data

There have been many studies focusing on analyzing imbalanced data in recent
years [11, 1, 2]. It is usually the case that real-world data exhibits an
imbalanced distribution, and highly skewed data can adversely affect the
effectiveness of machine learning [12, 3]. Re-sampling [13] and re-weighting
[14, 12, 15] are traditional methods towards addressing imbalanced data.
Through re-weighting strategies, modern works can make networks more sensitive
to minority categories by assigning a variable weight to each class. Moreover,
two re-sampling methods, under-sampling frequent classes and over-sampling
minority classes, have been extensively discussed in previous studies. Also,
we saw success with new perspectives, such as deferred re-balancing [3]
schedule and decoupled training [16].

###  2.2 Adversarial Examples

Christian Szegedy et al [17] refer to the image which is added with small
noise that cannot be perceived by human but makes the output of neural network
change explicitly as ”adversarial examples”. For object recognition, the
predicted label of the adversarial example is different from that of the
original example. Adversarial training technique had been introduced in [4],
and adversarial examples are commonly used in optimizing the adversarial
robustness of neural networks. It is worth noting that adversarial examples
are originally utilized to attack neural networks. Aleksander Madry et al [18]
use multi-step projected gradient descent (PGD) as an attack method to
generate adversarial examples to train the model and the robustness of the
result model is improved substantially. However, adversarial examples
techniques have rarely benefited other areas. In this paper, We demonstrate
that adversarial examples can also be utilized for good to improve the
performance of imbalanced learning.

Fig. 3: An illustration of our method. By training with GAEs, the biased
decision boundary can be futher corrected.

##  3 Method

In this section, we start with the concept of guiding adversarial examples
(GAEs), and then describe the proposed method to benefit long-tailed
classification.

###  3.1 Guiding Adversarial Examples

The projected gradient descent (PGD) [18] we adopted is the most common method
for finding adversarial examples. Given $x^{0}\in\mathcal{X}$,

$x^{t+1}=\Pi_{\mathcal{B}[x^{0}]}\left(x^{t}+\alpha\,\text{sgn}(\nabla_{x^{t}}L(\theta,x^{t},y))\right)$
(1)

until a threshold is met. $x^{0}$ denotes an original example and
$\Pi_{\mathcal{B}[x^{0}]}\left(\cdot\right)$ is the projection function. Since
PGD is a multi-step method, the hyper-parameter $k$ denotes the number of
steps. we refer to adversarial examples in minority classes that can be
transferred to majority classes within a few steps as the Guiding Adversarial
Examples (GAEs). These examples can guide the learning of the biased decision
boundary while other adversarial examples have little guiding significance. It
is important to note that with too many steps, the biased decision boundary
will be over-corrected. Therefore, we limit the number of steps $k$ to a small
integer (usually less than or equal to 3).

The GAEs are generated from the tail examples which are close to the biased
decision boundary that separates minority classes from majority classes. And
we only calculate the adversarial examples for the tail examples
$(x_{tail},y_{tail})$ with $k$ steps. Let $(x,y)$ denote a mini-batch of
examples in minority classes, and $(x_{adv},y_{adv})$ is the corresponding
adversarial examples. The confidence scores for the true label and
misclassified label are represented by $f_{t}(x)$ and $f_{m}(x)$,
respectively. In this way, we use PGD to generate adversarial examples as:

$\displaystyle s=i+1,\,\text{where}\;i+1\leq k\;\text{and}$ (2)
$\displaystyle\begin{cases}\,f_{t}(x_{adv}^{i})-f_{m}(x_{adv}^{i})>0\\\
\,f_{t}(x_{adv}^{i+1})-f_{m}(x_{adv}^{i+1})<0\end{cases},$

where $s$ means the number of steps required for a successful attack. However,
the adversarial examples may be located in other minority classes, which will
restrict the effect of improving accuracy. Consequently, not all adversarial
examples among $(x_{adv},y_{adv})$ participate in the training, but the
adversarial examples $(x_{cross},y_{cross})$ with frequent classes after the
attacks. As the classification boundary becomes more balanced during training,
the number of GAEs should gradually decrease.

###  3.2 Adversarial Examples Guided Learning

GAEs are utilized to direct the learning of a convergent and more or less
unbalanced model, from which we generate GAEs, trained in some way on an
unbalanced dataset. To simplify the procedure, Our criterion is merely the
cross-entropy loss

$L_{CE}(q,p)=-\sum_{i}^{m}p(x_{i})\log q(x_{i}).$ (3)

We believe combining effective training techniques with our method will result
in better performance, and we will futher investigate it in the future. In the
following is the cross-entropy loss aiming to guide the training of the
model’s decision boundary:

$L_{cross}=L_{CE}(h(x_{cross}),y_{tail}^{{}^{\prime}})$ (4)

where $h(x)$ are the softmax output of a model on the input $x$ and
$y_{tail}^{{}^{\prime}}$ denote the original labels of $x_{cross}$.

##  4 Experiments

Table 1: Accuracy (%) on imbalanced SVHN, FashionMNIST and CIFAR100. Datasets | SVHN | FashionMNIST | CIFAR100
---|---|---|---
Imbalance ratio $\rho$ | 0.1 | 0.05 | 0.02 | 0.01 | 0.1 | 0.05 | 0.02 | 0.01 | 0.1 | 0.05 | 0.02 | 0.01
CE | 93.93 | 92.21 | 90.12 | 87.41 | 88.89 | 88.25 | 86.37 | 85.35 | 56.29 | 51.24 | 43.27 | 38.17
Ours | 94.40 | 93.18 | 91.78 | 89.57 | 89.52 | 88.95 | 87.18 | 85.85 | 58.22 | 53.19 | 45.80 | 39.93

(a) On CIFAR10.

(b) On SVHN.

Fig. 4: Comparison of accuracy on each class.

Fig. 5: T-sne visulization of standard trainning and our method On CIFAR10.
The test accuracy is 71.36% and 80.23%, respectively.

Fig. 6: The number of GAEs during training.

###  4.1 Experiment Setups

In this study, we conducted a number of experiments on four popular image
classification benchmark datasets: SVHN [19], FashionMNIST [20], CIFAR-10 and
CIFAR-100 [21] with controllable degrees of data imbalance. Follow the setting
in [3], the magnitude of the long-tailed imbalance declines exponentially with
class size. The imbalance ratio $\rho$ represents the ratio between the
samples of the least and most frequent classes. In this paper, we compare our
method to standard training (using the Cross-Entropy (CE) loss function) as
well as the state-of-the-art LDAM-DRW [3]. Despite the fact that state-of-the-
art two-stage approaches [16] have achieved high accuracy; nevertheless, the
baselines we use is sufficient for our needs, as our goal is to evaluate the
effectiveness of our method, not achieve the best possible accuracy on each
task.

Table 2: Accuracy (%) on imbalanced CIFAR-10. Imbalance ratio $\rho$ | 0.01 | 0.02 | 0.1
---|---|---|---
CE | 71.36 | 76.89 | 86.76
CE+DRW | 76.01 | 80.11 | 87.35
LDAM | 73.72 | 78.98 | 86.34
LDAM+DRW | 77.83 | 81.67 | 87.74
Ours | 80.23 | 81.85 | 87.86
Table 3: Ablation study for the effect of $k$. Steps | 1 | 3 | 5 | 7 | 9
---|---|---|---|---|---
Ours($\rho$=0.01) | 74.11 | 80.23 | 77.81 | 76.74 | 70.15
Ours($\rho$=0.02) | 76.98 | 81.85 | 79.48 | 79.01 | 75.41
Ours($\rho$=0.1) | 86.57 | 87.86 | 86.48 | 86.16 | 85.14

###  4.2 Experimental Results

We use a CNN architecture as our initial model on FashionMNIST and use
ResNet-32 on other datasets. A popular re-weighting strategy, DRW [3], is also
used in our comparisons. We first report the results on CIFAR10 dataset in
Table 2. We demonstrate that adversarial samples can also be utilized for good
to improve the performance of imbalanced learning by a large margin, e.g. the
accuracy increases from 71.36% to 80.23% when the imbalance ratio $\rho=0.01$.
We also demonstrate that our algorithm consistently outperforms the previous
state-of-the-art methods.

More detailed evaluations on other datasets can be seen in Table 1. We report
the top-1 accuracy on four datasets under various imbalanced settings. As
shown in Table 1, our method demonstrates significant improvements over the
standard training, even on larger datasets CIFAR100 with 100 classes.
Similarly, our method achieves significant improvements in accuracy by 2% for
the complex dataset CIFAR100. It is also worth mentioning that when the
training data becomes increasingly unbalanced, our method is becoming
increasingly effective. We would like to argue that this is reasonable. As the
more imbalanced data can result in a more biased decision boundary, which is
exactly what our method attempts to tackle.

Furthermore, we present the test accuracy on each classes of SVHN and CIFAR10
in Figure 4. Obviously, we find that our method can significantly increase the
accuracy of minority classes while only sacrificing little accuracy on
majority classes. A T-sne visualization of the standard training and our
method can be seen in Figure 5, which indicates that training with GAEs can
produce a more accurate decision boundary.

###  4.3 Ablation Study

Our ablation study begins by investigating the contribution of the number of
steps introduced in our method to generate GAEs. We vary the steps
$k=\\{1,3,5,7,9\\}$ and report the corresponding top-1 accuracy on imbalanced
CIFAR10 dataset in Table 3. Clearly, the number of steps required to generate
GAEs directly impacts the results of model training. A small $k=1$ will
slightly correct the classification boundary with a few GAEs. When a large
value of k is used (e.g. k=9), our method can generate too many adversarial
examples, resulting in an over-correction of the decision boundary, which
reduces the performance of the model. We also report the number of GAEs during
training for different number of steps as an additional support for our
method. In Figure 6, we demonstrate that a larger $k$ can lead to more GAEs
and during training the number of GAEs will decrease. The explanation for this
phenomenon is that the decision boundary is constantly being adjusted, and
thus the number of effective GAEs is declining.

##  5 Conclusion

In this work, we demonstrate that adversarial examples can also be utilized
for good to improve the performance of imbalanced learning. To our best
knowledge, we are the first to deal with imbalanced learning with adversarial
examples. On several benchmark datasets, our proposed method is comparable to
the state-of-the-art method.

##  6 Acknowledgement

This work was supported by the National Key Research and Development Project
of China (2021ZD0110400 No. 2018AAA0101900), National Natural Science
Foundation of China (U19B2042), The University Synergy Innovation Program of
Anhui Province (GXXT-2021-004), Zhejiang Lab (2021KE0AC02), Academy Of Social
Governance Zhejiang University, Fundamental Research Funds for the Central
Universities (226-2022-00064), Artificial Intelligence Research Foundation of
Baidu Inc., Program of ZJU and Tongdun Joint Research Lab.

## References

  * [1] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X Yu,  “Large-scale long-tailed recognition in an open world,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 2537–2546.
  * [2] Yongshun Zhang, Xiu-Shen Wei, Boyan Zhou, and Jianxin Wu,  “Bag of tricks for long-tailed visual recognition with deep convolutional neural networks,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2021, vol. 35, pp. 3447–3455.
  * [3] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma,  “Learning imbalanced datasets with label-distribution-aware margin loss,”  arXiv preprint arXiv:1906.07413, 2019.
  * [4] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy,  “Explaining and harnessing adversarial examples,”  arXiv preprint arXiv:1412.6572, 2014.
  * [5] Hao Huang, Yongtao Wang, Zhaoyu Chen, Yuze Zhang, Yuheng Li, Zhi Tang, Wei Chu, Jingdong Chen, Weisi Lin, and Kai-Kuang Ma,  “Cmua-watermark: A cross-model universal adversarial watermark for combating deepfakes,”  in Proceedings of the AAAI Conference on Artificial Intelligence, 2022, vol. 36, pp. 989–997.
  * [6] Siao Liu, Zhaoyu Chen, Wei Li, Jiwei Zhu, Jiafeng Wang, Wenqiang Zhang, and Zhongxue Gan,  “Efficient universal shuffle attack for visual object tracking,”  in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 2739–2743.
  * [7] Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, and Chao Wu,  “Towards efficient data free black-box adversarial attack,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15115–15125.
  * [8] Hadi Salman, Greg Yang, Jungshian Li, Pengchuan Zhang, Huan Zhang, Ilya P. Razenshteyn, and Sébastien Bubeck,  “Provably robust deep learning via adversarially trained smoothed classifiers,”  in NeurIPS, 2019.
  * [9] Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric P. Xing, Laurent El Ghaoui, and Michael I. Jordan,  “Theoretically principled trade-off between robustness and accuracy,” 2019.
  * [10] Zhaoyu Chen, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, and Wenqiang Zhang,  “Towards practical certifiable patch defense with vision transformer,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 15148–15158.
  * [11] Haibo He and Edwardo A Garcia,  “Learning from imbalanced data,”  IEEE Transactions on knowledge and data engineering, vol. 21, no. 9, pp. 1263–1284, 2009.
  * [12] Muhammad Abdullah Jamal, Matthew Brown, Ming-Hsuan Yang, Liqiang Wang, and Boqing Gong,  “Rethinking class-balanced methods for long-tailed visual recognition from a domain adaptation perspective,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 7610–7619.
  * [13] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer,  “Smote: synthetic minority over-sampling technique,”  Journal of artificial intelligence research, vol. 16, pp. 321–357, 2002.
  * [14] Byungju Kim and Junmo Kim,  “Adjusting decision boundary for class imbalanced learning,”  IEEE Access, vol. 8, pp. 81674–81685, 2020.
  * [15] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie,  “Class-balanced loss based on effective number of samples,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268–9277.
  * [16] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis,  “Decoupling representation and classifier for long-tailed recognition,”  arXiv preprint arXiv:1910.09217, 2019.
  * [17] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus,  “Intriguing properties of neural networks,”  arXiv preprint arXiv:1312.6199, 2013.
  * [18] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu,  “Towards deep learning models resistant to adversarial attacks,”  arXiv preprint arXiv:1706.06083, 2017.
  * [19] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng,  “Reading digits in natural images with unsupervised feature learning,”  2011\.
  * [20] Han Xiao, Kashif Rasul, and Roland Vollgraf,  “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,”  arXiv preprint arXiv:1708.07747, 2017.
  * [21] Alex Krizhevsky, Geoffrey Hinton, et al.,  “Learning multiple layers of features from tiny images,”  2009\.