# Membership Inference Attacks Against
Time-Series Models

Noam Koren<EMAIL_ADDRESS>
AI Privacy Group
IBM Research  Haifa  Israel  Abigail Goldsteen<EMAIL_ADDRESS>
AI Privacy Group
IBM Research  Haifa  Israel  Ariel Farkash<EMAIL_ADDRESS>
AI Privacy Group
IBM Research  Haifa  Israel  Guy Amit<EMAIL_ADDRESS>
AI Privacy Group
IBM Research  Haifa  Israel

###### Abstract

Analyzing time-series data that may contain personal information, particularly
in the medical field, presents serious privacy concerns. Sensitive health data
from patients is often used to train machine-learning models for diagnostics
and ongoing care. Assessing the privacy risk of such models is crucial to
making knowledgeable decisions on whether to use a model in production, share
it with third parties, or deploy it in patients’ homes. Membership Inference
Attacks (MIA) are a key method for this kind of evaluation, however time-
series prediction models have not been thoroughly studied in this context. We
explore existing MIA techniques on time-series models, and introduce new
features, focusing on the seasonality and trend components of the data.
Seasonality is estimated using a multivariate Fourier transform, and a low-
degree polynomial is used to approximate trends. We applied these techniques
to various types of time-series models, using datasets from the health domain.
Our results demonstrate that these new features enhance the effectiveness of
MIAs in identifying membership, improving the understanding of privacy risks
in medical data applications.

###### keywords:

Privacy, Machine Learning, Time-Series, Membership Inference

##  1 Introduction

There is a clear conflict between the ever-increasing interest in analyzing
personal data to enhance and improve processes, and the need to preserve the
privacy of data subjects. In the medical domain, sensitive health data from
real patients is often used to train machine learning (ML) models that aid
physicians in diagnostics and treatment. ML models are also utilized within
medical devices and applications to predict malfunctions and improve ongoing
care.

Assessing the privacy risk of such models is crucial to enable making
knowledgeable decisions on whether to use a model in production, share it with
third parties, or deploy it in patients’ homes. Privacy risk assessment is
often achieved by running membership inference attacks against the models and
measuring their success rate.

Membership inference attacks (MIA) attempt to distinguish between samples that
were part of a target model’s training data (called members) and samples that
were not (non-members), based on the model’s outputs. Many such attacks are
based on training a binary classifier as an attack model Shokri et al. (2016).
These attacks can be applied to various model types, including classification
Shokri et al. (2016), regression Truex et al. (2019), graph He et al. (2021),
and generative Hayes et al. (2017) models. However, MIA against time-series
prediction models, has not yet been properly researched.

This paper addresses this gap by evaluating existing membership inference
approaches on time-series forecasting models and introducing new features
specifically designed for these models. Our main contribution is the addition
of two novel features that exploit the trend and seasonality components of
time-series data. The trend is approximated by fitting a low-degree polynomial
and the seasonality is estimated using the Discrete Fourier Transform (DFT).

Since time series fundamentally consist of trend and seasonality components,
it is reasonable to assume that time-series models are more adept at
accurately estimating these elements in series encountered during training.
Additionally, various state-of-the-art forecasting models, such as Neural
Fourier Transform (NFT) Koren and Radinsky (2024), TimesNet Wu et al. (2023)
and N-BEATS Oreshkin et al. (2019), specifically incorporate those components
into their design. Consequently, when targeting a time-series prediction
model, there is a significant likelihood that the model will precisely
estimate the series’ seasonality and trend of its training data, providing a
strategic advantage in MIAs. This underscores the importance of considering
these features when assessing the vulnerability of time-series models.

The impact of adding seasonality and trend as input features to MIA models is
empirically evaluated by testing different combinations of existing and new
features. The evaluation is performed on six time-series prediction models,
using two medical datasets. The results demonstrate significant improvements
across multiple prediction horizons, ranging from 3% to 26%, confirming the
efficacy of the proposed attack features. This is an important first step
towards proper privacy assessment methods for time-series models, which have
so far been mostly overlooked.

##  2 Background

###  2.1 Time-Series Forecasting models

Time series forecasting has evolved significantly, initially relying on linear
models like ARIMA Zhang (2003) and Exponential Smoothing Gardner Jr (1985).
However, with deep learning advancements, neural network architectures such as
LSTM Yu et al. (2019) and GRU Dey and Salem (2017) showed superior performance
over traditional methods.

Recently, Convolutional Neural Networks (CNNs) Alzubaidi et al. (2021) and
Temporal Convolutional Networks (TCNs) Hewage et al. (2020) have demonstrated
state-of-the-art results. The Transformer architecture Wen et al. (2022) was
also adapted for forecasting, with models like AutoFormer Wu et al. (2021) and
FEDformer Zhou et al. (2022) as leading architectures. However, Zeng et al.
(2022) proposed DLinear, a simple linear model, challenging the efficacy of
transformers. The subsequent models TimesNet Wu et al. (2023), and PatchTST
Nie et al. (2022) improved upon DLinear, while the NFT model Koren and
Radinsky (2024) emerged as a top-performing multivariate time-series model.

Multidimensional Fourier Transform. The Fourier Transform has been widely used
in time series analysis to identify periodic patterns or cycles in the data Yi
et al. (2023). By converting time-series data into the frequency domain, one
can identify the main frequencies at which these cycles occur Nussbaumer and
Nussbaumer (1982).

Several forecasting models use Fourier Transforms for better performance.
Autoformer employs Fast Fourier Transform for autocorrelation Wu et al.
(2021), FEDformer focuses on key frequencies Zhou et al. (2022), and the
Fourier Neural Operator approximates partial differential equations operators
with Fourier Transforms Li et al. (2020). TimesNet uses Fourier Transforms for
feature decomposition to capture periodic patterns Wu et al. (2023).

The _Multidimensional Fourier Transform (MFT)_ Tolimieri et al. (2012) extends
the traditional Fourier Transform to handle multi-dimensional data. We drew
inspiration from the Neural Fourier Transform (NFT) Koren and Radinsky (2024),
and used a 2-dimensional Discrete Fourier Transform (DFT) to extract the
seasonality of the time series.

###  2.2 Membership Inference Attacks

Membership inference attacks (MIAs) represent a significant privacy threat in
machine learning. In these attacks, an adversary aims to determine whether a
specific data record, $x$, was included in the training set of a model, $D$.
If successful, the attack can reveal sensitive information about individuals,
such as their medical history, financial status, or personal preferences.
Moreover, membership inference attacks can also be used to identify
individuals who are part of a specific group or community, potentially leading
to discrimination, stigma, or even physical harm.

Formally, given access to a machine learning model $M$, the attacker seeks to
ascertain the membership of a data sample $x$ in $D$, i.e; to check if $x\in
D$. To this end the attacker typically analyze $M$’s outputs, and produce
numeric characteristics, i.e. features, that will enable it to distinguish
member of the training data from none members. Such features, include the
$M$’s loss Shokri et al. (2016) given the sample, the log-probabilities
Carlini et al. (2022b) and the outputs entropy.

In the context of time-series forecasting models, a malicious attacker seeks
to determine whether a specific time series was utilized in the model’s
training dataset, such as a patient’s ECG test results. This type of attack
poses a significant threat in industries like healthcare and finance, where
sensitive time-series data is frequently leveraged to develop predictive
models, and the unauthorized disclosure of such information could have severe
consequences.

##  3 Related work

In the realm of time series, Hisamoto et al. studied membership inference on
sequence-to-sequence (seq2seq) models in the context of machine translation,
where the output is a chained sequence of classifications Hisamoto et al.
(2020). This differs from medical sequence modeling whose input features and
outputs are numerical and continuous.

Pyrgelis et al. Pyrgelis et al. (2017) presented the first study on the
feasibility of membership inference attacks on aggregate location time series,
modeling the problem as a classification task to distinguish whether a target
user is part of an aggregate. Their empirical evaluation on mobility datasets
shows that MIAs are a privacy threat, influenced by the adversary’s prior
knowledge, data characteristics, number of users, and aggregation timeframe.

Similarly, Voyez et al. Voyez et al. (2022) explored the vulnerability of
aggregated time-series data to MIAs, introducing a linear programming-based
attack that leverages the correlation between the length of the published time
series and the size of the aggregated data. Their experiments demonstrate that
aggregated time series data can be highly susceptible to privacy breaches,
emphasizing the need for better privacy-preserving techniques, particularly in
the medical domain.

However, to our knowledge, risk assessment in general and MIA specifically has
not been thoroughly explored on ML models trained on numerical time-series
data. This presented us with an opportunity for novel applications and
advancements in attacking time-series models, potentially unlocking new
insights and methodologies in this area.

##  4 Methodology

###  4.1 Problem Statement

This study focuses on MIA on multivariate time-series forecasting models. We
assume that the attacker can access a complete sample that was either used in
model training or not.

In time-series data, training samples consist of data points up to time $T$
(lookback), denoted as $\mathbf{X}=[y_{1},\ldots,y_{T}]\in\mathbb{R}^{M\times
T}$, and the model predicts $H$ data points onward (horizon), denoted as
$\mathbf{Y}=[y_{T+1},\ldots,y_{T+H}]\in\mathbb{R}^{M\times H}$, where
$y_{t}\in\mathbb{R}^{M}$ for $t=1,\ldots,T+H$, and $M$ is the number of
variables. The predicted values are $\hat{\mathbf{Y}}$.

Our task is to determine if a specific sample, $\mathbf{X}$, is part of the
training data, $\mathbf{D}$, i.e; $\mathbf{X\in D}$ by comparing the real
future values, $\mathbf{Y}$, and the models predicted values,
$\hat{\mathbf{Y}}$.

###  4.2 Features for MIA

In the context of MIA, the attack features are the set of attributes or
characteristics that the attack model leverages to determine whether a given
data sample was a part of the training set (member) or not (non-member). As in
any ML model, selecting the correct attack features is critical, as they form
the basis upon which the attack model makes its predictions. An optimal set of
attack features can significantly improve the success of the attack,
potentially posing a much higher privacy risk.

Our goal is to find attack features that will provide good membership
inference results for time-series models. This involves leveraging the
characteristics of time-series data by identifying features that capture its
unique aspects. Hence, the introduced features isolate the seasonality and
trend components of the model’s prediction, contrasted with the same
components of the true data (labels).

Given that time series data inherently include components such as trend and
seasonality, models trained on such data are particularly good at capturing
those elements. This proficiency is leveraged by a variety of forecasting
models, including Neural Fourier Transform (NFT) Koren and Radinsky (2024),
TimesNet Wu et al. (2023), Fedformer Zhou et al. (2022), Autoformer Wu et al.
(2021), Fourier Neural Operator (FNO) Li et al. (2020), N-BEATS Oreshkin et
al. (2019), NeuralProphet Triebe et al. (2021), ARIMA Zhang (2003), etc., all
of which explicitly integrate these elements into their predictions. Thus,
when attacking a time-series prediction model, we aim to take advantage of the
model’s ability to accurately predict the series’ trend and seasonality, thus
offering a tactical advantage in MIAs. This highlights the critical need to
consider these characteristics when evaluating the susceptibility of time
series models to such privacy threats.

In this work, to effectively capture the seasonality, we employ the
Multidimensional Fourier Transform, which excels in extracting periodic
patterns from time-series data Musbah and El-Hawary (2019). We identify the
predominant trend through a low-degree polynomial fit, allowing us to find the
principal direction while filtering out variations Masry (1996).

####  4.2.1 Seasonality

We detect seasonality in multivariate temporal data by utilizing the
2-dimensional Discrete Fourier Transform (2-DFT) Tolimieri et al. (2012),
inspired by NFT Koren and Radinsky (2024). This method breaks down the dataset
into its fundamental frequency components, considering both the range of
variables and the timeline. This approach is essential for datasets where the
interaction between different variables can create new seasonality patterns
that are not explicitly seen in a univariate context.

Consider how 2-DFT is applied to the matrix $\mathbf{Y}\in\mathbb{R}^{M\times
H}$. The 2-DFT allows for a breakdown into two sequential 1-DFTs. The first
stage applies a 1-DFT to columns of $Y$, yielding an intermediate matrix $Z$:

$\mathbf{Z=F_{M}Y}$ (1)

where $F_{M}$ reflects the transformations applied across different variables
($M$ is the number of variables).

The subsequent phase involves a row-wise 1-DFT on $Z$, deriving the matrix
$C$:

$\mathbf{C=ZF_{H}^{\top}}$ (2)

where $F_{H}$ relates to the temporal structure of the data ($H$ denotes the
number of predicted time points).

Concisely, this can be denoted as:

$\mathbf{C}=\mathbf{F_{M}YF_{H}}^{\top}$ (3)

In this matrix representation, $C$ contains the Fourier coefficients.

Following are the Fourier matrices $F_{M}$ and $F_{H}$ that achieve the
desired Fourier transformation:

$F_{M}=\begin{bmatrix}\cos(2\pi\cdot
0\cdot\frac{0}{M})&\mathinner{\cdot\cdot}&\cos(2\pi\cdot
0\cdot\frac{M-1}{M})\\\
\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}\\\
\cos(2\pi\cdot\frac{M}{2}\cdot\frac{0}{M})&\mathinner{\cdot\cdot}&\cos(2\pi\cdot\frac{M}{2}\cdot\frac{M-1}{M})\\\
\sin(2\pi\cdot 0\cdot\frac{0}{M})&\mathinner{\cdot\cdot}&\sin(2\pi\cdot
0\cdot\frac{M-1}{M})\\\
\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}\\\
\sin(2\pi\cdot\frac{M}{2}\cdot\frac{0}{M})&\mathinner{\cdot\cdot}&\sin(2\pi\cdot\frac{M}{2}\cdot\frac{M-1}{M})\\\
\end{bmatrix}$ $F_{H}=\begin{bmatrix}\cos(2\pi\cdot
0\cdot\frac{0}{H})&\mathinner{\cdot\cdot}&\cos(2\pi\cdot
0\cdot\frac{H-1}{H})\\\
\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}\\\
\cos(2\pi\cdot\frac{H}{2}\cdot\frac{0}{H})&\mathinner{\cdot\cdot}&\cos(2\pi\cdot\frac{H}{2}\cdot\frac{H-1}{H})\\\
\sin(2\pi\cdot 0\cdot\frac{0}{H})&\mathinner{\cdot\cdot}&\sin(2\pi\cdot
0\cdot\frac{H-1}{H})\\\
\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}&\mathinner{\vbox{\hbox{.}\hbox{.}}}\\\
\sin(2\pi\cdot\frac{H}{2}\cdot\frac{0}{H})&\mathinner{\cdot\cdot}&\sin(2\pi\cdot\frac{H}{2}\cdot\frac{H-1}{H})\\\
\end{bmatrix}$

The matrix $F_{M}$, corresponding to the number of variables, $M$ in the
dataset, has dimensions $M\times M$. This matrix reflects the transformations
applied across different variables. Conversely, the matrix $F_{H}$, that
relates to the temporal structure of the data, has dimensions $H\times H$,
where $H$ denotes the number of predicted time points in each series.
Together, these matrices facilitate a comprehensive multidimensional Fourier
analysis, enabling the decomposition of the multivariate time series into
sinusoidal components.

The input features to the attack derived from this method are:

  1. 1.

Coefficients of the Fourier series corresponding to the true values:

$\mathbf{C}=\mathbf{F_{1}}^{\top}\times\mathbf{Y}\times\mathbf{F_{2}}$

  2. 2.

Coefficients of the Fourier series corresponding to the model’s predicted
values:

$\mathbf{\hat{C}}=\mathbf{F_{1}}^{\top}\times\mathbf{\hat{Y}}\times\mathbf{F_{2}}$

  3. 3.

The $L_{2}$ norm between the coefficients of the true and predicted values:

$||\mathbf{C}-\mathbf{\hat{C}}||_{2}$

####  4.2.2 Trend

Consider a multivariate time series $Y$ with $H$ time points and $M$
variables. Each variable’s series is approximated using a polynomial of degree
$d$. This approximation can be represented as:

$\mathbf{Y=P\times A}$ (4)

where $A$ is the coefficients matrix, and $P$ is the Vandermonde matrix,
constructed from the time vector $t=\frac{[0,1,\dots,H-1]}{H}$. $P$ contains
powers of $t$ up to $d-1$, has dimensions $d\times H$, and is defined as:

$P=\begin{bmatrix}1&1&\cdots&1\\\ t_{1}&t_{2}&\cdots&t_{H}\\\
t_{1}^{2}&t_{2}^{2}&\cdots&t_{H}^{2}\\\ \vdots&\vdots&\ddots&\vdots\\\
t_{1}^{d-1}&t_{2}^{d-1}&\cdots&t_{H}^{d-1}\end{bmatrix}$

where $t_{i}=\frac{i-1}{H}$ for $i=1,2,\ldots,H$.

The coefficients matrix $A$ is obtained by the least squares solution:

$\mathbf{A}=(\mathbf{P}^{T}\mathbf{P})^{-1}\mathbf{P}^{T}\mathbf{Y}$

The input features to the attack derived from this method are:

  1. 1.

Coefficients of the polynomial outlining the trend of the true values:

$\mathbf{A}=(\mathbf{P}^{T}\mathbf{P})^{-1}\mathbf{P}^{T}\mathbf{Y}$

  2. 2.

Coefficients of the polynomial outlining the trend of the model’s predicted
values:

$\mathbf{\hat{A}}=(\mathbf{P}^{T}\mathbf{P})^{-1}\mathbf{P}^{T}\mathbf{\hat{Y}}$

  3. 3.

The $L_{2}$ norm between the coefficients of the true and predicted values:

$||\mathbf{A}-\mathbf{\hat{A}}||_{2}$

####  4.2.3 Mean Absolute Scaled Error (MASE)

Additionally, this study investigates incorporating the Mean Absolute Scaled
Error (MASE) metric Hyndman and Koehler (2006) as a feature for the attack
model. MASE is a scaled measure of the accuracy of forecasts. It compares the
mean absolute error of the forecast to the mean absolute error of a naïve
forecasting method.

MASE
$\displaystyle=\frac{\frac{1}{H}\sum_{i=1}^{H}|y_{T+i}-\hat{y}_{T+i}|}{\frac{1}{H-1}\sum_{i=2}^{H}|y_{T+i}-y_{T+i-1}|}$

This metric is extended to the multivariate case by averaging the respective
univariate MASE values across all variables.

##  5 Experimental Setup

###  5.1 Threat Model & MIA Attack Setup

Continuing the line of privacy assessment works Shachor et al. (2023); Amit et
al. (2024); Anderson et al. (2024), this paper assumes a gray-box threat
model. In the assumed threat model, the attacker has access to the training
data of the model and a set of samples that were not used for training. The
access to this data, allows estimating a worst case privacy risk for a model
before deployment, without the need of developing shadow models Shokri et al.
(2016).

To execute the attack, we leverage a privacy risk assessment framework Shachor
et al. (2023), which builds upon recent breakthroughs in membership inference
attacks (MIAs) Carlini et al. (2022a); Shokri et al. (2016). This framework
harnesses a diverse set of input features extracted from the target model’s
inputs and outputs. Through an exhaustive grid search, it systematically
explore various attack model architectures, hyperparameters, and preprocessing
techniques to identify the optimal configuration that yields maximum attack
performance.

The attack models trained by the risk assessment framework used different
combinations of the following features:

  1. 1.

Seasonality (includes all three seasonality features)

  2. 2.

Trend (includes all three trend features)

  3. 3.

Trend and Seasonality

  4. 4.

Mean Absolute Scaled Error (MASE)

  5. 5.

Mean Squared Error (MSE)

  6. 6.

MSE and MASE

  7. 7.

All four features: Seasonality, Trend, MSE, and MASE

In this evaluation, the framework was use to perform five attack instances,
each with three runs. An instance refers to executing the entire attack
optimization process on a different random data sample, while the runs involve
different splits of the data sample to fit and infer the attack model within
each instance. For each instance, a sample of 450 members and 450 non-members
was chosen at random. Results were averaged across all runs and instances.

###  5.2 Datasets

In this evaluation, two multivariate time-series medical datasets were used:

  * •

EEG: 36-lead EEG database, which contains more than 1000 EEG recordings dating
from 2002 to the present, sampled at a frequency of 250 Hz Obeid and Picone
(2016). Our subset includes data from 32 patients, and the first 3-leads for
each patient.

  * •

ECG: Georgia 12-Lead ECG Challenge Database, curated by Emory University
Goldberger et al. (2000). The complete database contains ECG recordings of
over 10,000 individuals, sampled at a frequency of 500 Hz, and collected from
various healthcare settings worldwide. Our subset features ECG time-series
data from 600 individuals.

Data preprocessing included outlier removal using the Interquartile Range
method, imputation of missing values via mean substitution, and data
standardization.

For both datasets, The data was partitioned into three distinct subsets: 42.5%
of the patients were used for training the model, 15% for validation, and the
remaining 42.5% were reserved as non-member data points for the attack model.
The validation set was used to tune the models parameters, thus creating
strong models to attack. The non-member data points remained uninvolved in
training or validating the models but were used in subsequent attack
experiments. Additionally, the data was split into lookbacks and horizons
using the sliding window approach. Statistics on the datasets can be found in
Table 1.

Table 1: Datasets Statistics Dataset | Num of Variables | Timesteps | Lookback | Prediction Horizons
---|---|---|---|---
EEG | 3 | 9620519 | 100 | 1, 5, 10, 15, 20
ECG | 12 | 2393563 | 100 | 1, 5, 10, 15, 20, 25, 30

###  5.3 Models

We performed attacks against various state-of-the-art time-series forecasting
architectures:

  * •

Neural Fourier Transform (NFT): Configured with Fourier granularity of 8 for
the seasonality blocks and a polynomial degree of 4 for the trend blocks,
comprising 2 blocks per stack Koren and Radinsky (2024).

  * •

TimesNet: An advanced neural network with the model dimension set to 16 and a
dropout rate of 0.1 Wu et al. (2023).

  * •

PatchTST: This transformer model included one encoder and decoder layer, a
model dimension of 16, and a dropout rate of 0.1 Nie et al. (2022).

  * •

DLinear: Consists of Linear layers, featured a dimension of 16 and a dropout
rate of 0.1 Zeng et al. (2022).

  * •

Temporal Convolutional Network (TCN): Configured with channels set to [2, 2],
a kernel size of 2, and a dropout rate of 0.2 Hewage et al. (2020).

  * •

Long Short Term Memory (LSTM): Featured a 2-layer structure with hidden
dimensions set to 50 Yu et al. (2019).

In Table 2, the number of parameters for each model is detailed. For all
models, the Mean Squared Error (MSE) loss was utilized during model training.
Figure 1 presents the performance (MSE) of each model on the test set (non-
members) for different prediction horizons.

Table 2: Approximate Number of Model Parameters Model | DLinear | TCN | LSTM | NFT | TimesNet | PatchTST
---|---|---|---|---|---|---
# Parameters | $8\times 10^{2}$ | $1\times 10^{3}$ | $34\times 10^{3}$ | $13\times 10^{4}$ | $16\times 10^{4}$ | $3\times 10^{6}$

$0$$10$$20$$5\cdot
10^{-2}$$0.1$HorizonMSEEEG$0$$10$$20$$30$$0$$0.2$$0.4$HorizonMSEECGNFTTimesNetPatchTSTDLinearTCNLSTM

Figure 1: MSE values of models with varying horizons on ECG and EEG datasets

##  6 Results

In all experiments, the MSE loss-based attack is used as the baseline and
compared to the new features.

###  6.1 AUC ROC Results

Figures 2 and 3 highlight the strong performance of the seasonality and trend
features across various time-series models and datasets. The highest AUC ROC
values were achieved with feature combinations that included the trend or
seasonality features, outperforming the MSE-only attack. For the EEG dataset,
when looking at the different models, the improvement percentage of the best-
performing feature combination compared to the MSE-only attack, averaged
across horizons, ranged from 8.44% (with average std 0.007) to 26.41% (with
average std 0.006). For the ECG dataset, improvements ranged from 2.97% (with
average std 0.008) to 24.55% (with averaged std 0.007). The TimesNet model
showed the best improvements in both datasets.

The MASE feature achieved the lowest attack performance, however, the
combination of MASE and MSE generally surpassed attacks that use the MSE
feature alone.

Overall, the results indicate that the Seasonality and Trend features provide
a robust solution for membership inference attacks for time-series forecasting
models. Its consistent performance across different models, datasets, and
horizons highlights its effectiveness.

Analysis of Attack Performance by Prediction Horizon. We analyzed the relative
attack AUC ROC as a function of the prediction horizon. To this end, we
computed, for each dataset, the correlation between the prediction horizon
(previously denoted by $H$) and the improvement percentage (MSE-only attack
relative to the highest attack value). See Table 3 for the resulting
correlations. We consistently observed a positive correlation for all the
models except for NFT and TCN on the ECG dataset. We assume that the negative
correlation can be traced back to the difference in architecture. Notably, the
NFT and TCN models both are based on convolutional layers, which capture the
trend and seasonality in a different manner. The results presented in the
table highlight the greater improvement in vulnerability of time series
prediction models to attacks as the prediction horizon increases, compared to
the MSE baseline attack. This suggests an enhanced attack vector against these
models, which is not present in other model architectures.

Table 3: Correlation between Prediction Horizons and Improvement Percentages Model | Correlation EEG | Correlation ECG
---|---|---
PatchTST | 0.859 | 0.942
TimesNet | 0.796 | 0.732
DLinear | 0.883 | 0.177
LSTM | 0.482 | 0.376
TCN | 0.780 | -0.150
NFT | 0.089 | -0.931

Analysis of Attack Performance by Model. PatchTST emerged as the most
vulnerable model, achieving an AUC ROC of 1 for high horizons, indicating it
is highly susceptible to attacks. In contrast, DLinear was the least
vulnerable and hardest to attack. Notably, PatchTST has the highest number of
parameters (3,000,000), while DLinear has the fewest (800). The vulnerability
of the other models varied across the datasets.

$0$$10$$20$$0.6$$0.8$HorizonAUC
ROCNFT$0$$10$$20$$0.5$$0.6$$0.7$$0.8$HorizonAUC
ROCTimesNet$0$$10$$20$$0.6$$0.8$$1$HorizonAUC
ROCPatchTST$0$$10$$20$$0.5$$0.55$$0.6$$0.65$HorizonAUC
ROCDLinear$0$$10$$20$$0.6$$0.8$HorizonAUC
ROCTCN$0$$10$$20$$0.6$$0.8$HorizonAUC ROCLSTMAllTrend and
SeasonalityTrendSeasonalityMASEMSEMSE and MASE

Figure 2: AUC ROC Performance Across Different Models on EEG Data

$0$$10$$20$$30$$0.5$$0.55$$0.6$HorizonAUC
ROCNFT$0$$10$$20$$30$$0.6$$0.8$$1$HorizonAUC
ROCTimesNet$0$$10$$20$$30$$0.6$$0.8$$1$HorizonAUC
ROCPatchTST$0$$10$$20$$30$$0.5$$0.55$$0.6$$0.65$HorizonAUC
ROCDLinear$0$$10$$20$$30$$0.5$$0.55$$0.6$HorizonAUC
ROCTCN$0$$10$$20$$30$$0.5$$0.55$$0.6$HorizonAUC ROCLSTMAllTrend and
SeasonalityTrendSeasonalityMASEMSEMSE and MASE

Figure 3: AUC ROC Performance Across Different Models on ECG Data

###  6.2 TPR Results

In addition to AUC ROC, we measured the attack’s True Positive Rate at a False
Positive Rate of 1% for a fixed horizon of 5. The results are presented in
Tables 4 and 5.

For the EEG dataset, using the seasonality feature with the PatchTST model
substantially improved the TPR, showing a 9x increase. Significant
improvements were noted across NFT, TimesNet, and LSTM models with the trend
feature, ranging from 2.1x to 4x. The TCN model demonstrated a 2.1x
improvement with the combination of trend and seasonality, while the DLinear
model showed similar gains with both combined and standalone seasonality
features.

In the ECG dataset, the trend feature and its combination with seasonality in
the NFT model led to a 3x improvement in TPR. The seasonality feature provided
notable increases for the TimesNet and PatchTST models, with up to 11.6x
enhancement.

Table 4: Average TPR at FPR 1% for EEG Dataset at Horizon 5 Model | MSE | MASE | MSE and | Trend | Seasonality | Trend and | All
---|---|---|---|---|---|---|---
|  |  | MASE |  |  | Seasonality |
NFT | 0.20 | 0.06 | 0.21 | 0.42 | 0.37 | 0.41 | 0.38
TimesNet | 0.02 | 0.01 | 0.03 | 0.08 | 0.06 | 0.05 | 0.04
PatchTST | 0.07 | 0.02 | 0.06 | 0.28 | 0.63 | 0.39 | 0.35
DLinear | 0.03 | 0.02 | 0.04 | 0.06 | 0.07 | 0.07 | 0.06
TCN | 0.23 | 0.04 | 0.23 | 0.40 | 0.42 | 0.43 | 0.35
LSTM | 0.18 | 0.10 | 0.20 | 0.44 | 0.39 | 0.38 | 0.39
Table 5: Average TPR at FPR 1% for ECG Dataset at Horizon 5 Model | MSE | MASE | MSE and | Trend | Seasonality | Trend and | All
---|---|---|---|---|---|---|---
|  |  | MASE |  |  | Seasonality |
NFT | 0.01 | 0.02 | 0.02 | 0.03 | 0.02 | 0.03 | 0.02
TimesNet | 0.03 | 0.05 | 0.03 | 0.14 | 0.35 | 0.35 | 0.36
PatchTST | 0.34 | 0.03 | 0.32 | 0.52 | 0.99 | 0.51 | 0.66
DLinear | 0.01 | 0.01 | 0.02 | 0.03 | 0.03 | 0.03 | 0.03
TCN | 0.01 | 0.01 | 0.01 | 0.02 | 0.02 | 0.02 | 0.02
LSTM | 0.02 | 0.02 | 0.01 | 0.02 | 0.02 | 0.01 | 0.02

###  6.3 Ablation Study

In this ablation study, we examined the effects of incorporating the predicted
labels as a feature in the attack model, as is common with MIA against
classical ML models. The predicted labels feature was added to every feature
combination we used and tested independently. The results on the EEG and ECG
datasets are illustrated in Figure 4. The black curve, labeled ”Benchmark
Limit”, represents the upper bound of the AUC ROC values for the feature
combinations without predicted labels, as presented in the previous section.

The analysis of the results did not reveal a consistent trend across horizons
and datasets. For the EEG dataset, the addition of the predicted labels
feature consistently under-performed compared to the feature combinations
without the predicted labels.

In contrast, the results for the ECG dataset were more varied. Although the
addition of the predicted labels feature often surpassed the Benchmark Limit
across different models and horizons, the results varied widely and no feature
combination consistently exceeded the Benchmark Limit across all horizons.

Overall, the results indicate that while the predicted labels feature can in
some cases improve the performance of the attack model, its impact is not
straightforward and depends on the specific model, dataset and horizon. The
figures highlight the complexity and variability of incorporating predicted
labels in this context.

$0$$10$$20$$0.7$$0.8$$0.9$HorizonAUC ROCNFT -
EEG$0$$10$$20$$0.7$$0.8$$0.9$HorizonAUC ROCTCN -
EEG$0$$10$$20$$0.7$$0.8$$0.9$HorizonAUC ROCLSTM -
EEG$0$$10$$20$$30$$0.55$$0.6$$0.65$$0.7$HorizonAUC ROCNFT -
ECG$0$$10$$20$$30$$0.55$$0.6$$0.65$HorizonAUC ROCTCN -
ECG$0$$10$$20$$30$$0.5$$0.55$$0.6$$0.65$$0.7$HorizonAUC ROCLSTM - ECGBenchmark
LimitAllPredicted LabelsTrend and SeasonalityTrendSeasonalityMASEMSEMSE and
MASE

Figure 4: Addition of Predicted Labels feature to each feature combination

##  7 Conclusion and Future Work

This paper is the first to explore membership inference attacks on numeric
time-series machine-learning models. Our primary contribution is the
introduction of two novel features, Trend and Seasonality, derived from low-
degree polynomial fitting and Multivariate Fourier Transform respectively.
These features enhance MIA models by leveraging the inherent characteristics
of time-series data, improving the identification of member samples.

Various state-of-the-art forecasting models incorporate those components into
their design. Consequently, when targeting a time-series prediction model,
there is a significant likelihood that the model will precisely estimate the
series’ Seasonality and Trend of its training data, providing a strategic
advantage in MIAs.

The effectiveness of these features was tested on six diverse models using two
medical datasets. The Trend and Seasonality features showed superior accuracy,
with a 3% to 26% improvement in attack AUC ROC scores over traditional
features across various horizons.

We plan to explore additional MIA scenarios, including models pre-trained on
many patients and fine-tuned for home monitoring of a specific patient, to see
if such an attack can expose the original training data. Additionally, we aim
to investigate user-level attacks that exploit the fact that multiple samples
in the training set belong to the same person.

## References

  * Alzubaidi et al. (2021) Laith Alzubaidi, Jinglan Zhang, Amjad J Humaidi, Ayad Al-Dujaili, Ye Duan, Omran Al-Shamma, José Santamaría, Mohammed A Fadhel, Muthana Al-Amidie, and Laith Farhan.  Review of deep learning: concepts, cnn architectures, challenges, applications, future directions.  _Journal of big Data_ , 8:1–74, 2021.
  * Amit et al. (2024) Guy Amit, Abigail Goldsteen, and Ariel Farkash.  Sok: Reducing the vulnerability of fine-tuned language models to membership inference attacks.  2024\.  URL https://arxiv.org/abs/2403.08481.
  * Anderson et al. (2024) Maya Anderson, Guy Amit, and Abigail Goldsteen.  Is my data in your retrieval database? membership inference attacks against retrieval augmented generation.  _arXiv preprint arXiv:2405.20446_ , 2024.
  * Carlini et al. (2022a) Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer.  Membership inference attacks from first principles.  In _2022 IEEE Symposium on Security and Privacy (SP)_ , pages 1897–1914. IEEE, 2022a.
  * Carlini et al. (2022b) Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer.  Membership inference attacks from first principles.  In _2022 IEEE Symposium on Security and Privacy (SP)_ , pages 1897–1914. IEEE, 2022b.
  * Dey and Salem (2017) Rahul Dey and Fathi M Salem.  Gate-variants of gated recurrent unit (gru) neural networks.  In _2017 IEEE 60th international midwest symposium on circuits and systems (MWSCAS)_ , pages 1597–1600. IEEE, 2017.
  * Gardner Jr (1985) Everette S Gardner Jr.  Exponential smoothing: The state of the art.  _Journal of forecasting_ , 4(1):1–28, 1985.
  * Goldberger et al. (2000) A. L. Goldberger, L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C. K. Peng, and H. E. Stanley.  PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals.  _Circulation_ , 101(23):e215–e220, 2000.
  * Hayes et al. (2017) Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro.  Logan: Membership inference attacks against generative models.  _arXiv preprint arXiv:1705.07663_ , 2017.
  * He et al. (2021) Xinlei He, Rui Wen, Yixin Wu, Michael Backes, Yun Shen, and Yang Zhang.  Node-level membership inference attacks against graph neural networks.  _arXiv preprint arXiv:2102.05429_ , 2021.
  * Hewage et al. (2020) Pradeep Hewage, Ardhendu Behera, Marcello Trovati, Ella Pereira, Morteza Ghahremani, Francesco Palmieri, and Yonghuai Liu.  Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station.  _Soft Computing_ , 24:16453–16482, 2020.
  * Hisamoto et al. (2020) Sorami Hisamoto, Matt Post, and Kevin Duh.  Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?  _Transactions of the Association for Computational Linguistics_ , 8:49–63, 01 2020.  ISSN 2307-387X.  10.1162/tacl_a_00299.  URL https://doi.org/10.1162/tacl_a_00299.
  * Hyndman and Koehler (2006) Rob J Hyndman and Anne B Koehler.  Another look at measures of forecast accuracy.  _International journal of forecasting_ , 22(4):679–688, 2006.
  * Koren and Radinsky (2024) Noam Koren and Kira Radinsky.  Interpretable multivariate time series forecasting using neural fourier transform.  _arXiv preprint arXiv:2405.13812_ , 2024.
  * Li et al. (2020) Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar.  Fourier neural operator for parametric partial differential equations.  _arXiv preprint arXiv:2010.08895_ , 2020.
  * Masry (1996) Elias Masry.  Multivariate regression estimation local polynomial fitting for time series.  _Stochastic Processes and their Applications_ , 65(1):81–101, 1996.
  * Musbah and El-Hawary (2019) Hmeda Musbah and Mo El-Hawary.  SARIMA model forecasting of short-term electrical load data augmented by fast fourier transform seasonality detection.  In _2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE)_ , pages 1–4. IEEE, 2019.
  * Nie et al. (2022) Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam.  A time series is worth 64 words: Long-term forecasting with transformers.  _arXiv preprint arXiv:2211.14730_ , 2022.
  * Nussbaumer and Nussbaumer (1982) Henri J Nussbaumer and Henri J Nussbaumer.  _The fast Fourier transform_.  Springer, 1982.
  * Obeid and Picone (2016) Iyad Obeid and Joseph Picone.  The Temple University Hospital EEG data corpus.  _Frontiers in neuroscience_ , 10:196, 2016.
  * Oreshkin et al. (2019) Boris N Oreshkin, Dmitri Carpov, Nicolas Chapados, and Yoshua Bengio.  N-beats: Neural basis expansion analysis for interpretable time series forecasting.  _arXiv preprint arXiv:1905.10437_ , 2019.
  * Pyrgelis et al. (2017) Apostolos Pyrgelis, Carmela Troncoso, and Emiliano De Cristofaro.  Knock knock, who’s there? membership inference on aggregate location data.  _arXiv preprint arXiv:1708.06145_ , 2017.
  * Shachor et al. (2023) Shlomit Shachor, Natalia Razinkov, and Abigail Goldsteen.  Improved membership inference attacks against language classification models.  In _arXiv:2310.07219_ , 2023.  URL https://doi.org/10.48550/arXiv.2310.07219.
  * Shokri et al. (2016) Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models (s&p’17).  2016\.
  * Tolimieri et al. (2012) Richard Tolimieri, Myoung An, and Chao Lu.  _Mathematics of multidimensional Fourier transform algorithms_.  Springer Science & Business Media, 2012.
  * Triebe et al. (2021) Oskar Triebe, Hansika Hewamalage, Polina Pilyugina, Nikolay Laptev, Christoph Bergmeir, and Ram Rajagopal.  Neuralprophet: Explainable forecasting at scale.  _arXiv preprint arXiv:2111.15397_ , 2021.
  * Truex et al. (2019) Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei.  Demystifying membership inference attacks in machine learning as a service.  _IEEE transactions on services computing_ , 14(6):2073–2089, 2019.
  * Voyez et al. (2022) Antonin Voyez, Tristan Allard, Gildas Avoine, Pierre Cauchois, Elisa Fromont, and Matthieu Simonin.  Membership inference attacks on aggregated time series with linear programming.  In _SECRYPT 2022 - 19th International Conference on Security and Cryptography_ , pages 193–204. Springer, 2022.
  * Wen et al. (2022) Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun.  Transformers in time series: A survey.  _arXiv preprint arXiv:2202.07125_ , 2022.
  * Wu et al. (2021) Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long.  Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting.  _Advances in neural information processing systems_ , 34:22419–22430, 2021.
  * Wu et al. (2023) Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long.  Timesnet: Temporal 2d-variation modeling for general time series analysis.  In _The eleventh international conference on learning representations_ , 2023.
  * Yi et al. (2023) Kun Yi, Qi Zhang, Longbing Cao, Shoujin Wang, Guodong Long, Liang Hu, Hui He, Zhendong Niu, Wei Fan, and Hui Xiong.  A survey on deep learning based time series analysis with frequency transformation.  _arXiv preprint arXiv:2302.02173_ , 2023.
  * Yu et al. (2019) Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang.  A review of recurrent neural networks: Lstm cells and network architectures.  _Neural computation_ , 31(7):1235–1270, 2019.
  * Zeng et al. (2022) Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu.  Are transformers effective for time series forecasting?  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 37, pages 11121–11128, 2022.
  * Zhang (2003) G Peter Zhang.  Time series forecasting using a hybrid arima and neural network model.  _Neurocomputing_ , 50:159–175, 2003.
  * Zhou et al. (2022) Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin.  Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting.  In _International conference on machine learning_ , pages 27268–27286. PMLR, 2022.


* [146] J. W. Smith, J. E. Everhart, W. Dickson, W. C. Knowler, and R. S. Johannes, “Using the adap learning algorithm to forecast the onset of diabetes mellitus,” in _Proceedings of the annual symposium on computer application in medical care_. American Medical Informatics Association, 1988, p. 261.
  * [147] Kaggle, “Avazu dataset,” https://www.kaggle.com/c/avazu-ctr-prediction.
  * [148] Criteo-Labs., “Criteo dataset,” https://labs.criteo.com/2014/02/download-kaggle-display-advertising-challenge-dataset/.
  * [149] M. F. Duarte and Y. H. Hu, “Vehicle classification in distributed sensor networks,” _Journal of Parallel and Distributed Computing_ , vol. 64, no. 7, pp. 826–838, 2004.
  * [150] D. Dua and C. Graff, “UCI machine learning repository,” "http://archive.ics.uci.edu/ml", 2017.
  * [151] J. A. Blackard and D. J. Dean, “Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables,” _Computers and electronics in agriculture_ , vol. 24, no. 3, pp. 131–151, 1999.
  * [152] T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng, “NUS-WIDE: A real-world web image database from national university of singapore,” in _Proc. of ACM Conf. on Image and Video Retrieval (CIVR’09)_ , Santorini, Greece., Jul. 2009.
  * [153] M. van Breukelen, R. P. Duin, D. M. Tax, and J. Den Hartog, “Handwritten digit recognition by combined classifiers,” _Kybernetika_ , vol. 34, no. 4, pp. 381–386, 1998.
  * [154] PASCAL-Challenge-2008, “epsilon dataset,” https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/.
  * [155] P. Mooney, “Breast histopathology images,” https://www.kaggle.com/datasets/paultimothymooney/breast-histopathology-images.
  * [156] J. Irvin, P. Rajpurkar, M. Ko, Y. Yu, S. Ciurea-Ilcus, C. Chute, H. Marklund, B. Haghgoo, R. Ball, K. Shpanskaya _et al._ , “Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison,” in _Proceedings of the AAAI conference on artificial intelligence_ , vol. 33, no. 01, 2019, pp. 590–597.
  * [157] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao, “3D ShapeNets: A deep representation for volumetric shapes,” in _2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , Jun. 2015, pp. 1912–1920.
  * [158] P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Galligher, and T. Eliassi-Rad, “Collective classification in network data,” _AI magazine_ , vol. 29, no. 3, pp. 93–93, 2008.
  * [159] S. Rakshit, “Yahoo answers dataset,” https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset.
  * [160] S. S. Keerthi, D. DeCoste, and T. Joachims, “A modified finite newton method for fast solution of large scale linear svms.” _Journal of Machine Learning Research_ , vol. 6, no. 3, 2005.
  * [161] E. Shmueli and T. Tassa, “Secure multi-party protocols for item-based collaborative filtering,” in _Proceedings of the Eleventh ACM Conference on Recommender Systems_ , ser. RecSys ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 89–97.
  * [162] K. Atarashi and M. Ishihata, “Vertical federated learning for higher-order factorization machines,” in _Advances in Knowledge Discovery and Data Mining_ , K. Karlapalem, H. Cheng, N. Ramakrishnan, R. K. Agrawal, P. K. Reddy, J. Srivastava, and T. Chakraborty, Eds. Cham: Springer International Publishing, 2021, pp. 346–357.
  * [163] J. Cui, C. Chen, L. Lyu, C. Yang, and W. Li, “Exploiting data sparsity in secure cross-platform social recommendation,” _Advances in Neural Information Processing Systems_ , vol. 34, pp. 10 524–10 534, 2021.
  * [164] J. Zhang and Y. Jiang, “A vertical federation recommendation method based on clustering and latent factor model,” in _2021 International Conference on Electronic Information Engineering and Computer Science (EIECS)_. IEEE, 2021, pp. 362–366.
  * [165] H. Yuan, C. Ma, Z. Zhao, X. Xu, and Z. Wang, “A privacy-preserving oriented service recommendation approach based on personal data cloud and federated learning,” in _2022 IEEE International Conference on Web Services (ICWS)_. IEEE, 2022, pp. 322–330.
  * [166] J. Cai, Y. Liu, X. Liu, J. Li, and H. Zhuang, “Privacy-preserving federated cross-domain social recommendation,” _arXiv preprint arXiv:2206.03200_ , 2022\.
  * [167] F. Cai, “Bytedance breaks federated learning: Open source fedlearner framework, 209% increase in advertising efficiency.” _Available online: https://www.jiqizhixin.com/articles/2020-11-03-9 (accessed on 15 March 2021)_ , 2020.
  * [168] Y. Hou, “Jd’s exploration and practice of large-scale federated learning.” _Available online: https://zhuanlan.zhihu.com/p/376697402 (accessed on 31 May 2021)_ , 2021.
  * [169] Y. Lin, “The practice of federated learning in tencent wesee advertising.” _Available online: https://cloud.tencent.com/developer/article/1872819_ , 2021\.
  * [170] Y. Wu, “Huawei’s exploration and application in federated advertising algorithm.” _Available online: https://zhuanlan.zhihu.com/p/558684266_ , 2022\.
  * [171] G. Long, Y. Tan, J. Jiang, and C. Zhang, “Federated learning for open banking,” in _Federated learning_. Springer, 2020, pp. 240–254.
  * [172] T. Rooijakkers, “Convinced—enabling privacy-preserving survival analyses using multi-party computation,” 2020.
  * [173] Y. Song, Y. Xie, H. Zhang, Y. Liang, X. Ye, A. Yang, and Y. Ouyang, “Federated learning application on telecommunication-joint healthcare recommendation,” in _2021 IEEE 21st International Conference on Communication Technology (ICCT)_. IEEE, 2021, pp. 1443–1448.
  * [174] Z. Teimoori, A. Yassine, and M. S. Hossain, “A secure cloudlet-based charging station recommendation for electric vehicles empowered by federated learning,” _IEEE Transactions on Industrial Informatics_ , 2022.
  * [175] S. Niknam, H. S. Dhillon, and J. H. Reed, “Federated learning for wireless communications: Motivation, opportunities, and challenges,” _IEEE Communications Magazine_ , vol. 58, no. 6, pp. 46–51, 2020.
  * [176] Y. Zhang, Q. Wu, and M. Shikh-Bahaei, “Vertical federated learning based privacy-preserving cooperative sensing in cognitive radio networks,” in _2020 IEEE Globecom Workshops (GC Wkshps_. IEEE, 2020, pp. 1–6.
  * [177] N. Hashemi, P. Safari, B. Shariati, and J. K. Fischer, “Vertical federated learning for privacy-preserving ml model development in partially disaggregated networks,” in _2021 European Conference on Optical Communication (ECOC)_. IEEE, 2021, pp. 1–4.
  * [178] H. Liu, X. Zhang, X. Shen, and H. Sun, “A federated learning framework for smart grids: Securing power traces in collaborative learning,” _arXiv preprint arXiv:2103.11870_ , 2021.
  * [179] N. Ge, G. Li, L. Zhang, and Y. Liu, “Failure prediction in production line based on federated learning: an empirical study,” _Journal of Intelligent Manufacturing_ , pp. 1–18, 2021.
  * [180] F. Liu, X. Wu, S. Ge, W. Fan, and Y. Zou, “Federated learning for vision-and-language grounding problems,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 07, 2020, pp. 11 572–11 579.
  * [181] X. Ni, X. Xu, L. Lyu, C. Meng, and W. Wang, “A vertical federated learning framework for graph convolutional network,” _CoRR_ , vol. abs/2106.11593, 2021. [Online]. Available: https://arxiv.org/abs/2106.11593
  * [182] X. Zhang, Y. Kang, K. Chen, L. Fan, and Q. Yang, “Trading off privacy, utility and efficiency in federated learning,” 2022. [Online]. Available: https://arxiv.org/abs/2209.00230
  * [183] B. Li, L. Fan, H. Gu, J. Li, and Q. Yang, “Fedipr: Ownership verification for federated deep neural network models,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2022.
  * [184] Q. Yao, M. Wang, Y. Chen, W. Dai, Y.-F. Li, W.-W. Tu, Q. Yang, and Y. Yu, “Taking Human out of Learning Applications: A Survey on Automated Machine Learning,” Oct. 2018.
  * [185] H. Zhu, H. Zhang, and Y. Jin, “From federated learning to federated neural architecture search: a survey,” _Complex & Intelligent Systems_, vol. 7, no. 2, pp. 639–657, 2021.