# A Practical Recipe for Federated Learning Under Statistical Heterogeneity
Experimental Design

Mahdi Morafah  Weijia Wang  and Bill Lin M. Morafah, W. Wang and B. Lin are
all with Electrical and Computer Engineering Department of University of
California San Diego, USA (e-mail address: {mmorafah, wweijia,
billlin}@eng.ucsd.edu).Corresponding author: Mahdi Morafah.

###### Abstract

Federated Learning (FL) has been an area of active research in recent years.
There have been numerous studies in FL to make it more successful in the
presence of data heterogeneity. However, despite the existence of many
publications, the state of progress in the field is unknown. Many of the works
use inconsistent experimental settings and there are no comprehensive studies
on the effect of FL-specific experimental variables on the results and
practical insights for a more comparable and consistent FL experimental setup.
Furthermore, the existence of several benchmarks and confounding variables has
further complicated the issue of inconsistency and ambiguity. In this work, we
present the first comprehensive study on the effect of FL-specific
experimental variables in relation to each other and performance results,
bringing several insights and recommendations for designing a meaningful and
well-incentivized FL experimental setup. We further aid the community by
releasing FedZoo-Bench, an open-source library based on PyTorch with pre-
implementation of 22 state-of-the-art methods111We will continue the effort to
extend FedZoo-Bench by implementing more methods and adding more features. Any
contributions to FedZoo-Bench would be greatly appreciated as well., and a
broad set of standardized and customizable features available at
https://github.com/MMorafah/FedZoo-Bench. We also provide a comprehensive
comparison of several state-of-the-art (SOTA) methods to better understand the
current state of the field and existing limitations.

Federated Learning aims to train a machine learning model using the massive
decentralized data available at IoT and mobile devices, and different data
centers while maintaining data privacy. However, despite the existence of
numerous works, the state of progress in the field is not well-understood.
Papers use different methodologies and experimental setups that are hard to
compare and examine the effectiveness of methods in more general settings.
Moreover, the effect of federated learning experimental design factors such as
local epochs, and sample rate on the performance results have remained
unstudied in the field. Our work comprehensively studies the effect of
experimental design factors in federated learning, provides suggestions and
insights, introduces FedZoo-Bench with the pre-implementation of 22 state-of-
the-art algorithms under a unified setting, and finally measures the state of
progress in the field. The studies and findings discussed in our work can
significantly help the federated learning field by providing a more
comprehensive understanding of the impact of experimental design factors,
facilitating the design of better performing algorithms, and enabling a more
accurate evaluation of the effectiveness of different methods.

Benchmark, Data Heterogeneity, Experimental Design, Federated Learning,
Machine Learning, Non-IID Data.

##  1 Introduction

Federated Learning (FL) is a machine learning setting, which aims to
collaboratively train machine learning models with the participation of
several clients under the orchestration of a central server in a privacy-
preserving and communication efficient manner [62, 34]. FL has seen a surge of
interest in the machine learning research community in recent years, thanks to
its potential to improve the performance of edge users without compromising
data privacy. This innovative approach has been successfully applied to a wide
range of tasks, including image classification, natural language processing,
and more [34, 82, 73, 64, 15].

The ultimate goal of standard (global) FL is to train a shared global model
which uniformly performs well over almost the entire participating clients.
However, the inherent diversity and not independent and identical (Non-IID)
distribution of clients’ local data has made the global FL approach very
challenging [24, 47, 84, 28, 48, 68]. Indeed, clients’ incentives to
participate in FL can be to derive personalized models rather than learning a
shared global model. This client-centric perspective along with the challenges
in the global FL approach under Non-IID data distributions has motivated an
alternative personalized FL approach. Personalized FL aims to learn
personalized models performing well according to the distinct data
distribution of each participating client.

Despite the significant number of works that have been done in both global and
personalized FL approaches under data heterogeneity, the state of progress is
not well understood in the FL community. In particular, the following key
questions have remained unanswered in the existing literature: what factors
affect experimental results and how to control them? Which experimental design
setups are effective and well-incentivized for a fair comparison in each FL
approach? What are the best practices and remedies to compare different
methods and avoid evaluation failure? We primarily find that the
methodologies, experimental setups, and evaluation metrics are so inconsistent
between papers that a comprehensive comparison is impossible. For example,
some papers consider a specific FL approach, however, they use an experimental
setup that is not well-incentivized or has been created to match their
assumptions and is not applicable to other cases. Moreover, the existence of
numerous benchmarks and inconsistent implementation environments together with
different confounding variables such as data augmentation and pre-processing
techniques, choice of the optimizer, and learning rate schedule have made such
comparison even more difficult.

To address the mentioned issues in the current state of FL research, we
present the first comprehensive study, to the best of our knowledge, on FL-
specific experimental variables and provide new insights and best practices
for a meaningful and well-incentivized FL experimental design for each FL
approach. We also introduce FedZoo-Bench, an open-source library based on
PyTorch that provides a commonly used set of standardized and customizable
features in FL, and implementation of 22 state-of-the-art (SOTA) methods under
a unified setting to make FL research more reproducible and comparable.
Finally, we present a comprehensive evaluation of several SOTA methods in
terms of performance, fairness, and generalization to newcomers, to provide a
clear understanding of the promises and limitations of existing methods.

Contributions. Our study makes the following key contributions:

  * •

We conduct the first comprehensive analysis of FL experimental design
variables by running extensive experiments and provide new insights and best
practices for a meaningful and well-incentivized experimental setup for each
FL approach.

  * •

We introduce FedZoo-Bench, an open-source library for implementation and
evaluation of FL algorithms under data heterogeneity, consisting of the
implementation of 22 SOTA algorithms, available at
https://github.com/MMorafah/FedZoo-Bench.

  * •

Using FedZoo-Bench, we conduct a comprehensive evaluation of several SOTA
algorithms in terms of performance, fairness, and generalization to newcomers
to form a clear understanding of the strengths and limitations of existing
methods.

Organization. The rest of the paper is organized as follows. In Section 2 we
bring a concise literature review. In Section 3 we provide the background for
each FL approach and statistical heterogeneity. In Section 4 we provide our
comprehensive study on FL-specific experimental variables. In Section 5 we
discuss our recommendations. In Section 6 we introduce FedZoo-Bench and
compare 17 different algorithms. In Section 7 we conclude and provide future
works.

##  2 Literature Review

Global FL. McMahan et al. [62] proposed the first method for global FL called
FedAvg which simply averages the local models to update the server-side model.
This FL approach mainly suffers from poor convergence and degradation of the
results in the presence of data heterogeneity [75, 84]. Some works attempt to
address these issues by regularizing local training. FedProx [48] uses an L2
regularizer on the weight difference between the local and global model. MOON
[46] utilizes contrastive learning to preserve global knowledge during local
training. In FedDyn [4], an on-device dynamic regularization at each round has
been used to align the local and global solutions. Another set of works
studies the optimization issues of FedAvg in the presence of data
heterogeneity and proposes alternative optimization procedures with better
convergence guarantees [79, 36, 35, 67, 24]. FedNova [79] addresses the
objective inconsistency caused by heterogeneity in the local updates by
weighting the local models in server-side averaging to eliminate bias in the
solution and achieve fast error convergence. Scaffold [36] proposes control
variates to correct the local updates and eliminate the “client drift” which
happens because of data heterogeneity resulting in convergence rate
improvement. Other approaches have focused on proposing better model fusion
techniques to improve performance [52, 78, 53, 69, 23]. FedDF [52] adds a
server-side KL training step after averaging local models by using the average
of clients’ logits on a public dataset. FedMA [78] proposes averaging on the
matched neurons of the models at each layer. GAMF [53] formulates model fusion
as a graph matching task by considering neurons or channels as nodes and
weights as edges. For a more detailed review of methods on global FL
literature, we recommend reading the surveys [58, 33].

Personalized FL. The fundamental challenges of the global FL approach, such as
poor convergence in the presence of data heterogeneity and lack of
personalized solutions, have motivated the development of personalized FL.
Personalized FL aims to obtain personalized models for participating clients.
There are various efforts to solve this problem through different techniques.
Multi-Task Learning (MTL) based techniques have been proposed in a number of
works by considering clients as tasks and framing the personalized FL as an
MTL problem [72, 74, 25, 49, 60, 19]. Another group of studies proposes model
interpolation techniques by mixing the global and local models [59, 17, 61].
There are also works that utilize representation learning techniques by
decoupling parameters (or layers) into global and local parameters and then
averaging only the global parameters with other clients [51, 5, 14, 66].
Additionally, there are some meta-learning-based works that attempt to obtain
a global model with the capability of getting personalized fast by further
local fine-tuning [32, 21, 21, 11, 71]. Clustering-based methods have been
also shown to be effective in several studies by grouping clients with similar
data distribution to achieve better personalized models and faster convergence
[22, 8, 57, 20, 12, 68, 77]. More recently, personalized FL has been realized
with pruning-based techniques as well [43, 76, 44, 7, 30]. For a more detailed
review of methods on personalized FL literature, we recommend the surveys [39,
85].

FL Benchmarks. Current FL benchmarks primarily focus on enabling various
platforms and APIs to perform FL for different applications. They often only
realize the global FL approach and implement basic algorithms (e.g. FedAvg)
[55, 2, 3, 1], while other benchmarks,such as FLOWER [6], FedML [26], FLUTE
[18] and NIID-Bench [45], offer more customizable features, and implementation
of more algorithms. For a more detailed review of the applicability and
comparison of the existing benchmarks, we defer the reader to UniFed [54].
While the majority of existing benchmarks are for global FL, there are a few
recently released personalized FL benchmarks, including pFL-Bench [10] and
Motley [80]. However, these benchmarks do not investigate the effect of
experimental variables and only consider a few algorithms and their variants
for comparison. In contrast, FedZoo-Bench offers support for both global and
personalized FL approaches, providing implementation of 22 SOTA methods and
the ability to assess generalization to newcomers. Additionally, our study on
the effect of FL-specific experimental variables in relation to each other and
performance results, together with the identification of more meaningful
setups for each FL approach, is a new contribution to the field.

##  3 Background: Federated Learning and Statistical Heterogeneity

###  3.1 Overview of Federated Learning and Notations

Consider a server and a set of clients $S$ which participate in the federation
process. $f_{i}({\boldsymbol{x}};{\boldsymbol{\theta}}_{i})$ is the $L$ layers
neural network model of client $i$ with parameters
${\boldsymbol{\theta}}_{i}=({\boldsymbol{W}}_{i}^{1},\ldots,{\boldsymbol{W}}_{i}^{L})$
where ${\boldsymbol{W}}^{l}$ stands for the $l$-th layer weights, training
dataset $D_{i}^{train}$, and test dataset $D_{i}^{test}$. At the beginning of
communication round $t$, a subset of clients $S_{t}$ are randomly selected
with the sampling rate of $C\in(0,1]$ out of total $N$ available clients. The
selected clients receive the parameters from the server and perform local
training for $E$ epochs with the batch size of $B$ and learning rate of
$\eta$. At the end of communication round $t$, the selected clients send back
their updated parameters to the server for server-side processing and model
fusion. This federation process continues for $T$ total communication rounds.
Algorithm 1 shows this process in detail.

###  3.2 Problem Formulation

Global Federated Learning (gFL). The objective is to train a shared global
model at the server which uniformly performs well on each client and the
problem is defined as follow:

$\displaystyle\widehat{\boldsymbol{\theta}}_{g}=\underset{{\boldsymbol{\theta}}_{g}}{\mathrm{argmin}}\
\sum_{i}^{N}\mathbb{E}_{({\boldsymbol{x}},y)\sim
D_{i}^{\text{train}}}[\ell(f_{i}({\boldsymbol{x}};{\boldsymbol{\theta}}_{g}),y)].$
(1)

FedAvg [62] is the first and most popular algorithm proposed to solve Equation
1, which uses parameter averaging at the server-side for model fusion.

Personalized Federated Learning (pFL). The objective is to train personalized
models to perform well on each client’s distinctive data distribution and the
problem is defined as follows:

$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\\{{\widehat{\boldsymbol{\theta}}_{i}}\\}_{1}^{N}=\underset{\\{{\boldsymbol{\theta}}_{i}\\}_{1}^{N}}{\mathrm{argmin}}\
\sum_{i}^{N}\mathbb{E}_{({\boldsymbol{x}},y)\sim
D_{i}^{\text{test}}}[\ell(f_{i}({\boldsymbol{x}};{\boldsymbol{\theta}}_{i}),y)].}$
(2)

FedAvg + Fine-Tunining (FT) [32] is the simplest algorithm proposed to solve
Equation 2, where each client fine-tunes the global model obtained via FedAvg
on their local data.

Algorithm 1 Federated Learning

0: number of clients ($N$), sampling rate ($C\in(0,1]$), number of
communication rounds ($T$), local dataset of client $k$ ($D_{k}$), local epoch
($E$), local batch size ($B$), learning rate ($\eta$).

1: Initialize the server model with ${\boldsymbol{\theta}}_{g}^{0}$

2: for each round $t=0,1,\ldots,T-1$ do

3: $m\leftarrow{\rm{max}}(C\cdot N,1)$

4: $S_{t}\leftarrow$(random set of m clients)

5: for each client $k\in S_{t}$ in parallel do

6:
${\boldsymbol{\theta}}^{t+1}_{k}\leftarrow{\mathtt{ClientUpdate}}(k;{\boldsymbol{\theta}}^{t}_{g})$

7: end for

8:
${\boldsymbol{\theta}}^{t+1}_{g}=\mathtt{ModelFusion}(\\{{\boldsymbol{\theta}}_{k}\\}_{k\in
S_{t}})$ {FedAvg [62]: $\theta^{t+1}_{g}=\sum_{k\in
S_{t}}{|D_{k}|\theta^{t+1}_{k}}/\sum_{k\in S_{t}}{|D_{k}|}$}

9: end for

10: function $\mathtt{ClientUpdate}(k,{\boldsymbol{\theta}}^{t}_{g})$

11: ${\boldsymbol{\theta}}^{t}_{k}\leftarrow{\boldsymbol{\theta}}^{t}_{g}$

12: $\mathcal{B}\leftarrow$ (randomly splitting $D_{k}^{train}$ into batches
of size $B$)

13: for each local epoch $\in\\{1,\ldots,E\\}$ do

14: for each batch $\mathbf{b}\in\mathcal{B}$ do

15:
${\boldsymbol{\theta}}^{t}_{k}\leftarrow{\boldsymbol{\theta}}^{t}_{k}-\eta\nabla_{{\boldsymbol{\theta}}^{t}_{k}}\ell(f_{k}({\boldsymbol{x}};{\boldsymbol{\theta}}^{t}_{k}),y)$

16: end for

17: end for

18: ${\boldsymbol{\theta}}^{t+1}_{k}\leftarrow{\boldsymbol{\theta}}^{t}_{k}$

19: end function

###  3.3 Statistical Heterogeneity

Consider the local data distribution of clients, denoted as
$\mathbb{P}_{i}({\boldsymbol{x}},y)$. Clients can have statistical
heterogeneity or Non-IID data distribution w.r.t each other when
$\mathbb{P}_{i}({\boldsymbol{x}},y)\neq\mathbb{P}_{j}({\boldsymbol{x}},y)$.
The most popular way to realize this heterogeneity in FL is label distribution
skew in which clients have different label distributions of a dataset (i.e.
$\mathbb{P}_{i}(y)\neq\mathbb{P}_{j}(y)$)222Other mechanisms to realize
statistical heterogeneity do exist, but are less commonly used in the FL
community.. Two commonly used mechanisms to create label distribution skew in
FL are:

  * •

Label skew($p$) [45]: each client receives $p\%$ of the total classes of a
dataset at random. The data points of each class are uniformly partitioned
amongst the clients owning that class.

  * •

Label Dir($\alpha$) [29]: each client draws a random vector with the length of
total classes of a dataset from Dirichlet distribution with a concentration
parameter $\alpha$. The data points of each class are uniformly partitioned
amongst clients according to each client’s class proportions.

##  4 Comprehensive Study on FL Experimental Variables

Overview. To design an effective FL experiment, it is crucial to understand
how the FL-specific variables which are clients’ data partitioning (type and
level of statistical heterogeneity), local epochs ($E$), sample rate ($C$),
and communication rounds ($T$) interact with each other and can affect the
results. While communication rounds ($T$) serve as the equivalent of epochs in
traditional centralized training and primarily determine the training budget,
the other variables have a more direct impact on performance. Hence, we focus
our analysis on these variables, in relation to each other and performance
results, and evaluation metric failure and derive new insights for the FL
community to design meaningful and well-incentivized FL experiments.

Baselines. We use three key baselines in our study: _FedAvg_ [62], the
standard FL baseline that has been widely used in the existing literature and
can serve as a good representative for the global FL approach 333We show in
Section 6.1 that the performance of this baseline is competitive to the SOTA
methods.; _FedAvg + Fine-Tuning (FT)_ [32], a simple personalized FL baseline
that has been shown to perform well in practice and can serve as a
representative for the personalized FL approach 3; and _SOLO_ , a solo (local
only) training of each client on their own dataset without participation in
federation, which serves as a baseline to evaluate the benefit of federation
under different experimental conditions.

Setup. We use CIFAR-10 [37] dataset and LeNet-5 [42] architecture which has
been used in the majority of existing works. We fix number of clients ($N$),
communication rounds ($R$), and sample rate ($C$) to $100$, $100$, and $0.1$
respectively; unless specified otherwise. We use SGD optimizer with a learning
rate of $0.01$, and momentum of $0.9$ 444This optimizer and learning rate have
been used in some works [14, 45, 77] under a similar setup and we also find
that it works the best for our studies.. We use this base setting for all of
our experimental studies in this section. The reported results are the average
results over 3 independent and different runs for a more fair and robust
assessment.

(a)

(b)

(c)

Figure 1: These figures show factors causing evaluation failures when
reporting results: (a) the accuracy variation of FedAvg over the last $10$
communication rounds, (b) the variation of FedAvg’s accuracy using different
test data percentages, (c) the variation of FedAvg + FT’s accuracy using
different local test data percentages.

###  4.1 Evaluation Metric

The evaluation metric for performance is a critical factor in making a fair
and consistent assessment in FL. However, the way in which evaluation metrics
are calculated in the current FL literature is often ambiguous and varies
significantly between papers. In this part we focus on identifying the causes
of evaluation metric failures for each FL approach and bring our suggestions
for avoiding them.

Global FL. The evaluation metric is the performance of the global model on the
test dataset at the server. We find that the causes for evaluation failures
are (1) the round used to report the result and (2) the test data percentage
used to evaluate the model. Figure 1(a) shows the global model accuracy over
the last $10$ rounds on Non-IID Label Dir($0.1$) partitioning. We can see
there is a maximum variation of $7\%$ in the results based on which round to
pick for reporting the result. Also, the difference of the final round result
with the average bar is about $4\%$. This shows that the round used to report
the result is important and to have a more robust metric for these variations,
it is better to report the average results over a number of rounds. Figure
1(b) shows the variations of the reported result for the same model using
different test data percentages. This clearly shows that using different test
data points can cause bias in the evaluation. To avoid the mentioned failures
and have a more reliable evaluation metric we suggest the following
definition:

Definition 1 (global FL evaluation metric). We define the average performance
of the global model on the entire test data at the server (if available) over
the last $[C\cdot N]$ communication rounds as the evaluation metric for global
FL approach 555We use this metric definition for all of our experiments..

Personalized FL. The evaluation metric is the final average performance of all
participating clients on their local test data. The factor which can cause
evaluation failure is the local test data percentage used to evaluate each
client’s model. Prior works have allocated different amounts of data as local
test sets to individual clients. Figure 1(c) shows the variability of the
reported results for the same clients under different local test data
percentages on Non-IID Label Dir($0.1$) partitioning. This highlights that the
use of a randomly selected portion for test data can lead to inaccurate and
biased evaluations based on the selected data points be easy or hard. To avoid
the mentioned failure in the evaluation metric we suggest the following
definition:

Definition 2 (personalized FL evaluation metric). We define the average of the
final performance of all the clients on their entire local test data (if
available) as the evaluation metric for personalized FL approach 5,666The
entire local test data consists of allocating all available test samples
belonging to the classes owned by the client..

###  4.2 Statistical Heterogeneity and Local Epochs

In this part, we focus our study to understand how different levels and types
of statistical heterogeneity together with local epochs can affect the results
and change the globalization and personalization incentives in FL.

(a)

(b)

(c)

Figure 2: These figures show the effect of the level of statistical
heterogeneity and the number of local epochs for the label Dir type of
statistical heterogeneity: (a) illustrates the incentives for globalization
and personalization under various levels of statistical heterogeneity with a
fixed number of $10$ local epochs, (b) and (c) show the performance of FedAvg
and FedAvg + FT under different levels of statistical heterogeneity and local
epochs, respectively.

Level of statistical heterogeneity. Figure 2(a) illustrates how the
performance of the baselines for a fixed label Dir type of statistical
heterogeneity with 10 local epochs varies with the level of statistical
heterogeneity. As the level of statistical heterogeneity decreases, the global
FL approach becomes more successful than the personalized one. The vertical
line in the plot indicates the approximate boundary between the incentives of
the two FL perspectives. We can see that in the extreme Non-IID case
($\alpha=0.05$), neither of the FL approaches is motivated, as the performance
of the SOLO baseline is competitive. Additionally, from $\alpha=0.8$ onwards,
the global FL approach seems to perform close to the end of the spectrum at
$\alpha=\infty$, which is IID partitioning. Furthermore, we find that the
incentives for globalization and personalization can vary with changes in the
number of local epochs for a fixed type of statistical heterogeneity (see
Section 8.1 for more results).

Local epochs. Figures 2(b), and 2(c) show how the performance of FedAvg and
FedAvg + FT for a fixed label Dir type of statistical heterogeneity varies
with different levels of statistical heterogeneity and the number of local
epochs. Figure 2(b) suggests that FedAvg favors fewer local epochs to achieve
higher performance. However, Figure 2(c) suggests that FedAvg + FT favors more
local epochs for achieving better results but no more than $5$ or $10$
depending on the level of statistical heterogeneity. Our findings support the
observation in [36] that client drift can have a significant impact on
performance, and increasing the number of local epochs amplifies this effect
in the results.

(a)

(b)

(c)

Figure 3: These figures show the effect of the level of statistical
heterogeneity and the number of local epochs for the label skew type of
statistical heterogeneity: (a) illustrates the incentives for globalization
and personalization under various levels of statistical heterogeneity with a
fixed number of $10$ local epochs, (b) and (c) show the performance of FedAvg
and FedAvg + FT under different levels of statistical heterogeneity and local
epochs, respectively.

Type of statistical heterogeneity. Figure 3 illustrates the results of an
experiment similar to that of Figure 2, but with a label skew type of
statistical heterogeneity. Figure 3(a) shows the performance of the baselines
with fixed $10$ local epochs at different levels of statistical heterogeneity.
Comparing this figure with Figure 2(a) reveals that this type of heterogeneity
favors personalization over globalization across a wider range of
heterogeneity levels. Figures 3(b) and 3(c) show the performance of FedAvg and
FedAvg + FT with different levels of statistical heterogeneity and local
epochs. Comparing these figures with Figures 2(b) and 2(c) for the label Dir
type of statistical heterogeneity reveals that this type of statistical
heterogeneity is less affected by an increase in local epochs at each level of
heterogeneity. This highlights another finding that the effect of client drift
may vary for different types of statistical heterogeneity (see Section 8.1 for
more results).

###  4.3 Sample Rate

Figures 4(a), 4(b) and 4(c), 4(d) demonstrate the impact of sample rate and
local epochs on the performance for label Dir($0.1$) and label skew ($30$%)
types of statistical heterogeneity, respectively. Our observations show that
increasing the sample rate (i.e., averaging more models) can effectively
mitigate the negative impact of statistical heterogeneity on
performance777Model averaging is a standard component of many FL algorithms..
Additionally, it is essential to consider that averaging a higher number of
models (i.e., a high sample rate) reduces the approximation error associated
with the averaged models across all clients888It is also worth mentioning that
generally high sample rates are not favorable, as they would increase the
communication cost of the FL algorithms.. We find that sample rates of
$C>=0.4$ can significantly reduce the effect of heterogeneity on the
performance, while very small sample rates of $C<0.1$ result in poor
performance. Based on these findings, we suggest using a sample rate in the
range of $0.1<=C<0.4$ for experimental design to accurately evaluate an
algorithm’s success in the presence of data heterogeneity.

(a)

(b)

(c)

(d)

Figure 4: These figures illustrate the impact of sample rate ($C$) and local
epochs on the performance: (a), (b) and (c), (d) show the performance of
FedAvg and FedAvg + FT for Non-IID Label Dir($0.1$) and Non-IID Label
Skew($30$%), respectively.

##  5 Summary and Recommendations

In this section we identify a series of best practices and recommendations for
designing a well-incentivized FL experimental setting base on our findings and
insights in Section 4.

Level of statistical heterogeneity and local epochs. The level of statistical
heterogeneity and the number of local epochs determine the incentives for
global and personalized FL. Generally, when the level of statistical
heterogeneity is low, global FL is more incentivized than personalized FL, and
vice versa (Figures 2(a) and 3(a)). Additionally, increasing the number of
local epochs can make personalized FL more incentivized (Figures 5 and 6). We
have identified the well-incentivized settings for each FL approach in Table
1.

Table 1: Well-incentivized settings for personalized and global FL approaches.

Local Epoch | Non-IID Label Dir | Non-IID Label Skew
---|---|---
pFL | gFL | pFL | gFL
$E=1$ | $\alpha<0.3$ | $\alpha>0.3$ | $p<0.8$ | $p>0.8$
$E=5$ | $\alpha<0.3$ | $\alpha>0.3$ | $p<0.8$ | $p>0.8$
$E=10$ | $\alpha<0.5$ | $\alpha>0.5$ | $p<0.8$ | $p>0.8$
$E=20$ | $\alpha<0.5$ | $\alpha>0.5$ | $p<0.9$ | $p>0.9$

Type of statistical heterogeneity. We observe that the nature of label skew
type of statistical heterogeneity favors personalized FL over a wider range of
heterogeneity levels compared to label Dir (Figures 2(a), 3(a), 5 and 6).
Additionally, we observe that the impact of local epochs on performance is
more pronounced for label Dir type of statistical heterogeneity compared to
label skew (Figures 2(b), 2(c), 3(b) and 3(c)). To provide a comprehensive
perspective on an algorithm’s success in the presence of statistical
heterogeneity, we recommend researchers conduct experiments with both types of
statistical heterogeneity.

Sample rate. This variable plays a crucial role in evaluating an algorithm’s
performance under statistical heterogeneity. Choosing a high sample rate
($C>0.4$) can mask the effect of statistical heterogeneity, thus
misrepresenting an algorithm’s true ability to handle it. Additionally, it can
lead to inaccurate representations of an algorithm’s capability to handle the
stochasticity resulting from random device selection and the inherent errors
caused by model averaging approximation. On the other hand, a low sample rate
($C<0.1$) may hinder convergence due to insufficient models for averaging and
high errors caused by model averaging approximation (Figure 4). To avoid these
pitfalls, we recommend researchers use a sample rate of $0.1\leq C\leq 0.4$
for their experiments.

Summary. To ensure consistency, comparability, and meaningful results in FL
experiments, we have compiled a set of recommended settings in Table 2. We
encourage researchers to adopt these settings, as well as the evaluation
metrics outlined in Section 4.1, for their experiments. This will facilitate
more consistent and fair comparisons with SOTA algorithms, and eliminate
concerns about evaluation failures and the impact of various experimental
settings.

Table 2: Recommended settings for pFL and gFL approaches.

Approach | Type of Heterogeneity | Level of Heterogeneity | Local Epoch | Number of Clients | Sample Rate
---|---|---|---|---|---
pFL | Label Dir | $\alpha\in[0.01,0.3]$ | $\\{1,5,10,20\\}$ | $\\{20,100\\}$ | $\\{0.1,0.2,0.3,0.4\\}$
Label Skew | $p\in\\{2,3,4,5\\}$ | $\\{1,5,10,20\\}$ | $\\{20,100,500\\}$ | $\\{0.1,0.2,0.3,0.4\\}$
gFL | Label Dir | $\alpha\in(0.3,1]$ | $\\{1,5,10,20\\}$ | $\\{20,100\\}$ | $\\{0.1,0.2,0.3,0.4\\}$
Label Skew | $p\in\\{8,9\\}$ | $\\{1,5,10,20\\}$ | $\\{20,100,500\\}$ | $\\{0.1,0.2,0.3,0.4\\}$

##  6 FedZoo-Bench

We introduce FedZoo-Bench, an open-source library based on PyTorch that
facilitates experimentation in federated learning by providing researchers
with a comprehensive set of standardized and customizable features such as
training, Non-IID data partitioning, fine-tuning, performance evaluation,
fairness assessment, and generalization to newcomers, for both global and
personalized FL approaches. Additionally, it comes pre-equipped with a set of
models and datasets, and pre-implemented 22 different SOTA methods, allowing
researchers to quickly test their ideas and hypotheses. FedZoo-Bench is a
powerful tool that empowers researchers to explore new frontiers in federated
learning and enables fair, consistent, and reproducible research in federated
learning. We have provided more details on the implemented algorithms and
available features in Appendix Section 10.

###  6.1 Comparison of SOTA methods

In this section, we present a comprehensive experimental comparison of several
SOTA FL methods using FedZoo-Bench. We evaluate their performance, fairness,
and generalization to new clients in a consistent experimental setting. Our
experiments aim to provide a better understanding of the current progress in
FL research.

Training setting. Following our recommended settings indicated in Table 2, we
choose two different training settings presented in Table 3 to conduct the
experimental comparison of the baselines for each FL approach. We run each of
the baselines 3 times and report the average and standard deviation of the
results.

Table 3: Training settings

Setting | Dataset | Architecture | clients | sample rate | local epochs | partitioning | communication rounds
---|---|---|---|---|---|---|---
gFL #1 | CIFAR-10 | LeNet-5 | 100 | 0.1 | 5 | Non-IID Label Skew(80%) | 100
gFL #2 | CIFAR-100 | ResNet-9 | 20 | 0.2 | 10 | Non-IID Label Dir(0.5) | 100
pFL #1 | CIFAR-10 | LeNet-5 | 100 | 0.1 | 10 | Non-IID Label Skew(30%) | 100
pFL #2 | CIFAR-100 | ResNet-9 | 20 | 0.2 | 10 | Non-IID Label Dir(0.1) | 100

Table 4: Performance and fairness comparison for personalized FL baselines.

(a) Performance Comparison

Algorithm | Setting (pFL #1) | Setting (pFL #2)
---|---|---
FedAvg + FT [32] | $69.26\pm 0.42$ | $49.03\pm 0.40$
LG-FedAvg [51] | $54.03\pm 2.16$ | $25.30\pm 0.50$
PerFedAvg [21] | $76.03\pm 0.31$ | $2.29\pm 0.07$
IFCA [22] | $64.84\pm 3.41$ | $46.73\pm 1.82$
Ditto [49] | $70.97\pm 1.27$ | $48.16\pm 3.25$
FedPer [5] | $64.64\pm 0.45$ | $42.87\pm 1.66$
FedRep [14] | $54.99\pm 3.16$ | $29.39\pm 0.31$
APFL [17] | $68.91\pm 0.59$ | $55.18\pm 0.65$
pFedMe [74] | $10.00\pm 0.98$ | $2.10\pm 0.30$
CFL [68] | $16.83\pm 1.6$ | $1.52\pm 0.17$
SubFedAvg [76] | $69.95\pm 1.34$ | $49.83\pm 1.09$
PACFL [77] | $67.78\pm 0.11$ | $50.11\pm 1.10$

(b) Fairness Comparison

Algorithm | Setting (pFL #1) | Setting (pFL #2)
---|---|---
FedAvg + FT [32] | $8.05\pm 0.32$ | $5.40\pm 0.77$
LG-FedAvg [51] | $12.79\pm 0.64$ | $5.11\pm 0.38$
PerFedAvg [21] | $7.32\pm 0.27$ | $--$
IFCA [22] | $10.56\pm 2.75$ | $5.03\pm 0.07$
Ditto [49] | $7.50\pm 0.37$ | $4.10\pm 0.73$
FedPer [5] | $7.64\pm 0.59$ | $6.19\pm 0.59$
FedRep [14] | $9.18\pm 0.50$ | $5.19\pm 0.73$
APFL [17] | $8.22\pm 0.96$ | $4.60\pm 1.15$
pFedMe [74] | $--$ | $--$
SubFedAvg [76] | $7.44\pm 0.46$ | $4.66\pm 0.56$
CFL [68] | $--$ | $--$
PACFL [77] | $8.82\pm 0.71$ | $4.68\pm 0.42$

Performance comparison. We use the evaluation metrics outlined in Section 4.1
to compare the performance results.

  * •

Global FL. Table 5 shows the performance results of 6 different global FL
methods. As it is noticeable Scaffold and FedProx have given the best results
in settings (gfl #1) and (gfl #2), respectively and FedDF has given the worst
results in both settings. FedAvg which is the simplest method has appeared to
be competitive in both settings and even better than 4 other algorithms in
setting (gfl #2).

  * •

Personalized FL. In Table 4(a), we present the performance results of 12
different personalized FL methods. Similar to global FL methods (Table 5), we
observe that each method performs differently in different settings. No single
method consistently achieves the best results across all settings. For
example, PerFedAvg performs well in setting (pfl #1), but poorly in setting
(pfl #2). Additionally, CFL and pFedMe perform poorly in both settings. On the
other hand, FedAvg + FT, the simplest baseline, performs fairly well in both
settings and is competitive or even superior to several other methods.

Fairness comparison. Fairness is another important aspect of the personalized
FL approach. We use the fairness metric mentioned in [49, 63] which is the
standard deviation of final local test accuracies. Table 4(b) shows the
fairness comparison of the methods. SubFedAvg and Ditto have achieved the best
fairness results in (pfl #1) and (pfl #2) settings, respectively. FedAvg + FT
also demonstrated competitive fairness results in both settings. For
algorithms having poor results we did not report the fairness results as they
would not be meaningful.

Generalization to newcomers. To evaluate the generalization capabilities of
personalized FL methods to newcomers, we reserve 20% of the clients as
newcomers and train the FL models using the remaining 80% clients. While the
adaptation process for many methods is not explicitly clear, we follow the
same procedure as in [60, 77] and allow the newcomers to receive the trained
FL model and perform local fine-tuning. For methods like PACFL that have a
different adaptation strategy, we follow their original approach. Table 6
shows the results of this evaluation.

Discussion. The experimental comparison between several SOTA methods for each
FL approach outlined in this section highlights the progress that has been
made in FL research. While we can see that many methods have improved compared
to the simple FedAvg and FedAvg + FT baselines for global and personalized FL
approaches, respectively, there are also some limitations that are worth
noting:

  * •

There is no method that consistently performs the best across all experimental
settings. Furthermore, for the personalized FL approach, a method may achieve
good fairness results but lack generalization to newcomers. Thus, evaluating
FL methods from different perspectives and developing algorithms that can
provide a better trade-off is crucial.

  * •

Despite the existence of numerous works for each FL approach, the performance
of the simple methods of FedAvg and FedAvg + FT are still competitive or even
better compared to several methods. Thus, there is a need for new methods that
can achieve consistent improvements across different types of statistical
heterogeneity.

  * •

Fairness and generalization to newcomers are two important aspects of the
personalized FL approach that are often overlooked in the literature and are
only focused on performance improvement. Therefore, it is crucial to consider
these aspects in addition to performance improvement when designing new
personalized FL methods.

Table 5: gFL accuracy comparison

Algorithm | Setting (gFL #1) | Setting (gFL #2)
---|---|---
FedAvg [62] | $44.89\pm 0.20$ | $56.47\pm 0.57$
FedProx [48] | $46.01\pm 0.46$ | $56.85\pm 0.36$
FedNova [79] | $44.59\pm 0.60$ | $53.20\pm 0.33$
Scaffold [36] | $56.85\pm 1.06$ | $51.71\pm 0.65$
FedDF [52] | $27.43\pm 2.32$ | $30.24\pm 0.26$
MOON [46] | $45.60\pm 0.31$ | $50.23\pm 0.55$

Table 6: pFL generalization to newcomers

Algorithm | Setting (pFL #1) | Setting (pFL #2)
---|---|---
FedAvg + FT [32] | $64.19\pm 4.64$ | $37.14\pm 0.43$
LG-FedAvg [51] | $40.39\pm 17.98$ | $21.22\pm 2.56$
PerFedAvg [21] | $74.97\pm 1.10$ | $2.22\pm 0.20$
IFCA [22] | $62.64\pm 1.03$ | $14.84\pm 1.86$
Ditto [49] | $62.55\pm 3.10$ | $38.96\pm 0.26$
FedPer [5] | $65.3\pm 2.41$ | $35.66\pm 1.61$
FedRep [14] | $64.50\pm 0.62$ | $23.85\pm 1.49$
APFL [17] | $66.38\pm 1.25$ | $39.52\pm 1.11$
SubFedAvg [76] | $63.54\pm 1.42$ | $30.81\pm 1.28$
PACFL [77] | $68.54\pm 1.33$ | $36.50\pm 1.42$

##  7 Conclusion and Future Works

In this paper, we present a thorough examination of key variables that
influence the success of FL experiments. Firstly, we provide new insights and
analysis of the FL-specific variables in relation to each other and
performance results by running several experiments. We then use our analysis
to identify recommendations and best practices for a meaningful and well-
incentivized FL experimental design. We have also developed FedZoo-Bench, an
open-source library based on PyTorch that provides a comprehensive set of
standardized and customizable features, different evaluation metrics, and
implementation of 22 SOTA methods. FedZoo-Bench facilitates a more consistent
and reproducible FL research. Lastly, we conduct a comprehensive evaluation of
several SOTA methods in terms of performance, fairness, and generalization to
newcomers using FedZoo-Bench. We hope that our work will help the FL community
to better understand the state of progress in the field and encourage a more
comparable and consistent FL research.

In future work, we plan to expand our study to other domains such as natural
language processing and graph neural networks to understand how FL
experimental settings behave in those areas and to assess its versatility and
applicability across different problem domains. Additionally, we will continue
to improve our benchmark by implementing more algorithms and adding new
features. We also have plans to establish an open leaderboard using FedZoo-
Bench, enabling systematic evaluations of FL methods across a wide variety of
datasets and settings. Based on the comparison results presented in Section
6.1, we also believe that the development of new algorithms for both global
and personalized FL approaches to achieve even greater improvement and more
consistent results across different experimental settings would be an exciting
future avenue. Also, more studies on evaluation metrics and the development of
new metrics that can better assess different aspects of an FL algorithm would
be a valuable future work.

## APPENDIX

Organization. We organize the supplementary materials as follow:

  * •

In Section 8, we present addition experimental results to complete our
analysis in Section 4 of the main paper.

  * •

In Section 9, we discuss an experimental checklist to facilitate an easier
comparison of FL methods.

  * •

In Section 10, we provide more details about the available algorithms,
datasets, architectures and data partitionings in FedZoo-Bench.

##  8 Additional Results

###  8.1 Globalization and Personalization Incentives

The additional results in this part complements the results discussed in
Section 4.2. Comparing Figures 5 and 6 further corroborates our finding
mentioned in Section 4.2 that Non-IID Label Skew partitioning has higher
incentive for personalization compared to the other type of heterogeneity.
Moreover, increase of local epochs has incentivized personalization more for
both types of heterogeneity.

Figure 5: These figures show globalization and personalization incentives at
different level of heterogeneity and local epochs for Non-IID Label Dir
partitioning. The approximate boundary shifts from $0.3$ to $0.5$ with the
increase of local epochs.

Figure 6: These figures show globalization and personalization incentives at
different level of heterogeneity and local epochs for Non-IID Label Skew
partitioning. The approximate boundary shifts from $80\%$ to $90\%$ with the
increase of local epochs.

##  9 Experimental Checklist

To facilitate an easier comparison between FL methods in future studies, we
recommend the following checklist:

  * •

Make sure that the used experimental setting is meaningful and well-incentived
for the considered FL approach.

  * •

State the exact setting including local epochs, sample rate, number of
clients, type of data partitioning, level of heterogeneity, communication
rounds, dataset, architecture, evaluation metrics, any pre-processing on the
dataset, any learning rate scheduling if used, and initialization.

  * •

Report the average results over at least $3$ independent and different runs.

  * •

Mention the hyperparameters used to obtain the results.

##  10 FedZoo-Bench

We introduced FedZoo-Bench in Section 6. In this section we provide more
details about the available features in FedZoo-Bench. For more information on
FedZoo-Bench’s implementation and use cases for different settings, refer to
the project’s documentation at https://github.com/MMorafah/FedZoo-Bench.

###  10.1 Available Baselines

  * •

Global FL (7 algorithms)

    * –

FedAvg [62]

    * –

FedProx [48]

    * –

FedNova [79]

    * –

Scaffold [36]

    * –

FedDF [52]

    * –

MOON [46]

    * –

FedBN [50]

  * •

Personalized FL (15 algorithms)

    * –

FedAvg + FT [32]

    * –

LG-FedAVg [51]

    * –

PerFedAvg [21]

    * –

FedPer [5]

    * –

FedRep [14]

    * –

Ditto [49]

    * –

APFL [17]

    * –

IFCA [22]

    * –

SubFedAvg [76]

    * –

pFedMe [74]

    * –

CFL [68]

    * –

PACFL [77]

    * –

MTL [72]

    * –

FedEM [60]

    * –

FedFOMO [83]

Additionally, FedZoo-Bench can be easily used for other variations of FedAvg
[67] and different choice of optimizers.

###  10.2 Available Datasets

  * •

MNIST [16]

  * •

CIFAR-10 [38]

  * •

CFIAR-100

  * •

USPS [31]

  * •

SVHN [65]

  * •

CelebA [56]

  * •

FMNIST [81]

  * •

FEMNIST [9]

  * •

Tiny-ImageNet [40]

  * •

STL-10 [13]

###  10.3 Available Architectures

  * •

MLP as in FedAvg [62]

  * •

LeNet-5 [41]

  * •

ResNet Family [27]

  * •

ResNet-50

  * •

VGG Family [70]

###  10.4 Available Data Partitionings

  * •

IID

  * •

Non-IID Label Dir [29]

  * •

Non-IID Label Skew [45]

  * •

Non-IID Random Shard [62]

  * •

Non-IID Quantity Skew [45]

## References

  * [1] Fedlearner.  https://github.com/bytedance/fedlearner.  Accessed: 2023-01-10.
  * [2] Paddlefl.  https://github.com/PaddlePaddle/PaddleFL.  Accessed: 2023-01-10.
  * [3] Tff.  https://github.com/tensorflow/federated.  Accessed: 2023-01-10.
  * Acar et al. [2021] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N Whatmough, and Venkatesh Saligrama.  Federated learning based on dynamic regularization.  _arXiv preprint arXiv:2111.04263_ , 2021.
  * Arivazhagan et al. [2019] Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary.  Federated learning with personalization layers.  _arXiv preprint arXiv:1912.00818_ , 2019.
  * Beutel et al. [2020] Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, Pedro PB de Gusmão, and Nicholas D Lane.  Flower: A friendly federated learning research framework.  _arXiv preprint arXiv:2007.14390_ , 2020.
  * Bibikar et al. [2022] Sameer Bibikar, Haris Vikalo, Zhangyang Wang, and Xiaohan Chen.  Federated dynamic sparse training: Computing less, communicating less, yet learning better.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 6080–6088, 2022.
  * Briggs et al. [2020] Christopher Briggs, Zhong Fan, and Peter Andras.  Federated learning with hierarchical clustering of local updates to improve training on non-iid data.  In _2020 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–9. IEEE, 2020.
  * Caldas et al. [2018] Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H. Brendan McMahan, Virginia Smith, and Ameet Talwalkar.  LEAF: A benchmark for federated settings.  _CoRR_ , abs/1812.01097, 2018.  URL http://arxiv.org/abs/1812.01097.
  * Chen et al. [2022] Daoyuan Chen, Dawei Gao, Weirui Kuang, Yaliang Li, and Bolin Ding.  pfl-bench: A comprehensive benchmark for personalized federated learning.  _arXiv preprint arXiv:2206.03655_ , 2022.
  * Chen et al. [2018] Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, and Xiuqiang He.  Federated meta-learning with fast convergence and efficient communication.  _arXiv preprint arXiv:1802.07876_ , 2018.
  * Cho et al. [2021] Yae Jee Cho, Jianyu Wang, Tarun Chiruvolu, and Gauri Joshi.  Personalized federated learning for heterogeneous clients with clustered knowledge transfer.  _arXiv preprint arXiv:2109.08119_ , 2021.
  * Coates et al. [2011] Adam Coates, Andrew Ng, and Honglak Lee.  An analysis of single-layer networks in unsupervised feature learning.  In Geoffrey Gordon, David Dunson, and Miroslav Dudík, editors, _Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics_ , volume 15 of _Proceedings of Machine Learning Research_ , pages 215–223, Fort Lauderdale, FL, USA, 11–13 Apr 2011\. PMLR.  URL https://proceedings.mlr.press/v15/coates11a.html.
  * Collins et al. [2021] Liam Collins, Hamed Hassani, Aryan Mokhtari, and Sanjay Shakkottai.  Exploiting shared representations for personalized federated learning.  In _International Conference on Machine Learning_ , pages 2089–2099. PMLR, 2021.
  * Dayan et al. [2021] Ittai Dayan, Holger R Roth, Aoxiao Zhong, Ahmed Harouni, Amilcare Gentili, Anas Z Abidin, Andrew Liu, Anthony Beardsworth Costa, Bradford J Wood, Chien-Sung Tsai, et al.  Federated learning for predicting clinical outcomes in patients with covid-19.  _Nature medicine_ , 27(10):1735–1743, 2021.
  * Deng [2012] Li Deng.  The mnist database of handwritten digit images for machine learning research.  _IEEE Signal Processing Magazine_ , 29(6):141–142, 2012.
  * Deng et al. [2020] Yuyang Deng, Mohammad Mahdi Kamani, and Mehrdad Mahdavi.  Adaptive personalized federated learning.  _arXiv preprint arXiv:2003.13461_ , 2020.
  * Dimitriadis et al. [2022] Dimitrios Dimitriadis, Mirian Hipolito Garcia, Daniel Madrigal Diaz, Andre Manoel, and Robert Sim.  Flute: A scalable, extensible framework for high-performance federated learning simulations.  _arXiv preprint arXiv:2203.13789_ , 2022.
  * Dinh et al. [2021] Canh T Dinh, Tung T Vu, Nguyen H Tran, Minh N Dao, and Hongyu Zhang.  Fedu: A unified framework for federated multi-task learning with laplacian regularization.  _arXiv preprint arXiv:2102.07148_ , 400, 2021.
  * Duan et al. [2021] Moming Duan, Duo Liu, Xinyuan Ji, Yu Wu, Liang Liang, Xianzhang Chen, Yujuan Tan, and Ao Ren.  Flexible clustered federated learning for client-level data distribution shift.  _IEEE Transactions on Parallel and Distributed Systems_ , 2021.
  * Fallah et al. [2020] Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar.  Personalized federated learning: A meta-learning approach.  _arXiv preprint arXiv:2002.07948_ , 2020.
  * Ghosh et al. [2020] Avishek Ghosh, Jichan Chung, Dong Yin, and Kannan Ramchandran.  An efficient framework for clustered federated learning.  _Advances in Neural Information Processing Systems_ , 33:19586–19597, 2020.
  * Gong et al. [2021] Xuan Gong, Abhishek Sharma, Srikrishna Karanam, Ziyan Wu, Terrence Chen, David Doermann, and Arun Innanje.  Ensemble attention distillation for privacy-preserving federated learning.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 15076–15086, 2021.
  * Haddadpour and Mahdavi [2019] Farzin Haddadpour and Mehrdad Mahdavi.  On the convergence of local descent methods in federated learning.  _arXiv preprint arXiv:1910.14425_ , 2019.
  * Hanzely et al. [2020] Filip Hanzely, Slavomír Hanzely, Samuel Horváth, and Peter Richtárik.  Lower bounds and optimal algorithms for personalized federated learning.  _Advances in Neural Information Processing Systems_ , 33:2304–2315, 2020.
  * He et al. [2020] Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, et al.  Fedml: A research library and benchmark for federated machine learning.  _arXiv preprint arXiv:2007.13518_ , 2020.
  * He et al. [2015] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition, 2015.  URL https://arxiv.org/abs/1512.03385.
  * Hsieh et al. [2020] Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons.  The non-iid data quagmire of decentralized machine learning.  In _International Conference on Machine Learning_ , pages 4387–4398. PMLR, 2020.
  * Hsu et al. [2019] Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown.  Measuring the effects of non-identical data distribution for federated visual classification.  _arXiv preprint arXiv:1909.06335_ , 2019.
  * Huang et al. [2022] Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, and Dacheng Tao.  Achieving personalized federated learning with sparse local models.  _arXiv preprint arXiv:2201.11380_ , 2022.
  * Hull [1994] J. J. Hull.  A database for handwritten text recognition research.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 16(5):550–554, 1994.  10.1109/34.291440.
  * Jiang et al. [2019] Yihan Jiang, Jakub Konečnỳ, Keith Rush, and Sreeram Kannan.  Improving federated learning personalization via model agnostic meta learning.  _arXiv preprint arXiv:1909.12488_ , 2019.
  * Kairouz et al. [2019] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.  Advances and open problems in federated learning.  _arXiv preprint arXiv:1912.04977_ , 2019.
  * Kairouz et al. [2021] Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.  Advances and open problems in federated learning.  _Foundations and Trends® in Machine Learning_ , 14(1–2):1–210, 2021.
  * Karimireddy et al. [2020a] Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J Reddi, Sebastian U Stich, and Ananda Theertha Suresh.  Mime: Mimicking centralized stochastic algorithms in federated learning.  _arXiv preprint arXiv:2008.03606_ , 2020a.
  * Karimireddy et al. [2020b] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh.  Scaffold: Stochastic controlled averaging for federated learning.  In _International Conference on Machine Learning_ , pages 5132–5143. PMLR, 2020b.
  * Krizhevsky et al. [2009a] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009a.
  * Krizhevsky et al. [2009b] Alex Krizhevsky et al.  Learning multiple layers of features from tiny images.  2009b.
  * Kulkarni et al. [2020] Viraj Kulkarni, Milind Kulkarni, and Aniruddha Pant.  Survey of personalization techniques for federated learning.  In _2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4)_ , pages 794–797. IEEE, 2020.
  * [40] Ya Le and Xuan Yang.  Tiny imagenet visual recognition challenge.
  * Lecun et al. [1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.  10.1109/5.726791.
  * LeCun et al. [1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.
  * Li et al. [2020a] Ang Li, Jingwei Sun, Binghui Wang, Lin Duan, Sicheng Li, Yiran Chen, and Hai Li.  Lotteryfl: Personalized and communication-efficient federated learning with lottery ticket hypothesis on non-iid datasets.  _arXiv preprint arXiv:2008.03371_ , 2020a.
  * Li et al. [2021a] Ang Li, Jingwei Sun, Xiao Zeng, Mi Zhang, Hai Li, and Yiran Chen.  Fedmask: Joint computation and communication-efficient personalized federated learning via heterogeneous masking.  In _Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems_ , pages 42–55, 2021a.
  * Li et al. [2021b] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He.  Federated learning on non-iid data silos: An experimental study.  _arXiv preprint arXiv:2102.02079_ , 2021b.
  * Li et al. [2021c] Qinbin Li, Bingsheng He, and Dawn Song.  Model-contrastive federated learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10713–10722, 2021c.
  * Li et al. [2019] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smithy.  Feddane: A federated newton-type method.  In _2019 53rd Asilomar Conference on Signals, Systems, and Computers_ , pages 1227–1231. IEEE, 2019.
  * Li et al. [2020b] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith.  Federated optimization in heterogeneous networks.  _Proceedings of Machine Learning and Systems_ , 2:429–450, 2020b.
  * Li et al. [2021d] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith.  Ditto: Fair and robust federated learning through personalization.  In _International Conference on Machine Learning_ , pages 6357–6368. PMLR, 2021d.
  * Li et al. [2021e] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou.  Fedbn: Federated learning on non-iid features via local batch normalization.  _arXiv preprint arXiv:2102.07623_ , 2021e.
  * Liang et al. [2020] Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B Allen, Randy P Auerbach, David Brent, Ruslan Salakhutdinov, and Louis-Philippe Morency.  Think locally, act globally: Federated learning with local and global representations.  _arXiv preprint arXiv:2001.01523_ , 2020.
  * Lin et al. [2020] Tao Lin, Lingjing Kong, Sebastian U Stich, and Martin Jaggi.  Ensemble distillation for robust model fusion in federated learning.  _Advances in Neural Information Processing Systems_ , 33:2351–2363, 2020.
  * Liu et al. [2022a] Chang Liu, Chenfei Lou, Runzhong Wang, Alan Yuhan Xi, Li Shen, and Junchi Yan.  Deep neural network fusion via graph matching with applications to model ensemble and federated learning.  In _International Conference on Machine Learning_ , pages 13857–13869. PMLR, 2022a.
  * Liu et al. [2022b] Xiaoyuan Liu, Tianneng Shi, Chulin Xie, Qinbin Li, Kangping Hu, Haoyu Kim, Xiaojun Xu, Bo Li, and Dawn Song.  Unifed: A benchmark for federated learning frameworks.  _arXiv preprint arXiv:2207.10308_ , 2022b.
  * Liu et al. [2021] Yang Liu, Tao Fan, Tianjian Chen, Qian Xu, and Qiang Yang.  Fate: An industrial grade platform for collaborative learning with data protection.  _J. Mach. Learn. Res._ , 22(226):1–6, 2021.
  * Liu et al. [2015] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In _Proceedings of International Conference on Computer Vision (ICCV)_ , December 2015.
  * Lyu et al. [2022] Boxiang Lyu, Filip Hanzely, and Mladen Kolar.  Personalized federated learning with multiple known clusters.  _arXiv preprint arXiv:2204.13619_ , 2022.
  * Mahlool and Abed [2022] Dhurgham Hassan Mahlool and Mohammed Hamzah Abed.  A comprehensive survey on federated learning: Concept and applications.  _arXiv preprint arXiv:2201.09384_ , 2022.
  * Mansour et al. [2020] Yishay Mansour, Mehryar Mohri, Jae Ro, and Ananda Theertha Suresh.  Three approaches for personalization with applications to federated learning.  _arXiv preprint arXiv:2002.10619_ , 2020.
  * Marfoq et al. [2021] Othmane Marfoq, Giovanni Neglia, Aurélien Bellet, Laetitia Kameni, and Richard Vidal.  Federated multi-task learning under a mixture of distributions.  _Advances in Neural Information Processing Systems_ , 34:15434–15447, 2021.
  * Marfoq et al. [2022] Othmane Marfoq, Giovanni Neglia, Richard Vidal, and Laetitia Kameni.  Personalized federated learning through local memorization.  In _International Conference on Machine Learning_ , pages 15070–15092. PMLR, 2022.
  * McMahan et al. [2017] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-efficient learning of deep networks from decentralized data.  In _Artificial intelligence and statistics_ , pages 1273–1282. PMLR, 2017.
  * Mohri et al. [2019] Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh.  Agnostic federated learning.  In _International Conference on Machine Learning_ , pages 4615–4625. PMLR, 2019.
  * Muhammad et al. [2020] Khalil Muhammad, Qinqin Wang, Diarmuid O’Reilly-Morgan, Elias Tragos, Barry Smyth, Neil Hurley, James Geraci, and Aonghus Lawlor.  Fedfast: Going beyond average for faster training of federated recommender systems.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pages 1234–1242, 2020.
  * Netzer et al. [2011] Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng.  Reading digits in natural images with unsupervised feature learning.  In _NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011_ , 2011.  URL http://ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf.
  * Pillutla et al. [2022] Krishna Pillutla, Kshitiz Malik, Abdelrahman Mohamed, Michael Rabbat, Maziar Sanjabi, and Lin Xiao.  Federated learning with partial model personalization.  _arXiv preprint arXiv:2204.03809_ , 2022.
  * Reddi et al. [2020] Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečnỳ, Sanjiv Kumar, and H Brendan McMahan.  Adaptive federated optimization.  _arXiv preprint arXiv:2003.00295_ , 2020.
  * Sattler et al. [2020] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek.  Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints.  _IEEE transactions on neural networks and learning systems_ , 32(8):3710–3722, 2020.
  * Sattler et al. [2021] Felix Sattler, Tim Korjakow, Roman Rischke, and Wojciech Samek.  Fedaux: Leveraging unlabeled auxiliary data in federated learning.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2021\.
  * Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition, 2014\.  URL https://arxiv.org/abs/1409.1556.
  * Singhal et al. [2021] Karan Singhal, Hakim Sidahmed, Zachary Garrett, Shanshan Wu, John Rush, and Sushant Prakash.  Federated reconstruction: Partially local federated learning.  _Advances in Neural Information Processing Systems_ , 34:11220–11232, 2021.
  * Smith et al. [2017] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar.  Federated multi-task learning.  _Advances in neural information processing systems_ , 30, 2017.
  * Sui et al. [2020] Dianbo Sui, Yubo Chen, Jun Zhao, Yantao Jia, Yuantao Xie, and Weijian Sun.  Feded: Federated learning via ensemble distillation for medical relation extraction.  In _Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)_ , pages 2118–2128, 2020.
  * T Dinh et al. [2020] Canh T Dinh, Nguyen Tran, and Josh Nguyen.  Personalized federated learning with moreau envelopes.  _Advances in Neural Information Processing Systems_ , 33:21394–21405, 2020.
  * Tan et al. [2022] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang.  Towards personalized federated learning.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2022\.
  * Vahidian et al. [2021] Saeed Vahidian, Mahdi Morafah, and Bill Lin.  Personalized federated learning by structured and unstructured pruning under data heterogeneity.  In _2021 IEEE 41st International Conference on Distributed Computing Systems Workshops (ICDCSW)_ , pages 27–34. IEEE, 2021.
  * Vahidian et al. [2022] Saeed Vahidian, Mahdi Morafah, Weijia Wang, Vyacheslav Kungurtsev, Chen Chen, Mubarak Shah, and Bill Lin.  Efficient distribution similarity identification in clustered federated learning via principal angles between client data subspaces.  _arXiv preprint arXiv:2209.10526_ , 2022.
  * Wang et al. [2020a] Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni.  Federated learning with matched averaging.  _arXiv preprint arXiv:2002.06440_ , 2020a.
  * Wang et al. [2020b] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H Vincent Poor.  Tackling the objective inconsistency problem in heterogeneous federated optimization.  _Advances in neural information processing systems_ , 33:7611–7623, 2020b.
  * Wu et al. [2022] Shanshan Wu, Tian Li, Zachary Charles, Yu Xiao, Ziyu Liu, Zheng Xu, and Virginia Smith.  Motley: Benchmarking heterogeneity and personalization in federated learning.  _arXiv preprint arXiv:2206.09262_ , 2022.
  * Xiao et al. [2017] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
  * Yang et al. [2018] Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, Nicholas Kong, Daniel Ramage, and Françoise Beaufays.  Applied federated learning: Improving google keyboard query suggestions.  _arXiv preprint arXiv:1812.02903_ , 2018.
  * Zhang et al. [2020] Michael Zhang, Karan Sapra, Sanja Fidler, Serena Yeung, and Jose M Alvarez.  Personalized federated learning with first order model optimization.  _arXiv preprint arXiv:2012.08565_ , 2020.
  * Zhao et al. [2018] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra.  Federated learning with non-iid data.  _arXiv preprint arXiv:1806.00582_ , 2018.
  * Zhu et al. [2021] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin.  Federated learning on non-iid data: A survey.  _Neurocomputing_ , 465:371–390, 2021.

[]Mahdi Morafah received the BS in Electrical Engineering from the Tehran
Polytechnic University, Tehran, Iran, in 2019, and the MS degree in Electrical
and Computer Engineering from the University of California, San Diego, in
2021. He is currently pursuing the PhD degree in Electrical and Computer
Engineering at the University of California, San Diego. His research interests
include Machine Learning and Deep Learning, Distributed Learning, Federated
Learning, Continual Learning, and Optimization.

[]Weijia Wang received the B.S. degree in Electrical Engineering from Zhejiang
University, Hangzhou, China, in 2016 and the M.S. degree in Electrical and
Computer Engineering from the University of California, San Diego, in 2018. He
is currently pursuing the Ph.D. degree in Electrical and Computer Engineering
at the University of California, San Diego. His research interest is machine
learning and deep learning, including the compression and acceleration of deep
convolutional neural networks, algorithms of meta-learning and federated
learning, and explainable artificial intelligence.

[]Bill Lin received the BS, MS, and the PhD degrees in electrical engineering
and computer sciences from the University of California, Berkeley in 1985,
1988, and 1991, respectively. He is a Professor in Electrical and Computer
Engineering at the University of California, San Diego, where he is actively
involved with the Center for Wireless Communications (CWC), the Center for
Networked Systems (CNS), and the Qualcomm Institute in industry-sponsored
research efforts. His research has led to over 200 journal and conference
publications, including a number of Best Paper awards and nominations. He also
holds 5 awarded patents. He has served as the General Chair and on the
executive and technical program committee of many IEEE and ACM conferences,
and he has served as an Associate or Guest Editor for several IEEE and ACM
journals as well.


Benedicks-Amrein-Berthier’s Theorem Related to the Quaternion Fourier
Transform

Youssef El Haoui1,111Corresponding author., Said Fahlaoui1

1Department of Mathematics and Computer Sciences, Faculty of Sciences,
University Moulay Ismail, Meknes 11201, Morocco

E-MAIL<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The main objective of the present paper is to establish a new uncertainty
principle (UP) for the two-sided quaternion Fourier transform (QFT). This
result is an extension of a result of Benedicks, Amrein and Berthier, which
states that a nonzero function in
$L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)$ and its two-sided QFT cannot
both have support of finite measure.

Key words: Quaternion Fourier transform, Benedicks–Amrein–Berthier’s theorem.

##  1 Introduction

In harmonic analysis, It is known that if the supports of a function $f\in
L^{1}\left({\mathbb{R}}^{d}\right)$ and its Fourier transform
${{\mathcal{F}}}\\{f\\}$ are contained in bounded rectangles, then $f$
vanishes almost every where.
Benedicks [2] relaxed the requirements for this conclusion by sharing that the
supports of $f$ and ${\mathcal{F}}\\{f\\}$ need only have finite measure,
whose proof, first circulated as a preprint in 1974 but not formally published
for another decade, Amrein and Berthier [1] later gave a different proof.
The classical Fourier transform, defined over the real line, is given by

${\mathcal{F}}\left\\{f\left(x\right)\right\\}\left(y\right)=\int^{+\infty}_{-\infty}{e^{-2\pi
ixy}}\ f\left(x\right)dx,$

for $f\in L^{1}(\mathbb{R}).$
The Benedicks-Amrein-Berthier’s theorem is the following:
Let $f\in L^{1}\left({\mathbb{R}}\right)$ , and $\ {\mathcal{F}}\\{f\\}$ be
its Fourier transform.
Suppose that the sets $\\{x\in{\mathbb{R}}:f\left({x}\right)\neq 0\\}$ and
$\\{\xi\in{\mathbb{R}}:{\mathcal{F}}\\{f\\}\left(\xi\right)\neq 0\\}$ both
have finite Lebesgue measure, then $f=0.$
In this paper, we extend the validity of the Benedicks-Amrein-Berthier’s
theorem to the two-sided QFT.
Other theorems of the UPs: Hardy, Beurling, Cowling-Price, Gelfand-Shilov,
Miyachi have been extended for this transform (e.g. [6, 7, 8]) .
We further emphasize that our generalization is non-trivial because of the
non-commutative property of quaternion multiplication.

Our paper is organized as follows: In the next section, we collect some basic
concepts in quaternion algebra. In section 3, we recall the definition and
some results for the two-sided QFT useful in the sequel. In section 4, we
prove Benedicks-Amrein-Berthier’s theorem for the two-sided QFT.
For the sake of further simplicity, we will use the real vector notations
$x=(x_{1},x_{2})\in{\mathbb{R}}^{2},\ dx=dx_{1}dx_{2}$ the Lebesgue-measure,
$|A|$ is the normed Lebesgue measure of a measurable set
$A\subseteq\mathbb{R}^{n}$.

##  2 The algebra of quaternions

The quaternion algebra $\mathbb{H}$ was formally invented by by Irish
mathematician Sir William Rowan Hamilton in 1843, it is an associative non-
commutative four-dimensional algebra which is a generalization of complex
numbers.

$\mathbb{H}=\\{q=q_{0}+iq_{1}+jq_{2}+kq_{3};\
q_{0},q_{1},q_{2},q_{3}\in\mathbb{R}\\}$

where the elements $i,j,k$ satisfy Hamilton’s rules

$ij=-ji=k;\ jk=-kj=i;\ ki=-ik=j;$ $i^{2}=j^{2}=k^{2}=-1.$

The algebra of quaternions is isomorphic to the Clifford algebra
${Cl}_{(0,2)}$ of ${\mathbb{R}}^{(0,2)}$:

$\mathbb{H}\cong{Cl}_{(0,2)}.$ (2.1)

The scalar part of $q=q_{0}+iq_{1}+jq_{2}+kq_{3}\in{\mathbb{H}}$ denoted by
$Sc(q)$ is $q_{0}$ , and $iq_{1}+jq_{2}+kq_{3}$ is called the vector part(or
pure quaternion) of $q$ denoted by $Vec(q)$.

The conjugate $\overline{q}$ of $q$ is defined by :

$\overline{q}=q_{0}-iq_{1}-jq_{2}-kq_{3}.$

and we get the norm od modulus of $q\in\mathbb{H}$ as

${|q|}_{Q}=\sqrt{q\overline{q}}=\sqrt{{q_{0}}^{2}{+}\
{q_{1}}^{2}{+}{q_{2}}^{2}+{q_{3}}^{2}}.$

Then we get

${|pq|}_{Q}={|p|}_{Q}{|q|}_{Q}.$

Applying the conjugate and the modulus of $q$, we can define the inverse of
$q\in\mathbb{H}\backslash\\{0\\}$ as

$q^{-1}=\frac{\overline{q}}{|q|^{2}_{Q}}.$

Thus, $\mathbb{H}$ is a normed division algebra.

Let $q\in\mathbb{H}$ , we have a natural generalization of Euler’s formula for
quaternions

$q=|q|_{Q}e^{\mu\theta},$ (2.2)

where

$e^{\mu\theta}={\cos\left(\theta\right)\ }+\mu\ {\sin\left(\theta\right),\ }$

with

$\theta=\ artan\frac{\left|Vec\left(q\right)\right|_{Q}}{Sc\left(q\right)},\
0\leq\theta\leq\pi,\ \mbox{and}\
\mu:=\frac{Vec\left(q\right)}{\left|{Vec}\left({q}\right)\right|_{Q}}\mbox{verifying}\
{\mu}^{2}=\ -1.$

We introduce the quaternion space
$L^{p}\left({\mathbb{R}}^{2},\mathbb{H}\right)$, $1\leq p\leq\infty$, as the
Banach space of all quaternion-valued functions, given by

$L^{p}\left({\mathbb{R}}^{2},\mathbb{H}\right)=\\{f:\mathbb{R}^{2}{{\underset{}{\rightarrow}}}\mathbb{H},f=f_{0}+if_{1}+jf_{2}+kf_{3}|f_{m}\in
L^{p}(\mathbb{R}^{2},\mathbb{R}),m=0,1,2,3\\}$

The $L^{p}\left({\mathbb{R}}^{2},\mathbb{H}\right)-$ inner product and norrm
are defined as

$\forall f,g\in L^{p}\left(\mathbb{R}^{2},\mathbb{H}\right),\ \
(f,g)=\int_{\mathbb{R}^{2}}f(x)\overline{g}(x)dx$
${\left|f\right|}_{p,Q}=({\int_{{\mathbb{R}}^{2}}{{\left|f(x)\right|}^{p}_{Q}\
dx}})^{\frac{1}{p}}<\infty\ for\ 1\leq p<\infty,$
${\left|f\right|}_{\infty,Q}=ess\
{sup}_{x\in{\mathbb{R}^{2}}}{{\left|f(x)\right|}_{Q}}<\infty,p=+\infty$

##  3 Quaternion Fourier transforms

Ell [9] introduced the quaternion Fourier transform (QFT) as an hyper-complex
transform which generalizes the classical Fourier transform (CFT) using the
framework of quaternion algebra. Several known and useful properties, and
theorems of this extended transform are generalizations of the corresponding
properties, and theorems of the CFT with some modifications (e.g., [3, 4, 9,
11]).
The QFT plays a significant role in sgnal analysis due to transforming a real
2D signal into a quaternion-valued frequency domain signal, and it belongs to
the family of Clifford Fourier transformations because of (2.1).

There are three different types of QFT, the left-sided QFT , the right-sided
QFT , and two-sided QFT. For more details, we refer the reader to [12].
In this paper we only treat the two-sided QFT.
In the following, we will recall the definition and some properties of the
two-sided QFT needed to prove the main theorem.

######  Definition 3.1.

Let$\ f\in L^{1}\left(\mathbb{R}^{2},\mathbb{H}\right)$.
Then two-sided quarternion Fourier transform of the function $f$ is given by

${\mathcal{F}}\\{f(x)\\}(\xi)=\int_{\mathbb{R}^{2}}{e^{-i{2\pi\xi}_{1}x_{1}}}f(x)e^{-j{2\pi\xi}_{2}x_{2}}dx,$
(3.1)

Where $\xi,x\in\mathbb{R}^{2}.$

######  Lemma 3.2.

(Continuity)
If $f\in L^{1}(\mathbb{R}^{2},\mathbb{H}),$ then ${\mathcal{F}}\\{f\\}$ is a
continuous function on $\mathbb{R}^{2}$.

Proof. See [[4], Proposition 3.1(ix)].

######  Lemma 3.3.

Inverse QFT ([3], Thm. 2.5)

If $f\in L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right),and\
{\mathcal{F}}\\{f\\}\in L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)$,
then the two-sided QFT is an invertible transform and its inverse is given by

$f(x)=\int_{{\mathbb{R}}^{2}}{e^{i2\pi{\xi}_{1}x_{1}}}{\mathcal{F}}\\{f(x)\\}(\xi)e^{j{2\pi\xi}_{2}x_{2}}d\xi,\
d\xi=d\xi_{1}d\xi_{2}.$

######  Lemma 3.4.

(Dilation property), see example 2 on page 50 [3]
Let $k_{1},k_{2}$ be a positive scalar constants, we have

${\mathcal{F}}\left\\{f(t_{1},t_{2})\right\\}\left(\frac{u_{1}}{k_{1}},\frac{u_{2}}{k_{2}}\right)=k_{1}k_{2}{\mathcal{F}}\left\\{f(k_{1}t_{1},k_{2}t_{2})\right\\}\left(u_{1},u_{2}\right).$

##  4 Benedicks–Amrein–Berthier type theorem

The uncertainty principle is a significant mathematical result that gives
limitations on the exact measurement of a signal in both the time domain and
frequency domain simultaneously. It has implications in different disciplines
such as quantum physics, information processing, signal analysis, etc.
Because of their great importance for signal and image analysis, researchers
interested to extend the validity of the of the uncertainty principles in the
framework of the algebra of Quaternions (see, e.g., [11, 4, 6].
In this context, this section deal with proving Benedicks–Amrein–Berthier’s
uncertainty principle the two-sided QFT by virtue of the quaternion Fourier
series expansion .

######  Definition 4.1.

The quaternion-valued signal $f:{\mathbb{R}}^{2}\to\mathbb{H}$ is said to be
periodic of period
$K=(2L,2L),\ L>0,$ if it verifies

$f\left({x+K}\right)=f(x)\ \ \ \forall{x}\in\mathbb{R}^{2}.$

The Fourier series in the quaternion case may be written as :

$f\left(x\right)\simeq\frac{a_{0}}{2}+\sum_{(r,s)\in\mathbb{Z}^{2}}{{cos}\left(\frac{{r\pi
x}_{1}}{L}\right){cos}\left(\frac{{s\pi x}_{2}}{L}\right)A_{rs}+\
}\sum_{(r,s)\in\mathbb{Z}^{2}}{{sin}\left(\frac{{r\pi
x}_{1}}{L}\right){cos}\left(\frac{{s\pi x}_{2}}{L}\right)B_{rs}}$
$+\sum_{(r,s)\in\mathbb{Z}^{2}}{{cos}\left(\frac{{r\pi
x}_{1}}{L}\right){sin}\left(\frac{{s\pi x}_{2}}{L}\right)C_{rs}{+}\
}\sum_{(r,s)\in\mathbb{Z}^{2}}{{sin}\left(\frac{{r\pi
x}_{1}}{L}\right){sin}\left(\frac{{s\pi x}_{2}}{L}\right)D_{rs}\ },$

with $a_{0},A_{rs},B_{rs},C_{rs},D_{rs}\in\mathbb{H}.$
We have according to the quaternion Euler formula (2.2)

$cos(\frac{{n\pi x}_{r}}{L})=\frac{e^{i\frac{{n\pi
x}_{r}}{L}}+e^{-i\frac{{n\pi x}_{1}}{L}}}{2}$

and

$sin(\frac{{n\pi x}_{r}}{L})=\frac{e^{j\frac{{n\pi
x}_{r}}{L}}-e^{-j\frac{{n\pi x}_{r}}{L}}}{2j},\ \ \ r=1,2.$

Therfore , we can rewrite the Fourier series in the exponential notations:

$f\left(x\right)\simeq\sum_{(r,s)\in\mathbb{Z}^{2}}{e^{i\frac{{r\pi
x}_{1}}{L}}C_{rs}e^{j\frac{{s\pi x}_{2}}{L}}},\ with\ C_{rs}\in\mathbb{H}.$
(4.1)

######  Lemma 4.2.

The quaternion Fourier coefficients are given by:

$C_{rs}\simeq\frac{1}{{(4L)}^{2}}\int^{L}_{-L}{\int^{L}_{-L}{e^{-i\frac{{r\pi
x}_{1}}{L}}f(x)}}e^{-j\frac{{s\pi x}_{2}}{L}}dx$

Proof. we have by (4.1) :

$\displaystyle\frac{1}{{(4L)}^{2}}\int^{L}_{-L}{\int^{L}_{-L}{e^{-i\frac{{r\pi
x}_{1}}{L}}f(x)}}e^{-j\frac{{s\pi x}_{2}}{L}}dx$ $\displaystyle\simeq$
$\displaystyle\frac{1}{{(4L)}^{2}}\int^{L}_{-L}{\int^{L}_{-L}{e^{-i\frac{{r\pi
x}_{1}}{L}}\sum_{(m_{1},m_{2})\in\mathbb{Z}^{2}}{e^{i\frac{{m_{1}\pi
x}_{1}}{L}}{C_{m_{1}m_{2}}}e^{j\frac{{m_{2}\pi x}_{2}}{L}}}}}e^{-j\frac{{s\pi
x}_{2}}{L}}dx_{1}dx_{2}$ $\displaystyle=$
$\displaystyle\frac{1}{{(4L)}^{2}}\sum_{(m_{1},m_{2})\in\mathbb{Z}^{2}}\int^{L}_{-L}{e^{-i\frac{{r\pi
x}_{1}}{L}}e^{i\frac{{m_{1}\pi
x}_{1}}{L}}dx_{1}C_{m_{1}m_{2}}\int^{L}_{-L}{e^{j\frac{{m_{2}\pi
x}_{2}}{L}}}}e^{-j\frac{{s\pi x}_{2}}{L}}dx_{2}$ $\displaystyle=$
$\displaystyle\frac{1}{{(4L)}^{2}}\sum_{(m_{1},m_{2})\in\mathbb{Z}^{2}}{C_{m_{1}m_{2}}}2L{\delta}_{rm_{1}}2L{\delta}_{sm_{2}}$
$\displaystyle=$ $\displaystyle\ C_{rs}.$

$\Box$

The next two lemmas will be needed to prove the main theorem.

######  Lemma 4.3.

Let ${\mathbb{T}}=[0,1].$ If $\ f\in L^{1}(\mathbb{R}^{2},\mathbb{H})$,

Then

$\varphi\left(t\right)=\sum_{k\in\mathbb{Z}^{2}}{f\left(t+k\right)}\
\mbox{converges in}L^{1}({\mathbb{T}}^{2},\mathbb{H}),$

and
the quaternion Fourier series of $\varphi$ is

$\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{e^{2\pi
it_{1}k_{1}}{\mathcal{F}}\\{f\\}(k)e^{2\pi jt_{2}k_{2}}}$

Proof. As $\ f\in L^{1}(\mathbb{R}^{2},\mathbb{H})$, we have

$\displaystyle\int_{{[0,1]}^{2}}{{|\sum_{\left|k\right|\leq
N}{f\left(t+k\right)}|_{Q}}}$ $\displaystyle\leq$
$\displaystyle\int_{{[0,1]}^{2}}{{\sum_{\left|k\right|\leq
N}{{|f\left(t+k\right)|}_{Q}dt}}}$ $\displaystyle=$
$\displaystyle\sum_{\left|k\right|\leq
N}{\int_{{[0,1]}^{2}}{{|f\left(x+k\right)|}_{Q}dt}}$ $\displaystyle=$
$\displaystyle\sum_{\left|k\right|\leq
N}{\int^{k_{1}+1}_{k_{1}}{\int^{k_{2}+1}_{k_{2}}{{|f\left(t\right)|}_{Q}dt}}}$
$\displaystyle=$ $\displaystyle\sum_{\left|k\right|\leq
N}{\int^{k_{1}+1}_{k_{1}}{\int^{k_{2}+1}_{k_{2}}{{|f\left(t\right)|}_{Q}dt}}}$
$\displaystyle\leq$
$\displaystyle\int_{\mathbb{R}^{2}}{{|f\left(t\right)|}_{Q}dt}<\infty.$

Where $\left|k\right|=\left|k_{1}\right|+\left|k_{2}\right|$ and
$N\in{\mathbb{N}}^{2}$ is big enough.
Thus we have shown that $\ \ \varphi\ \in L^{1}(\mathbb{T}^{2},\mathbb{H}).$

Since $\varphi$ is periodic with period = 1, according to lemma 4.2 the
qutaernion Fourier coefficients are

$\displaystyle\int^{\frac{1}{2}}_{-\frac{1}{2}}{\int^{\frac{1}{2}}_{-\frac{1}{2}}{e^{-2\pi
it_{1}m_{1}}\varphi(t)}}e^{-2\pi jt_{2}m_{2}}dt$ $\displaystyle=$
$\displaystyle\int^{\frac{1}{2}}_{-\frac{1}{2}}{\int^{\frac{1}{2}}_{-\frac{1}{2}}{e^{-2\pi
it_{1}m_{1}}\sum_{k\in\mathbb{Z}^{2}}{f\left(t+k\right)}}}e^{-2\pi
jt_{2}m_{2}}dt$ $\displaystyle=$
$\displaystyle\int^{\frac{1}{2}}_{-\frac{1}{2}}{\int^{\frac{1}{2}}_{-\frac{1}{2}}{e^{-2\pi
i{(t}_{1}{+k_{1})m}_{1}}\sum_{k=(k_{1},k_{2})\in
Z^{2}}{f\left(t+k\right)}}}e^{-2\pi j{(t}_{2}+k_{2})m_{2}}dt$ $\displaystyle=$
$\displaystyle\sum_{k\in\mathbb{Z}^{2}}{\int^{k_{2}+\frac{1}{2}}_{k_{2}-\frac{1}{2}}{\int^{k_{1}+\frac{1}{2}}_{k_{1}-\frac{1}{2}}{e^{-2\pi
i{m_{1}y}_{1}}}}\ f\left(y\right)e^{-2\pi j{m_{2}y}_{2}}dy_{1}dy_{2}}$
$\displaystyle=$
$\displaystyle{\mathcal{F}}\\{f\left(y\right)\\}(m_{1},m_{2}).$

$\Box$

######  Lemma 4.4.

Let $f:{[0,1]}^{2}\to\mathbb{H}\ $ be a measurable function.

If $\int_{{[0,1]}^{2}}{{|f(x)|}_{Q}\ dx<\infty},$

then there is a set ${E}\subseteq{[0,1]}^{2}\ $ with $|E|=1$ and
${|f(a)|}_{Q}<\infty\ \ \forall{a}\in{E.}$

Proof. Let

$E=\\{x\in{\left[0,1\right]}^{2}:\ {|f(x)|}_{Q}<\infty\\}\mbox{and}\
F={\left[0,1\right]}^{2}\backslash E.$

By absurd, if $|F|>0$ then $\int_{F}{{|f(x)|}_{Q}dx=\infty},$ contradiction
with $\int_{[0,1]^{2}}{{|f(x)|}_{Q}dx<\infty}.$

Then necessarily

$|F|=0$, therefore $|E|=1.$ $\Box$

will also need the following result of the zeros of an analytic function.

######  Lemma 4.5.

Let${\ f}$ be an analytic function on ${\mathbb{C}}^{2}$, and
$Z(f):=\\{z\in{\mathbb{C}}^{2}{\ :\ f(z)=0}\\}.$

One of these properties can occur

  1. (i)

$Z(f)$ is of zero measure on ${\mathbb{C}}^{2}$.

  2. (ii)

$Z(f)={\mathbb{C}}^{2}$.

Proof. The lemma holds for a function of one variable, see [[13], Thm. 10.18].

Let ${\mu}_{r}$ be the Lebesgue-measure on ${\mathbb{C}}^{r},\ r=1,2$ and $f$
be a function of two complex variables $z_{1},z_{2}$.

We define $f_{z_{2}}(z_{1})=f(z_{1},z_{2})$, and
$f_{z_{1}}(z_{2})=f(z_{1},z_{2})$.

We have

$Z(f)=Z(f_{z_{1}})\otimes Z(f_{z_{2}})\ and\
{\mu}_{2}\left(Z\left(f\right)\right)={\mu}_{1}(Z(f_{z_{2}}))\
{\mu}_{1}(Z(f_{z_{1}})).$

Hence, if either ${\mu}_{1}\left({Z}\left(f_{z_{2}}\right)\right)=0$ or
${\mu}_{1}\left({Z}\left(f_{z_{1}}\right)\right)=0$
then ${\mu}_{2}\left({Z}\left(f\right)\right)=0.$

If not, then
$Z\left(f_{z_{2}}\right)={\mathbb{C}}$ and
$Z\left(f_{z_{1}}\right)={\mathbb{C}}$ such that $Z(f)={\mathbb{C}}^{2}.$
$\Box$

######  Theorem 4.6.

Let $f\in L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)$,
We define $\Sigma(f):=\\{x\in{\mathbb{R}}^{2}:f(x)\neq 0\\}$ and
$\Sigma({{\mathcal{F}}\\{f\\}}):=\\{\xi\in{\mathbb{R}}^{2}:{{\mathcal{F}}\\{f\\}}(\xi)\neq
0\\}.$
If

$|\Sigma(f)|\ |\Sigma({\mathcal{F}}\\{f\\})|<\infty,$

then $f=0.$

Proof. Since $|\Sigma({f)}|$ is finite, by composing $f$ with a dilation,
lemma 3.4, we can assume that

$|\Sigma(f)|<1$ without loss of generality.

Hence we have

$\displaystyle\int_{{[0,1]}^{2}}{\sum_{k\in\mathbb{Z}^{2}}{1_{\Sigma\left({F}\left\\{f\right\\}\right)}\left(y+k\right)dy}}$
$\displaystyle=$
$\displaystyle\sum_{k\in\mathbb{Z}^{2}}{\int_{{[0,1]}^{2}}{1_{\Sigma\left({\mathcal{F}}\left\\{f\right\\}\right)}\left(y+k\right)dy}}$
$\displaystyle=$
$\displaystyle\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{\int^{1-k_{1}}_{-k_{1}}{\int^{1-k_{2}}_{-k_{2}}{1_{\Sigma\left({\mathcal{F}}\left\\{f\right\\}\right)}\left(z\right)dz}}}$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{2}}{1_{\Sigma\left({\mathcal{F}}\left\\{f\right\\}\right)}\left(z\right)dz}$
$\hskip 28.45274pt\ =|\Sigma({\mathcal{F}}\\{f\\})|\ <\infty$ (4.2)

(by assumption)

Also

$\displaystyle\int_{{[0,1]}^{2}}{\sum_{k\in\mathbb{Z}^{2}}{1_{\Sigma\left(f\right)}\left(x+k\right)dx}}$
$\displaystyle=$
$\displaystyle\int_{\mathbb{R}^{2}}{1_{\Sigma\left(f\right)}\left(x\right)dx}$
$\hskip 28.45274pt=|\Sigma(f)|\ <1$ (4.3)

According to (4.2) and lemma 4.4, there exists an $E\subseteq{[0,1]}^{2}$,
with $|E|=1$

and
$\sum_{k\in\mathbb{Z}^{2}}{1_{\Sigma\left({\mathcal{F}}\left\\{f\right\\}\right)}\left(a+k\right)<\
\infty}$ for all $a\in E$.

Let
$F=\left\\{x\in{[0,1]}^{2}:\sum_{k\in\mathbb{Z}^{2}}{1_{\Sigma\left(f\right)}\left(x+k\right)=0}\
\right\\}$

We have
$|F|>0$ ( otherwise we would have
$\int_{{[0,1]}^{2}}{\sum_{k\in\mathbb{Z}^{2}}{1_{\Sigma\left(f\right)}\left(x+k\right)dx\geq
1}},$ contradiction with (4.3)).

Given $a=(a_{1},a_{2})\in E,$ We define

${\varphi}_{a}\left(t\right)=\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{e^{-2\pi
ia_{1}(t_{1}+k_{1})}f(t+k)e^{-2\pi ja_{2}(t_{2}+k_{2})}},$

for $t=(t_{1},t_{2})\in\mathbb{R}^{2}\ \ $.

As $f\in L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)$ by assumption,
applying lemma 4.3 gives

${\varphi}_{a}\in\ L^{1}(\mathbb{T}^{2},\mathbb{H}),$

Now, Let $g\left(t\right)=e^{-2\pi ia_{1}t_{1}}f(t)e^{-2\pi ja_{2}t_{2}}.$

Straightforward computations show that

${\mathcal{F}}\\{g\\}(y)={\mathcal{F}}\\{f\\}(y+a).$ (4.4)

Therefore, the quaternion Fourier series of ${\varphi}_{a}$ is

$\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{e^{2\pi
it_{1}k_{1}}{\mathcal{F}}\\{f\\}(a+k)e^{2\pi jt_{2}k_{2}}}.$

Indeed,

$\displaystyle{\varphi}_{a}\left(t\right)$ $\displaystyle=$
$\displaystyle\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{g(t+k)}$
$\displaystyle=$ $\displaystyle\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{e^{2\pi
it_{1}k_{1}}{\mathcal{F}}\\{g\\}(k)e^{2\pi jt_{2}k_{2}}}$ $\displaystyle=$
$\displaystyle\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}{e^{2\pi
it_{1}k_{1}}{\mathcal{F}}\\{f\\}(a+k)e^{2\pi jt_{2}k_{2}}}.$

The second equality follows from lemma 4.3, the last by (4.4).

Complexifying the variable$\ \ t=a+i_{\mathbb{C}}{\ b\ }$;${\ a}$
=($a_{1},a_{2}$),${\ b}$ =($b_{1},b_{2}$)$\ \in\mathbb{R}^{2}$ (we note by
$i_{\mathbb{C}}$ the complex number checking $i^{2}_{\mathbb{C}}$= -1).
Since a$\ \in E$, ${\varphi}_{a}\left(t\right)$ is well defined for all
$t\in{\mathbb{C}}^{2}$, and it is a trigonometric polynomial (with quaternion
coefficients).

Hence ${\varphi}_{a}$ is analytic.
So, by lemma 4.5 either ${\varphi}_{a}=0$ everywhere, or ${\varphi}_{a}\neq 0$
a.e.
On the other hand,

$\displaystyle|{\varphi}_{a}(t)|_{Q}$ $\displaystyle\leq$
$\displaystyle\sum_{k=(k_{1},k_{2})\in\mathbb{Z}^{2}}|f(t+k)|_{Q}$
$\displaystyle=$ $\displaystyle 0\ \ for\ t\in F,$

and $F$ has measure greater than zero, thus ${\varphi}_{a}=0$ everywhere for
all $a\in E$.

Whence ${\mathcal{F}}\\{f\\}(a+k)=0$ for all $a\in E$ and
$k\in\mathbb{Z}^{2}$.
Then ${\mathcal{F}}\\{f\\}=0$ a.e.
Since ${\mathcal{F}}\\{f\\}$ is continuous (lemma 3.2), we obtain
${\mathcal{F}}\\{f\\}=0.$
Then by lemma 3.3, we conclude that $f=0$.
This completes the proof of Benedicks–Amrein–Berthier type theorem. $\Box$

######  Corollary 4.7.

Let $f\in L^{p}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)\ for\ p>1$,
If

$|\Sigma(f)|\ |\Sigma({\mathcal{F}}\\{f\\})|<\infty,$

then $f=0.$

Proof. Suppose $f\in L^{p}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)\ with\
p>1$,
since $|\Sigma(f)|<\infty\ $, then by Hôlder inequality we obtain $f\in
L^{1}\left({\mathbb{R}}^{2},{\mathbb{H}}\right)$.
So by the theorem 4.6 we get $f=0.$

which was to be proved. $\Box$

##  5 Conclusion

In this paper, based on quaternionic Fourier series representation, a
generalization of the Benedicks-Amrein-Berthier’s UP associated with the QFT
is proposed. The extension of this qualitative UP to the quaternionic algebra
framework shows that a quaternionic 2D signal and its QFT cannot both
simultaneously have support of finite measure .
In the future work, we will consider Beurling’s UP, Hardy’s UP, Cowling-
Price’s UP and Gelfand-Shilov’s UP for the two-dimensional quaternionic
windowed Fourier transform.

Acknowledgements

The authors would like to thank Eckhard Hitzer for several helpful comments
and Rajakumar Roopkumar for proofreading and discussions related to this work.

## References

  * [1] W.O. Amrein and A.M. Berthier, On support properties of Lp-functions and their Fourier transforms. J. Funct. Anal. 24 (1977), pp. 258-267.
  * [2] M. Benedicks, On Fourier transforms of functions supported on sets of finite Lebesgue measure. J. Math. Anal. Appl. 106 (1985), pp. 180-183.
  * [3] T. Bülow: (1999) Hypercomplex spectral signal representations for the processing and analysis of images. Ph.D. Thesis, Institut für Informatik und Praktische Mathematik, University of Kiel, Germany
  * [4] L.P. Chen, K.I. Kou, M.S. Liu, Pitt’s inequality and the UP associated with the quaternion Fourier transform, J. Math.Anal.Appl.423(2015) pp. 681–700.
  * [5] J.G. Christensen, Uncertainty Principles. Master’s Thesis, Institute for Mathematical Sciences, University of Copenhagen, 2003.
  * [6] Y. El Haoui and S. Fahlaoui, The Uncertainty principle for the two-sided quaternion Fourier transform, Mediterr. J. Math. (2017) 14: 221. https://doi.org/10.1007/s00009-017-1024-5.
  * [7] Y. El Haoui and S. Fahlaoui, Miyachi’s Theorem for the Quaternion Fourier Transform, https://arxiv.org/abs/1711.07519
  * [8] Y. El Haoui and S. Fahlaoui, Beurling’s theorem for the quaternion Fourier transform, J. Pseudo-Differ. Oper. Appl. (2019) DOI: 10.1007/s11868-019-00281-7.
  * [9] T.A. TEll, Quaternion-Fourier transfotms for analysis of two-dimensional linear time-invariant partial differential systems. In: Proceeding of the 32nd Conference on Decision and Control, San Antonio, Texas, pp. 1830–1841 (1993).
  * [10] G. B. Folland and A. Sitaram. The uncertainty principle: a mathematical survey. J. Fourier Anal. Appl., 3(3): pp. 207–238, 1997.
  * [11] E. Hitzer, Quaternion Fourier transform on quaternion fields and generalizations, Adv. Appl. Clifford Algebr., 17 (3), pp. 497-517, 2007.
  * [12] S.C. Pei, J.J. Ding, J.H. Chang, Efficient implementation of quaternion Fourier transform, convolution, and correlation by 2-D complex FFT. IEEE Trans. Signal Process. 49(11), pp. 2783–2797 (2001).
  * [13] W. Rudin. Real and complex analysis. McGraw-Hill Book Co., New York, 1987.
  * [14] E. Hitzer, S. J. Sangwine, The Orthogonal 2D Planes Split of Quaternions and Steerable Quaternion Fourier Transformations, in E. Hitzer, S.J. Sangwine (eds.), ”Quaternion and Clifford Fourier transforms and wavelets”, Trends in Mathematics 27, Birkhauser, Basel, 2013, pp. 15-39.
  * [15] F. Brackx, E. Hitzer, S. Sangwine, History of Quaternion and Clifford-Fourier Transforms, in: E. Hitzer, S.J. Sangwine (eds.), ”Quaternion and Clifford Fourier Transforms and Wavelets”, Trends in Mathematics (TIM) Vol. 27, Birkhauser, Basel, 2013, pp. xi-xxvii.