# FederatedTrust: A Solution for Trustworthy Federated Learning

Pedro Miguel Sánchez Sánchez  Alberto Huertas Celdrán  Ning Xie  Gérôme Bovet
Gregorio Martínez Pérez  Burkhard Stiller  Department of Information and
Communications Engineering, University of Murcia, Murcia 30100, Spain
Communication Systems Group (CSG), Department of Informatics (IfI), University
of Zurich UZH, 8050 Zürich, Switzerland  Cyber-Defence Campus, armasuisse
Science & Technology, 3602 Thun, Switzerland

###### Abstract

With the ever-widening spread of the Internet of Things (IoT) and Edge
Computing paradigms, centralized Machine and Deep Learning (ML/DL) have become
challenging due to existing distributed data silos containing sensitive
information. The rising concern for data privacy is promoting the development
of collaborative and privacy-preserving ML/DL techniques such as Federated
Learning (FL). FL enables data privacy by design since the local data of
participants are not exposed during the creation of the global and
collaborative model. However, data privacy and performance are no longer
sufficient, and there is a real necessity to trust model predictions. The
literature has proposed some works on trustworthy ML/DL (without data
privacy), where robustness, fairness, explainability, and accountability are
identified as relevant pillars. However, more efforts are needed to identify
trustworthiness pillars and evaluation metrics relevant to FL models, as well
as to create solutions computing the trustworthiness level of FL models. Thus,
this work analyzes the existing requirements for trustworthiness evaluation in
FL and proposes a comprehensive taxonomy of six pillars (privacy, robustness,
fairness, explainability, accountability, and federation) with notions and
more than 30 metrics for computing the trustworthiness of FL models. Then, an
algorithm called FederatedTrust has been designed according to the pillars and
metrics identified in the previous taxonomy to compute the trustworthiness
score of FL models. A prototype of FederatedTrust has been implemented and
deployed into the learning process of FederatedScope, a well-known FL
framework. Finally, four experiments performed with different configurations
of FederatedScope using the FEMNIST dataset under different federation
configurations (participant numbers, participants selection rate, training
rounds, and differential privacy) demonstrated the usefulness of
FederatedTrust when computing the trustworthiness of FL models.

###### keywords:

Trustworthy Federated Learning , Data Privacy , Federation Trustworthiness ,
Robustness , Fairness , Explainability , Accountability

##  1 Introduction

The last decade has been a revolutionary time for Artificial Intelligence (AI)
[1]. IBM Watson, ImageNet, or AlphaGo were some of the first successful AI
solutions that defined the path towards the recent ChatGPT, DALL·E 2, or Tesla
Autopilot, among many others. This journey has allowed Machine and Deep
Learning (ML/DL) models to learn how to play, see, speak, paint, drive, and do
many other things, almost like humans. Traditionally, the AI hype has been
focused on achieving ever-higher accuracy and performance. However,
performance is no longer sufficient. In the last few years, we started to hear
more mishaps and situations in which wrong AI-based decisions negatively
affect human lives. Some examples are i) ML/DL-based systems supporting judges
in pretrial recidivism scoring racially biased [2], ii) ML/DL models of
autonomous vehicles not prepared nor trained for uncommon fatalities [3], or
iii) AI-powered chatbots giving wrong answers to straightforward questions and
problems [4]. In conclusion, these situations erode the trustworthiness of AI
and raise concerns about Responsible AI (RAI) [5].

Trustworthy AI is an emerging concept towards RAI that embraces several
existing terms such as explainable AI (XAI) [6], ethical AI [7], robust AI
[8], or fair AI [9], among others. In 2019, the European Commission published
the ethics guidelines for AI with high-level foundations, principles, and
requirements that AI systems should fulfill to be trustworthy [10]. According
to these guidelines, three main foundations should be met throughout the AI
system life cycle. First, AI should be lawful and comply with existing
regulations. Second, it should ensure adherence to ethical principles. Last
but not least, AI should be robust from technical and social perspectives.
Under these three foundations, respect for human autonomy, prevention of harm,
fairness, and explainability are four ethical principles that must be
respected by trustworthy AI systems. Finally, the European Commission
translated these principles into the following seven requirements to achieve
trustworthy AI: i) human agency and oversight, ii) technical robustness and
safety, iii) privacy and data governance, iv) transparency, v) diversity, non-
discrimination, and fairness, vi) societal and environmental wellbeing, and
vii) accountability. In parallel to the European Commission, researchers have
also developed specific approaches and techniques that AI systems should adopt
to be trustworthy. In this context, the systematic reviews on Trustworthy AI
conducted in [11, 12] identified robustness, privacy, fairness,
explainability, and accountability as the five key pillars of trustworthy AI.

Trustworthiness is a critical aspect influencing AI, but nowadays, it is not
the only one, and data privacy and protection are also highly demanded by our
society. In this context, new laws and regulations have been drawn in response
to this necessity. The General Data Protection Regulation (GDPR) in the
European Union and the California Consumer Privacy Act (CCPA) in the state of
California (USA) are well-known examples of new data protection regulation
[13]. As expected, these changes affect AI systems since most ML/DL models are
trained with data belonging and maintained by different stakeholders in
different silos. Therefore, to deal with the challenge of preserving data
privacy in AI, Federated Learning (FL) [14] was proposed in 2016 by Google as
a decentralized ML paradigm. FL builds collaborative models between the
federation members while keeping sensitive data within the premises and
control of each participant. In summary, FL is one solution to data silo and
fragmentation issues caused by the new legislation that prohibits the free
sharing of data and forces data to be maintained by isolated data owners [15].
FL is highlighted as a key player in the present and future of AI since its
global market size is projected to be USD 210 million by 2028 [16].

The relevance of FL drives academic and industrial actors to consider the
trustworthiness of FL model predictions. To better understand the magnitude of
trustworthiness in FL, a comparison with centralized ML/DL has to be made.
Like traditional centralized ML/DL, FL is subject to common risks of
algorithmic bias, adversarial attacks, data privacy breaches, and reliability
issues, among others. However, unlike centralized ML/DL, FL involves different
and more stakeholders, actors, information exchanges, communication
infrastructures, and attack surfaces that must be considered to assess the
trustworthiness of FL. In addition, FL presents new challenges regarding
architectural designs, opportunities for privacy-preserving standards, and new
perspectives on fairness and explainability beyond the underlying ML/DL models
[17]. In this context, the literature lacks work studying and analyzing
trustworthiness pillars and metrics specifically relevant to the context of
FL. Furthermore, there is a lack of tools, algorithms, or solutions assessing
the trustworthiness level of heterogeneous FL models. Last but not least,
there is a need for solutions dealing with trustworthy AI that can be
transparently integrated into existing FL frameworks to provide not only
traditional performance metrics but also metrics dealing with trustworthy FL.

To cover the previous literature gaps, the work at hand presents the following
contributions:

  * 1.

The creation of a taxonomy with the most relevant pillars, notions, and
metrics to compute the trustworthiness of FL models. To create such a
taxonomy, pillars, notions, and metrics used to evaluate the trustworthiness
of classical and federated ML/DL models were studied, analyzed, and compared.
More in detail, the following six pillars were identified as the main building
blocks of the taxonomy: privacy, robustness, fairness, explainability,
accountability, and federation. Each pillar represents a dimension of trust
broken down into different notions per pillar, which are then quantified by
more than 30 metrics.

  * 2.

The design and implementation of FederatedTrust, an algorithm quantifying the
trustworthiness of FL models based on the pillars, notions, and metrics
presented in the proposed taxonomy. FederatedTrust has been implemented as a
prototype and deployed in a well-known FL framework called FederatedScope.
Considering the inner workings of the FederatedScope framework, the prototype
algorithm is a Python library (publicly available in [18]) that can be
imported into FederatedScope (and other FL frameworks) to generate the inputs
needed for the trust evaluation process.

  * 3.

The validation of the FederatedTrust prototype in FederatedScope through four
experiments classifying hand-written digits in a cross-device FL context using
the FEMNIST dataset. Experiments introduced differences in the number of
participants in the federation, training rounds, sample rates, and
countermeasures against attacks. For all experiments, the trustworthiness
values of the FL models were computed (together with the other performance
evaluation results) and compared to identify the factors impacting the
trustworthiness level of FL.

The remainder of this paper is structured as follows. Section 2 contains
findings from the literature review on trustworthy FL. Section 3 identifies
and presents a detailed analysis of the following six trustworthy FL pillars
and their metrics: robustness, privacy, fairness, explainability,
accountability, and federation. Section 4 presents the design detail of the
proposed algorithm, while Section 5 focuses on its implementation and
deployment on a real FL framework. Section 6 validates the algorithm in a use
case and presents results from the performed experiments. Finally, Section 7
provides conclusions and future work.

##  2 Related Work

This section reviews existing solutions focused on trustworthy FL and well-
defined pillars relevant to trustworthy AI, such as robustness, privacy,
fairness, explainability, and accountability [11]. It is important to mention
that a large body of literature on trustworthy centralized ML/DL has emerged
in recent years. However, trustworthy FL is a nascent research field that
requires more effort.

FedEval [19] is the closest solution to the one proposed in this paper because
it combines several aspects relevant to trustworthy AI. More in detail,
FedEval is an open-source framework for FL systems that evaluates the
accuracy, communication, time efficiency, privacy, and robustness of FL models
to compute their trustworthiness level. FedEval was designed as a benchmark
system with a built-in evaluation model. Regarding accuracy, it compares the
performance of FL and the centralized training to determine whether the FL
model has achieved better or worse accuracy. The communication metric
measurement relies on the number of communication rounds and the total amount
of data transmission during training. The time efficiency metric measures the
overall time needed for getting a converged model and the time needed for sub-
modules in the model. The privacy metric is calculated by implementing two
state-of-the-art gradient attacks in the framework, checking the model
accuracy, or reviewing attack results. Finally, robustness metrics are simply
the performance under Non-IID data. Another solution focused on quantifying
the trustworthiness of AL models is presented in [20]. The authors propose an
extensible, adaptive, and parameterized algorithm to quantify the
trustworthiness level of supervised ML/DL models with tabular data according
to their robustness, explainability, fairness, and accountability. The main
limitation of this work is that it is not suitable for FL models.

Privacy is the central point of FL since its main objective is to protect data
privacy among the federation participants. Therefore, it is crucial to
preserve and quantify data privacy effectively to trust FL model predictions.
In this context, several works and techniques can be categorized into three
main families: i) encryption-based, ii) perturbation-based, and iii)
anonymization-based. In the encryption-based category, [21] designed a
privacy-preserving protocol against a semi-honest adversary by combining
Ternary Gradients [22] with secret sharing and homomorphic encryption. In
addition, the authors of [23] developed a privacy-preserving entity resolution
and an additively homomorphic encryption scheme for vertical FL scenarios.
[24] designed the Secure Aggregation aggregator by leveraging secure
multiparty computation to perform sums of model parameter updates from
individual users’ devices in a secure manner. In the perturbation category,
the global differential privacy scheme has been widely used in many FL
methods. In this sense, [25] demonstrated that global differential privacy
offered a strong level of privacy when protecting sensitive health data in an
FL scenario. In addition, [26] proposed a procedure using differential privacy
to ensure that a learned model does not reveal whether a client participated
during decentralized training. Unlike perturbation-based techniques,
anonymization-based techniques can provide privacy defense without
compromising data utility. In this category, [27] proposed using the
k-anonymity scheme at the client level with a global anonymization mapping
process to achieve better privacy preservation and model performance than
differential privacy-based FL models. Finally, to measure privacy in FL, [28]
proposed a novel method to approximate the mutual information between local
gradient updates and batched input data during each round of training.

Although FL provides a first level of data protection by not sharing training
data, the FL paradigm is still vulnerable to adversarial attacks affecting
data privacy (inference attacks) and model performance (poisoning attacks)
[29]. Therefore, robustness is an important pillar to consider in trustworthy
FL. To improve the FL robustness against adversarial attacks affecting data
privacy and model performance, the authors of [29] proposed the usage of
differential privacy, robust aggregation, and outlier detection as primary
defenses. In the same direction, [30] showed empirical evidence that
differential privacy could defend against backdoor attacks and mitigate white-
box membership inference attacks in FL. [31] introduced Adaptive Federated
Averaging (AFA), a Byzantine-robust FL algorithm that detects and discards bad
or malicious client updates at every iteration by comparing the similarity of
the individual updates to the one for the aggregated model. From a similar
perspective, [32] proposed Robust Filtering of one-dimensional Outliers
(RFOut-1d), a new FL approach resilient to model-poisoning backdoor attacks by
filtering out the model updates of the adversarial clients. Finally, regarding
mechanisms and metrics able to evaluate the robustness of FL, the literature
has focused on methods to quantify robustness once the model is trained. Some
examples of these metrics are loss sensitivity, empirical robustness, or
CLEVER score [33].

Fairness is another essential pillar for trustworthy FL, as multiple parties
contribute data to the model training and are eventually rewarded with the
same aggregated global model. In this sense, [34] provided a survey and
overview of fairness notions adopted in FL-based solutions. The notions
include: i) accuracy parity, which measures the degree of uniformity in
performance across FL client devices; ii) selection fairness, which aims to
mitigate bias and reduce under-representation and never representation; and
iii) contribution fairness, which aims to distribute payoff proportionately to
the contributions of clients. Apart from that, the authors of [35] proposed
GIFAIR-FL. This framework imposed group and individual fairness to FL settings
by penalizing the spread in the loss of clients to drive the optimizer to fair
solutions. FairFL [36] is another solution that facilitated fairness across
all demographic groups by employing a Multi-Agent Reinforcement Learning-based
scheme. This approach solved the fair classification problem in FL by
enforcing an optimal client selection policy on each client. The authors of
[37] proposed a long-term fairness constraint that considered an expected
guaranteed chosen rate of clients that the selection scheme must fulfill.
Finally, [38] proposed the Completed Federated Shapley Value (ComFedSV) to
evaluate data owners’ contributions in FL based on solving a low-rank matrix
completion problem for the utility matrix.

Even with active research on XAI, there are still challenges specifically for
FL models, as most client data are private and cannot be read or analyzed. In
this context, some explainability methods, such as feature importance, reveals
underlying feature information from other parties. For horizontal FL models,
since clients share the same feature space, the authors of [39] suggested that
predictions could be explained by calculating the Shapley value of each
feature using the definition provided by [40]. For vertical FL models, their
work proposed a variant version of SHAP [41] by combining the participant
features into individual united feature space. Therefore, participants do not
get information about the features of other participants. Another solution
called EVFL was proposed in [42], where authors presented a credible federated
counterfactual explanation method to evaluate feature importance for vertical
FL models and minimize the distribution of the counterfactual and query
instances in the client party.

Even though FL models are promising regarding data privacy, they require
transparency and accountability, as in the case of classical centralized ML/DL
models. In this sense, IBM introduced the Accountable FL FactSheet framework
[43] that instruments accountability in FL models by fusing verifiable claims
with tamper-evident facts. The framework requires different actors, like the
project owner, data owner, or aggregator, to log claims about the various
processes occurring during the FL training lifecycle. They also expanded the
IBM AI FactSheet 360 [44] project to account for the complex model
compositions of FL. Finally, [45], [46] and [47] incorporated blockchain and
smart contracts to add different auditing and accountability mechanisms to FL
models by leveraging the immutability and decentralized trust properties of
blockchain.

In conclusion, this section has reviewed the work done in each dimension or
pillar relevant to trustworthy AI. As can be seen, there is a lack of
solutions quantifying the trustworthiness level of FL models by combining the
different pillars identified by related work. Most solutions focus on isolated
pillars and improving the pillar aspect instead of assessing or quantifying
its status, which is needed before deploying countermeasures. In addition,
there is no solution dealing with aspects related to the architectural design
of the federation.

##  3 Trustworthy FL: Pillars, Notions, and Metrics

Despite the research done in the field of trustworthy AI, there is a lack of
work focused on assessing the trustworthiness of FL models. Therefore, this
work identifies, introduces, and explains for the first time (to the best of
our knowledge) the most relevant pillars, notions, and metrics for Trustworthy
FL. Additionally, it proposes a novel pillar called Federation, which is not
considered in the current literature. This pillar focuses on capturing complex
compositions and design challenges of FL architectures to compute their
trustworthiness.

Figure 1 presents a taxonomy describing the pillars, notions, and metrics
relevant for trustworthy FL. Under each pillar, the major aspects defining it
are grouped into notions. Under each notion, the specific metrics that can be
calculated to quantify the level of utility towards trustworthiness are
defined. This taxonomy serves as the baseline for evaluating and assessing the
trustworthiness level of FL models.

Figure 1: Trustworthy FL Taxonomy

###  3.1 Privacy

Data privacy is the most prominent driving force for the development of FL.
Therefore, FL models must preserve data privacy within their lifecycle to gain
participants’ trust. Even though FL already elicits a degree of data privacy
by definition, assumptions have to be made about the integrity of the multiple
actors and parties making up the federation. If participants are honest, but
the aggregating server is honest-but-curious, prevention of information
leakage from model parameter exchanges needs to be in place. If all federation
members are honest-but-curious, then the prevention of information leakage
should focus on secure communication. Moreover, information can still be
leaked by malicious attacks from outside.

To cover these aspects, the first notion of this pillar is the usage of
privacy-preserving approaches for improved resilience to privacy attacks. The
second focuses on metrics measuring information gain/loss based on the
information leakage risk derived from the FL process. Finally, two additional
notions arise from the probability of knowledge inference from the client
updates.

Privacy-preserving approaches. This notion considers the following main
approaches to protect data privacy in FL.

  * 1.

Perturbation. It adds noise to the raw data, so the perturbated data are
statically indistinguishable from the raw data. The most widely adopted scheme
is Differential Privacy. In FL, there are global and local Differential
Privacy [48]. The global approach adds noise to the parameters shared after
the local model is trained, while the local one adds noise to each client data
sample used for training.

  * 2.

Encryption. It encrypts the model parameters of each participant before
sharing it. The most widely adopted scheme is Homomorphic Encryption because
the aggregation server does not have to decrypt the parameters to aggregate
them into the global model [49]. Another popular scheme is Secure Multiparty
Computation (SMC), which allows participants to collaboratively calculate an
objective function without revealing their data [50].

  * 3.

Anonymization. The most widely adopted schemes in this approach are
k-anonymity and l-diversity. K-anonymity is satisfied if each sample in the
dataset cannot be re-identified from the revealed data of at least $k-1$
clients [27]. L-diversity extends on k-anonymity so that the sensitive
attributes of the samples are protected.

Information Gain/Loss. This notion focuses on measuring the amount of privacy
lost by participants or the amount of information gained by adversaries due to
leakage or disclosure of information [51].

  * 1.

Information Leakage Risk. In an FL system, the gradients can carry enough
information for adversaries to reconstruct the model or infer original data.
H-MINE [28] is a hierarchical mutual information estimation metric to measure
the mutual information between the high-dimensional gradients and batched
input data. The amount of leaked information (counting the information items
disclosed by a system), relative entropy (measuring the distance between two
probability distributions), or mutual information (quantifying the shared
information between two random variables) are other methods to compute this
metric.

Uncertainty. The uncertainty of data estimation by adversaries makes a
difference in the level and effectiveness of a data privacy breach. Under
normal circumstances, high uncertainty estimation correlates with high
privacy. This notion considers metrics to measure uncertainty, most of which
are based on entropy.

  * 1.

Entropy. In general, entropy measures the uncertainty in predicting the value
of a random variable. In FL, an adversary may be interested in identifying
which data samples belong to a particular client or organization participating
in the training. Equation 1 calculates the entropy of $X$, where $X$ is a
participating client and $p(x_{i})$ is the estimated probability of this
client being the target.

$priv_{ENT}\equiv H(X)=-\sum_{x\in{X}}p(x)\log_{2}p(x)$ (1)

Indistinguishability. Some adversaries are also interested in distinguishing
between two data samples of interest, and the indistinguishability notion
analyzes whether an adversary can do so. In general, privacy is high if the
adversary cannot distinguish between two outcomes of the model.

  * 1.

Global Privacy Risk. It enables indistinguishability in the training data by
adding random noise. It is a formal statistical guarantee that any disclosure
is equally likely whether a sample is in the dataset or not. Equation 2
describes the formal differential privacy proof for privacy mechanism. A
randomized function $K$ checks if the output of random variables for two
datasets $D_{1}$, $D_{2}$ that differ at most to some extent (e.g., one row of
data), differ by at most $exp(\epsilon)$:

$\begin{split}priv_{DP}\equiv\forall{S}\subseteq
Range(\textit{K}):p(K(D_{1})\in{S})\leq\exp{(\epsilon)}\\\
\cdot{p(K(D_{2}\in{S})}\end{split}$ (2)

###  3.2 Robustness

Robustness is one of the three foundations of trustworthy AI, together with
lawfulness and ethics, as defined by the European Commission AI ethics
guidelines [10]. AI systems must be technically robust to ensure that they are
not vulnerable to malicious use or bring harm to humans. In this sense, the
literature has considered three different notions, and this work proposes a
new one to assess the robustness of FL models. According to the literature
[12], FL models must be resilient to adversarial attacks adding perturbations
or erroneous inputs. Secondly, the hardware and software of participants
training and deploying FL models must be robust to avoid cyberattacks [52].
Thirdly, FL algorithms performance and customization must be reliable and
robust [53]. Last but not least, this work proposes adding client and data
reliability as a novel notion since reliable clients and data increase the
probability of robust and reliable FL models. More details of each notion and
its metrics are provided below.

Resilience to Attacks. FL models are susceptible to poisoning attacks
affecting the model training and its robustness. Poisoning attacks can be
categorized into data poisoning and model poisoning. In data poisoning, the
integrity of the training data is compromised. Common methods are flipping or
permuting the labels and inserting backdoor patterns or perturbations into the
training data. Model poisoning attacks have a broader range, and the goal is
to manipulate the training procedure. In FL, this could be gradient
manipulation or model update poisoning attack, which is performed by
corrupting the updates of a participant directly or during model exchanges.
Therefore, evaluation of this notion is usually performed by first checking
whether the FL model is equipped with any defense mechanism and, if so,
verifying the model defense capabilities against representative attacks
empirically.

  * 1.

Poisoning Defense. It focuses on providing defense mechanisms against
poisoning attacks. On the one hand, Byzantine-resilient Defense is a popular
defense mechanism where various robust aggregation methods have demonstrated
their effectiveness in detecting malicious client updates and reducing their
impact [17]. On the other hand, Outlier Detection explicitly identifies and
denies negative influence as a more proactive defense against poisoning
attacks. Existing approaches include rejecting updates with too large error
rates, measuring the distribution of parameter updates, or looking for dormant
neurons that are not frequently activated [54].

  * 2.

Empirical Robustness. If small changes in the input data cause significant
output deviations, adversarial perturbations can be used to generate undesired
outcomes. It can be measured by implementing a model poisoning attack, a
typical poisoning attack altering some local data (data poisoning), or the
gradients (model poisoning) [55]. A mathematical explanation of how a model
replacement attack works is provided by [54] and illustrated as follows.

It is assumed that at least one compromised client could apply the backdoor
patterns to perform a model replacement attack. Equation 3 describes how a
poisoned model update is generated. Where $w_{t}$ is the current model and $N$
the number of all clients. The global model $w$ at time $t+1$ is an averaged
mean of model updates from $N$ clients at time $t+1$, and the goal is to
replace the global model $w$ at $t+1$ with the attacker’s model $x_{atk}$:

$x_{atk}=w_{t+1}=w_{t}+\frac{1}{N}\sum_{i=1}^{N}(x_{t+1}^{i}-w_{t})$ (3)

Now, $x_{t+1}^{m}$ is the update from the malicious client $m$ at time $t+1$,
then rearranging Equation 3, we have:

$x_{t+1}^{m}=N\cdot x_{atk}-N\cdot
w_{t}-\sum_{i=1}^{N-1}(x_{t+1}^{i}-w_{t})+w_{t}$ (4)

Assuming $\sum_{i=1}^{N}(x_{t+1}^{i}-w_{t})\approx{0}$ as explained in [56],
we have the attacker’s update be simplified as the following:

$x_{t+1}^{m}=N\cdot(x_{atk}-w_{t})+w_{t}$ (5)

  * 3.

Certified Robustness. It defines the least amount of perturbation required for
the attacker to succeed (change the model prediction). In other words, a model
is certifiably robust for an upper-bounded amount of perturbations. The CLEVER
(Cross Lipschitz Extreme Value for nEtwork Robustness) metric [57] using the
local Lipschitz constant for neural networks is one of the most well-known
metrics. CLEVER is an attack-agnostic derivation of the universal lower bound
on the minimal distortion required for a successful attack.

System-level Robustness. This notion deals with hardware and software
robustness and must be considered in production environments with FL models
where proper software development and deployment standards are needed. Testing
coverage and system reliability are the main metrics for this notion.

  * 1.

Testing Coverage. It guarantees that the developed functionaries in the client
federation adhere to the FL requirements. This is important to FL because an
FL model evolves critical functionality like broadcasting messages to
distributed clients, client selection, and model aggregation. This metric can
be implemented using different methodologies, such as robust system delivery,
ranging from code review, unit testing, integration testing, system testing,
and acceptance testing [58].

  * 2.

System Reliability. It deals with the probability and the duration of time of
failure-free operation [59]. It is measured based on error, timeout, and
dropout rates. The error rate is normally calculated as the number of failures
over a given amount of time. The maximum timeout measures the time the server
should wait to receive client model updates. Finally, the dropout rate is the
number of clients leaving the federation (to speed up convergence, optimize
resources, or due to errors) divided by the total amount of clients. A high
dropout rate can also indicate a less reliable FL system.

Algorithm-level Robustness. This notion deals with the performance and
generalization of FL algorithms. Performance is widely used to showcase how
good an ML/DL model is. However, good performance does not necessarily imply
generalization. Generalization is a major challenge in FL because each client
has different local data heterogeneity, and the aggregated global model might
not be able to capture the data pattern of each of them. Non-IID data [60] may
cause severe learning divergence to parametric models. Therefore, it is
desirable to have metrics to measure the performance, and generalization of FL
[61].

  * 1.

Performance. In FL, there are two approaches to measuring performance. One is
reserving a set of test or validation data on the server side to be used for
global model evaluation. The other is by evaluating test accuracy at each
client’s device and aggregating the test accuracy. This metric considers these
two aspects.

  * 2.

Personalization. Several methods have been proposed for this category of
metrics. Regularization is one of the personalized FL approaches aiming to
minimize the disparity between global and local models. Multi-tasking Learning
is a method where multiple learning tasks are solved simultaneously. This
benefits FL models since multiple organizations participating in the FL models
can train their personalized models to achieve better performance. For
example, MOCHA [62] generates separated but related models on local client
devices using data of related tasks. Clustering is another personalization
approach where clients are allocated into clusters based on their similarity
[63]. Personalized Layer is a metric for neural network models that introduces
custom layers for each client in the model. FedPer [64] is an example that
uses the base layers as the shallow layers and personalized layers as the deep
layers while keeping everything else the same as the baseline FedAvg
algorithm.

Client and Data Reliability. This novel notion of robustness deals with client
and data reliability. In this sense, metrics like federation scale, clients’
reputation, and data quality are essential for robust and trustworthy FL
models.

  * 1.

Federation Scale. The number of clients impacts the reliability of the system.
In the case of FL, the number of clients determines the number of distributed
networks, devices, or machines that must work together seamlessly to co-train
FL models. With more clients, the network stability, computation power, and
availability of clients are all influential factors in the success and
reliability of the FL system.

  * 2.

Client Reputation. In [53], a novel reputation metric was proposed, and a
subjective logic model was used to calculate the reputation score for each
client interaction. After each training iteration, the server uses a poisoning
attack detection scheme and the elapsed time to determine if the local update
from the client is reliable. Reliable updates are treated as positive
interactions and improve the reputation value, and vice versa. The client is
treated as malicious and unreliable when the reputation value is below a
certain threshold.

  * 3.

Data Quality. It compares, in each training round, the local updates with the
current global model to see if the new local updates are better or worse than
the global model [65]. If a round of training at one client improves the
performance of the global model, then the data from that client has a high-
quality score.

###  3.3 Fairness

One primary source of unfairness in AI is coming from the data. In FL,
different clients might contribute with heterogeneous amounts and quality of
data. When data are not representative of the wider federation, participant
selection bias is propagated into the model. The selection bias can be
manifested by feature distribution skew or label distribution skew [17], both
of which are major challenges in FL. Therefore, Client Selection Fairness is
the first notion of this pillar. Apart from that, in fair AI [9], fairness is
broken down into group-level and individual-level fairness. Group-level
fairness means that members of a particular group should not be subject to
discrimination. Individual-level fairness means that similar individuals
should receive similar treatment regardless of their membership group. These
requirements do not change when moving to FL. Therefore, the Group-level
Fairness notion deals with the group-level aspects, and the Performance
Fairness and Class Distribution notion covers individual-level ones. More in
detail, Performance Fairness ensures that each client’s reward should be
proportional to their data contribution. Finally, Class Distribution looks at
the label imbalance in the dataset of each participant.

Client Selection. Usually, in FL, only a fraction of clients are selected to
participate in the training process of each round. In practice, there are
several criteria for selection, such as availability, network speed,
computation power, or battery level, among others. In the worst case, clients
located in regions with low network speed or models with weaker computation
power could never get selected and represented in the training data.
Therefore, under-representation is a source of selection bias that could lead
to unfair model outcomes.

  * 1.

Participation Variation. In statistics, the Coefficient of Variation (CV)
measures how far the data values from a set are dispersed from the mean. This
metric analyzes the distribution of participation rates among all clients.
With similar clients, the more dispersed the distribution of participation
rate, the less fair the client selection mechanism is, and vice versa [66].
Equation 6 calculates the CV in the participation rate, where $\sigma$
represents the standard deviation, and $\mu$ represents the average of the
participation rate.

$CV=\frac{\sigma}{\mu}$ (6)

Performance. Even though performance fairness already exists in AI, in FL,
another grouping needs to be considered, which is client-level fairness. [67]
suggests a definition that a model provides a more fair solution to the FL
learning objective on the clients if the performance is more uniform than that
of another model.

  * 1.

Accuracy Variation. This metric considers the test accuracy as a
representation of performance. More in detail, the aggregated global model and
test data from each client are used to measure the test accuracy. The more
uniform the test accuracy among the clients, the more fair the model
performance is.

Group-level. Evaluating and mitigating demographic bias in FL is more
difficult than in centralized learning. First, raw training data, labels, and
sensitive demographic information of each participant cannot be revealed.
Second, in centralized learning, all the training data can be analyzed and
pre-processed to balance the class distribution before training. In contrast,
different clients pre-process their data locally in FL, so additional
mechanisms are needed to adjust the global data distribution in a secure and
protected manner.

  * 1.

Discrimination Index. The discrimination index metric [36] measures the
difference in the F1 score between a particular demographic group ($\sigma$)
and the rest of the population. The metric value falls between $[-1,1]$, where
the ideal discrimination index should be as close to 0 as possible.
Calculating this index globally would reveal sensitive attributes and
statistics of the demographic group in the client data. Equation 7 calculates
the discrimination index, where $F1(X^{+}_{\nu})$ represents the F1 score of
all the samples from the protected group and $F1(X^{-}_{\nu})$ represents the
F1 score of the rest of the samples.

$\Phi_{\sigma}=F1(w(X^{+}_{\sigma}))-F1(w(X^{-}_{\sigma}))$ (7)

Class Distribution. Analyzing the class distribution of the training data used
in an ML/DL model provides insight into whether the data samples are selected
properly to reflect a fair representation of the wider group. In theory, this
should also apply to FL models, except that in practice, this often requires
access and analysis of the raw training data, which goes against the priority
of privacy protection in FL. However, in [68], an estimation scheme was
proposed to reveal the class distribution without the awareness of raw data
specifically for the FL setting. Furthermore, secure aggregation can also be
considered for aggregating class distribution information among clients.

  * 1.

Class Imbalance. Two approaches can be used to evaluate the class imbalance.
One is the estimation scheme using a well-balanced auxiliary dataset and the
gradients of a neural network model [68]. Another general way to get the class
imbalance information in FL is to ask every client to submit their class
distribution to the server. The secure aggregation method combines all the
class distributions into one unified distribution. Then the coefficient of
variation of the class distribution can be used to calculate the level of
variation of the class sample sizes to determine the class imbalance level.

###  3.4 Explainability

AI guidelines demand that AI processes should be transparent, with the
capabilities and the purpose of AI systems openly communicated and decisions
explainable to those directly and indirectly impacted. Additionally,
transparency is one of the key requirements for achieving Trustworthy AI.
Transparency is often expressed as interpretability which is often wrongly
mistaken as interchangeable with explainability. Interpretability is the first
notion of this pillar and can be described as a passive characteristic of a
model referring to the level of understandability for humans. In contrast,
explainability is the ability to provide a description of the technical
process of an AI system. Interpretable models can be explained by analyzing
the model itself, but for non-interpretable models, Post-hoc Explainability
Methods are diverse means to enhance interpretability, being the second notion
of this pillar. For FL, since ML/DL models are also used in the training
process, the requirement of explainability for the algorithmic model also
applies. However, the privacy constraint makes it difficult to access and
analyze raw data or protected model features directly.

Interpretability. This notion combines the algorithm transparency and the
model size to evaluate the FL model interpretability. As already identified in
the literature, some ML/DL models are interpretable by design, and some are
not. In addition, model size is also widely used as a metric of
interpretability. More details about these two metrics are provided below.

  * 1.

Algorithmic Transparency. A model is considered transparent if it is
understandable by itself. This definition could be very subjective to
different levels of intellectual grasp. However, algorithmically transparent
models must first be fully explorable through mathematical analysis and
methods. Then the assessment considers model complexity (in terms of the
number of variables and interactions) and decomposability (in terms of the
interpretability of each component of the model) [69].

  * 2.

Model Size. Different algorithms have diverse definitions of model size. For
instance, it could be the number of decision rules, the depth of a decision
tree, the number of features in a linear/logistic regression model or the
number of trainable parameters in a neural network [70]. The larger the model
size, the harder it is to understand and explain the causal relationship
between input and output.

Post-hoc Methods. The three most common post-hoc explanation methods are
simplification, feature importance, and visualization [69]. Simplification
means reducing the complexity of a non-interpretable model into a simpler
surrogate model that is more easily interpreted and understood. The feature
relevance explanation method measures each feature influence, relevance, or
importance in the prediction output, hoping to explain what aspect of the
training population causes certain model behavior. Finally, visualization
methods provide visual representations of AI models to facilitate better
understanding by humans. Below more details about the two last metrics are
provided.

  * 1.

Simplification. The idea of simplification lies in reducing the number of
architectural elements or parameters in a model. One of the main techniques
applied for model simplification is knowledge distillation [71].

  * 2.

Feature Importance. Most model explanation methods can be directly used for
horizontal FL because all participants share the full feature space in their
local data [39]. However, exposing feature information to the server for
calculating feature importance score is not ideal. For vertical FL, methods
like SHAP cannot be directly used because parties do not share the full
feature space. The variant version of SHAP for Vertical FL combines one
party’s features into a federated feature space when referenced by another
party for the feature importance calculation [39].

  * 3.

Visualization. A lifecycle dashboard that visualizes information from the
server, starting from client registration to training, validation, and
deployment, was proposed in [72]. The dashboard shows which clients
participated in which round of training and the current status of the model.

###  3.5 Accountability

Accountability is another of the seven critical requirements of Trustworthy AI
defined by the EU guidelines [10]. The first main notion about accountability
is FactSheet Completeness. IBM Research was the first to propose a document
called FactSheet in charge of recording facts about the overall ML/DL pipeline
[73]. Another important notion of accountability is Monitoring. Even with
complete and detailed documentation, every participant has to make an effort
to ensure that FL models are built strictly following the intended
architecture, development, and deployment processes.

FactSheet Completeness. IBM extended the FactSheet approach to enable
accountability in FL as well [43]. The accountable FL FactSheet template is a
comprehensive document that contains meta-information about the project,
participants, data, model configurations, and performance. Since FL is more
complicated in architecture and more privacy-preserving, the FactSheet should
contain information about the additional layer of configurations and avoid
sensitive information about participating clients. Below more details about
the aspects considered in FactSheets are provided.

  * 1.

Project Specification. This section of the FactSheet documents the project
overview, purpose, and background. The overview explains what the project is
about. The purpose details the goals, and the background elaborates on the
relevant information and knowledge motivating the project.

  * 2.

Participants. It contains the participants of the FL process. The template
considers the participants’ names and their organization unit names for
identity verification.

  * 3.

Data. It documents the information regarding the data used in the FL process.
Two aspects are included: data provenance and pre-processing procedures. Data
provenance helps trace the data origin and flow to access validity and even
reputation. Pre-processing steps can tell us how the raw data have been
handled before being fed to the model for training.

  * 4.

Configuration. It deals with the information about the FL model configuration.
First, it contains the type of optimization algorithm and the ML/DL model.
Then, it indicates the global hyper-parameters the aggregator uses, for
instance, the number of rounds, the maximum timeout, and the termination
accuracy. Lastly, it contains the local hyper-parameters used by the trainer
at each client, such as the learning rate and the number of epochs.

  * 5.

System. This FactSheet section documents the system information for the
learning process. It includes the average time spent on training, the model
size, the model upload, and the download speed in bytes. This information
indicates the number of resources expected to be utilized for the process.

Monitoring. For AI systems, algorithmic auditing is a range of approaches to
auditing algorithmic processing systems. The evaluation of the metrics under
this notion uses a checklist-based approach, verifying if the FL system
employs any external or internal algorithmic auditing.

  * 1.

Algorithmic Auditing. This metric can be implemented in different ways. For
example, it could be functional testing, performance testing, user acceptance
testing, etc. It could also be system anomaly or attack monitoring. Some
organizations even invite or hire external hackers to find vulnerabilities as
a measure of monitoring. The SMACTR framework [74] is a good option for a more
systematic approach. It is an internal auditing framework with five stages:
scoping, mapping, artifact collection, testing, and reflection. Each stage
yields a set of documents that form a comprehensive audit report.

###  3.6 Federation

The major management challenges of FL deal with communication, efficiency,
resource limitation, and security. In this context, it is very challenging to
coordinate the learning process of thousands of clients while ensuring model
integrity and security. Global models might converge slowly due to
heterogeneous client data. Inconsistent clients, networks, and limited
resources might cause clients to drop out, and training failures could impact
model quality. In conclusion, although there is active research in FL
algorithms, there is still a lack of research and guideline on the
architectural design of FL systems. In this sense, the main notions of this
pillar are Client and Model Management, which considers how client and model
information is administrated in the system, and Optimization Algorithm, which
may impact the model performance and robustness.

Client Management. This notion proposed a Client Registry, where participants
can register themselves for training, and a Client Selector to filter eligible
clients for training. More details are provided below.

  * 1.

Client Registry. It enables the system to manage client connections and track
the status of all client devices. The proposed design pattern maintains the
client registry in the central server for the client-server architecture. The
server sends a request for information along with the initial local model to
the clients when they first connect to the system. The information requested
includes device ID, connection up and down time, and device computation power
storage, among others.

  * 2.

Client Selector. It optimizes resource usage and reduces the risk of client
dropout and communication latency. The proposed design pattern also maintains
the client selector in the central server where the selection occurs. Before
each round of training, the client selector actively selects a certain number
of clients for the training according to predefined criteria to reduce
convergence time and optimize the model performance.

Model Management. In a distributed learning process like FL, multiple rounds
of training and aggregation of models generate numerous local model updates
and aggregated global models during the process. Without recording the local
and global intermediary models, there is neither traceability nor fallback
when something goes wrong in the training process. A model co-versioning
registry and replacement can help trace the model quality and improve system
accountability.

  * 1.

Model Co-versioning. It aligns the local model versions with their
corresponding aggregated global models. It can be a registry where local model
versions are stored and mapped to the associated global models. With this
registry, model updates and aggregations do not always have to be synchronous
because the server can refer to the mapping to perform asynchronous
aggregations. Another advantage is that it allows early stopping if a model
converges before the specified number of rounds.

  * 2.

Model Replacement. It detects the global model performance dropping below a
certain threshold level. For that, it compares the global modal performance in
all clients to see if the performance degradation is a global event. The new
global model training task is triggered if the degradation is global and
persistent.

Optimization. According to the goal and the context of FL models, the choice
of an optimization algorithm can impact the model performance. Therefore,
various studies have conducted performance benchmarking of FL optimization
algorithms [75]. This benchmarking comparison serves as a reference for this
metric.

  * 1.

Aggregation Algorithm. FedAvg is considered the baseline aggregation
algorithm, but some other optimization algorithms have been proposed as an
extension for various purposes [75].

##  4 FederatedTrust Algorithm Design

This section details the design of FederatedTrust, the algorithm proposed to
quantify the trustworthiness level of FL models according to the pillars,
notions, and metrics presented in Section 3. To the best of our knowledge,
this algorithm is the first attempt to evaluate the overall trustworthiness
level of FL models.

Before presenting the algorithm details, it is important to mention that this
work assumes that the central server, aggregating the clients’ models, is
honest and maintained by a trusted system administrator. Therefore, the server
does not maliciously interfere with the trust calculation process. In
addition, the following functional requirements (FR), non-functional
requirements (NF), and privacy constraints (PC) have been considered before
designing and implementing the proposed algorithm.

  1. FR-1:

Each of the six trustworthy FL pillars must be represented in the algorithm,
meaning that at least one metric from each pillar must be considered in the
final score.

  2. FR-2:

The final trustworthiness score must be a combination of the trustworthiness
scores from all notions and pillars.

  3. NF-1:

The algorithm should add minimal computation overhead and complexity to the
server, participants, and FL model.

  4. NF-2:

The algorithm should be modular and configurable.

  5. PC-1:

The algorithm must not store any sensitive data from the FL model.

  6. PC-2:

The algorithm must not leak or share any sensitive data from clients, the
server, and the FL model with third parties.

  7. PC-3:

The metrics calculations can occur at the client’s local devices, the central
server, or collaboratively between both.

  8. PC-4:

When metrics are calculated collaboratively between clients and the server,
the computation should be performed securely and privately if the individual
client metrics contain sensitive information.

Once assumptions and requirements are defined, Figure 2 shows the overview of
the FederatedTrust algorithm design. First, the proposed algorithm considers
the following input sources to compute the trustworthiness of FL models.

  * 1.

FL Model. The FL model trained in a collaborative and privacy-preserving way
between the federation participants. This input contains information about the
model configuration and model personalization.

  * 2.

FL Framework Configuration. The configuration parameters of the tool
implementing the protocol needed to train and evaluate the FL model. This
input contains information about the number of clients, the client selection
mechanisms, the aggregation algorithm, and the model hyperparameters.

  * 3.

FactSheet. A document containing relevant information about the training
process, the federation, and its participants. More in detail, this input
contains information about the FL project overview, its purpose and
background, the data provenance, pre-processing techniques, and differential
privacy mechanisms.

  * 4.

Statistics. The statistical information extracted from the training dataset of
each client and its model performance. It does not contain sensitive
information. This input contains information about the client class balance,
client test performance loss, client test accuracy, client clever score,
coefficient of variation for client feature importance, coefficient of
variation for client test accuracy, coefficient of variation for clients
participation rate, coefficient of variation for client class imbalance,
client average training time, average model size, average upload bytes, and
average download bytes.

These input sources are used to compute the metrics indicated in Section 3,
which output values are then normalized to have a common range of values. It
is important to mention that each metric can consider different input sources
and can be calculated in a different phase of the FL model creation process
(pre-training, during-training, or post-training) and by a different actor of
the federation (client or server). These details regarding when and who
computes each metric are provided later. Once the normalized outputs of
metrics are calculated, they are weighted and aggregated to compute one score
per notion (see Section 3 for more information about notions). Each pillar has
one or more notions calculated according to predefined but configurable
weights per metric. Therefore, the same process is repeated to obtain the
pillar scores from weighting and aggregating notion scores. Finally, the final
trust score of the FL model is a configurable combination of the pillar
scores. The implementation details of the previous steps are provided in
Section 5.

Figure 2: Design of the FederatedTrust Algorithm

As indicated, some metrics of the algorithm are calculated during the training
phase of the FL model. Therefore, it is necessary to integrate the proposed
algorithm into an FL framework in charge of creating FL models. In this
context, Figure 3 shows the interactions between the main actors involved in
the computation of the trustworthy level of FL models. As it can be seen, the
central server of the federation hosts i) the Aggregator, in charge of
combining the parameters of the clients’ models to create the FL model, ii)
the FactSheet, accounting the most important aspects of the FL project and
participants federation, iii) the FL Framework Configuration, detailing
aspects of the framework, and iv) the FederatedTrust algorithm, computing the
FL model trust score. To compute the trust score, during the pre-training
phase, the content of the FactSheet and the FL framework configuration
(detailed in Section 5) is sent to the FederatedTrust algorithm (steps 1 and 2
in Figure 3) and metrics depending on them are calculated as previously
explained. Then, the model training process starts with the server sharing the
type of model and its characteristics with the clients of the federation (step
3). Once clients locally train their models with their private datasets, they
send the models parameters and statistics of their data to the server for
aggregation (step 4). At that point, the parameters are aggregated, and the FL
Model and statistics (about training datasets and model performance) are sent
to the FederatedTrust algorithm for computing metrics during training (step
5). Steps 3, 4, and 5 are repeated until the FL model converges or the
training rounds are over. At this point, the training process concludes, and
FederatedTrust computes the post-training metrics. Finally, the FederatedTrust
algorithm provides a score per pillar and a global one, together with a
trustworthiness report. The detailed version of the previous steps is provided
by Algorithm 1.

Figure 3: Overview of the Integration of FederatedTrust into an FL Framework
Algorithm 1 Training a FL model equipped with FederatedTrust

1:Input: clients $N$, sampling size $m$, a central server $S$, number of
iterations $T$, initial model $\bar{w}^{(0)}$, FL Framework configuration $C$,
FederatedTrust $ft$, FactSheet $fs$

2:Output: global score, pillars scores, trustworthiness report

3:$S$ sends the hashed IDs of all clients $i\in[N]$, $C$, and $fs$ to $ft$

4:$ft$ creates a map of hashed client IDs to values of 0, representing the
initial selection rate

5:$S$ sends the model metadata to $ft$

6:$S$ request class distribution information from all clients $i\in[N]$

7:for clients $i\in[N]$ do

8: Client $i$ uses $ft$ function to calculate the sample size per class of
local data

9: $ft$ creates or updates the class distribution map of hashed labels to
sample size

10:end for

11:for $t$ = 0 to $T$ do

12: $S$ randomly samples $D^{(t)}\subset{[N]}$ clients with size of $m$

13: $S$ sends the hashed IDs of the selected clients to $ft$

14: $ft$ updates the client selection rate map

15: $S$ broadcasts the current model $\bar{w}^{(t)}$ to all clients $i\in
D^{(t)}$

16: for clients $i\in D^{(t)}$ do

17: Client $i$ performs local training with $\bar{w}^{(t)}$

18: Client $i$ sends new model updates $w_{i}^{(t+1)}$ back to $S$

19: Client $i$ computes evaluation metrics with local test data and local
model $\bar{w_{i}}^{\;{}^{\prime}}$

20: Client $i$ sends the evaluation results back to $S$

21: end for

22: $S$ performs secure aggregation of all updates into a new global model
$\bar{w}^{(t+1)}$

23:end for

24:$S$ aggregates the evaluation results and sends them to $ft$

25:$ft$ receives the evaluation results and populates them

26:$S$ asks $ft$ to evaluate the trustworthiness of the model

27:$ft$ computes the trustworthiness score and generates a report JSON and
print message

##  5 FederatedTrust Algorithm Prototype

This section contains the implementation details of the FederatedTrust
algorithm when dealing with a given problem presented as a use case. In more
detail, first, a well-known and powerful FL framework able to train
heterogeneous FL models has been selected. After that, a set of metrics, from
the list provided in Section 4, common for different FL use cases, has been
implemented in the FederatedTrust algorithm to compute the trustworthiness
level of different FL models. Then, a use case where ML models must be trained
in a collaborative and privacy-preserving way has been selected. Finally,
FederatedTrust has been deployed and integrated into the FL framework to solve
the use case. The implemented prototype is available in [18].

###  5.1 FL Framework Selection and Use Case

Regarding the selection of the FL framework, after careful comparison and
analysis of the most relevant and used frameworks in charge of training
diverse FL models (TensorFlow Federated, PySyft, Flower, FLUTE, LEAF,
FederatedScope, FedEval, and FedML), FederatedScope [76] was chosen as a
reference tool. FederatedScope is an open-source FL framework developed in
2022 by the Data Analytics and Intelligence Lab (DAIL) of Alibaba DAMO Academy
[77]. FederatedScope employs an event-driven architecture to provide users
with great flexibility. It is a comprehensive framework that builds on
existing research work [78] and offers a suite of functionalities for privacy
and adversarial attacks. More in detail, the following reasons lead to the
selection of FederatedScope in this work.

  1. 1.

Provides standalone and distributed modes to set up clients experimentally or
realistically.

  2. 2.

Provides differential privacy, customization, and inference attacks.

  3. 3.

Provides well-documented evaluation metrics.

  4. 4.

Includes Data zoo, a suite of well-known federated datasets such as FEMNIST
[78].

  5. 5.

Includes Algo zoo, a list of state-of-the-art optimization algorithms such as
FedAvg, FedProx, FedOpt, or pFedMe, among others.

  6. 6.

Includes Model zoo, a set of state-of-the-art computer vision and language
models.

Then, a basic use case was chosen for implementing and testing the deployment
of the FederatedTrust algorithm as a proof of concept. In this context, the
use case focuses on classifying handwritten digits and numbers in a federated
and privacy-preserving way. For that, FederatedScope runs an FL training
process in standalone mode with a variable number of clients and one central
server over a defined number of iterations. The function to aggregate
individual models and create the global FL Model is FedAVg, and the model is a
convolutional neural network (CovNet2). The federated datasets that the
clients hold locally are from the FEMNIST image dataset of handwritten digits
and letters. The dataset has 62 classes (10 digits, 26 lowercase, 26
uppercase), and the images are 28 by 28 pixels. The client selector strategy
is random sampling.

To compute the trustworthiness level of FL models classifying handwritten
numbers in a privacy-preserving and federated fashion, the following metrics
were implemented in the FederatedTrust prototype.

###  5.2 Metrics Selection and FederatedTrust Implementation

As seen in Section 3, not all metrics have a standardized way of being
computed, available equations, or feasible calculation methods. Therefore, the
objective of this section is to implement a lightweight prototype of the
FederatedTrust algorithm with the basic pillars, notions, and metrics that can
be calculated in any FL project created with the FederatedScope framework.
More in detail, the list of omitted notions and metrics and the reasons for
not including them in the prototype implementation are the following.

  * 1.

Encryption and anonymization (Privacy). The usage of these two techniques is
not documented in the configuration file of the FederatedScope framework.
Furthermore, the implemented prototype considers differential privacy as a
perturbation technique to protect data privacy.

  * 2.

Information Leakage Risk (Privacy). This metric needs to be calculated during
every round of the training process by running extra neural networks, which
would incur high computation overhead.

  * 3.

Poisoning Defense (Robustness). It is often integrated into the FL training
protocol, for example, with robust aggregation. Therefore, it is complicated
to quantify the usage of the poisoning defense mechanism unless the
information is documented in the FL framework, which is not the case in
FederatedScope. In addition, another way to verify this metric could be by
checking the certified robustness against poisoning attacks, which is
considered in the prototype implementation.

  * 4.

Empirical Robustness (Robustness). This metric depends on each attack type,
and the number of possibilities and complexity is huge. Furthermore, certified
robustness could help to cover this metric.

  * 5.

Testing Coverage and System Reliability (Robustness). Neither testing data nor
error rates are available in the FederatedScope framework.

  * 6.

Client Reputation (Robustness). There is no clear way to measure client
reputation in a simulated environment. It requires more inspections of the
client data provenance.

  * 7.

Data Quality (Robustness). The proposed calculation for this metric needs to
be executed during every round of the training process, creating a high
computation overhead for the FL model training process.

  * 8.

Discrimination Index (Fairness). This metric requires the knowledge of one
protected sensitive attribute. It is unclear how this calculation can be done
without sharing sensitive information between clients and the server.

  * 9.

Visualization (Explainability). Graphical capabilities to show the
explainability of FL models are not included in the FederatedScope framework.

  * 10.

Algorithmic Auditing (Accountability). It is not implemented as the validation
scenario uses simulation data, there are no real end users, and it is assumed
no attackers in the environment.

  * 11.

Client Registry (Federation). The FederatedScope framework simulates the
clients participating in the federation. Therefore, no client registration is
required, as in real scenarios.

  * 12.

Model co-versioning and replacement (Federation). Versioning and model
replacement trigger functionality is not implemented as the validation
scenario is simulated.

Once those metrics excluded for the prototype implementation are indicated,
Table 1 describes the list of implemented metrics, with descriptions, inputs,
output values, calculation moment, and federation actor in charge of computing
them.

Table 1: Metrics Implemented by the FederatedTrust Algorithm Prototype Metric |  Description |  Input |  Output |  When |  Who
---|---|---|---|---|---
Privacy
Differential Privacy |  Use of global or local differential privacy as a privacy defense |  FactSheet |  0/1 |  Pre-training |  Server
Entropy |  Uncertainty in predicting the value of a random variable |  FL Framework Conf |  [0, 1] |  Pre-training |  Server
Global Privacy Risk |  Maximum privacy risk with differential privacy based on $\epsilon$ |  Client Statistics |  % |  Pre-training |  Server
Robustness
Certified Robustness |  Minimum perturbation required to change the neural network prediction |  FL Model |  Real |  Post-training |  Server
Performance |  Test accuracy of the global model |  Statistics, FL Model |  % |  During-training |  Clients
Personalization |  Use of personalized FL techniques |  FactSheet |  0/1 |  Pre-training |  Server
Federation Scale |  Number of clients representing the scale of the federation |  FactSheet |  Integer |  Pre-training |  Server
Fairness
Participation Variation |  Uniformity of distribution of participation rate among clients |  FL Framework |  [0, 1] |  Post-training |  Server
Accuracy Variation |  Uniformity of distribution of performance among clients |  Client Statistics, FL Model |  [0, 1] |  During-training |  Clients
Class Imbalance |  Average class imbalance estimation among clients |  Client Statistics |  [0, 1] |  Pre-training |  Clients
Explainability
Algorithmic Transparency |  Interpretability of the model by design |  FL Model |  [1, 5] |  Pre-training |  Server
Model Size |  Model Features dimensionality, depth of decision tree, or number of parameters in neural networks |  FL Model |  Integer |  Post-training |  Server
Feature Importance |  Average variance of feature importance scores |  FL Model |  [0, 1] |  Post-training |  Server
Accountability
Project Specification |  Project details and purpose |  FactSheet |  0/1 |  Pre-training |  Server
Participants |  Participants number, identifiers, and their organizations |  FL Framework Conf |  0/1 |  Pre-training |  Server
Data |  Contains Data origin and data-preprocesing steps |  FactSheet |  0/1 |  Pre-training |  Server
Configuration |  Information about the FL model |  FL Framework Conf, FactSheet |  0/1 |  Pre-training |  Server
System |  Contains training time, FL model size, and network performance |  FL Framework Conf, Statistics |  0/1 |  Post-training |  Server
Federation
Client Selector |  Use of a client selector scheme rather than random selection |  FactSheet |  0/1 |  Pre-training |  Server
Aggregation Algorithm |  Selected aggregation function |  FL Framework Conf |  % |  Pre-training |  Server

In order to understand how the trust FL score is computed, it is important to
mention that the output value of each metric can have different ranges, as
indicated in the metric definitions (Table 1). Therefore, to combine these
values into an understandable trustworthiness score, different functions are
used to translate the output values into a normalized score between [0, 1].
The logic of each normalization function is explained in Table 2. It shows for
each metric the type of values it gets, the range or set of possible values,
and how these values are mapped into the final output between 0 and 1.

Then, once all metrics outputs are normalized, they are grouped (first by
notions and then by pillars) to compute the Trust FL score. The algorithm
prototype uses the mean average of all the metrics under a notion to calculate
its final score. In the same way, the mean average of the notions of one
pillar is employed as the pillar score. Finally, the Trust Score of the entire
setup is the mean average of all the pillars. How metrics, notions, and
pillars are combined can be changed to different aggregation methods based on
the requirements of the environment, giving more importance to some metrics,
notions, or pillars. The implementation of the previous life-cycle is done by
using the numpy, scipy, and sklearn libraries. More in detail, FederatedTrust
is designed as a third-party library.

Table 2:  Normalization of the Metrics Outputs Metric |  Type |  Output |  Normalized Output
---|---|---|---
Privacy
Differential Privacy |  True/False |  0/1 |  0/1
Entropy |  Uncertainty |  [0, 1] |  [0,1]
Global Privacy Risk |  Percentage |  [0, 100] |  [0, 1]
Robustness
Certified Robustness |  CLEVER Score |  [0, 0.2, 0.4, 0.6, 0.8, 1, 1.2, 1.4, 1.6, 1.8, 2, 2.2, 2.4, 2.6, 2.8, 3, 3.2, 3.4, 3.6, 3.8, 4.0] |  {0, 0.05, 0.1, … 1}
Performance |  Accuracy |  [0, 1] |  [0, 1]
Personalization |  True/False |  0/1 |  0/1
Federation Scale |  Number of clients |  [10, $10^{2}$, $10^{3}$, $10^{4}$, $10^{5}$, $10^{6}$] |  {0, 0.2, 0.4, … 1}
Fairness
Participation Variation |  Coefficient of Variation |  [0, 1] |  [0, 1]
Accuracy Variation |  Coefficient of Variation |  [0, 1] |  [0, 1]
Class Imbalance |  Balance Ratio |  [0, 1] |  [0, 1]
Explainability
Algorithm Transparency |  Random Forest, K-Nearest Neighbors, Support Vector Machine, GaussianProcessClassifier, Decision Tree, Multilayer Perceptron, AdaBoost, GaussianNB, Quadratic Discriminant Analysis, Logistic Regression, Linear Regression, Sequential, Convolutional Neural Network |  {4, 3, 2, 3, 5, 1, 3, 3.5, 3,4, 3.5, 1, 1} |  {0, 0.2, 0.4, 0.5, … 1}
Model Size |  Number of Model Parameters |  {1, 10, 50, 100, 500, 1000, 5000, 10000, 50000, 100000, 500000} |  {0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0}
Feature Importance |  SHAP Importance |  [0, 1] |  [0, 1]
Accountability
Project Specification |  True/False |  0/1 |  0/1
Participation |  True/False |  0/1 |  0/1
Data |  True/False |  0/1 |  0/1
Configuration |  True/False |  0/1 |  0/1
System |  True/False |  0/1 |  0/1
Federation
Client Selector |  True/False |  0/1 |  0/1
Aggregation Algorithm |  FedAvg, FedOpt, FedProx, FedBN, pFedMe, Ditto, FedEM |  {0.8493, 0.8492, 0.8477, 0.8548, 0.8765, 0.8661, 0.8479} |  {0.8493, 0.8492, 0.8477, 0.8548, 0.8765, 0.8661, 0.8479}
[] denotes a continuous range
{} denotes a list of values

##  6 Experiments

The previously defined setup combining FederatedTrust and FederatedScope was
used to perform four experiments that consist of training FL models that
classify hand-written digits using the FEMNIST dataset. The details of each
experiment are provided below.

  * 1.

Experiment 1: The experiment considers a federation of 10 clients with a
selection rate of 50% clients per training round. In other words, 5 randomly
selected clients out of the 10 available ones participate in each round. The
training of the FL model runs 5 rounds to produce an aggregated global model.
Finally, the experiment does not use personalization techniques. Therefore,
all clients share the same global model.

  * 2.

Experiment 2: It considers 50 clients for the federation. In every iteration,
the server randomly selects 60% clients to participate in the training. The
experiment runs 25 training iterations. As in the previous experiment, no
personalization techniques are used. However, the main novelty of this
experiment lies in the inclusion of differential privacy with $\epsilon$ of
20.

  * 3.

Experiment 3: This experiment has a similar configuration to Experiment 2 but
with more data privacy due to a lower value of $\epsilon$ (6) used in
differential privacy. It means that the expected privacy protection would be
higher.

  * 4.

Experiment 4: This experiment is medium-scale among all the previous ones. It
consists of 100 clients with a 40% client selection rate and 50 rounds of
training. The $\epsilon$ value of differential privacy is set to 6.

For all experiments, the same values about the project specifications, data,
and participants are extracted. The purpose and the background of the FL
project are not specified in the model. Finally, all models used a random
sampling client selector and ran the FedAvg aggregation algorithm. Because of
that, the federation pillar score does not change between the four
experiments.

###  6.1 Trustworthiness Scores for Experiments 1 and 2

As can be seen in Figure 4 and Figure 5, the trustworthiness score for both
experiments is 0.56. However, there are differences in terms of pillars and
metrics.

Starting from the privacy pillar, its score increased from 0.31 (Experiment 1)
to 0.64 (Experiment 2) because of using differential privacy. However, the
effectiveness of the privacy mechanism was not good enough because of the
large value of $\epsilon$ chosen (20). It can be seen in the
indistinguishability notion (covered by the global privacy risk metric), with
a score of 0 in both experiments. In this context, the larger the $\epsilon$
value, the less noise was added to the data, and therefore, the higher the
probability of being identified by adversaries.

Dealing with the fairness pillar, its score from Experiment 1 to 2 was also
increased from 0.25 to 0.47. It was mainly due to a significant increment in
the selection variation metric (from 0.08 to 0.83), which resulted from the
overall increase in the number of clients, the client sampling rate, and the
number of rounds. The selection fairness improved with more clients
participating and more training rounds. However, the performance variation
metric (Fairness pillar) dropped from 0.58 to 0.50 from Experiment 1 to 2. It
could also be due to the increased number of clients. More clients with
different levels of heterogeneity in their data could influence the
generalizability of the global model hence affecting the individual test
accuracy at the client level. Furthermore, personalization techniques were not
used, so the global model was not adapted to the clients.

Regarding the explainability pillar, its global score increased from 0.61
(Experiment 1) to 0.73 (Experiment 2), with an increase in the feature
importance metric score from 0.67 to 0.9. It is impacted by the different
numbers in terms of clients and the differences between their datasets.

Finally, the robustness pillar score dropped from 0.5 (Experiment 1) to 0.38
(Experiment 2). It was mainly due to the decrement in the certified robustness
metric (from 0.48 to 0.19). The drop in the certified robustness metric could
be related to the increase in the number of clients and the number of rounds.
In theory, more aggregating parties provide more entries and surfaces for
adversaries to insert backdoor perturbations for poisoning attacks. There are
also higher chances for parties to collude when they are more in number. The
higher number of rounds also means that adversaries have more chances to
attack. The federation scale metric for both experiments has a similar score
since both have less than 50 clients.

Figure 4: Experiment 1 - 10 clients, 20% sampling rate, 5 rounds, no
differential privacy Figure 5: Experiment 2 - 50 clients, 60% sampling rate,
25 rounds, differential privacy $\epsilon$ = 20

###  6.2 Trustworthiness Scores for Experiments 3 and 4

Figure 6 and Figure 7 show the trustworthiness score for both experiments with
global scores of 0.64 and 0.67, respectively. Before comparing these two
experiments, it is important to mention that the main difference between
Experiments 2 and 3 is the privacy pillar (due to the $\epsilon$ change). The
remaining pillar scores are very similar.

Focusing on Experiments 2 and 3, from the robustness pillar perspective, its
overall score only increased from 0.36 (Experiment 3) to 0.38 (Experiment 4),
even though there was a significant increase in the model performance from
0.09 to 0.36. It was mainly due to the less relevance of that metric in the
pillar than the rest. Furthermore, the federation scale metric score did not
change because the number of clients in both experiments fell within the range
[50, 100], which is defined in this work as robust and reliable.

The privacy pillar score increased from 0.68 (Experiment 3) to 0.71
(Experiment 4) due to the increase in the global privacy risk metric. The
increase in the number of clients caused an improvement in the
indistinguishability notion (covered by the previous metric). Assuming that
random guessing was used, in Experiment 4 was twice as challenging to guess
the correct target among 100 clients compared to 50 clients (used in
Experiment 3).

In terms of explainability, the score increased from 0.65 (Experiment 3) to
0.73 (Experiment 4) due to the feature importance score, which continued to
increase to 0.92 in Experiment 4. It might be related to the increase in data
samples from more clients and the better model performance. When the model
learned the data better, the features had more impact on the prediction
outcomes.

Finally, the fairness score was almost identical for both experiments (0.47
and 0.5). It might be because the ratio between the increase in the number of
clients and the increase in the number of rounds was the same, while the
clients’ sampling rate remained the same as well.

Figure 6: Experiment 3 - 50 clients, 60% sampling rate, 20 rounds,
differential privacy $\epsilon$ = 6 Figure 7: Experiment 4 - 100 clients, 60%
sampling rate, 50 rounds, differential privacy $\epsilon$ = 6

###  6.3 Limitations

As seen from the experiment results, there are several limitations to the
FederatedTrust prototype in terms of quantifying the trustworthiness level of
FL models. On the one hand, some metrics are easily quantifiable and can
represent a component of the trustworthiness level well. For example,
certified robustness represents the attacker lower bound, which is directly
associated with resilience to attacks. On the other hand, other metrics, like
the federation scale (robustness pillar), often have to be considered with
other factors to represent the notion well. For example, the analysis
performed in Section 3 shows that, in practice, the client reputation metric
is also an essential factor for the client reliability notion. However, it was
difficult to quantify the client reputation in the performed experiments.
Another example would be the client selection fairness notion. Although the
client participation variation metric is easily quantifiable by computing the
dispersion of selection rate among the clients, this variation metric alone
might not be the best representation of fairness for client selection in FL.
Increasing the number of clients, the sample rate, and the round of training
could easily bring up the participation variation metric, but equality does
not necessarily imply equity which is important for true fairness.

Another limitation of the algorithm pertains to the scoring system. To
aggregate all the scores into a trustworthiness level between 0 and 1, all the
metric values had to go through the metric operations and the scoring
functions. The logic of the scoring functions greatly impacts how the trust
score of each metric is calculated. Based on the pillar analysis in Section 3,
there were general directions of how every metric should impact the overall
trustworthiness level. However, the concrete scoring maps and ranges were
created based on subjective understanding and standards used from other
studies. Their generalizability for other systems was not fully evaluated.
Furthermore, the current metric weighing system does not implement varied
weights, meaning that every metric has the same weight under one notion, and
every notion has the same weight when aggregated into the final
trustworthiness level. The weighing system could be enhanced by further
analysis of the trade-offs between pillars and metrics to produce a more
balanced trustworthiness level of FL models.

##  7 Conclusions and Future Work

This work has studied and analyzed the pillars, notions, and metrics relevant
to trustworthy FL to later create a comprehensive taxonomy with the most
relevant ones. The proposed taxonomy extends the pillars already identified by
previous work (privacy, robustness, fairness, explainability, and
accountability) with a new one called federation. This new pillar quantifies
the trustworthiness of FL models from the participants and FL model
perspectives. Furthermore, those pillars identified by the literature have
also been extended with different notions and new metrics dealing with FL
models. Then, based on the proposed taxonomy, a trustworthiness evaluation
algorithm for FL models, named FederatedTrust, has been designed to be
extensible, configurable, and flexible. One of the most relevant and powerful
FL frameworks, FederatedScope, was selected as the reference FL framework to
deploy and test the prototypical implementation of FederatedTrust. Four
experiments were performed to validate the FederatedTrust prototype with
different FL settings varying the number of clients, training rounds, and
differential privacy parameters. For each experiment, the trustworthiness
level of each pillar and metric were compared and discussed while solving the
problem of classifying handwritten digits (using the FEMNIST dataset) in a
collaborative and privacy-preserving way. The experiments demonstrated the
complexity of the task of quantifying the trustworthiness level of FL models,
being the FederatedTrust algorithm the first attempt to holistically assess an
FL model based on a comprehensive trustworthiness taxonomy. To conclude,
existing limitations of the current version of the algorithm prototype have
also been discussed.

As future work, it is planned to extend the prototype by implementing new
metrics identified in the taxonomy. It is also intended to deploy the
algorithm on FL frameworks training FL models in a decentralized fashion.
Furthermore, the score aggregation step could be improved by analyzing the
trade-offs between pillars and metrics to handle the weight assignments.
Finally, the normalization functions could also be improved by designing
better score ranges and mappings.

## ACKNOWLEDGMENT

This work has been partially supported by (a) the Swiss Federal Office for
Defense Procurement (armasuisse) with the DEFENDIS and CyberForce
(CYD-C-2020003) projects and (b) the University of Zürich UZH.

## References

  * [1] Britta Daffner.  Why Artificial Intelligence (AI) will be the technology of 2023, 2023\.
  * [2] Cynthia Rudin.  The age of secrecy and unfairness in recidivism prediction.  2019\.
  * [3] Muhammad Uzair.  Who is liable when a driverless car crashes?  2021\.
  * [4] Garling Wu.  5 Big Problems With OpenAI’s ChatGPT, 2022.
  * [5] Virginia Dignum.  Responsible artificial intelligence: how to develop and use AI in a responsible way.  Springer Nature, 2019.
  * [6] Gabrielle Ras, Ning Xie, Marcel van Gerven, and Derek Doran.  Explainable deep learning: A field guide for the uninitiated, 2020.
  * [7] Luciano Floridi, Josh Cowls, et al.  AI4People - An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations.  Minds and Machines, 28(4):689–707, Dec 2018.
  * [8] Gary Marcus.  The next decade in ai: four steps towards robust artificial intelligence.  arXiv preprint arXiv:2002.06177, 2020.
  * [9] Stefan Feuerriegel, Mateusz Dolata, and Gerhard Schwabe.  Fair AI.  Business & information systems engineering, 62(4):379–384, 2020\.
  * [10] AI HLEG of the European Commission.  Ethics guidelines for trustworthy ai, 2019.
  * [11] Haochen Liu, Yiqi Wang, et al.  Trustworthy AI: A Computational Perspective, 2021.
  * [12] Bo Li, Peng Qi, et al.  Trustworthy AI: From Principles to Practices.  ACM Comput. Surv., aug 2022.  Just Accepted.
  * [13] Alberto Huertas Celdrán, Manuel Gil Pérez, Izidor Mlakar, Jose M. Alcaraz Calero, Félix J. García Clemente, Gregorio Martínez Pérez, and Zakirul A. Bhuiyan.  PROTECTOR: Towards the protection of sensitive data in Europe and the US.  Computer Networks, 181:107448, 2020.
  * [14] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas.  Communication-Efficient Learning of Deep Networks from Decentralized Data.  In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 1273–1282. PMLR, 20–22 Apr 2017.
  * [15] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu.  Federated learning.  Synthesis Lectures on Artificial Intelligence and Machine Learning, 13(3):1–207, 2019.
  * [16] researchandmarkets.com.  Global Federated Learning Market by Application (Drug Discovery, Industrial IoT, Risk Management), Vertical (Healthcare & Life Sciences, BFSI, Manufacturing, Automotive & Transportation, Energy & Utilities), and Region - Forecast to 2028, 2022.
  * [17] Peter Kairouz, H Brendan McMahan, et al.  Advances and open problems in federated learning.  Foundations and Trends® in Machine Learning, 14(1–2):1–210, 2021.
  * [18] Xie, Ning.  FederatedTrust: A Trustworthiness Evaluation Framework, 2022.
  * [19] Di Chai, Leye Wang, Kai Chen, and Qiang Yang.  Fedeval: A benchmark system with a comprehensive evaluation model for federated learning.  arXiv preprint arXiv:2011.09655, 2020.
  * [20] Alberto Huertas Celdran, Jan Kreischer, Melike Demirci, Joel Leupp, Pedro M. Sanchez Sanchez, Muriel Figueredo Franco, Gérôme Bovet, Gregorio Martinez Perez, and Burkhard Stiller.  A Framework Quantifying Trustworthiness of Supervised Machine and Deep Learning Models.  In SafeAI2023: The AAAI’s Workshop on Artificial Intelligence Safety, pages 2938–2948. PMLR, 2023.
  * [21] Ye Dong, Xiaojun Chen, Liyan Shen, and Dakui Wang.  EaSTFLy: Efficient and secure ternary federated learning.  Computers & Security, 94:101824, 2020.
  * [22] Wei Wen, Cong Xu, Feng Yan, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li.  Terngrad: Ternary gradients to reduce communication in distributed deep learning.  Advances in neural information processing systems, 30, 2017.
  * [23] Stephen Hardy, Wilko Henecka, et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption.  arXiv preprint arXiv:1711.10677, 2017.
  * [24] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth.  Practical secure aggregation for privacy-preserving machine learning.  In proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175–1191, 2017.
  * [25] Olivia Choudhury, Aris Gkoulalas-Divanis, Theodoros Salonidis, Issa Sylla, Yoonyoung Park, Grace Hsu, and Amar Das.  Differential privacy-enabled federated learning for sensitive health data.  arXiv preprint arXiv:1910.02578, 2019.
  * [26] Robin C Geyer, Tassilo Klein, and Moin Nabi.  Differentially private federated learning: A client level perspective.  arXiv preprint arXiv:1712.07557, 2017.
  * [27] Olivia Choudhury, Aris Gkoulalas-Divanis, et al.  A syntactic approach for privacy-preserving federated learning.  In ECAI 2020, pages 1762–1769. IOS Press, 2020.
  * [28] Yong Liu, Xinghua Zhu, Jianzong Wang, and Jing Xiao.  A quantitative metric for privacy leakage in federated learning.  In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3065–3069. IEEE, 2021.
  * [29] Malhar S Jere, Tyler Farnan, and Farinaz Koushanfar.  A taxonomy of attacks on federated learning.  IEEE Security & Privacy, 19(2):20–28, 2020.
  * [30] Mohammad Naseri, Jamie Hayes, and Emiliano De Cristofaro.  Local and central differential privacy for robustness and privacy in federated learning.  arXiv preprint arXiv:2009.03561, 2020.
  * [31] Luis Muñoz-González, Kenneth T Co, and Emil C Lupu.  Byzantine-robust federated machine learning through adaptive model averaging.  arXiv preprint arXiv:1909.05125, 2019.
  * [32] Nuria Rodríguez-Barroso, Eugenio Martínez-Cámara, M Victoria Luzón, and Francisco Herrera.  Backdoor attacks-resilient aggregation based on Robust Filtering of Outliers in federated learning for image classification.  Knowledge-Based Systems, 245:108588, 2022.
  * [33] Aleksandar Jankovic and Rudolf Mayer.  An empirical evaluation of adversarial examples defences, combinations and robustness scores.  In Proceedings of the 2022 ACM on International Workshop on Security and Privacy Analytics, pages 86–92, 2022.
  * [34] Yuxin Shi, Han Yu, and Cyril Leung.  A survey of fairness-aware federated learning.  arXiv preprint arXiv:2111.01872, 2021.
  * [35] Xubo Yue, Maher Nouiehed, and Raed Al Kontar.  Gifair-fl: An approach for group and individual fairness in federated learning.  arXiv preprint arXiv:2108.02741, 2021.
  * [36] Daniel Yue Zhang, Ziyi Kou, and Dong Wang.  Fairfl: A fair federated learning approach to reducing demographic bias in privacy-sensitive classification models.  In 2020 IEEE International Conference on Big Data (Big Data), pages 1051–1060. IEEE, 2020.
  * [37] Tiansheng Huang, Weiwei Lin, Wentai Wu, Ligang He, Keqin Li, and Albert Y Zomaya.  An efficiency-boosting client selection scheme for federated learning with fairness guarantee.  IEEE Transactions on Parallel and Distributed Systems, 32(7):1552–1564, 2020.
  * [38] Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P Friedlander, Changxin Liu, and Yong Zhang.  Improving fairness for data valuation in horizontal federated learning.  In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 2440–2453. IEEE, 2022.
  * [39] Guan Wang.  Interpret federated learning with shapley values.  arXiv preprint arXiv:1905.04519, 2019.
  * [40] Christoph Molnar.  Interpretable machine learning.  BOOKDOWN, 2020.
  * [41] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  Advances in neural information processing systems, 30, 2017.
  * [42] Peng Chen, Xin Du, Zhihui Lu, Jie Wu, and Patrick CK Hung.  Evfl: An explainable vertical federated learning for data-oriented artificial intelligence systems.  Journal of Systems Architecture, 126:102474, 2022.
  * [43] Nathalie Baracaldo, Ali Anwar, Mark Purcell, Ambrish Rawat, Mathieu Sinn, Bashar Altakrouri, Dian Balta, Mahdi Sellami, Peter Kuhn, Ulrich Schopp, et al.  Towards an accountable and reproducible federated learning: A factsheets approach.  arXiv preprint arXiv:2202.12443, 2022.
  * [44] IBM Research.  AI FactSheets 360, 2022.
  * [45] Harsh Bimal Desai, Mustafa Safa Ozdayi, and Murat Kantarcioglu.  Blockfla: Accountable federated learning via hybrid blockchain architecture.  In Proceedings of the eleventh ACM conference on data and application security and privacy, pages 101–112, 2021.
  * [46] Vaikkunth Mugunthan, Ravi Rahman, and Lalana Kagal.  Blockflow: An accountable and privacy-preserving solution for federated learning.  arXiv preprint arXiv:2007.03856, 2020.
  * [47] Sana Awan, Fengjun Li, Bo Luo, and Mei Liu.  Poster: A reliable and accountable privacy-preserving federated learning framework using the blockchain.  In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 2561–2563, 2019.
  * [48] Kang Wei, Jun Li, Ming Ding, Chuan Ma, Howard H Yang, Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor.  Federated learning with differential privacy: Algorithms and performance analysis.  IEEE Transactions on Information Forensics and Security, 15:3454–3469, 2020.
  * [49] Haokun Fang and Quan Qian.  Privacy preserving machine learning with homomorphic encryption and federated learning.  Future Internet, 13(4):94, 2021.
  * [50] Yong Li, Yipeng Zhou, Alireza Jolfaei, Dongjin Yu, Gaochao Xu, and Xi Zheng.  Privacy-preserving federated learning framework based on chained secure multiparty computing.  IEEE Internet of Things Journal, 8(8):6178–6186, 2020.
  * [51] Isabel Wagner and David Eckhoff.  Technical privacy metrics: a systematic survey.  ACM Computing Surveys (CSUR), 51(3):1–38, 2018.
  * [52] Sin Kit Lo, Qinghua Lu, Chen Wang, Hye-Young Paik, and Liming Zhu.  A systematic literature review on federated machine learning: From a software engineering perspective.  ACM Computing Surveys (CSUR), 54(5):1–39, 2021.
  * [53] Jiawen Kang, Zehui Xiong, Dusit Niyato, Yuze Zou, Yang Zhang, and Mohsen Guizani.  Reliable federated learning for mobile networks.  IEEE Wireless Communications, 27(2):72–80, 2020.
  * [54] Chen Wu, Xian Yang, Sencun Zhu, and Prasenjit Mitra.  Mitigating backdoor attacks in federated learning.  arXiv preprint arXiv:2011.01767, 2020.
  * [55] Lingjuan Lyu, Han Yu, Xingjun Ma, Lichao Sun, Jun Zhao, Qiang Yang, and Philip S Yu.  Privacy and robustness in federated learning: Attacks and defenses.  arXiv preprint arXiv:2012.06337, 2020.
  * [56] Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov.  How to backdoor federated learning.  In International Conference on Artificial Intelligence and Statistics, pages 2938–2948. PMLR, 2020.
  * [57] Tsui-Wei Weng, Huan Zhang, Pin-Yu Chen, Jinfeng Yi, Dong Su, Yupeng Gao, Cho-Jui Hsieh, and Luca Daniel.  Evaluating the robustness of neural networks: An extreme value theory approach.  arXiv preprint arXiv:1801.10578, 2018.
  * [58] Xavier Gitiaux, Aditya Khant, Ebrahim Beyrami, Chandan Reddy, Jayant Gupchup, and Ross Cutler.  Aura: Privacy-preserving augmentation to improve test set diversity in noise suppression applications.  arXiv preprint arXiv:2110.04391, 2021.
  * [59] Sheikh Umar Farooq, SMK Quadri, and Nesar Ahmad.  Metrics, models and measurements in software reliability.  In 2012 IEEE 10th international symposium on applied machine intelligence and informatics (SAMI), pages 441–449. IEEE, 2012.
  * [60] Hangyu Zhu, Jinjin Xu, Shiqing Liu, and Yaochu Jin.  Federated learning on non-IID data: A survey.  Neurocomputing, 465:371–390, 2021.
  * [61] Sin Kit Lo, Qinghua Lu, et al.  Architectural patterns for the design of federated learning systems.  Journal of Systems and Software, 191:111357, 2022.
  * [62] Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar.  Federated multi-task learning.  Advances in neural information processing systems, 30, 2017.
  * [63] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek.  Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints.  IEEE transactions on neural networks and learning systems, 32(8):3710–3722, 2020.
  * [64] Manoj Ghuhan Arivazhagan, Vinay Aggarwal, Aaditya Kumar Singh, and Sunav Choudhary.  Federated learning with personalization layers.  arXiv preprint arXiv:1912.00818, 2019.
  * [65] Balázs Pejó and Gergely Biczók.  Quality inference in federated learning with secure aggregation.  arXiv preprint arXiv:2007.06236, 2020.
  * [66] Sheeraz A Alvi, Yi Hong, and Salman Durrani.  Federated learning cost disparity for iot devices.  arXiv preprint arXiv:2204.08036, 2022.
  * [67] Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith.  Fair resource allocation in federated learning.  arXiv preprint arXiv:1905.10497, 2019.
  * [68] Miao Yang, Ximin Wang, Hongbin Zhu, Haifeng Wang, and Hua Qian.  Federated learning with class imbalance reduction.  In 2021 29th European Signal Processing Conference (EUSIPCO), pages 2174–2178. IEEE, 2021.
  * [69] Alejandro Barredo Arrieta et al.  Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai, 2019.
  * [70] Sheikh Rabiul Islam, William Eberle, and Sheikh K Ghafoor.  Towards quantification of explainability in explainable artificial intelligence methods.  In The thirty-third international flairs conference, 2020.
  * [71] Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao.  Knowledge distillation: A survey.  International Journal of Computer Vision, 129(6):1789–1819, 2021\.
  * [72] Michael Ungersböck, Thomas Hiessl, et al.  Explainable Federated Learning: A Lifecycle Dashboard for Industrial Settings.  TechRxiv, 2022.
  * [73] Matthew Arnold, Rachel KE Bellamy, et al.  Factsheets: Increasing trust in ai services through supplier’s declarations of conformity.  IBM Journal of Research and Development, 63(4/5):6–1, 2019.
  * [74] Inioluwa Deborah Raji, Andrew Smart, Rebecca N White, Margaret Mitchell, Timnit Gebru, Ben Hutchinson, Jamila Smith-Loud, Daniel Theron, and Parker Barnes.  Closing the ai accountability gap: Defining an end-to-end framework for internal algorithmic auditing.  In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 33–44, 2020.
  * [75] Adrian Nilsson, Simon Smith, Gregor Ulm, Emil Gustavsson, and Mats Jirstrand.  A performance evaluation of federated learning algorithms.  In Proceedings of the second workshop on distributed infrastructures for deep learning, pages 1–8, 2018.
  * [76] Yuexiang Xie, Zhen Wang, Daoyuan Chen, Dawei Gao, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, and Jingren Zhou.  Federatedscope: A flexible federated learning platform for heterogeneity.  arXiv preprint arxiv.2204.05011, 2022.
  * [77] Alibaba DAMO Academy.  Data Analytics and Intelligence Lab.
  * [78] Sebastian Caldas, Sai Meher Karthik Duddu, et al.  Leaf: A benchmark for federated settings, 2018.