# The Cost of Training Machine Learning Models over Distributed Data Sources

Elia Guerra, Francesc Wilhelmi, Marco Miozzo, Paolo Dini Elia Guerra, Marco
Miozzo, and Paolo Dini are with Centre Tecnològic de Telecomunicacions de
Catalunya (CTTC/CERCA). Francesc Wilhelmi is with Nokia Bell-Labs, Stuttgart,
Germany. Corresponding author<EMAIL_ADDRESS>This publication has been
partially funded by the Spanish project
PID2020-113832RB-C22(ORIGIN)/MCIN/AEI/10.13039/50110001103, European Union
Horizon 2020 research and innovation programme under Grant Agreement No.
953775 (GREENEDGE) and the grant CHIST-ERA-20-SICT-004 (SONATA) by
PCI2021-122043-2A/AEI/10.13039/501100011033

###### Abstract

Federated learning is one of the most appealing alternatives to the standard
centralized learning paradigm, allowing a heterogeneous set of devices to
train a machine learning model without sharing their raw data. However, it
requires a central server to coordinate the learning process, thus introducing
potential scalability and security issues. In the literature, server-less
federated learning approaches like gossip federated learning and blockchain-
enabled federated learning have been proposed to mitigate these issues. In
this work, we propose a complete overview of these three techniques proposing
a comparison according to an integral set of performance indicators, including
model accuracy, time complexity, communication overhead, convergence time, and
energy consumption. An extensive simulation campaign permits to draw a
quantitative analysis considering both feedforward and convolutional neural
network models. Results show that gossip federated learning and standard
federated solution are able to reach a similar level of accuracy, and their
energy consumption is influenced by the machine learning model adopted, the
software library, and the hardware used. Differently, blockchain-enabled
federated learning represents a viable solution for implementing decentralized
learning with a higher level of security, at the cost of an extra energy usage
and data sharing. Finally, we identify open issues on the two decentralized
federated learning implementations and provide insights on potential
extensions and possible research directions in this new research field.

blockchain, decentralized learning, edge computing, energy efficiency,
federated learning, machine learning.

##  I Introduction

learning (ML) models, and in particular deep neural networks, require a
substantial amount of data and computational power that might not be available
on a single machine. As a consequence, ML operations are normally run at cloud
servers (or data centers), where batteries of powerful processing units enable
short training and inference computation times. However, training ML models in
a data center requires moving data from the information sources (e.g., edge
devices) to the central system. This approach runs into several issues:

  * •

Communication overhead. Nowadays, the huge pervasiveness of mobile services,
devices, and network infrastructures makes data sources mainly distributed. As
testified recently by the Ericsson Mobility Report [1], mobile network data
traffic grew exponentially over the last $10$ years, with a remarkable
increase of $42\%$ between Q3 2020 and Q3 2021. Mobile data traffic is
projected to grow by over $4$ times to reach $288$ EB per month by 2027 [1].
Moving such a big amount of data from distributed sources to a central
location for ML operations may create network congestion and service outage.

  * •

Latency. In several real-life scenarios, transmitting data requires a stable
and reliable connection to minimize latency and ensure updated models, which
cannot be always guaranteed. For example, minimizing communication latency in
connected vehicles is essential to guarantee road safety [2].

  * •

Energy consumption. Running ML models in cloud data centers consumes a
significant amount of energy and cannot be considered sustainable from an
environmental perspective. As reported in [3], from 2012 to 2018, the
computations required for training a deep learning (DL) model have been
doubling every 3.4 months, with an estimated increase of $300000$x. Estimates
show that training a state-of-the-art natural language processing model
produces more $\text{CO}_{2}$ than an average car in one year lifetime [4].

  * •

Privacy. With the growing awareness of data privacy and security, it is often
undesirable, or even unfeasible, to collect and centralize users’ data [5].
For instance, a single hospital may not be able to train a high-quality model
for a specific task on its own (due to the lack of data), but it cannot share
raw data due to various policies or regulations on privacy [6]. Another
example could be the case of a mobile user that would like to employ a good
next-word predictor model without sharing his/her private historical text
data.

###  I-A Edge AI and Federated Learning

To address the challenges that stem from cloud-based centralized ML, edge
computing pushes cloud services to the network edge and enables distributed ML
operations, i.e., the so-called edge intelligence [7]. In particular, AI on
Edge [8] is the paradigm of running AI models with a device-edge-cloud
synergy. It allows to relax the massive communication requirements and privacy
of cloud-based ML operations [9]. Moreover, distributing ML computation over
the edge has been demonstrated to save up to the $25\%$ of the energy
consumption [10]. In fact, data may be directly processed at the edge with
smaller and more energy efficient devices (no need of air conditioning
systems) and the energy cost related to communication is limited due to
unnecessary data transmission.

Among the several training paradigms enabled by edge intelligence, federated
learning (FL) has emerged as a popular solution by providing low communication
overhead, enhanced user privacy, and security to distributed learning [11].
With FL, the ML model is trained cooperatively by edge devices without sharing
local data, but exchanging only model parameters. The usual implementation
envisages an iterative procedure whereby a central server collects local
updates from the clients (e.g., edge devices) and returns an aggregated global
model. In the rest of the paper, we refer to centralized FL (CFL) to the
traditional server-dependent FL scenario. In this setting, the server has to
wait for all the clients before returning a new global update. Therefore, high
network latency, unreliable links, or straggled clients may slow down the
training process and even worsen model accuracy [12]. In addition, the central
server represents a single point of failure, i.e., if it is unreachable due to
network problems or an attack, the training process cannot continue.
Furthermore, it may also become a bottleneck when the number of clients is
very large [13].

Decentralized and server-less solutions for federated learning have been
introduced in the literature, mainly to overcome the single point of failure
and the security problems [14, 15]. In [16] a decentralized FL mechanism was
proposed by enabling one-hop communication among FL clients. Similarly, gossip
FL (GFL) extends device-to-device (D2D) communications to compensate for the
lack of an orchestrating central server [17, 18]. It guarantees a low
communication overhead thanks to the reduced number of messages [19].

Beyond, we find more sophisticated proposals, like blockchain-enabled
federated learning (BFL), which adopts blockchain to share FL information
among devices, thus removing the figure of the orchestrating central server.
In this way, blockchain removes the single point of failure for the sake of
openness and decentralization and provides enhanced security via tampered-
proof properties [20].

###  I-B Contributions

Despite in the literature it is possible to find papers comparing classical
centralized learning in data center with CFL [21], [22], a comparison among
the different federated learning approaches (centralized versus decentralized)
is still missing. In this work, we aim to fill this gap and, thus, we focus on
two of the most popular and widely adopted approaches for decentralizing FL:
GFL and BFL. In particular, we provide a comprehensive analysis of both
methods and compare them to traditional FL, i.e., CFL. Note that we combine
standard performance indicators for ML models, i.e., accuracy, with indicators
that quantify the efficiency of these algorithms, i.e., time complexity,
communication overhead, convergence time, and energy consumption. With our
comparison under fair conditions, we would like to provide the research
community with a complete overview of the three approaches together with all
the information to choose the best model according to the specific use cases.

The contributions of this paper may be summarized as follows:

  * •

We overview the traditional FL setting and delve into two approaches for
decentralizing it. They are selected since are two of the most popular in the
literature and are kind of diverging into two completely different solutions,
which are based on gossip communication and blockchain technology,
respectively.

  * •

We provide a thorough analysis to derive the running time complexity, the
communication overhead and the convergence time of each overviewed mechanism
for FL, including CFL, BFL, and GFL.

  * •

We provide an energy model to measure the energy consumption of each solution,
based on the associated communication and computation overheads.

  * •

We assess the performance of each method (CFL, GFL, and BFL) through extensive
simulations on widely used TensorFlow libraries [23].

  * •

We delve into the open aspects of decentralized FL, providing insights on
potential extensions, considerations, and software implementations for GFL and
BFL.

The rest of the paper is structured as follows: Section II reviews the related
work. Section III describes the three studied algorithms (CFL, BFL, and GFL).
Section IV analyzes their time complexity, the communication cost, introduces
the communication model and the convergence time. Section V provides the
energy model used in this paper. Then, Section VI compares the three
mechanisms through simulation results. In Section VII, we provide some open
issues of GFL and BFL, and we discuss the correspondent optimizations and
future research directions. Finally, Section VIII concludes the paper with
final remarks.

##  II Related Work

Distributing and decentralizing ML operations at the edge has been embraced as
an appealing solution for addressing the issues of centralization
(connectivity, privacy, and security) [24]. With FL, different devices
collaborate to train an ML model by sharing local updates obtained from local
and private data. The traditional FL algorithm (FedAvg), referred to as CFL in
this paper, is introduced in [25]. In [11], the authors propose techniques to
improve its communication efficiency. Nevertheless, CFL still requires a
central server responsible for clients orchestration and model aggregation.
The star topology is a weak aspect of CFL, since the central entity represents
a single point of failure, it may limit the number of devices that can
participate in the training process, augments the communication cost of the
learning process, and presents privacy concerns [26, 13].

To address these challenges, decentralized federated learning has been
proposed in [16]. The authors present a fully decentralized model, in which
each device can communicate only with its one-hop neighbors. The authors also
provide a theoretical upper bound on the mean square error. Ormandi et al.
[17] introduce gossip FL, a generic approach for peer-to-peer (P2P) learning
on fully distributed data, i.e., every device has only a single input sample
to process at each round. The same algorithm has been tested in [18] under
real-world conditions, i.e., devices have multiple input samples available
(rather than only one point, as originally stated in [17]), restricted
communication topology and, heterogeneous communication and computation
capabilities.

Another prominent solution to decentralize FL is blockchain-enabled FL [27,
28, 29]. A blockchain system allows clients to submit and retrieve model
updates without the central server. Additionally, the usage of blockchain
guarantees security, trust, privacy, and traceability, however, it introduces
delays due to the distributed ledger technology. An analysis of end-to-end
latency and the effects of blockchain parameters on the training procedure of
BFL is proposed in [20].

In the literature, there exist some comparisons across FL techniques. The
authors of [30] compare GFL and CFL with a logistic regression model in terms
of convergence time, proportion of the misclassified examples in the test set
(0-1 error), and used communication resources. When nodes have a random subset
of the learning samples, GFL performance is comparable with CFL; instead, CFL
converges faster when a node has only labels from one class. Another
comparison is proposed in [31], where the performance of FL algorithms that
require a central server, e.g., FedAvg and Federated Stochastic Reduced
Gradient are analyzed. Results show that FedAvg achieves the highest accuracy
among the FL algorithms regardless of how data are partitioned. In addition,
the comparison between FedAvg and the standard centralized algorithm shows
that they are equivalent when independent and identically distributed (IID)
datasets are used.

In [19], the authors compare GFL with the standard centralized data center
based architecture in terms of accuracy and energy consumption for two radio
access network use-cases. To achieve this goal, they use the machine learning
emission calculator [32] and green algorithms [33]. In [21], the authors
compare centralized data center based learning and CFL in terms of carbon
footprint using different datasets. The assessment is done by sampling the CPU
and GPU power consumption. In [22], the authors propose a framework to
evaluate the energy consumption and the carbon footprint of distributed ML
models with focus on industrial Internet of Things applications. The paper
identifies specific requirements on the communication network, dataset and
model size to guarantee the energy efficiency of CFL over centralized learning
approaches, i.e., bounds on the local dataset or model size. Differently from
our work, the authors do not consider Blockchain-enabled FL and evaluate the
algorithm performance in scenarios with limited number of devices (i.e., 100).
Moreover, we also empirically measure the energy consumption of the devices
based on the real load of the computations realized during the training phase;
we provide a communication model to estimate overheads and convergence time.
Additionally, here we introduce an analysis on the computational complexity of
the three federated algorithms under study.

In summary, in this paper, we bridge the gap in the literature by providing a
thorough comparison including performance analysis and cost of the different
federated approaches listed above, i.e., CFL, BFL, and GFL. Differently from
the other works in the literature, we combine standard metrics, i.e.,
accuracy, with indicators of the efficiency of these algorithms, i.e.,
computational complexity, communication overhead, convergence time and energy
consumption. Our final aim is to contribute to the development of Green AI
[34].

##  III Federated Learning Implementations

Let us consider a set of $N$ clients (or devices) $\mathcal{N}=\\{1,...,N\\}$
with their datasets $D_{1},...,D_{N}$. Each local dataset $D_{i},\forall
i\in\mathcal{N}$, contains pairs $(x_{i},y_{i})$, where $x_{i}$ is the feature
vector, and $y_{i}$ its true label. The goal of a federated setting is to
train a global model (e.g., a set of weights $w$), that minimizes the weighted
global loss function:

$\ell=\sum_{i=1}^{N}\ell_{i}(w,x_{i},y_{i}),$ (1)

where $\ell_{i}$ is the local loss experienced by client $i$. In this
scenario, devices do not share raw local data with other devices. Instead,
they exchange model parameter updates, computed during several iterations by
training the global model on local data. In this paper, we study three
different implementations to solve the federated problem stated above, namely:
CFL, BFL, and GFL. The investigated solutions are depicted in Fig. 1 and we
will introduce them in what follows. Though several variants are available in
the literature, the three algorithms described next are baseline
representations of the approaches studied and well suitable for our purposes.

Figure 1: Overview of the different FL scenarios.

###  III-A Centralized Federated Learning (CFL)

At the beginning of a round $t$, a random subset of $m$ devices
$\mathcal{S}^{t}\subseteq\mathcal{N}$ is selected, and the server sends the
current global model to the parties. Each client makes $E$ epochs on the local
dataset with a mini-batch size of $B$, updates its local model $w_{k}^{t+1}$
and sends it to the server. The server aggregates the local updates and
generates the new global model by computing the weighted average of the local
updates as follows:

$w^{t+1}=\sum_{k\in\mathcal{S}^{t}}\frac{|D_{k}|}{|D|}w_{k}^{t+1},$ (2)

where $|D|=\sum_{k\in\mathcal{S}^{t}}|D_{k}|$. The process is repeated until
the model reaches convergence, e.g., the loss function does not improve across
subsequent epochs or a specific number of training rounds have been executed.
In this work, we consider the FedAvg algorithm [25] as a merging method to
generate global model updates.

Algorithm 1 describes the CFL with FedAvg mechanism. The procedure Main is
executed by the server that coordinates the whole training process. Each
client executes the procedure ClientUpdate and applies the stochastic gradient
descent (SGD) algorithm on its local dataset with a learning rate $\eta$.

Algorithm 1 CFL

1:procedure Main

2: initialize $w_{0}$

3: $t\leftarrow 0$

4: while convergence is not reached do

5: $\mathcal{S}^{t}\leftarrow$ random set of $m$ clients

6: for each client $k\in\mathcal{S}^{t}$ in parallel do

7: Download the global model $w^{t}$

8: $w^{t+1}_{k}\leftarrow$ ClientUpdate($k$, $w^{t}$)

9: Send $w_{k}^{t+1}$ to the server

10: end for

11:
$w^{t+1}\leftarrow\sum_{k\in\mathcal{S}^{t}}\frac{|D_{k}|}{|D|}w^{t+1}_{k}$

12: $t\leftarrow t+1$

13: end while

14:end procedure

1:procedure ClientUpdate$(k,w)$$\triangleright$ Run on client $k$

2: $\mathcal{B}\leftarrow$ split the local dataset into batches of size $B$

3: for $E$ local epochs do

4: for batch $b\in\mathcal{B}$ do

5: $w\leftarrow w-\eta\nabla\ell(w,b)$

6: end for

7: end for

8: return $w$

9:end procedure

###  III-B Blockchain-enabled Federated Learning (BFL)

BFL is based on distributed ledger technology, which collects data in form of
transactions and organizes it in blocks. Indeed, a blockchain is a sequence of
blocks chained one after the other through advanced cryptographic techniques.
Each block contains the hash value of the previous one, leading to a tampered-
proof sequence and providing properties that are essential to building trust
in decentralized settings, such as transparency and immutability. In a
blockchain, a set of participant nodes (miners) apply certain mining protocols
and consensus mechanisms to append new blocks to the blockchain and agree on
the status of the same. This procedure allows devices to write concurrently on
a distributed database and guarantees that any malicious change on data would
not be accepted by the majority, so that data in a blockchain is secured.

When a blockchain is applied to a federated setting, the process is going as
follows [20]:

  1. 1.

Each device submits its local model updates in the form of transactions to the
blockchain peer-to-peer (P2P) network of miners.

  2. 2.

The transactions are shared and verified by miners.

  3. 3.

Miners execute certain tasks to decide which node updates the chain. One of
the most popular mining mechanisms, and studied in this paper, is Proof-of-
Work (PoW) [35], whereby miners spend their computational power (denoted by
$\lambda$) to solve computation-intensive mathematical puzzles.

  4. 4.

As a result of the concurrent mining operation, a new block is created and
propagated throughout the P2P blockchain network every $BI$ seconds (on
average). The block size $S_{B}$ is selected such that can include a maximum
of $m$ transactions, each one representing a local model submitted by a
client.

  5. 5.

Clients download the latest block from its associated miner (as in [27, 36]),
which would allow performing on-device global model aggregation and local
training.

An important consequence of the blockchain decentralized consensus is forking.
A fork occurs when two or more miners generate a valid block simultaneously
(i.e., before the winning block succeeds to be propagated). The existence of
forks can be seen as a waste of resources, as it may lead to extra computation
and delay overheads [37]. In this work, we consider the version of BFL
reported in Algorithm 2 [20], which entails the participation of multi-access
edge computing (MEC) servers and edge devices. Each client downloads the
updates ${w^{t}_{1}...w^{t}_{m}}\in b^{t}$ contained in the latest block,
computes the new global $w^{t}$, and trains it on its local dataset with the
ClientUpdate procedure described in Section III-A. The parameters of the new
updated model $w^{t+1}_{k}$ are then submitted with the method
SubmitLocalUpdate, where $S_{tr}$ is the transaction size. Once all the local
updates are uploaded to the blockchain, a new block $b^{t+1}$ is mined with
MineBlock, where the block generation rate, $\lambda=\frac{1}{BI}$, is derived
from the total computational power of blockchain nodes. Finally, the new block
is shared across all the blockchain nodes with the procedure PropagateBLock,
which depends on the size of block $b^{t+1}$ (fixed to $S_{B}$). The process
is repeated until convergence.

Algorithm 2 BFL

1:procedure Main

2: $t\leftarrow 0$

3: initialize $w_{0}$

4: while convergence is not reached do

5: $\mathcal{S}^{t}\leftarrow$ random set of $m$ clients

6: for each client $k\in\mathcal{S}^{t}$ in parallel do

7: Download the latest block, $b^{t}$

8: $w^{t}\leftarrow\sum_{j\in b^{t}}\frac{|D_{j}|}{|D|}w_{j}^{t}$

9: $w^{t+1}_{k}\leftarrow$ ClientUpdate($k$, $w^{t}$)

10: SubmitLocalUpdate($S_{tr}$)

11: end for

12: $b^{t+1}\leftarrow$MineBlock($\lambda$)

13: PropagateBlock($b^{t+1}$)

14: if $b^{t+1}$ is not valid then

15: Go to line 12

16: end if

17: $t\leftarrow t+1$

18: end while

19:end procedure

###  III-C Gossip Federated Learning (GFL)

GFL is an asynchronous protocol that trains a global model over decentralized
data using a gossip communication algorithm [17, 18].

We consider the general skeleton proposed in [18] and [19]. Overall, the
participating clients start from a common initialization. The global model is
then trained sequentially on local data and following a given path (e.g.,
random walk) of visiting clients.

Algorithm 3 describes the GFL procedure. At each round $t$, $m$ nodes are
randomly selected and ordered in a sequence
$\mathcal{S}^{t}=[k_{1},...,k_{m}]$. Every node $k_{i}\in\mathcal{S}^{t}$
receives the model $w_{k_{i-1}}^{t}$ from the previous node in the sequence
and performs the steps below, also reported in Fig. 2 for a better
understanding:

  1. 1.

Run the procedure Merge to combine $w_{k_{i-1}}^{t}$ and the model saved in
its local cache, $\rm{lastModel}_{k_{i}}$, i.e., the model from the previous
round in which the node have been selected.

  2. 2.

Train the model generated in 1) on the local dataset with the procedure
ClientUpdate described in Section III-A.

  3. 3.

Update the local cache ($\rm{lastModel}_{k_{i}}$) with the model received from
the previous node $w^{t}_{k_{i-1}}$.

  4. 4.

Share the model trained on the local dataset $w_{k_{i}}^{t}$ with the
following node in the sequence.

A round is completed when the model has visited all the nodes in the sequence.
The algorithm stops when convergence is reached (after a given number of
rounds).

Algorithm 3 GFL

1:procedure Main

2: initialize $lastModel_{k}$ for each client $k$

3: $t\leftarrow 0$

4: while convergence is not reached do

5: $\mathcal{S}^{t}\leftarrow$ random set of $m$ clients

6: $[k_{1},...,k_{m}]\leftarrow\textsc{GetSequence}(\mathcal{S}^{t})$

7: for $i=1,...,m$ do

8: $w^{t}_{k_{i-1}}$ $\triangleright$ Model trained by the previous node in
the sequence

9:
$w^{t}_{k_{i}}\leftarrow\textsc{Merge($w^{t}_{k_{i-1}},\rm{lastModel}_{k_{i}}$)}$

10: $w^{t}_{k_{i}}\leftarrow\textsc{ClientUpdate}(k_{i},w^{t}_{k_{i}})$

11: $\rm{lastModel}_{k_{i}}$ $\leftarrow$ $w^{t}_{k_{i-1}}$

12: Send model to the next client

13: end for

14: $t\leftarrow t+1$

15: end while

16:end procedure

17:

18:procedure Merge$(w,w^{\prime})$

19: $w\leftarrow\frac{w+w^{\prime}}{2}$

20: return $w$

21:end procedure

22:procedure GetSequence($S^{t}$)

23: $[k_{1},\ldots,k_{m}]\overset{\text{i.i.d.}}{\sim}U(S^{t})$

24: return $[k_{1},\ldots,k_{m}]$

25:end procedure

Figure 2: Overview of the operations executed by a node in the GFL algorithm.

##  IV Computational and Communication Costs

In this section, we introduce the mathematical statements for the calculation
of the time complexity of the three federated algorithms considered in Section
III. We elaborate also on the data overhead due to the communication of the
different model updates during the several rounds of the process of each
implementation. Finally, we derive the equations for the calculation of the
time to reach the convergence of the three analyzed federated approaches. The
results proposed hereafter are derived using the following assumptions:

  1. 1.

Scalar operations (sums and products) cost $O(1)$.

  2. 2.

The time complexity of the matrix multiplication is linear with the matrix
size, i.e., $A\in\mathbb{R}^{i\times j}$ and $B\in\mathbb{R}^{j\times k}$, the
cost of the product is $O(i\cdot j\cdot k)$.

  3. 3.

For a single input pair $(x_{i},y_{i})$, the time complexity required to
compute $\nabla\ell$ is linear with the number of model’s weights, $O(|w|)$.

  4. 4.

During the mining process, with the PoW, a miner computes the nonce of a block
using brute force until finding a hash value lower or equal to a certain
threshold [38], referred to as the mining difficulty. Assume that the hash
value has $b$ bits, and that its solution should be smaller than $2^{b-l}$
bits (being $l$ a value determined by the mining difficulty), if the miner
samples the nonce values at random, the probability of a valid value is
$2^{-l}$. Henceforth, $2^{l}$ sampling operations are required for mining a
block. The time complexity is $O(2^{l})$.

  5. 5.

The set of nodes that have a local copy of the blockchain is
$\mathcal{N}_{B}=\\{1,...,N_{B}\\}$, without loss of generality, is assumed to
be $\mathcal{N}\cap\mathcal{N}_{B}=\varnothing$.

  6. 6.

We assume that convergence of the FL training procedure is reached after $R$
rounds.

######  Theorem 1.

The time complexity of CFL is:

$O(RmE|D_{\max}||w|),$ (3)

where $D_{\max}=\max_{k\in\mathcal{N}}|D_{k}|$. The communication overhead is
given by

$2Rm|w|$ (4)

###### Proof.

See Appendix A-A. ∎

######  Theorem 2.

The time complexity of BFL is:

$O(R(|w|m^{2}+E|D_{\max}||w|m+2^{l}+m|w|N_{B})),$ (5)

where $N_{B}$ is the number of nodes that have a local copy of the blockchain
and $l$ is related to the PoW difficulty (see Assumption 4). Its communication
overhead is

$R\left(|w|m^{2}+|w|m+m|w|N_{B}\right)$ (6)

###### Proof.

See Appendix A-B. ∎

######  Theorem 3.

The time complexity of GFL is

$O(RmE|D_{\max}||w|)$ (7)

and its communication overhead is

$Rm|w|$ (8)

###### Proof.

See Appendix A-C. ∎

TABLE 1: Computational complexity and communication overhead for CFL, BFL and GFL. Algorithm | Time complexity | Communication Overhead
---|---|---
CFL | $O(RmE|D_{\max}||w|)$ | $2Rm|w|$
BFL | $O(R(|w|m^{2}+E|D_{\max}||w|m+2^{l}+m|w|N_{B}))$ | $R(|w|m^{2}+|w|m+m|w|N_{B})$
GFL | $O(RmE|D_{\max}||w|)$ | $Rm|w|$

The three algorithms have a time complexity that depends on the dataset size
$|D_{\max}|$. Additionally, the time complexity of BFL (5) is also a function
of the blockchain parameters $N_{B}$ and $l$. In particular, the dominant term
in (5) is $R2^{l}$. Hence, the time complexity of BFL is exponential in the
PoW difficulty $l$, while for CFL and GFL is polynomial in $RmE|D_{\max}||w|$.
Table 1 summarizes the different results obtained for time complexity and
communication overhead.

To finalize our analysis, we compute the total execution time of each
algorithm till convergence (convergence time) as a function of the delay
introduced by the communication rounds and the computational operations. To do
that, we characterize the links whereby the different types of nodes exchange
information (e.g., local model updates, blocks), having in mind the topology
introduced in Fig. 1. We classify two different types of connections: cloud
(solid arrows) and edge (dotted and dashed arrows). Cloud connections (assumed
to be wired) are used by miners in the blockchain; instead, edge connections
(assumed to be wireless) are used by edge nodes to upload/download models.
Given its popularity and easiness of deployment, we adopt IEEE 802.11ax links
for edge connections [39]. Since edge devices are often energy-constrained, we
consider different values of transmission power for the edge connections. The
central server and blockchain node use a transmission power of
$P_{\text{TX}}^{c}$, instead, edge devices use $P_{\text{TX}}^{e}$, with
$P_{\text{TX}}^{e}\leq P_{\text{TX}}^{c}$. The wired connection has a capacity
$C_{P2P}$. Additionally, we identify three main types of computational
operations during the federated learning processes: local model training,
model parameters exchange, and blockchain data sharing. Based on this, we can
compute the convergence time of CFL, BFL, and GFL as follows:

$T_{\rm CFL}=T_{\rm train}+T_{\rm{Tx}}^{e}+T_{\rm{Tx}}^{c},$ (9) $T_{\rm
BFL}=T_{\rm BC}+T_{\rm train}+T_{\rm{Tx}}^{e}+T_{\rm{Tx}}^{c},$ (10) $T_{\rm
GFL}=T_{\rm train}+T_{\rm{Tx}}^{e},$ (11)

where $T_{\rm train}$ is the total amount of time spent for training the ML
model locally, $T_{\rm{Tx}}^{c/e}$ is the total transmission time of the
central server/blockchain nodes $(c)$ or the edge devices $(e)$, and computed
according to the model detailed in Appendix B. $T_{\rm BC}$ is the total delay
introduced by blockchain and described in steps 2-4 of the process in Section
III-B.

##  V Energy Footprint

In this section, we define the models used to characterize the energy
consumption that results from the FL operations. Driven by (9), (10) and (11),
the total amount of energy consumed in each scenario is:

$\mathcal{E}_{\rm CFL}=\mathcal{E}_{\rm
train}+\mathcal{E}_{\rm{Tx}}^{e}+\mathcal{E}_{\rm{Tx}}^{c},$ (12)
$\mathcal{E}_{\rm BFL}=\mathcal{E}_{\rm BC}+\mathcal{E}_{\rm
train}+\mathcal{E}_{\rm{Tx}}^{e}+\mathcal{E}_{\rm{Tx}}^{c},$ (13)
$\mathcal{E}_{\rm GFL}=\mathcal{E}_{\rm train}+\mathcal{E}_{\rm{Tx}}^{e},$
(14)

where $\mathcal{E}_{\rm train}$ is the energy consumed by all the nodes during
the local training, and $\mathcal{E}_{\rm{Tx}}^{c/e}$ the energy required to
transmit the model through the IEEE 802.11ax wireless links during the whole
algorithm execution, from either a central server/blockchain node ($c$) or an
edge device ($e$). $\mathcal{E}_{train}$ is calculated as:

$\mathcal{E}_{\rm train}=\sum_{r=0}^{R-1}\sum_{i\in\mathcal{S}^{r}}P_{{\rm
CPU}_{i}}^{r}\Delta_{i}^{r},$ (15)

where $P_{{\rm CPU}_{i}}^{r}$ is the average power consumed by the CPU and
DRAM during a round $r$, and $\Delta_{i}^{r}$ is the duration of the operation
for the $i-th$ client. As described in Section IV, we may have two types of
communication links: cloud and edge. Considering that cloud links are wired,
we assume that their energy consumption is negligible. Instead, we compute the
energy consumption of the edge connections according to the following
equation:

$\mathcal{E}_{\rm{Tx}}^{c/e}=T_{\rm{Tx}}^{c/e}P_{\rm{Tx}}^{c/e},$ (16)

where $T_{\rm{Tx}}^{c/e}$ and $P_{\rm{Tx}}^{c/e}$ are the transmission time
and power of a central server/miner $(c)$ or an edge device $(e)$. The
additional term $\mathcal{E}_{\rm BC}$ for BFL is associated to mining
operations of the PoW. We measure that consumed energy based on the model
proposed in [40] and according to the following equation:

$\mathcal{E}_{\rm BC}=P_{h}\frac{1}{\lambda}N_{\rm chain},$ (17)

where $P_{h}$ is the total hashing power of the network, $\lambda$ is the
block generation rate, and $N_{\rm chain}$ is the number of blocks on the main
chain.

##  VI Performance Evaluation

In this section, we first describe the experimental settings adopted to
compare the three federated approaches and then, we present numerical results.

###  VI-A Simulation Setup

We use the Extended MNIST (EMNIST) dataset available on Tensorflow Federated
(TFF) library [41]. The input features are black and white images that
represent handwritten digits in $[0,9]$, coded in a matrix of $28\times 28$
pixels. Considering only digits, it contains $341\,873$ training examples and
$40\,832$ test samples, both divided across $3\,383$ users. The training and
the test sets share the same users’ list so that each user has at least one
sample. In the EMNIST dataset, all the clients have a rich number of samples
for all the classes, thus data distribution across them can be considered as
IID. To evaluate the targeted federated mechanisms in more challenging
settings, we create a new version of the EMNIST dataset, called EMNISTp, by
randomly restricting each client dataset to $3$ classes only. Fig. 3 shows the
available samples of the first $4$ clients, for both versions of the dataset.

Figure 3: Distribution of samples across the four first clients for both
EMNIST and EMNISTp federated datasets.

To correctly classify these samples we choose two models proposed in [25]. The
first one is a feed-forward neural network (FFNN) with an input layer of $784$
neurons, two hidden layers of $200$ neurons activated with the rectified
linear unit (ReLU) function, and an output layer of $10$ neurons with
_Softmax_ activation function. The number of trainable parameters for the FFNN
$(|w^{\prime}|)$ is $199\,210$. Assuming that each parameter requires $4$
bytes in memory, i.e., size of a _float32_ variable, the total amount of space
required ($S_{w^{\prime}}$) is $796.84$ KB. The second one is a convolutional
neural network (CNN) with the following structure:

  1. 1.

A $5\times 5$ convolutional layer with $32$ channels and ReLU activation
function.

  2. 2.

A $2\times 2$ max pooling layer.

  3. 3.

A $5\times 5$ convolutional layer with $64$ channels and ReLU activation
function.

  4. 4.

A $2\times 2$ max pooling layer.

  5. 5.

A fully connected layer with $512$ units and ReLu activation.

  6. 6.

An output layer with $10$ neurons and Softmax activation function.

In total, the number of trainable parameters for the CNN
$(|w^{\prime\prime}|)$ is $582\,026$ and the size in memory
$(S_{w^{\prime\prime}})$ is $2.33$ MB. We opted for a FFNN to reproduce a
realistic scenario where edge devices might not have enough computational
power to train more sophisticated deep learning mechanisms, like NNs based on
convolutional architectures. Despite of its simplicity, the selected FFNN
model accurately classifies the digits of the EMNIST dataset, as shown next.
Then, we use also a CNN to evaluate the multiple algorithms’ performance using
a more complex model (details in Section VII-C).

The three FL algorithms are implemented with Tensorflow (TF) [23], Tensorflow
Federated (TFF) [42] and Keras [43] libraries. We have extended the Bitcoin
model provided by BlockSim [44] to simulate the blockchain behavior. BlockSim
is an event-based simulator that characterizes the operations carried out to
store data in a blockchain, from the submission of transactions to mining
blocks and reaching consensus in a decentralized manner. Accordingly, BlockSim
allows simulating the delays added by the blockchain in a BFL application,
i.e., the $T_{BC}$ parameter defined in Section IV.

We create a validation set by choosing a subset of $200$ clients from the test
set. Following the TFF documentation [45], the training accuracy is computed
at the beginning of each round; instead the validation accuracy is calculated
at the end. For this reason may happen that the validation accuracy is higher
than the training one. At the end of each simulation, we evaluate the
performance on the test set.

As for $\mathcal{E}_{\rm train}$ and $T_{\rm train}$, we have used
Carbontracker [46], a Python library that periodically samples the hardware
energy consumption and measures the execution time. Moreover, $P_{\rm Tx}^{c}$
is set to $20\text{ dBm}$ and $P_{\rm Tx}^{e}=9\text{ dBm}$. Table 2 reports
all the other parameters used in our simulations. We note here that we have
used the same FL parameters for a fair comparison, being the number of rounds
$R$ of CFL and GFL equivalent to the main chain’s length ($N_{\rm chain}$) in
BFL. In such a way, we guarantee that the number of global rounds of each
learning algorithm is the same. We run the experiments with the following
hardware configurations: Intel i5-6600 with $8\text{GB}$ of RAM (HW1) and two
Intel Xeon 6230 with $188\text{GB}$ of RAM (HW2).

TABLE 2: Simulation parameters.

| Parameter | Description | Value
---|---|---|---
Fed. Learning | $|w^{\prime}|$ | Number of FFNN model parameters | $199\,210$
$|w^{\prime\prime}|$ | Number of CNN model parameters | $582\,026$
$S_{w^{^{\prime}}}$ | FFNN model parameters size |  $796.84$ KB
$S_{w^{\prime\prime}}$ | CNN model parameters size |  $2.33$ MB
$\eta$ | Learning rate | $0.2$
$N$ | Number of total clients | $3382$
$E$ | Local epochs number | $5$
$R$ | Number of rounds | $200$
$m$ | Number of clients for each round | $200$
$B$ | Batch size | $20$
$\ell_{i}$ | Local loss function | Sparse Cat. Crossentropy
Blockchain | $N_{\rm chain}$ | Number of blocks in the main chain | 200
$BI$ | Block interval |  $15$ s
$N_{B}$ | Number of blockchain nodes | $200$
$N_{m}$ | Number of miners | $10$
$C_{\rm P2P}$ | Capacity of P2P links |  $100$ Mbps
$S_{H}$ | Block header size |  $25$ KB
$S^{\prime}_{B}$ | Block size with FFNN |  $160.368$ MB
$S^{\prime\prime}_{B}$ | Block size with CNN |  $467$ MB
$S^{\prime}_{\rm tr}$ | Transaction size with FFNN |  $796.84$ KB
$S^{\prime\prime}_{\rm tr}$ | Transaction size with CNN |  $2.33$ MB
$P_{h}$ | Total hashing power |  $1350$ W
Communication (IEEE 802.11ax) | $P_{\rm{Tx}}^{e}$ | Tx power for edge devices |  $9$ dBm
$P_{\rm{tx}}^{c}$ | Tx power for a central server |  $20$ dBm
$\sigma_{\text{leg}}$ | Legacy OFDM symbol duration |  $4$ µs
$N_{\rm sc}$ | Number of subcarriers ($20$ MHz) | $234$
$N_{\rm ss}$ | Number of spatial streams | $1$
$T_{\rm{e}}$ | Empty slot duration |  $9$ µs
$T_{\rm{SIFS}}$ | SIFS duration |  $16$ µs
$T_{\rm{DIFS}}$ | DIFS duration |  $34$ µs
$T_{\rm{PHY}}$ | Preamble duration |  $20$ µs
$T_{\rm{HE-SU}}$ | HE single-user field duration |  $100$ µs
$L_{s}$ | Size OFDM symbol |  $24$ bits
$L_{\rm{RTS}}$ | Length of an RTS packet |  $160$ bits
$L_{\rm{CTS}}$ | Length of a CTS packet |  $112$ bits
$L_{\rm{ACK}}$ | Length of an ACK packet |  $240$ bits
$L_{\rm{SF}}$ | Length of service field |  $16$ bits
$L_{\rm{MAC}}$ | Length of MAC header |  $320$ bits
CW | Contention window (fixed) | $15$

###  VI-B Result Analysis

Table 3 reports the accuracy of each algorithm implementation with the FFNN
model on the two considered datasets executed on HW1. CFL and BFL achieve the
best accuracy (both close to $0.9$), instead, GFL presents lower values. This
result validates the claim that, under similar setups as in our simulations
(i.e., each block contains $m$ local updates organized in transactions), the
central parameter server of CFL can be replaced by a blockchain network,
properly dimensioned, without compromising the learning accuracy. On the other
hand, GFL achieves a validation accuracy around $0.5$ after $200$ rounds. We
justify this behavior by noticing that, before the ClientUpdate procedure, the
model received from the previous node in the sequence is merged with that in
the previous round. For the earliest training rounds, there is a high
probability that the merging procedure is with a fresh model that has never
been trained, hence disrupting the knowledge from the previous clients. We
analyze more in-depth this phenomenon in Section VII-A.

TABLE 3: FFNN simulation results on EMNIST (EMNISTp) datasets. | Acc. Training | Acc. Validation | Acc. Test | Conv. Time (s) | Comp. Energy (%) | Tot. Energy (Wh) | Comm. Overhead (GB)
---|---|---|---|---|---|---|---
CFL |  $0.9$ $(0.76)$ |  $0.87$ $(0.77)$ |  $0.86$ $(0.76)$ |  $46458.56$ $(45571.86)$ |  $98.91$ $(98.61)$ |  $21.84$ $(17.2)$ | $63.75$
BFL |  $0.88$ $(0.74)$ |  $0.87$ $(0.78)$ |  $0.86$ $(0.77)$ |  $51036.87$ $(50077.75)$ |  $99.98$ $(99.98)$ |  $1147.21$ $(1142.65)$ | 12781.31
GFL |  $0.44$ $(0.36)$ |  $0.42$ $(0.12)$ |  $0.41$ $(0.11)$ |  $38201.67$ $(36821.67)$ |  $99.57$ $(99.14)$ |  $17.83$ $(8.98)$ | $31.87$

Table 3 also details the convergence time of each algorithm, the percentage of
energy consumed in the computations (as a percentage of the total energy
consumed), the total amount of energy needed and the communication overhead
(i.e., data to be shared during the rounds of the algorithms). The fastest and
the most energy-efficient algorithm is GFL: it is able to save the 18% of
training time, the 18% of energy, and the 51% of data to be shared with
respect to the CFL solution. However, GFL main drawback resides in the poor
accuracy achieved, as stated above. BFL is the slowest and the most energy-
hungry federated implementation, mainly due to the overhead introduced by the
blockchain network to secure data in a decentralized way. Additionally, it is
to be noted that computation is the most energy-consuming task for CFL, BFL,
and GFL. For BFL, the mining process drains $1125$ Wh, i.e., $98\%$ of the
total energy. We highlight here that our comparison may be unfair in this
respect, since both CFL and GFL are not including any security mechanism.
However, we believe that it is worth to include BFL in our analysis on
distributed versus centralized federated learning because, from our results,
it emerges that the secure and decentralized method introduced by the
blockchain network, despite increasing algorithm costs, does not jeopardize
its accuracy compared to its centralized counterpart CFL. Finally, GFL is the
implementation that requires the lowest communication overhead. To be more
precise, in this case, we need to include an extra cost to share the global
model across the nodes at the end of the last round (not considered in the
table), which is approximately 0.16 GB (the cost of one extra round)

##  VII Open Issues and Research Directions

###  VII-A Open Aspects of GFL

TABLE 4: FFNN simulation results of GFL on EMNIST (EMNISTp) datasets with higher number of rounds and local computations. R | E | Acc. Training | Acc. Validation | Acc. Test | Conv. Time (s) | Comp. Energy (%) | Tot. Energy (Wh)
---|---|---|---|---|---|---|---
$200$ | $5$ |  $0.44$ $(0.36)$ |  $0.42$ $(0.12)$ |  $0.41$ $(0.11)$ |  $36401.67$ $(35021.67)$ |  $99.59$ $(99.19)$ |  $17.83$ $(8.98)$
$400$ | $5$ |  $0.55$ $(0.41)$ |  $0.51$ $(0.16)$ |  $0.5$ $(0.15)$ |  $72791.81$ $(70029.31)$ |  $99.58$ $(99.18)$ |  $35.08$ $(17.91)$
$800$ | $5$ |  $0.85$ $(0.64)$ |  $0.67$ $(0.19)$ |  $0.66$ $(0.17)$ |  $145651.71$ $(140069.23)$ |  $99.59$ $(99.18)$ |  $70.95$ $(35.8)$
$200$ | $10$ |  $0.57$ $(0.58)$ |  $0.4$ $(0.09)$ |  $0.41$ $(0.1)$ |  $37377.47$ $(35448.58)$ |  $99.7$ $(99.38)$ |  $24.55$ $(11.77)$
$400$ | $10$ |  $0.66$ $(0.37)$ |  $0.47$ $(0.17)$ |  $0.48$ $(0.16)$ |  $74989.72$ $(70857.46)$ |  $99.7$ $(99.37)$ |  $49.17$ $(23.23)$
$800$ | $10$ |  $0.94$ $(0.58)$ |  $0.67$ $(0.3)$ |  $0.67$ $(0.29)$ |  $149448.14$ $(141730.77)$ |  $99.7$ $(99.38)$ |  $98.37$ $(46.87)$

As described before, GFL is not able to achieve the same accuracy level as CFL
and BFL. We identify two possible reasons for this behavior:

  1. 1.

The number of rounds is not enough to converge: the number of visited nodes
might not be sufficient to hit an acceptable accuracy.

  2. 2.

The merge step negatively impacts the performance of the learning algorithm:
the model received in the previous round and stored in the local cache slows
down the learning process.

To verify the first hypothesis, we execute GFL algorithm, with the FFNN model
on HW1, changing the number of rounds ($R=\\{200,400,800\\}$) and varying the
number of local computations ($E=\\{5,10\\}$). Table 4 shows the results
obtained. Considering the EMNIST dataset, the best results are achieved with
$R=800$ and $E=\\{5,10\\}$, i.e., a higher test accuracy of $0.66$, but still
lower than CFL and BFL. Moreover, the model is overfitting with $R=800$
rounds; hence, a regularization method would be needed, when increasing the
number of rounds. On the EMNISTp dataset, the accuracy is even lower for each
combination of the hyperparameters tested.

To verify the second hypothesis, we run GFL algorithm without the merge step
(_GFL-NM_). The pseudocode of this algorithm is the same in Algorithm 3 but
replacing the old Line 9 with the new command $w_{k_{i}}^{t}\leftarrow
w_{k_{i-1}}^{t}$. Thus, in GFL-NM, given a sequence of clients
$\mathcal{S}_{t}$ the model is trained incrementally on the client’s datasets.
GFL-NM achieves a training accuracy of $\sim 1.0\text{ }(0.94)$, a validation
accuracy of $0.94\text{ }(0.78)$ and a test accuracy of $0.93\text{ }(0.78)$
on the EMNIST (EMNISTp) dataset (see Fig. 4(b)), higher than CFL and BFL.
These results suggest that the Merge step compromises the training
performance. In fact, at the beginning of the learning process, there is a
high probability that a model visits a node that has never been visited before
and with $\rm lastModel$ storing initialization values. In this case, the
received model is merged with a model that has never been trained before, as
shown in Algorithm 3, which negatively impacts the resulting merged weights.
Figure 4 shows the comparison between the learning curves of GFL and GFL-NM.

(a) GFL

(b) GFL-NM

Figure 4: Training and validation accuracy on EMNIST and EMNISTp

In conclusion, we have seen that both 1) and 2) influence the achieved
accuracy. Moreover, GFL-NM solves the accuracy problem of standard GFL and
reaches the best performance from all the metrics point of view. In our
opinion, and encouraged by our results, the investigation of new methods for
merging the model updates from the distributed sources to achieve faster and
higher accuracy is an interesting and open research line. To the best of our
knowledge, there are still very few works that go in this direction in the
literature. In [19], the authors implement an incremental version of GFL with
a single round on the edge devices using the entire local dataset, and, hence,
without requiring any merge step. Similarly, [47] proposes an iterative
continual learning algorithm, where a model is trained incrementally on the
local datasets without applying any merge operation.

###  VII-B Open Aspects of BFL

Blockchain technology, while enabling a reliable and secure FL operation,
entails very high overheads in terms of time and energy for the sake of
keeping decentralization. The performance of a blockchain, typically measured
in transactions per second (tps), together with the granted degree of
security, strongly depends on the nature of the blockchain (e.g., degree of
visibility, type of consensus, mining protocol), its configurable parameters
(e.g., block interval, difficulty), and the size of the P2P network
maintaining it. Furthermore, the necessary energy to maintain a blockchain is
correlated to its performance in tps and security, thus leading to the well-
known performance, security, and energy trilemma.

To showcase the effect of using different types of blockchain networks, Fig. 5
shows the total delay incurred by the blockchain to the FL operation to
generate up to 200 blocks under different blockchain configurations. Notice
that, in the proposed setting, each block is equivalent to an FL round. In
particular, we vary the total number of miners ($N_{m}=\\{1,10,100\\}$) and
the block interval ($BI=\\{5,15,600\\}$ s), which affect the time required to
achieve consensus.

First, a higher number of miners leads to a higher fork probability, provided
that more nodes need to agree on the same status of the ledger. By contrast, a
higher block interval allows mitigating the effect of forks, since the
probability that two miners mine a block simultaneously is lower [48].

Figure 5: Blockchain delay as a function of the number of miners ($N_{m}$) and
the block interval ($BI$). The fork probability associated with each $N_{m}$
is shown in red.

As shown in Fig. 5, the blockchain delay increases with the block interval
($BI$), which indicates the average time for mining a block. Notice that, in a
PoW-based blockchain, the block interval is fixed by tuning the mining
difficulty according to the total computational power of miners. As for the
impact of $N_{m}$ on the delay, its effects on the delay are more noticeable
for low $BI$ values. In particular, a higher fork probability is observed as
$N_{m}$ increases, thus incurring additional delays to the FL application
operating on top of the blockchain.

To optimize the performance of a blockchain, a widely adopted approach
consists of finding the best block generation rate [27], which is controlled
by tuning the mining difficulty. Other approaches consider optimizing the
block size [49], which fits better scenarios where the intensity of
transaction arrivals depends on the nature of the application running on top
of the blockchain (e.g., FL updates provided by clients).

Regarding the communication cost of BFL, it can be improved by leveraging the
computational capacity of blockchain miners to speed up the FL operation. In
particular, instead of including individual local models in a block, each
block can bring a global model, aggregated by the miner responsible for
building the block. This approach has been widely adopted in the literature
(see, e.g., [50]), and would lead to a reduced time complexity and
communication cost (see Appendix A-B).

Finally, another important open aspect regarding blockchain-enabled FL lies in
the implications of decentralization on the learning procedure. In this paper,
we have assumed that the blockchain is perfectly shared and accessed by FL
devices to carry out training, thus acting as a central orchestrating server.
However, the decentralized data sharing in blockchain naturally leads to model
inconsistencies, provided that different FL devices can use the information
from different blocks to compute local model updates.

###  VII-C Model and implementation dependencies

Table 5 and Table 6 report the performance of the CNN model on EMNIST
(EMNISTp) datasets executed on two different platforms HW1 and HW2,
respectively. Similar to the previous FFNN case, BFL is the slowest and the
most energy demanding algorithm. Instead in this case, GFL reaches higher
validation accuracy on EMNIST, i.e., $0.8$, but is still not able to get the
performance of the other two algorithms. Moreover, using CNN, GFL is the
fastest algorithm and saves up to $16\%$ of the execution time, with respect
to CFL on HW1. Hence, model selection plays a key role for the algorithm
performance and may facilitate the training process, as in the case of GFL.
Finally, it is confirmed that the communication overhead of GFL is the lowest.

However, we report here some inconsistencies in performing energy
measurements. In fact on HW1, differently from the FFNN case, CFL is the most
energy efficient on EMNIST and saves $14\%$ of energy with respect to GFL. On
EMNISTp, instead, the situation is different since GFL saves $15\%$ of the
energy. Moreover, when using HW2 (Table 6), CFL results to be the most energy
efficient for both EMNIST and EMNISTp. Such inconsistencies are mainly due to
the fact that the average computational power consumption in CFL
implementation is higher than GFL (around $103$W and $93$W on EMNIST,
respectively); however GFL requires longer training time ($T_{\rm train}$).
Instead in the FFNN model implementation, the average computational power
consumption is higher for GFL (around $19\text{W}$ for CFL and $13\text{W}$
for GFL), but GFL requires lower training time. The reason lays mainly in the
software
implementations111https://github.com/eliaguerra/Federated_comparison_cttc. In
fact, CFL and BFL are based on TFF, which executes the training process for
all the participating clients in parallel. Differently, GFL is based on the
standard TF libraries and the training process is executed sequentially one
client after the other.

In view of the above, we state here that hardware and software implementation
play a key role in the energy assessment. Therefore, it is essential that
future research directions will focus on: i) joint optimization of federated
algorithms and their software implementations, ii) definition of standard
libraries for the three categories of algorithms studied in this paper, and
iii) design of effective and open test platforms for experiment comparison.

TABLE 5: CNN simulation results on HW1 and EMNIST (EMNISTp) datasets. | Acc. Training | Acc. Validation | Acc. Test | Time (s) | Comp. Energy (%) | Tot. Energy (Wh) | Comm. overhead (GB)
---|---|---|---|---|---|---|---
CFL |  $0.99$ $(0.97)$ |  $0.97$ $(0.9)$ |  $0.96$ $(0.91)$ |  $132761.16$ $(126691.59)$ |  $99.04$ $(98.1)$ |  $72.35$ $(36.72)$ | $186.4$
BFL |  $0.99$ $(0.97)$ |  $0.97$ $(0.9)$ |  $0.96$ $(0.91)$ |  $138452.61$ $(132284.94)$ |  $99.94$ $(99.94)$ |  $1198.11$ $(1161.99)$ | $37373.2$
GFL |  $0.99$ $(0.67)$ |  $0.81$ $(0.22)$ |  $0.8$ $(0.22)$ |  $113573.11$ $(106616.64)$ |  $99.73$ $(99.28)$ |  $83.95$ $(31.22)$ | $93.2$
TABLE 6: CNN simulation results on HW2 and EMNIST (EMNISTp) datasets. | Acc. Training | Acc. Validation | Acc. Test | Conv. Time (s) | Comp. Energy (%) | Tot. Energy (Wh) | Comm. overhead (GB)
---|---|---|---|---|---|---|---
CFL |  $0.99$ $(0.97)$ |  $0.97$ $(0.9)$ |  $0.96$ $(0.91)$ |  $125883.47$ $(124869.07)$ |  $99.65$ $(99.51)$ |  $201.68$ $(141.69)$ | $186.4$
BFL |  $0.99$ $(0.97)$ |  $0.97$ $(0.9)$ |  $0.96$ $(0.91)$ |  $131488.87$ $(130555.3)$ |  $99.95$ $(99.95)$ |  $1329.65$ $(1273.95)$ | $37373.2$
GFL |  $0.99$ $(0.67)$ |  $0.8$ $(0.22)$ |  $0.8$ $(0.22)$ |  $114217.66$ $(107854.86)$ |  $99.93$ $(99.84)$ |  $319.77$ $(143.22)$ | $93.2$

##  VIII Conclusions

Decentralized server-less federated learning is an appealing solution to
overcome CFL limitations. However, finding the best approach for each scenario
is not trivial due to the lack of comprehensive comparisons. In this work, we
have proposed a complete overview of these techniques and evaluated them
through several key performance indicators: accuracy, computational
complexity, communication overhead, convergence time, and energy consumption.
To do so, we have proposed a comprehensive theoretical analysis and an
implementation of these algorithms.

An extensive simulation campaign has driven our analysis. From numerical
results, it emerges that GFL is the algorithm that requires less communication
overhead to reach convergence. Then, CFL and GFL have similar behavior in
terms of energy consumption and accuracy, but slightly differ based on the DL
model adopted and the hardware used. BFL represents a viable solution for
implementing decentralized learning with a high accuracy and level of security
at the cost of an extra energy usage and data sharing.

Moreover, we have discussed some open issues and future research directions
for the two decentralized federated methods, like the poor accuracy achieved
by GFL and the blockchain overhead in BFL. Regarding GFL, we have argued that
the main drawback lies in the method used to merge model updates across the
algorithm steps. We have demonstrated that with an incremental approach, the
modified version of GFL is able to outperform CFL and BFL. As for BFL, we have
indicated that possible optimizations go in the direction of finding the best
block generation rate and block size. Moreover, we have reasoned on the
possibility of reducing the time complexity by including the global model in a
block, which is aggregated by the same miner building the block. In addition,
we have pointed out the importance of further studies on the implication of
model inconsistencies due to the fact that the blockchain cannot be perfectly
shared and accessed by (all) the FL devices.

Finally, we have argued on the key role played by the libraries used for the
implementation and their influence on the energy consumption on different
hardware platforms. We call for the definition of standard libraries and open
test platforms to be used for research purposes.

##  Appendix A Proofs

###  A-A Proof of Theorem I

Let us consider the procedure ClientUpdate, whose time complexity is
$E\left(|D_{\max}||w|+2\frac{|D_{\max}|}{B}|w|\right)$. In fact, a single
client $k$ performs the training phase on its local dataset $D_{k}$ along $E$
local epochs and updates the model parameters. The first operation has a time
complexity of $|D_{k}||w|$ and the second $2\frac{|D_{k}|}{B}|w|$. The update
it is executed a number of times equal to $\frac{|D_{k}|}{B}$, and requires a
product and a sum. Each client performs $E$ local epochs, so the total cost
is:

$\sum_{k\in S_{t}}E\left(|D_{k}||w|+2\frac{|D_{k}|}{B}|w|\right)$ (18)

To obtain an upper bound that does not depend on $k$, we can use $|D_{\max}|$
as an upper bound of $|D_{k}|$:

$\begin{split}\sum_{k\in
S_{t}}E&\left(|D_{k}||w|+2\frac{|D_{k}|}{B}|w|\right)\leq\\\
&mE\left(|D_{\max}||w|+2\frac{|D_{\max}|}{B}|w|\right).\end{split}$ (19)

We can divide the Main procedure in Algorithm 1 into two blocks. The first, up
to Line 10, has a cost upper bounded by

$mE\left(|D_{\max}||w|+2\frac{|D_{\max}|}{B}|w|\right)+2|w|m.$ (20)

In parallel every client downloads the global model, executes ClientUpdate,
and sends the updated parameters back to the server. The download and upload
operations have a time complexity proportional to $|w|$. Considering that the
same procedure is repeated by $m$ clients, the upper bound in (20) easily
follows. The second block starts from Line 10, where the server aggregates the
local updates and computes the new global model. The number of arithmetical
operations performed is:

$2|w|m.$ (21)

Combining (20) and (21), and considering the number of total rounds $R$
required to reach convergence, the total cost of CFL is given by:

$\begin{split}&R\left[mE\left(|D_{\max}||w|+2\frac{|D_{k}|}{B}|w|\right)+4m|w|\right]=\\\
&RmE|D_{\max}||w|+2RmE\frac{|D_{k}|}{B}|w|+4Rm|w|.\end{split}$ (22)

The first addend in (22) is the dominant term for the asymptotic time
analysis, so this completes the proof to obtain (3).

When it comes to the communications overhead of CFL, the result easily follows
considering that, for each round, each clients downloads and uploads the model
parameters.

###  A-B Proof of Theorem II

In each algorithm’s round, every client in $\mathcal{S}^{t}$ has to download
the latest block from the closest edge server (miner) to obtain the current
global model. These operations, as described before, have a cost of $|w|m$ and
$2|w|m$, respectively. Then, after running the ClientUpdate procedure in
Algorithm 2, clients submit the new model weights with a cost of $|w|$. These
steps are done by each node in $\mathcal{S}^{t}$ (in total, $m$ nodes), so the
total cost is:

$\begin{split}m\Bigg{(}&2|w|m+|w|m+E|D_{\max}||w|+\\\
&2E\frac{|D_{\max}|}{B}|w|+|w|\Bigg{)}.\end{split}$ (23)

When all the local updates have been computed, it is necessary to create a
block, reach consensus throughout the mining operation, and propagate the
block across all the blockchain nodes. The cost of these operations is given
by:

$2^{l}+m|w|N_{B}.$ (24)

If we combine together (23) and (24), we obtain the total time complexity of
the algorithm

$\begin{split}R\bigg{(}&3|w|m^{2}+E|D_{\max}||w|m+\\\
&2E\frac{|D_{\max}|}{B}|w|m+|w|m+2^{l}+m|w|N_{B}\bigg{)}.\end{split}$ (25)

The dominant addends are reported in (5).

The communication overhead of BFL can be easily derived from the algorithm
description.

In this analysis, we considered the less efficient implementation, whereby
each client has to perform the computation of the new global model given the
updates in the latest block. To improve this, we can move the instruction in
Line 8 outside the _for_ loop and execute it before the MineBlock procedure.
In this way, the new block has size $|w|$, since it contains only the
parameters of the new model. Following the same analysis described before, the
computational complexity is:

$O(R(mE|D_{\max}||w|+2^{l}+N_{B}|w|)).$ (26)

And the communication overhead is:

$R(2|w|m+N_{B}|w|).$ (27)

###  A-C Proof of Theorem III

Let $k_{i}$ be a client in the sequence $[k_{1},...,k_{m}]$. Following the
steps of Algorithm 3, three main operations are performed: 1) Merge, 2)
ClientUpdate and 3) send of the model parameters to the next client of the
sequence. The first one is the average of two model parameters, so its cost is
$2|w|$. The cost of the second operation has already been computed in (19) and
the cost of parameter sharing is $|w|$. By summing up these contributions we
obtain:

$m\left[E\left(|D_{\max}||w|+2\frac{|D_{\max}|}{B}|w|\right)+3|w|\right].$
(28)

This process is repeated for $R$ rounds, so the time complexity is:

$Rm\left[E\left(|D_{\max}||w|+2\frac{|D_{\max}|}{B}|w|\right)+3|w|\right],$
(29)

where the first addend is the dominant one.

Given that each client shares its local model only with the following node in
the sequence, the communication overhead is given by (8).

##  Appendix B Edge connection model

To compute the total duration for transmitting model weights, we assume IEEE
802.11ax channel access procedures [39], which also include the overheads to
carry out the distributed coordination function (DCF) operation. In
particular, the duration of a packet transmission is defined as:

$\begin{split}T_{\rm{Tx}}=&Rm(T_{\rm{RTS}}+T_{\rm{SIFS}}+T_{\rm{CTS}}+T_{\rm{DATA}}+\\\
&T_{\rm{SIFS}}+T_{\rm{ACK}}+T_{\rm{DIFS}}+T_{\rm{e}}),\end{split}$ (30)

where $T_{\rm{RTS}}$ is the duration of the ready-to-send (RTS) control frame,
$T_{\rm{SIFS}}$ is the short interframe space (SIFS) duration, $T_{\rm{CTS}}$
is the duration of the clear-to-send (CTS) control frame, $T_{\rm{DATA}}$ is
the duration of the data payload, $T_{\rm{ACK}}$ is the duration of the
acknowledgement (ACK) frame, $T_{\rm{e}}$ is the duration of an empty slot,
$R$ is the number of FL rounds, and $m$ the number of participating clients.

To compute the duration of each type of IEEE 802.11ax control frame, i.e.,
RTS, CTS, and ACK, we compute them as:

$T_{\rm{RTS/CTS/ACK}}=T_{\rm{PHY}}+\bigg{\lceil}\frac{L_{\rm{SF}}+L_{\rm{RTS/CTS/ACK}}}{L_{\rm{s}}}\bigg{\rceil}\sigma_{\rm{leg}},$
(31)

where $T_{\rm{PHY}}$ is the duration of the PHY preamble, $L_{\rm{SF}}$ is the
length of the service field (SF), $L_{\rm{RTS/CTS/ACK}}$ is the length of the
control frame, $L_{\rm{s}}$ is the length of an orthogonal frequency division
multiplexing (OFDM) symbol, and $\sigma_{\rm{leg}}$ is the duration of a
legacy OFDM symbol.

As for the duration of the data payload, it is computed as:

$T_{\rm{DATA}}=T_{\rm{HE-
SU}}+\bigg{\lceil}\frac{L_{\rm{SF}}+L_{\rm{MAC}}+L_{\rm{DATA}}}{L_{\rm{s}}}\bigg{\rceil}\sigma,$
(32)

where $T_{\rm{HE-SU}}$ is the duration of the high-efficiency (HE) single-user
field, $L_{\rm{MAC}}$ is the length of the MAC header, $L_{\rm{DATA}}$ is the
length of a single data packet (in our case, it matches with the model size,
$S_{w}$), and $\sigma$ is the duration of an OFDM symbol. The number of bits
per OFDM symbol will vary, so as the effective data rate, based on the
employed modulation and coding scheme (MCS), which depends on the transmission
power used.

## References

  * [1] “Ericsson mobility report november 2021,” 2021.
  * [2] J. Wang, J. Liu, and N. Kato, “Networking and communications in autonomous driving: A survey,” _IEEE Communications Surveys Tutorials_ , vol. 21, no. 2, pp. 1243–1274, 2019.
  * [3] “AI and Compute,” May 2018. [Online]. Available: https://openai.com/blog/ai-and-compute/
  * [4] E. Strubell, A. Ganesh, and A. McCallum, “Energy and policy considerations for deep learning in NLP,” in _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2019, pp. 3645–3650.
  * [5] T. Zhang and S. Mao, “An introduction to the federated learning standard,” _GetMobile: Mobile Computing and Communications_ , vol. 25, no. 3, pp. 18–22, 2022.
  * [6] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, and B. He, “A survey on federated learning systems: vision, hype and reality for data privacy and protection,” _IEEE Transactions on Knowledge and Data Engineering_ , 2021\.
  * [7] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Distributed learning in wireless networks: Recent progress and future challenges,” _IEEE Journal on Selected Areas in Communications_ , 2021\.
  * [8] S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, “Edge intelligence: The confluence of edge computing and artificial intelligence,” _IEEE Internet of Things Journal_ , vol. 7, no. 8, pp. 7457–7469, 2020.
  * [9] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” _Proceedings of the IEEE_ , vol. 107, no. 8, pp. 1738–1762, 2019.
  * [10] E. Ahvar, A.-C. Orgerie, and A. Lebre, “Estimating energy consumption of cloud, fog and edge computing infrastructures,” _IEEE Transactions on Sustainable Computing_ , pp. 1–1, 2019.
  * [11] J. Konečnỳ, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” _arXiv preprint arXiv:1610.05492_ , 2016.
  * [12] C. Xie, S. Koyejo, and I. Gupta, “Asynchronous federated optimization,” _arXiv preprint arXiv:1903.03934_ , 2019.
  * [13] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings _et al._ , “Advances and open problems in federated learning,” _Foundations and Trends® in Machine Learning_ , vol. 14, no. 1–2, pp. 1–210, 2021\.
  * [14] V. Mothukuri, R. M. Parizi, S. Pouriyeh, Y. Huang, A. Dehghantanha, and G. Srivastava, “A survey on security and privacy of federated learning,” _Future Generation Computer Systems_ , vol. 115, pp. 619–640, 2021.
  * [15] L. Barbieri, S. Savazzi, M. Brambilla, and M. Nicoli, “Decentralized federated learning for extended sensing in 6G connected vehicles,” _Vehicular Communications_ , vol. 33, p. 100396, 2022.
  * [16] A. Lalitha, S. Shekhar, T. Javidi, and F. Koushanfar, “Fully decentralized federated learning,” in _Third workshop on Bayesian Deep Learning (NeurIPS)_ , 2018.
  * [17] R. Ormándi, I. Hegedüs, and M. Jelasity, “Gossip learning with linear models on fully distributed data,” _Concurrency and Computation: Practice and Experience_ , vol. 25, no. 4, pp. 556–571, 2013.
  * [18] L. Giaretta and Š. Girdzijauskas, “Gossip learning: Off the beaten path,” in _2019 IEEE International Conference on Big Data (Big Data)_. IEEE, 2019, pp. 1117–1124.
  * [19] M. Miozzo, Z. Ali, L. Giupponi, and P. Dini, “Distributed and multi-task learning at the edge for energy efficient radio access networks,” _IEEE Access_ , vol. 9, pp. 12 491–12 505, 2021.
  * [20] F. Wilhelmi, L. Giupponi, and P. Dini, “Blockchain-enabled server-less federated learning,” _arXiv preprint arXiv:2112.07938_ , 2021.
  * [21] X. Qiu, T. Parcollet, D. Beutel, T. Topal, A. Mathur, and N. Lane, “Can federated learning save the planet?” in _NeurIPS-Tackling Climate Change with Machine Learning_ , 2020.
  * [22] S. Savazzi, V. Rampa, S. Kianoush, and M. Bennis, “An energy and carbon footprint analysis of distributed and federated learning,” _arXiv preprint arXiv:2206.10380_ , 2022.
  * [23] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org. [Online]. Available: https://www.tensorflow.org/
  * [24] J. Verbraeken, M. Wolting, J. Katzy, J. Kloppenburg, T. Verbelen, and J. S. Rellermeyer, “A survey on distributed machine learning,” _ACM Computing Surveys (CSUR)_ , vol. 53, no. 2, pp. 1–33, 2020.
  * [25] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in _Artificial intelligence and statistics_. PMLR, 2017, pp. 1273–1282.
  * [26] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” _IEEE Signal Processing Magazine_ , vol. 37, no. 3, pp. 50–60, 2020.
  * [27] H. Kim, J. Park, M. Bennis, and S.-L. Kim, “Blockchained on-device federated learning,” _IEEE Communications Letters_ , vol. 24, no. 6, pp. 1279–1283, 2019.
  * [28] U. Majeed and C. S. Hong, “Flchain: Federated learning via mec-enabled blockchain network,” in _2019 20th Asia-Pacific Network Operations and Management Symposium (APNOMS)_. IEEE, 2019, pp. 1–4.
  * [29] X. Bao, C. Su, Y. Xiong, W. Huang, and Y. Hu, “Flchain: A blockchain for auditable federated learning with trust and incentive,” in _2019 5th International Conference on Big Data Computing and Communications (BIGCOM)_ , 2019, pp. 151–159.
  * [30] I. Hegedüs, G. Danner, and M. Jelasity, “Decentralized learning works: An empirical comparison of gossip learning and federated learning,” _Journal of Parallel and Distributed Computing_ , vol. 148, pp. 109–124, 2021\.
  * [31] A. Nilsson, S. Smith, G. Ulm, E. Gustavsson, and M. Jirstrand, “A performance evaluation of federated learning algorithms,” in _Proceedings of the second workshop on distributed infrastructures for deep learning_ , 2018, pp. 1–8.
  * [32] A. Lacoste, A. Luccioni, V. Schmidt, and T. Dandres, “Quantifying the carbon emissions of machine learning,” _arXiv preprint arXiv:1910.09700_ , 2019\.
  * [33] L. Lannelongue, J. Grealey, and M. Inouye, “Green algorithms: Quantifying the carbon footprint of computation,” _Advanced Science_ , vol. 8, no. 12, p. 2100707, 2021.
  * [34] R. Schwartz, J. Dodge, N. A. Smith, and O. Etzioni, “Green ai,” _Communications of the ACM_ , vol. 63, no. 12, pp. 54–63, 2020.
  * [35] S. Nakamoto, “Bitcoin: A peer-to-peer electronic cash system,” _Decentralized business review_ , p. 21260, 2008.
  * [36] D. C. Nguyen, M. Ding, Q.-V. Pham, P. N. Pathirana, L. B. Le, A. Seneviratne, J. Li, D. Niyato, and H. V. Poor, “Federated learning meets blockchain in edge computing: Opportunities and challenges,” _IEEE Internet of Things Journal_ , 2021.
  * [37] F. Wilhelmi and L. Giupponi, “Discrete-time analysis of wireless blockchain networks,” in _2021 IEEE 32nd Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC)_. IEEE, 2021, pp. 1011–1017.
  * [38] Z. Zheng, S. Xie, H. Dai, X. Chen, and H. Wang, “An overview of blockchain technology: Architecture, consensus, and future trends,” in _2017 IEEE international congress on big data (BigData congress)_. IEEE, 2017, pp. 557–564.
  * [39] B. Bellalta, “IEEE 802.11 ax: High-efficiency wlans,” _IEEE Wireless Communications_ , vol. 23, no. 1, pp. 38–46, 2016.
  * [40] N. Lasla, L. Al-Sahan, M. Abdallah, and M. Younis, “Green-pow: An energy-efficient blockchain proof-of-work consensus algorithm,” _Computer Networks_ , vol. 214, p. 109118, 2022.
  * [41] “Emnist | TensorFlow Federated,” accessed on December 2022. [Online]. Available: https://www.tensorflow.org/federated/api_docs/python/tff/simulation/datasets/emnist
  * [42] TensorFlow, “Tensorflow federated,” accessed on: December 2022. [Online]. Available: https://www.tensorflow.org/federated
  * [43] “Keras: the Python deep learning API.” [Online]. Available: https://keras.io/
  * [44] M. Alharby and A. van Moorsel, “Blocksim: An extensible simulation tool for blockchain systems,” _Frontiers in Blockchain_ , vol. 3, 2020. [Online]. Available: https://www.frontiersin.org/article/10.3389/fbloc.2020.00028
  * [45] TensorFlow, “Federated learning for image classification,” accessed on: December 2022. [Online]. Available: https://github.com/tensorflow/federated/blob/v0.17.0/docs/tutorials/federated_learning_for_image_classification.ipynb
  * [46] L. F. W. Anthony, B. Kanding, and R. Selvan, “Carbontracker: Tracking and predicting the carbon footprint of training deep learning models,” _arXiv preprint arXiv:2007.03051_ , 2020.
  * [47] Y. Huang, C. Bert, S. Fischer, M. Schmidt, A. Dörfler, A. Maier, R. Fietkau, and F. Putz, “Continual learning for peer-to-peer federated learning: A study on automated brain metastasis identification,” _arXiv preprint arXiv:2204.13591_ , 2022.
  * [48] Y. Shahsavari, K. Zhang, and C. Talhi, “A theoretical model for fork analysis in the bitcoin network,” in _2019 IEEE International Conference on Blockchain (Blockchain)_. IEEE, 2019, pp. 237–244.
  * [49] F. Wilhelmi, S. Barrachina-Muñoz, and P. Dini, “End-to-end latency analysis and optimal block size of proof-of-work blockchain applications,” _IEEE Communications Letters_ , vol. 26, no. 10, pp. 2332–2335, 2022.
  * [50] S. R. Pokhrel and J. Choi, “Federated learning with blockchain for autonomous vehicles: Analysis and design challenges,” _IEEE Transactions on Communications_ , vol. 68, no. 8, pp. 4734–4746, 2020.

|  Elia Guerra received his master’s degree in Computer Engineering at the
University of Padova (Italy) in 2021. Prior to this, he got his bachelor’s
degree in Information Engineering in 2019. During his studies, he developed a
passion for Machine Learning and Algorithms. He is a Ph.D. student at the
Technical University of Catalonia (UPC) and he is currently working at CTTC
for the GREENEDGE (MSCA ETN) project. His main research lines are
distributed/decentralized and sustainable machine learning algorithms.
---|---
|  Francesc Wilhelmi holds a Ph.D. in Information and Communication
Technologies (2020), from Universitat Pompeu Fabra (UPF). Previously, he
obtained a B.Sc. degree in Telematics Engineering (2015) and an M.Sc. in
Intelligent and Interactive Systems (2016), also from the UPF. He is currently
working as a researcher at Nokia Bell Labs.
---|---
|  Marco Miozzo received his M.Sc. degree in Telecommunication Engineering
from the University of Ferrara (Italy) in 2005 and the Ph.D. from the
Technical University of Catalonia (UPC) in 2018. In June 2008 he joined the
Centre Tecnologic de Telecomunicacions de Catalunya (CTTC). In CTTC he has
been involved in several EU founded projects. He participated in several R&D
projects, among them SCAVENGE, 5G-Crosshaul, Flex5Gware and SANSA, working on
environmental sustainable mobile networks with energy harvesting capabilities
through learning techniques. Currently he is collaborating with the EU founded
H2020 GREENEDGE (MSCA ETN) and SONATA (CHIST-ERA). His main research interests
are: sustainable mobile networks, green wireless networking, energy
harvesting, multi-agent systems, machine learning, green AI, explainable AI.
---|---
|  Paolo Dini received M.Sc. and Ph.D. from the Universit‘a di Roma La
Sapienza, in 2001 and 2005, respectively. He is currently a Senior Researcher
with the Centre Tecnologic de Telecomunicacions de Catalunya (CTTC). His
current research interests include sustainable networking and computing,
distributed optimization and optimal control, machine learning, multi-agent
systems and data analytics. His research activity is documented in almost 90
peer-reviewed scientific journals and international conference papers. He
received two awards from the Cisco Silicon Valley Foundation for his research
on heterogeneous mobile networks, in 2008 and 2011, respectively. He has been
involved in more than 25 research projects. He is currently the Coordinator of
CHIST-ERA SONATA project on sustainable computing and communication at the
edge and the Scientific Coordinator of the EU H2020 MSCA Greenedge European
Training Network on edge intelligence and sustainable computing. He serves as
a TPC in many international conferences and workshops and as a reviewer for
several scientific journals of the IEEE, Elsevier, ACM, Springer, Wiley.
---|---