# FedBot: Enhancing Privacy in Chatbots with Federated Learning

Addi Ait-Mlouk
School of Informatics, University of Skövde,
Skövde, Sweden
<EMAIL_ADDRESS>&Sadi Alawadi
School of Information Technology
Halmstad University, Halmstad, Sweden
<EMAIL_ADDRESS>&Salman Toor
Department of Information Technology
Division of Scientific Computing
Uppsala University, Sweden
Scaleout Systems, Sweden
<EMAIL_ADDRESS>&Andreas Hellander
Department of Information Technology
Division of Scientific Computing
Uppsala University, Sweden
Scaleout Systems, Sweden
<EMAIL_ADDRESS>Corresponding author.

###### Abstract

Chatbots are mainly data-driven and usually based on utterances that might be
sensitive. However, training deep learning models on shared data can violate
user privacy. Such issues have commonly existed in chatbots since their
inception. In the literature, there have been many approaches to deal with
privacy, such as differential privacy and secure multi-party computation, but
most of them need to have access to users’ data. In this context, Federated
Learning (FL) aims to protect data privacy through distributed learning
methods that keep the data in its location. This paper presents Fedbot, a
proof-of-concept (POC) privacy-preserving chatbot that leverages large-scale
customer support data. The POC combines Deep Bidirectional Transformer models
and federated learning algorithms to protect customer data privacy during
collaborative model training. The results of the proof-of-concept showcase the
potential for privacy-preserving chatbots to transform the customer support
industry by delivering personalized and efficient customer service that meets
data privacy regulations and legal requirements. Furthermore, the system is
specifically designed to improve its performance and accuracy over time by
leveraging its ability to learn from previous interactions.

_Keywords_ Chatbot $\cdot$ Natural Language Processing $\cdot$ Data Privacy
$\cdot$ Federated Learning $\cdot$ Transformer

##  1 Background

A chatbot is a virtual agent able to assist users by providing useful
information to the user’s request. Chatbots have been very popular since their
inception in 1960. After two decades, research and development have seen
impressive progress from Eliza in 1960 to AI chatbots such as Siri in 2010,
Cortona, and google assistant. Early chatbot systems, such as Eliza Weizenbaum
(1966), Parry Colby (1999), and Alice AbuShawar and Atwell (2015), was
designed based on a text conversation. These chatbots have been used in
Healthcare, e-commerce, and marketing. They are not just conversational
systems but also can carry out other tasks such as ordering, booking,
frequently asked questions, and symptom assessment. In the past several years,
large companies have invested in AI and developed several assistants, among
them Apple’s Siri 111https://www.apple.com/ios/siri/, Microsoft Cortana
222https://www.microsoft.com/en-us/cortana/, Google Assistant
333https://assistant.google.com/, Facebook Messenger 444https://developers.
Facebook.com/blog/post/2016/04/12/, Alexa
555https://developer.amazon.com/alexa/ and ChatGPT
666https://chat.openai.com/. In terms of security, fundamental challenges for
chatbots and Question answering systems are trustworthiness and data privacy.
There are concerns regarding the proper handling of data in terms of consent
and regulations; these concerns revolve around 1) where and how data is shared
with third parties, 2) how data is legally collected or stored, 3) how data is
regulated (GDPR, HIPAA, GLBA, CCPA).

Chatbots are mainly data-driven; training a chatbot model requires massive
training data. In most machine learning settings, users or companies must
upload their data to a central server in order to train chatbot models.
However, centralizing data in this way is not always possible. Reasons for
this include the data being large, sensitive, and legally regulated. Federated
learning offers several benefits for chatbot applications. First, it allows
for the training of chatbots on a large dataset without the need to transfer
all the data to a central server which is particularly useful for chatbot
applications that handle sensitive or private information. Second, federated
learning enables the training of chatbots on data from multiple sources, which
can improve the accuracy and robustness of the model, this is especially
important for chatbots that interact with diverse user populations. Third,
federated learning can reduce the computational burden on the central server,
since the training takes place on local devices to ensure that user privacy is
preserved, as the data never leaves the device. In recent work, we developed a
flexible and scalable framework for federated learning, FEDn, and demonstrated
it for a wide range of applications in a geographically distributed setting
Ekmefjord et al. (2021). To extend it to more sophisticated applications,
firstly, we propose in this paper our POC that leverages large customer
support datasets to train collaboratively Deep Bidirectional Transformer
models for customer chatbots. Secondly, we employed incremental online
learning to construct more demanding models that enable accurate local
training. This method enhances model performance during collaborative training
without sacrificing data privacy. It also enables cross-silo and cross-device
participation, allowing models to learn from distributed datasets while
sharing knowledge in a trusted environment.

The remainder of this paper is organized as follows. Section 2 surveys related
work. Section 3 details the proposed approach and architecture of Fedbot, with
an emphasis on its privacy and scalability properties. In Section 4, we
demonstrate the potential of the framework in an evaluation based on customer
support datasets. Finally, Section 4 concludes the work and outlines future
work.

##  2 Related work

Chatbots have become increasingly popular in recent years, as businesses look
for ways to automate customer service and improve customer engagement.
Generally, chatbots can be categorized into different categories, among them
educational, healthcare, and business. They are based on knowledge from those
domains to provide relevant support for the user. Educational chatbots are
being used to support students, teachers, and administrators, providing
personalized learning experiences and improving student outcomes. For example,
educational chatbots can provide information on academic subjects, answer
questions about the curriculum, and provide feedback on student performance.
One example is MedChatbot Kazi et al. (2012) for medical students, which is
based on open-source AIML. Authors in this chatbot deploy the widely available
Unified Medical Language System (UMLS) as the domain knowledge source for
generating responses to queries and converting natural language queries into
relevant SQL queries. These SQL queries are run against the knowledge base,
and results are returned to the user in the natural dialogue. Computer
Simulation in Educational Communication, CSIEC Jia (2009), is a system with
newly developed multiple functions for English instruction that can chat
interactively in English with English learners. It generates responses
according to the user input, the dialogue context, user and personality
knowledge, common sense knowledge, and inference knowledge. Freudbot Heller et
al. (2005) for the psychology domain, is designed to chat in the first person
about his theories, concepts, and biographical events using a resource that
contained dictionary-type definitions of Freudian terms and concepts. Other
chatbots for FAQs have been proposed, among them Qiu et al. (2017).

Healthcare chatbots are being used to improve the patient experience, reduce
wait times, and improve triage. For example, virtual health assistants can
provide medical information, schedule appointments, and even help diagnose
certain conditions. These chatbots are also being used to improve
communication between patients and healthcare providers, making it easier for
patients to access the information and support they need. Among those
chatbots, Mamabot for supporting women and families during pregnancy Vaira et
al. (2018) exploiting AI-based chatbot systems to understand and respond to
the needs of patients and their families. Also, Pharmabot Comendador et al.
(2015) in 2015 is a pediatric generic medicine consultant chatbot designed to
prescribe, suggest, and give information on generic medicines for children. A
Chatbot for Psychiatric Counseling in Mental Healthcare Service Based on
Emotional Dialogue Analysis and Sentence Generation Oh et al. (2017) that
suggests a conversational service for psychiatric counselling that is adapted
methodologies to understand counselling contents based on high-level natural
language understanding (NLU). Dyva Divya et al.  proposed a medical chatbot;
the idea is to create a text-to-text chatbot that engages patients in
conversation about their medical issues and provides basic information and
diagnosis based on their symptoms. To improve the quality of life for young
adults with food allergies, AllergyBot Hsu et al. (2017) was proposed as an
intelligent and humane chatbot that provides restaurants’ allergy
accommodation information based on users’ allergens.

In addition, chatbots from the business domain are also proposed to support
customers and improve companies’ services in e-commerce systems. Customer
service chatbots have been widely adopted by businesses of all sizes to
provide 24/7 support to customers. These chatbots use natural language
processing (NLP) and machine learning (ML) algorithms to understand customer
inquiries and provide relevant responses. They can be integrated with popular
messaging platforms, such as Facebook Messenger and WhatsApp, to make it easy
for customers to interact with the chatbot. Among those chatbots, SuperAgent
Cui et al. (2017) from Microsoft is a customer service chatbot that leverages
large-scale and publicly available e-commerce data. It takes advantage of data
from in-page product descriptions and user-generated content from e-commerce
websites, which is more practical and cost-effective when answering repetitive
questions. Another conversational system is from IBM to automatically
generates responses for users’ requests on social media integrated with state-
of-the-art deep learning techniques and trained by nearly 1M Twitter
conversations between users and agents from over 60 brands Xu et al. (2017).
This system adopts a word embedding method, word2vec neural network language
model Mikolov et al. (2013) to learn distributed representations of words from
the customer.

Many open-source frameworks that assist in easily creating centralized
conversational engines are proposed, among them, Microsoft Bot, Facebook
Messenger, Google Assistant, and Amazon Lex. These frameworks build and
connect intelligent conversation engines to interact with customers naturally
wherever they are by taking advantage of the wide range of users. Besides,
they are highly customizable in real scenarios with third-party data. However,
most of them are limited regarding privacy, trust, and data governance. There
are many situations where data cannot or should not be shared in this way,
such as when data is regulated, sensitive or private (medical text, business,
social media). An obvious example is biomedical use cases where a single
hospital or clinic may not have access to a sufficient amount of data. Thus
the performance of predictive models stands in direct conflict with patient
privacy.

In this context, we here explore the use of federated learning as a method for
distributed and collaborative machine learning to train large models for
chatbots and conversational agents. Organizations maintain and govern their
data locally and participate in learning a new global, federated model by
sending only their model updates (model weights) to a server for aggregation
into the global model. Hence, all participants (clients) can benefit from a
newly trained model without exposing their data publicly. Our contributions in
this paper can be summarized as follows: (1) We propose federated learning
models for a customer support chatbot using a Deep Bidirectional Transformer
architecture, (2) we design and develop Fedbot, a trustworthy federated
learning system for collaborative training (4) we improve the local training
with an incremental learning scheme and private online labelling, and (5) our
analyses of the models respect data privacy regulations and performs well
compared to a baseline centralized model on customer support data after a
couple of rounds.

##  3 Proposed approach

The overall architecture of our proposed Fedbot is shown in Figure 1; it is
based on a modular approach and takes advantage of scalable federated
learning, data privacy, and distributed ML. Fedbot is trained on customer
support data on Twitter, (e.g., over 3 million tweets and replies from the
biggest brands) and can respond to user queries related to customer support;
it can also retrieve information from multiple sources (information
retrieval). Users can interact with the Fedbot using a chit-chat system or
voice-based messages. The main modules are the private data pipeline,
federated learning settings, a dialogue manager, incremental learning, and
information retrieval (see Figure 1). The private data pipeline module allows
parties (clients) to process and prepare their data locally to be used by
federated learning methods. The federated learning module enables multiple
private clients to form an alliance to collaboratively train transformer
models and send parameters to the server for global model generation
(aggregation of local models). The information retrieval module allows users
to enrich the conversation with the chatbot by using open-linked data and
knowledge graphs. The system allows parties to add new data continuously and
train the model through a defined number of rounds to improve the performance
using incremental learning techniques. Finally, participating clients can use
the global model for customer support. The system is implemented using the
open source federated learning framework FEDn Ekmefjord et al. (2021) and a
web interface using Flask (https://flask.palletsprojects.com/ accessed on: 04
Avril 2023). FEDn provides a highly scalable federated learning run-time, and
Flask is used to develop interactive and user-friendly dialogue interfaces for
the different processes in the workflow. The list of available datasets
related to customer support used for the demo is placed in the local data
sources. Moreover, the developed system is scalable, flexible, and can be
expanded with new clients/data sets on-demand (without re-training the
federated model). The conversations and user feedback are stored in a database
anonymously for continuous learning and future improvement. More details about
the working mechanism of the proposed Fedbot will be explained in the
following subsections.

Figure 1: The Fedbot architecture is organized in three main logical layers,
the first one deals with privacy-preserving training using federated learning,
the second one is a dialogue manager, and the third is for incremental
learning and private data labelling.

###  3.1 Private Data Pipeline Module

Deep learning has become the main component of many popular conversational AI
and chatbot tasks, such as intent classification, speech recognition, natural
language understanding, and question-answering. With massive generated data,
these tasks require computational and storage resources. To support this
increased demand, cloud providers such as Google cloud cloud (2021a),
Microsoft Azure cloud (2021b), and Amazon cloud (2021c) provide machine
learning and deep learning pipelines as a service to train and serve models.
Customers must provide/upload training datasets to the cloud provider to use
these services. However, training datasets can contain sensitive and private
data (e.g., personal medical data, customer data, and financial data). Hence,
data privacy becomes a security problem in machine learning with its unique
challenges. We propose a customer support chatbot based on a federated
learning methodology to preserve data privacy and provide secure collaborative
training. Fedbot builds on federated learning in which participating clients
train local model updates and then share the gradient (model parameters) with
the central server. The server implements an aggregation strategy where local
models are combined into an updated global model. This whole process is
iterated until convergence. This approach ensures input data privacy, enables
collaborative training, and reduces the need for very large central computing
resources by distributing the workload across clients. Further, the proposed
system is capable of improving local learning by using incremental learning
and a local data labelling pipeline.

###  3.2 Federated Machine Learning Module

Federated learning is an emerging technology for collaborative and
decentralized machine learning Kairouz et al. (2019) in which multiple parties
(client) jointly train machine learning models using private data. These
parties could be mobile and IoT devices (cross-device FL) or larger entities
such as different organizations, subsidiaries or geographically separated data
silos (cross-silo FL). In FL, private data remains local at each party at all
times, and only the parameter updates are communicated with a server. In our
system, we use FL to develop a customer support chatbot based on transformer
architecture. Fedbot trains a global model using Federated Averaging McMahan
et al. (2017) (Algorithm 1 ) on large amounts of data from multiple
geographically distributed parties. Each party trains a local transformer
model on its data (Algorithm 2) and share parameters $W_{t}$ with the central
server for aggregation. In the aggregation part, the aggregator (running in
the _combiner_ in FEDn Ekmefjord et al. (2021)) combines shared parameters and
generates a single global model $M(W_{t})$ for each round using incremental
averaging. In this way, Fedbot can learn new knowledge from different clients
without exposing or sharing their private data.

Input: $W_{t}$

Output: $M(W_{t})$

1 Server executes:

2 initialized $W_{0}$

3 Function  _IncrementalFedAVG(_$k,W_{t-1},W_{t}$_)_:

4

5 foreach  _$t\leftarrow 1$ to $r$_ do

6

7 $S_{t}\leftarrow$ (sample a random set of clients)

8 foreach  _client $k\in S_{t}$ in parallel_ do

9 $W_{t+1}^{k}\leftarrow ClientUpdate(k,W_{t},N_{l})$

10 $W_{t+1}\leftarrow\sum_{k=1}^{k}\frac{n_{k}}{n}W_{t+1}^{k}$

11 end foreach

12 $W_{t}\leftarrow(W_{t-1}+(W_{t}-W_{t-1})/t)$

13 end foreach

14 return $M(W_{t})$

15

Algorithm 1 Incremental FedAVG algorithm. k: Number of clients, r: Number of
rounds, $W_{i}$: Local model weights and M: Global model weights.

Output: $W_{t}$

1

2 // Run on client k

3 Function  _ClientUpdate(_$k,W_{t}$_)_:

4

5 $\beta$ $\leftarrow$ (split $D^{k}$ into mini batches)

6 for  _$local~{}epoch~{}e_{i}\in 1,\dots e$_ do

7 for  _batch b $\in$ $\beta$_ do

8 $W_{t}\leftarrow W_{t}-\eta\nabla l(W_{t},b)$

9

10 end for

11

12 end for

13 return _$W_{t}$_

14

15

Algorithm 2 Local client update, k: Number of clients, $D^{k}$: Client k local
dataset, e: Number of local epochs, and $\eta$ is the learning rate.

###  3.3 Transformer architecture

Encoder-Decoder: In Fedbot we used the Transformer architecture Vaswani et al.
(2017) to handle client queries in customer support. A transformer is a model
architecture eschewing recurrence and instead relying on an attention
mechanism to draw global dependencies between input and output; the
transformer follows the encoder and decoder architecture using stacked self-
attention. The encoder maps an input sequence of symbol representations
$(x_{1},...,x_{n})$ to a sequence of continuous representations
$z=(z_{1},...,z_{n})$. The Decoder then generates an output sequence
$(y_{1},...,y_{m})$ of symbols one element at a time. Both the encoder (left-
side) and decoder (right-side) are shown in Figure 2. Both Encoder and Decoder
are composed of layers and sub-layers that can be stacked on top of each other
multiple times, which $Nx$ describes in Figure 2. The first is a multi-head
self-attention mechanism, and the second is a simple, position-wise, fully
connected feed-forward network. The inputs and outputs are first embedded into
an n-dimensional space.

Figure 2:  The Transformer model architecture Vaswani et al. (2017) \-
Adapting the transformer model to a Fedbot architecture for natural language
processing, featuring an encoder-decoder architecture that generates responses
to user inputs using self-attention and feed-forward neural networks

Attention layer: The attention layer is a key component of the transformer
model used in natural language processing tasks Shen et al. (2018); Parikh et
al. (2016). The attention mechanism enables the model to selectively focus on
the relevant parts of the input sequence, rather than treating all inputs
equally. The attention layer consists of three sets of parameters - query (Q),
key (K), and value (V) - that are learned during the training process. In this
paper, by applying the self-attention mechanism, we aim at capturing the long
dependencies in the input sentence following these steps:

  * •

Obtain each word’s weight by calculating the similitude between the query
matrix and every key matrix $(Q,K,V)$. The dot product, concatenation, and
perceptron are typically applied as similarity functions.

  * •

Normalize the similitude score of the preceding step and compute the weights
with the SoftMax function.

  * •

Finally, the weights and the $V$-values matrix are weighted to get the final
attention. Then, the dot product function is used to compute the similarity,
and the attention is defined as follows:

$\displaystyle Attention(Q,K,V)=softmax(\frac{QK^{T}}{\sqrt{d_{k}}})V$ (1)

where $QK^{T}$ represents the dot product between the query vector $Q$ and the
transpose of the key matrix $K$, and ${\sqrt{d_{k}}}$ represents the square
root of the dimensionality of the key vectors.

Multi-head Attention Vaswani et al. (2017) is a module for attention
mechanisms that run through an attention mechanism several times in parallel.
The independent attention outputs are then concatenated and linearly
transformed into the expected dimension. The computation of Multi-head
Attention proceeds as follows:

$\displaystyle Head_{i}=Attention(QW_{i}^{Q},KW_{i}^{K},VW_{i}^{V})$ (2)
$\displaystyle MultiHead(H^{\prime})=(Head_{i}\oplus,...,\oplus Head_{h})$

Where the projections are parameter matrices $W_{i}^{Q}\in R^{d_{model}d_{k}}$
and $W_{i}^{k}\in R^{d_{model}d_{k}}$ and $W_{i}^{V}\in R^{d_{model}d_{k}}$
and $W_{0}=R^{(hd_{v}d_{model})}$.

####  3.3.1 Datasets

To train the transformer model, we processed customer support on the Twitter
dataset 777https://www.kaggle.com/thoughtvector/customer-support-on-
twitter/data which is a large, modern corpus of tweets and replies to aid
innovation in natural language understanding and conversational AI models. The
dataset is a CSV file, where each row is a tweet; every conversation included
has at least one request from a consumer and at least one response from a
company support team. The dataset consists of conversation support for many
companies, among them Apple, Amazon, Uber, Delta, Spotify, Tesco, AmericanAir
and Spotify (see Figure 3). In this paper, we used these datasets as private
data and processed them to be used to train a transformer model
collaboratively in federated learning settings.

Figure 3: The Customer Support on the Twitter dataset is a large, modern
corpus of tweets and replies to aid innovation in natural language
understanding. In our Fedbot, each company dataset is considered to be a
client in a cross-silo federation that trains a transformer model locally and
sends model parameters to a central server for aggregation.

###  3.4 Information Retrieval (IR)

With the rapid progress of the semantic web, a huge amount of structured data
has become available on the web (web of data). Making these resources
available and useful for end-users is one of the main objectives of linked
data Berners-Lee (2009) and open-domain chatbots. Hence, using IR techniques
to access knowledge and enrich user conversations can be very useful to
strengthen the Fedbot performances through SPARQL queries on the top of the
known knowledge base (DBpedia, Wikidata, etc.). Fedbot uses Named Entity
Recognition (NER) and user intent to generate SPARQL queries regarding factoid
and recurrent questions to enrich and improve the chatbot’s performance.

###  3.5 Incremental learning and Feedback

In incremental learning, new input data is continuously used to train the
model further. A goal is to attempt to improve a model’s performance while
adding as few samples as possible. In Fedbot, adding data locally by clients
is an important task to improve the local model performance, which is then
propagated into the global model after new global training rounds. We have
developed an intuitive process for each local client to contribute to the
addition of new samples on top of their baseline local data (Figure 4. The
first step is to add a new data point that will remain on the local site,
allowing users to add private training data. The incremental learning module
will process and transform the private data locally and generate training
points usable for local model training. This process enables collaborative
data generation between organizations while activating data protection and
avoiding necessary sharing within the alliance. Furthermore, FedBot utilizes a
database layer to store feedback and queries from end users. These inputs are
collectively analyzed to enhance and optimize future usage. Additionally,
numerous modules and libraries can be readily incorporated with FedBot to
broaden its functionalities and effectively handle various tasks.

Figure 4: Clients can continuously add local private training data to extend
the existing model’s knowledge by additional training of the global model. The
generated data will be stored locally on the client side. $M_{t}$ represents
the global model. $X_{t}$ represents the subset of data at time $t$. $G_{t}$
represents the generator at time $t$, and $X_{t}^{P}$ is a subset generated by
it.

##  4 Experiment and results

We conduct the experiment on the customer support dataset, which contains
550.287 question-answer pairs on 11 datasets from different companies (Figure
3). To ensure collaborative training, we randomly select and split data
through 10 clients with 20% for the validation dataset for all clients. We
used an encoder and decoder to build our transformer model, and the
implementation is based on TensorFlow and FEDn. For the training process, we
set the initial learning rate to 0.001, the batch size to 32 and the maximum
number of epochs to 10. Texts are tokenized using Wordpieces Wu et al. (2016)
with a maximum length of 30. More configuration is shown in Table 1. The
hyper-parameters used in this model are shown in Table 2.

Table 1: Federated training configuration Rounds | Nbr clients | Update size | Nbr of parameters
---|---|---|---
30 | 10 | 72 MB | 19.639.619
Table 2:  Order in which different hyper-parameters are explored and the corresponding values considered for each hyper-parameter. Underlined values indicate the default value. Step | Hyper-parameter | Values
---|---|---
1 | embedding dimension | 256, 512, 1024
2 | attention heads | 1, 2, 4, 8, 16
3 | dropout | 0.1, 0.2, 0.3, 0.4, 0.5
4 | number of layers | 1, 2, 3, 4, 5, 6, 7, 8
5 | number of units | 128, 256, 512
6 | enc/dec layer dropout | 0, 0.1, 0.2, 0.3, 0.4
7 | attention dropout | 0, 0.1, 0.2, 0.3
8 | activation dropout | 0, 0.1, 0.2, 0.3, 0.4, 0.5
9 | batch size | 8, 32, 64
10 | learning rate scheduler |  Transformer standard, inverse square root
11 | warm-up steps | 2000, 4000, 5000, 6000, 8000, 10000
12 | learning rate | 0.01, 0.001, 0.0001, 0.00001
13 | epochs | 5, 10, 15, 20, 25
14 | maxlen | 15, 20, 25, 30
15 | activation function | relu

###  4.1 Evaluation

To evaluate the performance of Fedbot, we used accuracy and loss metrics. We
then evaluated performance using the accuracy function provided by Tensorflow
that falls between (0, 100), shown in Equation 3, and the categorical cross-
entropy as a loss function, shown in Equation 4.

$Accuracy=\frac{TP+TN}{TP+TN+FN+FP}*100\%$ (3)

where $TP$, $TN$,$FP$, and $FN$ are the True Positives, True Negatives, False
Positives and False Negatives respectively.

$Loss=-\sum_{i=1}^{N}y_{i}.\log\hat{y}_{i}$ (4)

where $\hat{y}_{i}$ is the model prediction for i-th pattern, $y_{i}$
represent the corresponding real value, and N is the total number of samples.

To validate our POC, we partitioned the dataset into ten equal chunks so that
each client has 20% of the total dataset. We then compare the federated
scenario to centralized model training using the entire dataset. The
convergence and divergence of training and testing accuracy on the customer
support dataset for 10 clients on 20 communication rounds are depicted in
Figure 6 and Figure 6, respectively. In addition Figure, 8 and Figure 8 show
the partial models compared to the global model (Fedbot). Partial models
represent the model updates contributed by individual clients participating in
the collaborative learning process. These updates capture the local insights
and unique data characteristics of each client, which are incorporated into
the global model to enhance its performance. The use of partial models in
federated learning highlights the idea that the model’s performance benefits
everyone involved in the collaborative learning process. Each device
contributes its knowledge to the shared model, which improves the overall
model’s accuracy, and this, in turn, benefits all the participants.

Figure 5: Convergence of training and testing accuracy on customer support
dataset for 10 clients on 20 communication rounds

Figure 6: Divergence of training and testing loss for 10 clients on 20
communication rounds.

Figure 7: Training accuracy for partial models trained on each customer
support dataset. The accuracy varies greatly between clients due to the amount
of available data. As can be seen, the federated Fedbot model improves on all
client models.

Figure 8: Testing accuracy for partial models on each customer support
dataset, resulted in a difference due to the amount of available data. In the
end, a partial model (client) with poor performance will benefit from
knowledge transfer via the global model

For demonstration, we consider a FEDn network consisting of a single, high-
powered aggregation server (known as a combiner) with 16 VCPU and 16GB RAM,
along with connected client instances (each with 16 VCPU and 16GB RAM) located
in SSC (SNIC Science Cloud Toor et al. (2017)). To have an intuitive
observation of the predictions, we give prediction examples using global
models in Figure 9, which shows the Fedbot performance on a given utterance.
Hence, the proposed approach has contributed overall to a better understanding
of text conversation, preserving data privacy, and contributing to low-cost
training as well as collaborative training using large models.

###  4.2 Implementation and demo environment

Designing and developing privacy-aware chatbots is not a trivial task; it
requires various design techniques for data governance and privacy
regulations. Third-party frameworks have been proposed for chatbot
development; it refers to open-source building blocks that help build
conversation engines, including Microsoft Bot, Facebook Messenger, Google
Assistant, and Amazon Lex. However, the increasing use of chatbot technologies
has led to a heightened focus on privacy issues. To tackle this concern, we
developed Febot, a system that employs natural language processing techniques
and federated learning to operate on private data. It includes the following
benefits:

  * •

Privacy Protection: Federated learning ensures that sensitive customer data
remains on the local device, reducing the risk of data breaches and providing
stronger privacy protection for customers.

  * •

Improved User Experience: By training models locally, federated learning
enables chatbots to provide more accurate and relevant responses to users.
This can result in a better customer experience and higher customer
satisfaction.

  * •

Scalability: Federated learning enables organizations to scale their chatbots
quickly and efficiently, as the models can be trained on multiple devices
simultaneously. This reduces the need for large amounts of centralized
computing resources and eliminates the need for complex data transfers.

  * •

Robust Models: Federated learning enables chatbots to leverage data from
multiple devices to train robust models, improving their accuracy and
performance.

  * •

Incremental learning: generate new knowledge and a new global model by
attaching more clients and annotating new data points;

  * •

Standalone: multiple platforms (i.e., guarantee for low disk and memory
footprint). It can be run on a standard laptop having two cores and 2GB of
RAM;

The proposed Fedbot is accessible from different platforms to engage a wide
range of users, and it is also optimized for both desktop and mobile. Figure 9
present some examples.

Figure 9: Fedbot use case including UI and federated incremental learning
where the model is trained incrementally on new data as it becomes available.
This approach allows the model to adapt to changes in the data over time,
improving its accuracy and relevance.

## Conclusion

In this paper, we have proposed Fedbot, a proof of concept customer support
chatbot system, to address the data-sharing issue in machine learning and to
handle client questions in customer support. Validation experiments were
implemented based on twenty training rounds with a transformer neural network.
The system consists of several components: the private data pipeline, dialogue
management, collaborative training, and private incremental learning.
Experiments on the customer support dataset using the transformer architecture
demonstrate that our approach performs well compared to the baseline
centralized model. The proposed Fedbot allows collaborative training
participants to control their sensitive and private data while training a
chatbot. The integration of federated learning within chatbot and conversation
AI provides a sustainable solution by preserving data privacy. We conclude
that the application of FL to NLP tasks such as dialogue understanding can
contribute to solving the problem that arises when using machine learning with
private and sensitive data. Furthermore, the system actively supports end-
users in joining training and improving their performance through incremental
learning on various local clients.
In future work, we aim to extend Fedbot to cover more customer support
datasets in a real geographically distributed manner and test the model with
other aggregation algorithms (e.g., FedOPT, FedProx, etc.) for federated
learning. Additionally, we intend to Fine-tune the pre-trained model LLaMA
Touvron et al. (2023) and investigate its efficacy in federated learning
settings.

## Acknowledgement

This work is funded by Uppsala University in Sweden on scalable federated
learning research and supported by the University of Skövde. The authors also
would like to thank SNIC for providing cloud resources.

## References

  * Weizenbaum [1966] Joseph Weizenbaum.  Eliza a computer program for the study of natural language communication between man and machine.  _Commun. ACM_ , 9(1):36–45, January 1966.  ISSN 0001-0782.  doi:10.1145/365153.365168.  URL http://doi.acm.org/10.1145/365153.365168.
  * Colby [1999] K. M. Colby.  _Human-Computer Conversation in A Cognitive Therapy Program_ , pages 9–19.  Springer US, Boston, MA, 1999.  ISBN 978-1-4757-5687-6.  doi:10.1007/978-1-4757-5687-6_3.  URL https://doi.org/10.1007/978-1-4757-5687-6_3.
  * AbuShawar and Atwell [2015] Bayan AbuShawar and Eric Atwell.  Alice chatbot: Trials and outputs.  _Computación y Sistemas_ , 19, 12 2015.  doi:10.13053/cys-19-4-2326.
  * Ekmefjord et al. [2021] Morgan Ekmefjord, Addi Ait-Mlouk, Sadi Alawadi, Mattias Åkesson, Desislava Stoyanova, Ola Spjuth, Salman Toor, and Andreas Hellander.  Scalable federated machine learning with fedn, 2021.
  * Kazi et al. [2012] Hameedullah Kazi, Bhawani Shankar Chowdhry, and Zeesha Memon.  Medchatbot: An umls based chatbot for medical students.  2012\.
  * Jia [2009] Jiyou Jia.  Csiec: A computer assisted english learning chatbot based on textual knowledge and reasoning.  _Knowledge-Based Systems_ , 22(4):249 – 255, 2009\.  ISSN 0950-7051.  doi:https://doi.org/10.1016/j.knosys.2008.09.001.  URL http://www.sciencedirect.com/science/article/pii/S0950705109000045.  Artificial Intelligence (AI) in Blended Learning.
  * Heller et al. [2005] Bob Heller, Mike D. Proctor, Dean Y.-O.-N. Mah, Lisa Jewell, and Bill Cheung.  Freudbot: An investigation of chatbot technology in distance education.  2005\.
  * Qiu et al. [2017] Minghui Qiu, Feng-Lin Li, Siyu Wang, Xing Gao, Yan Chen, Weipeng Zhao, Haiqing Chen, Jun Huang, and Wei Chu.  AliMe chat: A sequence to sequence and rerank based chatbot engine.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 498–503, Vancouver, Canada, July 2017. Association for Computational Linguistics.  doi:10.18653/v1/P17-2079.  URL https://www.aclweb.org/anthology/P17-2079.
  * Vaira et al. [2018] Lucia Vaira, Mario A. Bochicchio, Matteo Conte, Francesco Margiotta Casaluci, and Antonio Melpignano.  Mamabot: a system based on ml and nlp for supporting women and families during pregnancy.  In _IDEAS_ , 2018.
  * Comendador et al. [2015] Benilda Eleonor Comendador, Bien Francisco, Jefferson Medenilla, Sharleen Nacion, and Timothy Serac.  Pharmabot: A pediatric generic medicine consultant chatbot.  _Journal of Automation and Control Engineering_ , 3:137–140, 01 2015.  doi:10.12720/joace.3.2.137-140.
  * Oh et al. [2017] KyoJoong Oh, Dongkun Lee, ByungSoo Ko, and Ho-Jin Choi.  A chatbot for psychiatric counseling in mental healthcare service based on emotional dialogue analysis and sentence generation.  _2017 18th IEEE International Conference on Mobile Data Management (MDM)_ , pages 371–375, 2017.
  * [12] Divya, Indumathi, Ishwarya, Priyasankari, and Kalpana Devi.  A self-diagnosis medical chatbot using artificial intelligence.  _Journal of Web Development and Web Designing_ , 3.
  * Hsu et al. [2017] Paris (Pei-Ting) Hsu, Jingshu Zhao, Kehan Liao, Tianyi Liu, and Chen Wang.  Allergybot: A chatbot technology intervention for young adults with food allergies dining out.  In _Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems_ , CHI EA ’17, pages 74–79, New York, NY, USA, 2017. ACM.  ISBN 978-1-4503-4656-6.  doi:10.1145/3027063.3049270.  URL http://doi.acm.org/10.1145/3027063.3049270.
  * Cui et al. [2017] Lei Cui, Furu Wei, Shaohan Huang, Chuanqi Tan, Chaoqun Duan, and Ming Zhou.  Superagent: A customer service chatbot for e-commerce websites.  In _Proceedings of ACL 2017, System Demonstrations_ , pages 97–102. Association for Computational Linguistics, July 2017.  URL https://www.microsoft.com/en-us/research/publication/superagent-customer-service-chatbot-e-commerce-websites/.
  * Xu et al. [2017] Anbang Xu, Zhe Liu, Yufan Guo, Vibha Sinha, and Rama Akkiraju.  A new chatbot for customer service on social media.  In _Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems_ , CHI ’17, pages 3506–3510, New York, NY, USA, 2017. ACM.  ISBN 978-1-4503-4655-9.  doi:10.1145/3025453.3025496.  URL http://doi.acm.org/10.1145/3025453.3025496.
  * Mikolov et al. [2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean.  Distributed representations of words and phrases and their compositionality.  In _Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2_ , NIPS’13, pages 3111–3119, USA, 2013\. Curran Associates Inc.  URL http://dl.acm.org/citation.cfm?id=2999792.2999959.
  * cloud [2021a] Google cloud.  Google cloud, 2021a.  URL https://cloud.google.com/.
  * cloud [2021b] Azure cloud.  Azure cloud, 2021b.  URL https://azure.microsoft.com/en-us/.
  * cloud [2021c] AWS cloud.  Aws cloud, 2021c.  URL https://aws.amazon.com/.
  * Kairouz et al. [2019] Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D’Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaïd Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao.  Advances and open problems in federated learning.  _CoRR_ , abs/1912.04977, 2019.  URL http://arxiv.org/abs/1912.04977.
  * McMahan et al. [2017] H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas.  Communication-efficient learning of deep networks from decentralized data, 2017.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.  Attention is all you need, 2017.
  * Shen et al. [2018] Tao Shen, Jing Jiang, Tianyi Zhou, Shirui Pan, Guodong Long, and Chengqi Zhang.  Disan: Directional self-attention network for RnN/CNN-free language understanding.  In _32nd AAAI Conference on Artificial Intelligence, AAAI 2018_ , pages 5446–5455, 2018.  ISBN 9781577358008.
  * Parikh et al. [2016] Ankur P Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit.  A decomposable attention model for natural language inference.  In _EMNLP 2016 - Conference on Empirical Methods in Natural Language Processing, Proceedings_ , pages 2249–2255, 2016.  ISBN 9781945626258.  doi:10.18653/v1/d16-1244.
  * Berners-Lee [2009] Tim Berners-Lee.  https://www.w3.org/designissues/linkeddata.html, 2009.
  * Wu et al. [2016] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean.  Google’s neural machine translation system: Bridging the gap between human and machine translation, 2016.
  * Toor et al. [2017] Salman Toor, Mathias Lindberg, Ingemar Falman, Andreas Vallin, Olof Mohill, Pontus Freyhult, Linus Nilsson, Martin Agback, Lars Viklund, Henric Zazzik, Ola Spjuth, Marco Capuccini, Joakim Möller, Donal Murtagh, and Andreas Hellander.  Snic science cloud (ssc): A national-scale cloud infrastructure for swedish academia.  In _2017 IEEE 13th International Conference on e-Science (e-Science)_ , pages 219–227, 2017.  doi:10.1109/eScience.2017.35.
  * Touvron et al. [2023] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.  Llama: Open and efficient foundation language models, 2023.