# Functional Clustering of Neuronal Signals with FMM Mixture Models

Alejandro Rodríguez-Collado
Department of Statistics and Operations Research, Universidad de Valladolid
and
Cristina Rueda∗
Department of Statistics and Operations Research, Universidad de Valladolid
The authors gratefully acknowledge the financial support received by the
Spanish Ministerio de Ciencia e Innovación [PID2019-106363RB-I00 to AR-C and
CR]. This study was also partially supported by Fundación Eugenio Rodríguez
Pascual (Biomedical Research Grants 2021).Corresponding Author:
<EMAIL_ADDRESS>
###### Abstract

The identification of unlabelled neuronal electric signals is one of the most
challenging open problems in neuroscience, widely known as Spike Sorting.
Motivated to solve this problem, we propose a model-based approach within the
mixture modeling framework for clustering oscillatory functional data called
MixFMM. The core of the approach is the FMM waves, which are non-linear
parametric time functions, flexible enough to describe different oscillatory
patterns and simple enough to be estimated efficiently. In particular,
specific model parameters describe the waveforms’ phase, amplitude, and shape.
A mixture model is defined using FMM waves as basic functions and gaussian
errors, and an EM algorithm is proposed for estimating the parameters. In
addition, the approach includes a method for the number of clusters selection.
Spike Sorting has received considerable attention in the literature, and
different functional clustering approaches have traditionally been considered.
We compare those approaches with the MixFMM in a broad collection of datasets,
including benchmarking simulated and real data. The MixFMM approach achieves
outstanding results in a selection of indexes across datasets, and the
significant improvements attained in specific scenarios motivate interesting
neuronal insights.

Keywords: Functional Data Analysis, Spike Sorting, Oscillatory Signals, Phase-
Amplitude Variation

##  1 Introduction

The analysis of the electric activity of the neurons is regarded as one of the
most practical and effective approaches to studying the nervous system. The
electric signals recorded by electrodes register rapid voltage rises lasting a
few milliseconds called spikes. The voltage returns to the initial baseline
level in each spike, describing a single oscillation. The study of spikes is
crucial in neuroscience, as spikes serve as the informational unit between
neurons, and their firing rate and shape determine the cell’s morphological,
functional, and genetic type. In particular, spikes fired by a particular
neuron recorded under similar conditions, such as the electrode’s distance and
orientation, are assumed to have a specific shape. Spike Sorting (SS) is the
collection of techniques to identify spikes corresponding to different
neurons. The correct identification of spikes is crucial for studying the
connectivity patterns between close-by neurons [6], relating the firing of
certain neurons to the memory process [33], the treatment of epileptic
patients [42], or the development of high-accuracy brain-machine interfaces
[25], among many other questions. However, SS remains one of the most
challenging open problems in neuroscience. The main reasons are the low
signal-to-noise ratio, the waveform variability of a particular neuron’s
spikes, the selection of adequate features to characterize the spike shape, or
the overlapping of spikes fired nearby in time. Algorithms that simultaneously
address all of these issues are an ever-increasing necessity, as manual spike
curation has become unaffordable in light of the increased data availability
in the next generation of recording techniques [33].

The four traditional stages of SS are shown in Figure 1, top and bottom left.
In stage 0, voltage traces are preprocessed, often with a basic bandpass
filtering. Stage 1 entails the automatic spike detection and the signal
segmentation into various sub-signals, each containing a single spike.
Finally, stages 2 and 3 are devoted to the extraction of discriminant features
and clustering, respectively. The focus of this paper is on stages 2 and 3,
which may be quite different from one approach to another; features are mainly
extracted using principal component analysis (PCA) or wavelet coefficients,
while the most widely used clustering algorithms are k-means (KM) and Gaussian
mixture models (GM). However, a complete list of alternative techniques used
in stages 2 and 3 in SS is large. Some interesting references are [30, 14, 33,
8, 43, 32, 7, 44, 47, 45].

Figure 1: Overview of SS: stages 0 and 1 (top), stages 2 and 3 in traditional
SS procedures (bottom left), and single model-based clustering stage in MixFMM
model (bottom right).

We propose in this paper an original parametric model-based functional
clustering approach as an alternative to stages 2 and 3 in traditional SS
(Figure 1, bottom right).

Model-based functional clustering is an active topic in the literature [20,
17, 20, 21, 22, 50]. Different parametric models have been studied, mainly
regression models and additive models defined by basic time functions such as
splines, Fourier waves, or wavelets. The model choice depends on the available
information, in terms of explicative variables, on the nature of the data, and
on the sources of variation characterizing data, among other issues. For
instance, Fourier methods might not be suitable for nonstationary signals.
Specifically, two main sources of variation are considered in functional
analysis, the amplitude variation that measures vertical variability, and the
phase variation measuring horizontal (lateral) displacements [23, 26, 10].
Depending on the problem at hand, either both sources of variation are the
focus or one is just nuisance and must be removed. Besides, some methods do
not work adequately without prior correction for phase differences.

The model-based approach suitable for analyzing oscillatory signals that we
present is the FMM mixture (MixFMM). The core is a mixture of gaussian
distributions where the means are sums of FMM waves or components. FMM waves
are non-linear time functions that describe a single oscillation with four
parameters $(A,\alpha,\omega,\beta)$, which characterize the amplitude ($A$),
phase ($\alpha$), and shape ($\beta,\omega$) of the waveform. In particular,
the amplitude and phase source of variation issues can be easily handled using
this model. Moreover, the mean cluster waveforms or templates can be compared
in a simple way. Signals defined as combination of FMM waves have been
successfully used to analyze oscillatory signals arising in different fields,
such as ECG and gene expression data, as well as neuronal spikes [38, 37, 36,
35]. In all of these works, the method attains highly accurate predictions and
allows the extraction of interpretable features, among other assets. The
number of components is often associated with the prominent peaks and troughs
in the signal, these being five for ECG signals or three for neuronal spikes.
The first component, also called the dominant component, usually identifies
the prominent underlying biological process and, in some contexts, explains
most of the data variability, as in neuronal spikes. Finally, another
remarkable property of FMM waves, very useful in functional analysis, is that
they are truly dynamic that can be formulated as an ordinary differential
equations system.

Noise existing in data is taken into account using FMMm models, defined as
signal plus error models where the signal is a combination of $m$ FMM waves
and the error is assumed gaussian. The maximum likelihood estimator (MLE) of
the parameters are derived using a backfitting algorithm, in which FMM1 models
are repeatedly fitted to the residue until a stop criterion is attained [40].
The novel approach is based on MixFMMm models defined as an FMMm mixture. We
solve the MixFMMm model’s parameter estimation with an expectation-
maximization (EM) algorithm designed ad-hoc that makes use of the FMMm
estimation algorithm. In addition, we propose a likelihood-based method to
select the number of clusters. The MixFMM approach has been validated in 10
benchmarking SS datasets (8 simulated and 2 real), including a wide variety of
spike waveforms, noise levels, and recording situations. The results are
compared with two standard approaches in SS, PCA plus either KM or GM, using
indexes based on the associated groundtruth and internal cluster metrics. The
MixFMM achieves outstanding results, clearly superior to its competitors in
terms of accuracy, interpretability, robustness, cluster cohesion and
separability. Moreover, the proposed method estimates the number of clusters
better than other traditional approaches.

The rest of the paper is organized as follows. Section 2 introduces the
MixFMMm model, the estimation algorithm, and the number of clusters selection.
Section 3 describes the datasets analyzed, and Section 4 presents the main
results. Finally, in Section 5, concluding remarks and future work are
discussed.

##  2 Methods

In the following, let us assume that $\boldsymbol{t}=(t_{1},...,t_{p})$, with
$t_{0}\leq t_{1}<t_{2}<...<t_{p}\leq T$, and
$\boldsymbol{x}_{1},...,\boldsymbol{x}_{n}$,
$\boldsymbol{x}_{i}=(x_{i}(t_{1}),...,x_{i}(t_{p})),\forall i\in\\{1,...,n\\}$
are the time points and observed signals, also called curves. Without loss of
generality, we assume that $t\in[0,2\pi)$. Otherwise, consider
$t^{\prime}=\frac{(t-t_{0})2\pi}{T}$.

###  2.1 FMM background

An FMM wave is defined as follows,

$W(t;A,\alpha,\beta,\omega)=A\cos\left(\beta+2\arctan\left(\omega\tan\left(\frac{t-\alpha}{2}\right)\right)\right).$
(1)

Specifically, $A\in\Re^{+}$ and $\alpha\in[0,2\pi)$ are measures of the wave
amplitude and phase location, respectively, while $\beta\in[0,2\pi)$ and
$\omega\in[0,1]$ describe the shape. Figure 2 shows the waveform patterns for
different parameter configurations.

Figure 2: $W(t;A,\alpha,\beta,\omega)$ for various $(A,\alpha,\beta,\omega)$.
Unless stated otherwise, $A=1,\alpha=0,\beta=\pi,\omega=0.2$.

The multicomponent FMM model of order $m$, FMMm, is a Gaussian model in which
the mean, $\mu(t;\boldsymbol{\theta})$, is defined as a sum of FMM waves as
follows:

$\mu(t;\boldsymbol{\theta})=M+\sum_{J=1}^{m}W(t;A_{J},\alpha_{J},\beta_{J},\omega_{J})$
(2)

where
$\boldsymbol{\theta}=(M,A_{1},\alpha_{1},\beta_{1},\omega_{1},...,A_{m},\alpha_{m},\beta_{m},\omega_{m})$
verifies:

  1. 1.

$M\in\Re$;
$(A_{J},\alpha_{J},\beta_{J},\omega_{J})\in\Theta_{J}=\Re^{+}\times[0,2\pi)\times[0,2\pi)\times[0,1]$;
$J=1,...,m$

  2. 2.

$\alpha_{1}\leq\alpha_{2}\leq...\leq\alpha_{m-1}\leq\alpha_{m}\leq\alpha_{1}$.

  3. 3.

$A_{1}=\max_{1\leq J\leq m}A_{J}.$

The restrictions guarantee the identifiability of the model parameters and
provide biophysiologically interpretable solutions. For instance, the $\alpha$
restrictions correspond to the assumption that the atrial depolarization is
previous to the ventricle depolarization in the analysis of ECG signals, and
that the cell membrane repolarization precedes its hyperpolarization in
neuronal spikes [38, 40].

###  2.2 FMM Mixture Models

The MixFMMm is defined as a mixture model with density defined as follows:

$f(\boldsymbol{x}|\boldsymbol{\Psi})=\sum_{k=1}^{K}\gamma_{k}\;\mathcal{N}(\boldsymbol{x};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}),\sigma^{2}_{k}\boldsymbol{I}_{p}),$
(3)

where $\boldsymbol{I}_{p}$ is the $p\times p$ identity matrix, and
$\boldsymbol{\Psi}=(\gamma_{1},...,\gamma_{K},\boldsymbol{\theta}_{1},...,\boldsymbol{\theta}_{K},\sigma_{1},...,\sigma_{K})$
is the vector of the model’s parameters, where $\gamma_{1},...,\gamma_{K}$
with $\gamma_{k}>0$ and $\sum_{k=1}^{K}\gamma_{k}=1$, are the mixture
proportions. Besides, $K$ is assumed to be known and corresponds to the number
of clusters in our application, while
$\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k})=(\mu(t_{1};\boldsymbol{\theta}_{k}),...,\mu(t_{p};\boldsymbol{\theta}_{k})),1\leq
k\leq K,$ being $\mu(t_{j};\boldsymbol{\theta}_{k}),\forall
j\in\\{1,...,p\\},$ combinations of FMM waves defined as in equation (2).

The log-likelihood of (3) for a sample of size $n$ is given by:

$\log
L(\boldsymbol{\Psi})=\sum_{i=1}^{n}\log\left(\sum_{k=1}^{K}\gamma_{k}\;\mathcal{N}(\boldsymbol{x}_{i};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}),\sigma^{2}_{k}\boldsymbol{I}_{p})\right)$
(4)

####  2.2.1 MLE via EM algorithm

An EM algorithm is designed to find the MLE of the MixFMMm model following the
methodology in [11, 9]. As equation (4) cannot be maximized in a closed form,
the complete-data log-likelihood is maximized given the observed data, which
is defined as follows:

$\log
L_{c}(\boldsymbol{\Psi})=\sum_{i=1}^{n}\sum_{k=1}^{K}Z_{ik}\log(\gamma_{k}\;\mathcal{N}(\boldsymbol{x}_{i};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}),\sigma^{2}_{k}\boldsymbol{I}_{p}))$
(5)

where $Z_{ik}\in\\{1,...,K\\},\forall i\in\\{1,...,n\\}$ and $1\leq k\leq K,$
is an indicator variable such that $Z_{ik}=1$ if $\boldsymbol{x}_{i}$ has been
generated by the $k$th FMM model. As described below, the EM starts with an
initial solution $\boldsymbol{\Psi}^{(0)}$ and alternates iteratively two
steps, maximization and expectation, until convergence is attained. The E-Step
computes the expectation of the complete-data log-likelihood given the
observed data and the current $\boldsymbol{\Psi}$; whereas in the M-Step,
$\boldsymbol{\Psi}$ is updated to maximize the expectation of the complete-
data log-likelihood.

E-Step
The expectation of the complete-data log-likelihood given the observed data
and the current parameter vector $\boldsymbol{\Psi}^{(q)}$ is defined as:

$Q(\boldsymbol{\Psi};\boldsymbol{\Psi}^{(q)})=E[\log
L_{c}(\boldsymbol{\Psi})|\boldsymbol{x}_{1},...,\boldsymbol{x}_{n},\boldsymbol{\Psi}^{(q)}]=\sum_{i=1}^{n}\sum_{k=1}^{K}\tau_{ik}^{(q)}\log(\gamma_{k}^{(q)}\mathcal{N}(\boldsymbol{x}_{i};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}),\sigma^{2}_{k}\boldsymbol{I}_{p}))$
(6)

where $\tau_{ik}^{(q)}$ are the posterior probability of curve $i$ being
generated by the $k$th FMM model in the iteration $q$, defined as:

$\tau_{ik}^{(q)}=P(Z_{ik}=1|\boldsymbol{x}_{i};\boldsymbol{\Psi}^{(q)})=\frac{\gamma_{k}^{(q)}\mathcal{N}(\boldsymbol{x}_{i};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}),\sigma^{2}_{k}\boldsymbol{I}_{p})}{\sum_{h=1}^{K}\gamma_{h}^{(q)}\mathcal{N}(\boldsymbol{x}_{i};\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{h}),\sigma^{2}_{h}\boldsymbol{I}_{p})},\;\forall
i\in\\{1,...,n\\},\;1\leq k\leq K$ (7)

M-Step
The values of the parameter vector are updated maximizing equation (6):

$\boldsymbol{\Psi}^{(q+1)}=\mathrm{argmax}_{\boldsymbol{\Psi}}Q(\boldsymbol{\Psi};\boldsymbol{\Psi}^{(q)})$
(8)

The estimators of the FMM parameters that solve equation (8) are updated as
follows:

$\boldsymbol{\theta}_{k}^{(q+1)}=\mathrm{argmin}_{\boldsymbol{\theta}\in\Theta}||\boldsymbol{\overline{x}}^{(q)}_{k}-\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta})||^{2},\quad
1\leq k\leq K$ (9)

where
$\boldsymbol{\overline{x}}^{(q)}_{k}=\dfrac{\sum_{i=1}^{n}\tau_{ik}^{(q)}\boldsymbol{x}_{i}}{\sum_{i=1}^{n}\tau_{ik}^{(q)}}$
is the mean waveform for cluster $k$ in the iteration $q$. Equation (9) is
solved using the standard FMM backfitting algorithm [40].

Then, the estimators of the standard deviation terms are obtained as follows:

$\sigma_{k}^{(q+1)}=\sqrt{\frac{1}{p\sum_{i=1}^{n}\tau_{ik}^{(q)}}\sum_{i=1}^{n}\tau_{ik}^{(q)}||\boldsymbol{x}_{i}-\boldsymbol{\mu}(\boldsymbol{t};\boldsymbol{\theta}_{k}^{(q+1)})||^{2}},\quad
1\leq k\leq K$ (10)

Finally, the cluster proportions are updated as follows:

$\gamma_{k}^{(q+1)}=\frac{\sum_{i=1}^{n}\tau_{ik}^{(q)}}{n},\quad 1\leq k\leq
K$ (11)

The homoscedasticity restriction $\sigma=\sigma_{1}=...=\sigma_{K}$, often
assumed in SS problems, reduces equation (10) to a single one.

Our proposal for the initial values is: $\gamma_{k}^{(0)}=1/k,1\leq k\leq K$,
and $\boldsymbol{\theta}^{(0)}_{1},...,\boldsymbol{\theta}^{(0)}_{K}$ are the
FMM parameters obtained by fitting FMMm models to the mean waveforms of a
random cluster assignation.

Success in terms of convergence to the MLE is not initially guaranteed,
although the solution converges to a local minimum. We propose to initialize
several times randomly in order to avoid local minimums. We have included
numeric studies in the paper that show how the proposal reaches a good
solution in practice.

The predicted label for each curve can be obtained by assigning the curve to
the cluster with the highest estimated posterior probability. It is denoted as
$I_{i}$, and defined as follows $I_{i}=\mathrm{argmax}_{1\leq k\leq
K}\;\tau_{ik}^{(q)},\forall i\in\\{1,...,n\\}$.

###  2.3 Selection of the number of clusters

We propose to determine the number of clusters using a slope heuristic
approach, similar to those proposed in [4, 5, 16]. Specifically, the log-
likelihood $\log L(\boldsymbol{\widehat{\Psi}})$ values for different numbers
of clusters starting from one are calculated and plotted consecutively. An
exponential trend is expected at the beginning. The minimum value for which
the exponential is reduced to a linear trend is considered the optimal number
of clusters. In case of doubt, we suggest breaking ties using additional
alternative indexes.

###  2.4 Validity metrics

The procedure for evaluating the goodness of a clustering algorithm is
important in order to avoid finding random patterns in data, as well as to
compare two clustering partitions. Several works in the literature deal with
this question, such as [46, 12]. Clustering indexes can be classified as
external or internal. The former uses externally known information, mainly the
class labels; whereas the latter, suitable when the true class labels are
unknown, only uses the internal information of the clustering process to
evaluate its performance. In this paper, we consider the accuracy as external
index calculated by crossing the predicted and real labels in a confusion
matrix. As internal indexes, we consider the Ball-Hall index ($\mathcal{BH}$)
and the Davies-Bouldin index ($\mathcal{DB}$), both often considered in SS
applications, as well as the average silhouette width ($\mathcal{ASW}$); this
being the most simple, interpretable and widely used internal index [45, 1].
The definitions of the four indexes are included below to facilitate the
interpretation of the results.

Let $\boldsymbol{D}$ be the $r\times r$ confusion matrix obtained from
crossing predicted and real classes, in such a way that the diagonal
frequencies sum is maximum. The accuracy is defined as:

$\mathrm{Accuracy}=\frac{\sum_{k=1}^{r}d_{kk}}{n}$ (12)

Now, let $n_{k}$,
$\boldsymbol{c}^{k}=\dfrac{1}{n_{k}}\sum_{i/I_{i}=k}\boldsymbol{x}_{i},$ and
$\delta^{k}=\dfrac{1}{n_{k}}\sum_{i/I_{i}=k}||\boldsymbol{x}_{i}-\boldsymbol{c}^{k}||^{2}$
denote, respectively the number of observations, barycenter and dispersion of
a cluster $k$.

The $\mathcal{BH}$ is the mean of the clusters’ dispersion:

$\mathcal{BH}=\frac{1}{K}\sum_{k=1}^{K}\delta^{k},$ (13)

the $\mathcal{DB}$ is the mean of each cluster’s maximum ratio between the sum
of the cluster dispersions and the distances between their barycenters:

$\mathcal{DB}=\frac{1}{K}\sum_{k=1}^{K}\max_{k\neq
k^{\prime}}\left(\frac{\delta^{k}+\delta^{k^{\prime}}}{||\boldsymbol{c}^{k}-\boldsymbol{c}^{k^{\prime}}||}\right),$
(14)

and the $\mathcal{ASW}$ is defined as:

$\mathcal{ASW}=\frac{1}{K}\sum_{k=1}^{K}\frac{1}{n_{k}}\sum_{i/I_{i}=k}s_{i},$
(15)

where $s_{i}=\frac{b_{i}-a_{i}}{\max(b_{i},a_{i})}$ is the silhouette of
observation $i$, where
$a_{i}=\frac{1}{n_{k}-1}\sum_{i,i^{\prime}/I_{i}=k,I_{i^{\prime}}=k}||\boldsymbol{x}_{i}-\boldsymbol{x}_{i^{\prime}}||,\forall
i\in\\{1,...,n\\},$ and $b_{i}=\min_{k^{\prime}\neq
I_{i}}\left(\sum_{i^{\prime}/I_{i^{\prime}}=k^{\prime}}||\boldsymbol{x}_{i}-\boldsymbol{x}_{i^{\prime}}||\right),\forall
i\in\\{1,...,n\\}$.

The better the cluster partitions, the lower the expected values of
$\mathcal{BH}$ and $\mathcal{DB}$ and the higher the expected values of the
accuracy and $\mathcal{ASW}$.

###  2.5 Programming languages

Data from the Allen Cell Types Database was obtained using Python and the
Allen SDK [3]. The rest of the experimentation was developed with R, including
spike detection, spike segmentation, and functional clustering. The MixFMM
approach has been implemented using the functions from the FMM package [15].
The PCA and KM implementations used are from the package stats [31], whereas
the GM are from mclust [41]. Moreover, the package clusterCrit [12] was used
to calculate the clustering internal indexes, while ggplot2 [48] and plotly
[19] were used to create the visualizations of the manuscript.

The developed algorithm for the estimation of the MixFMMm models has been
implemented in R and is readily available in the Supplementary Material.
Furthermore, a vignette illustrating its basic use is included.

##  3 Spike Sorting Datasets

In this work, ten different datasets have been analyzed. They were selected
because they encompass a wide variety of spike waveforms -real and simulated-;
the real class labels (groundtruth) are known; and they are widely used as
benchmarking data in the literature, as in [43, 47, 25, 45], among others.
Table 1 gives information on the number of clusters and signals, and the main
references. Furthermore, 2D descriptions are shown in Figure 3 using the first
two principal components; while the mean spike waveform by class (in solid
line), along with the global mean (dashed line), are displayed in Figure 4.

Table 1: Summary of the used datasets: name, spike waveform similarity, total
number of spikes, number of spikes per class, and origin of the dataset.

Dataset Nº Spikes Origin E11 Easy1$\\_$Noise01 $\mathrm{n}=3396$ [28] Waveform
similarity: Low $\mathrm{n}_{1}=1123,\mathrm{n}_{2}=1061,\mathrm{n}_{3}=1212$
E12 Easy1$\\_$Noise02 $\mathrm{n}=1896$ [28] Waveform similarity: Low
$\mathrm{n}_{1}=824,\mathrm{n}_{2}=193,\mathrm{n}_{3}=879$ E21
Easy2$\\_$Noise01 $\mathrm{n}=3389$ [28] Waveform similarity: Middle
$\mathrm{n}_{1}=1125,\mathrm{n}_{2}=1092,\mathrm{n}_{3}=1172$ E22
Easy2$\\_$Noise02 $\mathrm{n}=2707$ [28] Waveform similarity: Middle
$\mathrm{n}_{1}=951,\mathrm{n}_{2}=898,\mathrm{n}_{3}=858$ D11
Difficult1$\\_$Noise01 $\mathrm{n}=3365$ [28] Waveform similarity: High
$\mathrm{n}_{1}=1136,\mathrm{n}_{2}=1126,\mathrm{n}_{3}=1103$ D12
Difficult1$\\_$Noise02 $\mathrm{n}=2449$ [28] Waveform similarity: High
$\mathrm{n}_{1}=799,\mathrm{n}_{2}=792,\mathrm{n}_{3}=858$ D21
Difficult2$\\_$Noise01 $\mathrm{n}=3384$ [28] Waveform similarity: High
$\mathrm{n}_{1}=1164,\mathrm{n}_{2}=1119,\mathrm{n}_{3}=1101$ D22
Difficult2$\\_$Noise02 $\mathrm{n}=2634$ [28] Waveform similarity: High
$\mathrm{n}_{1}=856,\mathrm{n}_{2}=915,\mathrm{n}_{3}=863$ EPT Epileptic
patient temporal lobe $\mathrm{n}=9195$ [29] Waveform similarity: High
$\mathrm{n}_{1}=1050,\mathrm{n}_{2}=6638,\mathrm{n}_{3}=1449,\mathrm{n}_{4}=58$
VGA Visual cortex GABAergic $\mathrm{n}=507$ [2] Waveform similarity: High
$\mathrm{n}_{1}=163,\mathrm{n}_{2}=221,\mathrm{n}_{3}=123$

Figure 3: First two principal components projections of the spikes in the
datasets coloured by class. Components’ explained variance is shown in the
corresponding axis.

Figure 4: Spike means by class (solid lines) and global spike means (dashed)
of datasets.

More specifically, the eight first datasets are simulated extracellular
recordings first presented in [30]. The spikes fired by three different
neurons are superimposed on background noise generated by small-amplitude
spikes, mimicking the firing of distant neurons. In each dataset pair (E11 and
E12, E21 and E22, etc.), the same three neurons are fired with mid and high
background noise levels. The lower signal-to-noise ratio generates confusion
between classes, as can be seen in Figure 3. Furthermore, these datasets are
ordered in terms of increasing difficulty of correct clustering due to the
similarity of the waveforms, as seen in Figure 4. The data have been
preprocessed as in the cited works: spikes are detected with a threshold,
$k=\mathrm{Med}(|\boldsymbol{z}|/0.6745)$ where $\boldsymbol{z}$ is the
filtered signal, and the whole voltage traces have been fragmented into
signals of 64 samples, each containing a single spike with its maximum aligned
with the point 20. The noise artifacts erroneously detected as spikes, less
than $4\%$ of the signals, have initially been discarded for the rest of the
study.

Finally, the datasets EPT and VGA correspond to real recordings. EPT is a
collection of extracellular spikes from four neurons of the temporal lobe of
an epileptic patient, openly available in [29]. In this case, no preprocessing
has been done, as the spikes are already cut into signals, with 64 samples and
maxima aligned with point 20. In addition, the VGA has been created with data
from [2], a repository of intracellular electrophysiological neuronal
recordings. We selected signals from three GABAergic neuron types (Pvalb,
Htr3a and Vip positive) of the mouse visual cortex; in particular, those
generated by the short square stimulus with the lowest stimulus amplitude that
elicited a single spike for each cell in the database. To facilitate the
comparison, the signals have been down-sampled to have 64 samples, as in other
datasets, aligning with the maximum at data point 20. Even though the mean
spike waveforms are highly similar, as can be seen in Figure 4, the
distinction of GABAergic neuron spikes is crucial as they radically differ in
terms of physiological function [18].

##  4 Numerical Studies

In this Section, the MixFMM approach and the PCA (4 components) plus either KM
(method PCA+KM) or GM (method PCA+GM), have been used to find clusters in the
ten datasets described above. Specifically, the MixFMM1 and MixFMM3
homoscedastic models have been considered. A preliminary analysis with
heteroscedastic models resulted in quite similar variance estimators across
the datasets, as well as less accurate and more unstable solutions than those
of the homoscedastic models. PCA+KM and PCA+GM have been considered as they
are the most widely used clustering algorithms in SS, with better results in
practice than wavelets, among other approaches, see [30, 14, 45]. The number
of principal components in PCA ranges from two to four in the different SS
applications. Here, only the results using four components are shown, as this
choice gives better results in nine out of the ten datasets. For each of the
clustering procedures and datasets, the best model from multiple
initializations has been selected. Only ten initial values are needed for the
MixFMM and PCA+KM approaches to obtain stable solutions; while the PCA+GM
needed fifty initializations. Furthermore, the optimal number of clusters has
been estimated with the most widely used method for each clustering procedure.
A slope BIC heuristic for PCA+KM, and a combined integrated complete-data
likelihood criterion plus BIC for PCA+GM, as proposed in [27, 41],
respectively. The log-likelihood based approach, described in Section 2, has
been used for the MixFMMm models. Figure 5 shows the log-likelihoods for
different numbers of clusters and the selection for each dataset.

Figure 5: Log-likelihood of the MixFMM1 and MixFMM3 models for
$K=\\{2,3,4,5\\}$ in each of the datasets. The selected number of clusters is
highlighted with a circle.  Table 2: Estimated number of clusters, and index
results for each dataset and method.

Number of clusters E11 E12 E21 E22 D11 D12 D21 D22 EPT VGA PCA+KM 3 2 3 3 3 3
3 3 3 3 PCA+GM 4 4 4 4 4 2 3 3 4 3 MixFMM1 3 3 3 3 3 2 3 3 2 3 MixFMM3 3 3 3 3
3 2 3 3 2 3 Accuracy E11 E12 E21 E22 D11 D12 D21 D22 EPT VGA PCA+KM 0.992
0.888 0.875 0.563 0.665 0.524 0.720 0.540 0.589 0.617 PCA+GM 0.960 0.958 0.851
0.347 0.905 0.510 0.640 0.613 0.680 0.710 MixFMM1 0.992 0.966 0.936 0.700
0.895 0.530 0.705 0.575 0.874 0.755 MixFMM3 0.994 0.976 0.966 0.761 0.941
0.541 0.743 0.606 0.874 0.746 Ball-Hall Index E11 E12 E21 E22 D11 D12 D21 D22
EPT VGA PCA+KM 1.153 3.344 1.162 2.852 1.182 2.492 0.967 2.471 18901 1400
PCA+GM 3.276 3.000 1.984 3.855 1.841 2.681 2.156 2.783 52804 1541 MixFMM1
1.154 2.971 1.102 2.807 1.088 2.697 0.967 2.542 21850 1230 MixFMM3 1.150 2.887
1.085 2.770 1.074 2.688 0.944 2.460 21734 1228 Davis-Bouldin Index E11 E12 E21
E22 D11 D12 D21 D22 EPT VGA PCA+KM 0.727 0.936 1.455 2.159 2.531 2.316 2.144
2.103 1.601 1.213 PCA+GM 2.411 1.140 2.421 1.735 3.149 2.118 3.291 2.883 2.355
1.580 MixFMM1 0.728 1.217 1.357 2.070 1.785 2.040 2.091 2.241 0.744 1.144
MixFMM3 0.726 1.159 1.341 1.993 1.717 2.024 2.070 2.112 0.742 1.144 Average
Silhouette Width E11 E12 E21 E22 D11 D12 D21 D22 EPT VGA PCA+KM 0.552 0.439
0.250 0.117 0.126 0.120 0.173 0.128 0.262 0.280 PCA+GM 0.348 0.356 0.160 0.172
0.108 0.151 0.147 0.098 0.112 0.253 MixFMM1 0.551 0.348 0.282 0.124 0.193
0.162 0.173 0.121 0.512 0.326 MixFMM3 0.552 0.365 0.293 0.134 0.205 0.164
0.187 0.133 0.514 0.325

Table 2 summarizes the main results across different approaches, specifically
the optimal number of clusters and the values of the validation indexes. The
optimal number of clusters with the MixFMM corresponds to the real clusters in
eight out of ten cases, which is comparable to or better than those of the
other approaches. Note that the incorrect cases are characterized by a high
level of noise. In terms of accuracy, which is likely the most relevant index,
the MixFMM approach attains the best results in nine out of the ten datasets,
achieving significant improvements over the PCA-based methods in some cases.
The exception is dataset D22, in which PCA+GM is marginally better.
Interestingly, PCA+KM achieves better results than PCA+GM, except in datasets
where the average curves of the different classes are quite similar.
Furthermore, the internal indexes confirm the better performance of the MixFMM
approach against the PCA-based methods traditionally used in SS. More
specifically, the low values of $\mathcal{BH}$ and $\mathcal{DB}$ for the
MixFMM approaches indicate, on the one hand, that the clusters have a high
cohesion, low dispersion and well-separated barycenters. On the other hand,
the MixFMM clusters are composed of similar spikes distinguishable from others
in near clusters by the high values of the $\mathcal{ASW}$.

While most of the results highlight the MixFMM3 as the best all-around model,
the MixFMM1 should be considered a suitable alternative in specific cases,
particularly in data with a high noise level, as the outstanding results in
the real datasets demonstrate.

Comparing the results in Table 2 with results given in [45] in the common
datasets, GM attains significantly better results in this work, probably due
to R’s mclust being more optimized, flexible, and updated (mclust last
version: December 2021; Matlab’s gmdistribution last version: before 2018).

Finally, in order to validate the robustness of the different approaches, the
distribution of the accuracy has been studied across a hundred repetitions for
the methods with ten initializations in each dataset. Figure 6 shows the
associated boxplots, which illustrate that the MixFMM approach achieve an
accurate solution independently of the initialization in all the scenarios
analyzed, while the PCA-based methods fail, especially when spike waveform
differences arise mostly from shape and not amplitude.

Figure 6: Boxplots of the accuracy across a hundred repetitions for each
dataset and method.

##  5 Discussion

We present here a novel model-based functional data clustering method and
prove its potential to solve the SS problem. In particular, we show that the
MixFMM approach achieves better results than PCA-based methodologies often
used in SS applications, achieving significant improvements in some scenarios,
including very noisy cases. The MixFMM approach is based on using combinations
of FMM waves to describe the signals, which allow a wide variety of waveforms
to be described in a very simple and interpretable way. In particular,
different FMM parameters account for variations in phase, amplitude, and
shape.

The results of the MixFMM clustering entails interesting contributions in
neuroscience. In particular, neuronal signals from the temporal lobe, such as
EPT, are related to specific visual stimuli [33]. The function in the nervous
system of the different kinds of GABAergic neurons is radically different, so
the distinction of their signals is crucial. In VGA, signals from three types
of neurons are differentiated: Pvalb-positive neurons, key in learning and
memory processes, as detailed in [13]; Vip-positive neurons, related to
circadian rhythms [24]; and Htr3a-positive neurons, related to bladder
dysfunction [34]. In addition, the methodology can be adapted to cluster
multichannel spike recordings [49], combining the information of FMM models
for each channel, or with multivariate FMM models similar to those introduced
in [39] to analyze ECG multilead signals.

Beyond its contribution to neuroscience, the MixFMM is presented as a general
approach in functional clustering when oscillatory or quasi-oscillatory
signals are the target. In a field in which the literature is ever-growing,
and many excessively specific and complex proposals are arising, the need for
flexible and robust proposals is vital. The MixFMM assembles all of these
characteristics with a simple but rich parametric formulation that allows
differences in phase and/or amplitude to be taken into account. Oscillatory
signals appear in astronomy, economy, spectrometry, environmental science,
medical imaging, or electrocardiography, among other fields. In addition, from
a theoretical perspective, many extensions of the basic mixture framework
could be explored, such as a regularized estimation algorithm or the inclusion
of mixed-effects [9].

Finally, one limitation of the proposed methodology is its computational time.
It could be significantly reduced by implementing the parameter search in a
more computationally efficient programming language, such as C, and using GPU-
based computing.

## References

  * [1] Serhat Akhanli and Christian Hennig.  Comparing clusterings and numbers of clusters by aggregation of calibrated clustering validity indexes.  Statistics and Computing, 30:1523–1544, 2020.
  * [2] Allen Institute for Brain Science.  Allen cell types database, 2021.  Available in https://celltypes.brain-map.org/data.
  * [3] Allen Institute for Brain Science.  Allen SDK.  Allen Institute, 2021.  Python package version v2.13.0.
  * [4] Lucien Birgé and Pascal Massart.  Minimal penalties for gaussian model selection.  Probability Theory and Related Fields, 138:33–73, 2007.
  * [5] Charles Bouveyron, Etienne Côme, and Julien Jacques.  The discriminative functional mixture model for a comparative analysis of bike sharing systems.  The Annals of Applied Statistics, 9(4):1726–1760, 2015.
  * [6] Gyorgy Buzsáki.  Large-scale recording of neuronal ensembles.  Nature Neuroscience, 7:446–51, 2004.
  * [7] David Carlson and Lawrence Carin.  Continuing progress of spike sorting in the era of big data.  Current Opinion in Neurobiology, 55:90–96, 2019.
  * [8] Carmen Rocío Caro-Martín, José M. Delgado-García, Agnès Gruart, and Raudel Sánchez-Campusano.  Spike sorting based on shape, phase, and distribution features, and k-tops clustering with validity and error indices.  Scientific reports, 8(1):1–28, 2018.
  * [9] Faicel Chamroukhi and Hien D. Nguyen.  Model-based clustering and classification of functional data.  WIREs Data Mining and Knowledge Discovery, 9(4):e1298, 2019.
  * [10] Gerda Claeskens, Emilie Devijver, and Irène Gijbels.  Nonlinear mixed effects modeling and warping for functional data using B-splines.  Electronic Journal of Statistics, 15(2):5245 – 5282, 2021.
  * [11] Arthur Pentland Dempster, Nan Laird, and Donald Rubin.  Maximum likelihood from incomplete data via the em algorithm.  Journal of the Royal Statistical Society: Series B (Methodological), 39(1):1–22, 1977.
  * [12] Bernard Desgraupes.  clusterCrit: Clustering Indices.  University of Paris Ouest, 2018.  R package version 1.2.8.
  * [13] Flavio Donato, Santiago Rompani, and Pico Caroni.  Parvalbumin-expressing basket-cell network plasticity induced by experience regulates adult learning.  Nature, 504:272–276, 2013.
  * [14] Chaitanya Ekanadham, Daniel Tranchina, and Eero P. Simoncelli.  A unified framework and method for automatic neural spike identification.  Journal of Neuroscience Methods, 222:47–55, 2014.
  * [15] Itziar Fernández, Alejandro Rodríguez-Collado, Yolanda Larriba, Adrian Lamela, et al.  FMM: An R package for modeling rhythmic patterns in oscillatory systems.  The R Journal, 2022.  In press. Preprint available in https://arxiv.org/abs/2105.10168.
  * [16] María Luz López García, Ricardo García-Ródenas, and Antonia González Gómez.  K-means algorithms for functional data.  Neurocomputing, 151:231–245, 2015.
  * [17] Madison Giacofci, Sophie Lambert-Lacroix, Guillemette Marot, and Franck Picard.  Wavelet-based clustering for mixed-effects functional models in high dimension.  Biometrics, 69(1):31–40, 2013.
  * [18] Nathan W. Gouwens, Staci A. Sorensen, Fahimeh Baftizadeh, Agata Budzillo, et al.  Integrated morphoelectric and transcriptomic classification of cortical gabaergic cells.  Cell, 183(4):935–953, 2020.
  * [19] Plotly Technologies Inc.  Collaborative data science.  Plotly Technologies Inc., Montreal, QC, 2015.
  * [20] Julien Jacques and Cristian Preda.  Functional data clustering: a survey.  Advances in Data Analysis and Classification, 8(3):24, 2014.
  * [21] Wonyul Lee, Michelle F. Miranda, Philip Rausch, Veerabhadran Baladandayuthapani, et al.  Bayesian semiparametric functional mixed models for serially correlated functional data, with application to glaucoma data.  Journal of the American Statistical Association, 114(526):495–513, 2019.
  * [22] Yaeji Lim, Hee-Seok Oh, and Ying Kuen Cheung.  Multiscale Clustering for Functional Data.  Journal of Classification, 36(2):368–391, 2019.
  * [23] J. S. Marron, James Ramsay, Laura M. Sangalli, and Anuj Srivastava.  Functional Data Analysis of Amplitude and Phase Variation.  Statistical Science, 30(4):468–484, 2015.
  * [24] Cristina Mazuski, Samantha Chen, and Erik Herzog.  Different roles for vip neurons in the neonatal and adult suprachiasmatic nucleus.  Journal of Biological Rhythms, 35(5):465–475, 2020.
  * [25] Masoud Moghaddasi, Mahdi Aliyari Shoorehdeli, Zahra Fatahi, and Abbas Haghparast.  Unsupervised automatic online spike sorting using reward-based online clustering.  Biomedical Signal Processing and Control, 56:101701, 2020.
  * [26] Juhyun Park and Jeongyoun Ahn.  Clustering multivariate functional data with phase variation.  Biometrics, 73(1):324–333, 2017.
  * [27] Dau Pelleg and Andrew Moore.  X-means: Extending k-means with efficient estimation of the number of clusters.  In Proceedings of the 17th International Conf. on Machine Learning, pages 727–734, 2000.
  * [28] Rodrigo Quian Quiroga.  Simulated datasets, 2009.  Available in https://leicester.figshare.com/articles/dataset/Simulated_dataset/11897595.
  * [29] Rodrigo Quian Quiroga.  Dataset: Human single-cell recording, 2019.  Available in https://leicester.figshare.com/articles/Dataset_Human_single-cell_recording/11302427/1.
  * [30] Rodrigo Quian Quiroga, Zoltan Nadasdy, and Yoram Ben-Shaul.  Unsupervised Spike Detection and Sorting with Wavelets and Superparamagnetic Clustering.  Neural Computation, 16(8):1661–1687, 2004.
  * [31] R Core Team.  R: A Language and Environment for Statistical Computing.  R Foundation for Statistical Computing, Vienna, Austria, 2020.
  * [32] Melinda Rácz, Csaba Liber, Erik Németh, Richárd Fiáth, et al.  Spike detection and sorting with deep learning.  Journal of Neural Engineering, 17:016038, 2019.
  * [33] Hernan Gonzalo Rey, Carlos Pedreira, and Rodrigo Quian Quiroga.  Past, present and future of spike sorting techniques.  Brain research bulletin, 119:106–117, 2015.
  * [34] Elaine Ritter and Michelle Southard-Smith.  Dynamic expression of serotonin receptor 5-ht3a in developing sensory innervation of the lower urinary tract.  Frontiers in Neuroscience, 10:592, 2017.
  * [35] Alejandro Rodríguez-Collado and Cristina Rueda.  Electrophysiological and transcriptomic features reveal a circular taxonomy of cortical neurons.  Frontiers in Human Neuroscience, 15:684950, 2021.
  * [36] Alejandro Rodríguez-Collado and Cristina Rueda.  A simple parametric representation of the Hodgkin-Huxley model.  PLOS ONE, 16(7):1–19, 2021.
  * [37] Cristina Rueda, Yolanda Larriba, and Adrian Lamela.  The hidden waves in the ecg uncovered revealing a sound automated interpretation method.  Scientific Reports, 11:3724, 2021.
  * [38] Cristina Rueda, Yolanda Larriba, and Shyamal D. Peddada.  Frequency modulated Möbius model accurately predicts rhythmic signals in biological and physical sciences.  Scientific Reports., 9(1):1–10, 2019.
  * [39] Cristina Rueda, Alejandro Rodríguez-Collado, Itziar Fernández, Christian Canedo, et al.  A unique cardiac electrophysiologic model, 2022.  Preprint. Available in https://arxiv.org/abs/2202.03938.
  * [40] Cristina Rueda, Alejandro Rodríguez-Collado, and Yolanda Larriba.  A novel wave decomposition for oscillatory signals.  IEEE Transactions on Signal Processing, 69:960–972, 2021.
  * [41] Luca Scrucca, Michael Fop, T. Brendan Murphy, and Adrian E. Raftery.  mclust 5: clustering, classification and density estimation using Gaussian finite mixture models.  The R Journal, 8(1):289–317, 2016.
  * [42] Niraj K. Sharma, Carlos Pedreira, Maria Centeno, Umair J. Chaudhary, et al.  A novel scheme for the validation of an automated classification method for epileptic spikes by comparison with multiple observers.  Clinical Neurophysiology, 128(7):1246–254, 2017.
  * [43] Bryan Souza, Vítor Lopes dos Santos, João Bacelo, and Adriano Tort.  Spike sorting with gaussian mixture models.  Scientific Reports, 9:3627, 2019.
  * [44] Jeyathevy Sukiban, Nicole Voges, Till A. Dembek, Robin Pauli, et al.  Evaluation of spike sorting algorithms: Application to human subthalamic nucleus recordings and simulations.  Neuroscience, 414:168–185, 2019.
  * [45] Rakesh Veerabhadrappa, Masood Ul Hassan, James Zhang, and Asim Bhatti.  Compatibility evaluation of clustering algorithms for contemporary extracellular neural spike sorting.  Frontiers in Systems Neuroscience, 14:34, 2020.
  * [46] Lucas Vendramin, Ricardo J. G. B. Campello, and Eduardo R. Hruschka.  Relative clustering validity criteria: A comparative overview.  Statistical Analysis and Data Mining: The ASA Data Science Journal, 3(4):209–235, 2010.
  * [47] Pan Ke Wang, Sio Hang Pun, Chang Hao Chen, Elizabeth A. McCullagh, et al.  Low-latency single channel real-time neural spike sorting system based on template matching.  PLOS ONE, 14(11):1–30, 2019.
  * [48] Hadley Wickham.  ggplot2: Elegant Graphics for Data Analysis.  Springer - Verlag New York, second edition, 2016.
  * [49] Pierre Yger, Giulia LB Spampinato, Elric Esposito, Baptiste Lefebvre, et al.  A spike sorting toolbox for up to thousands of electrodes validated with ground truth recordings in vitro and in vivo.  eLife, 7:e34518, 2018.
  * [50] Qingzhi Zhong, Huazhen Lin, and Yi Li.  Cluster non-gaussian functional data.  Biometrics, 77(3):852–865, 2020.


representations that capture underlying patterns across different but related
tasks, leading to several key benefits [128]. Typically, MTL encourages the
ASR system to learn representations that are beneficial across multiple tasks.
This can lead to more robust feature extraction, as the model is not optimized
solely for transcribing speech but also for other related tasks, such as
speaker identification or emotion recognition. This shared learning process
helps in capturing a broader range of speech characteristics, which can
improve the ASR system’s ability to handle varied speech inputs. Moving one,
By simultaneously learning related tasks, MTL acts as a form of
regularization, reducing the risk of overfitting on the primary ASR task. This
is because the model must find a solution that performs well across all tasks,
preventing it from relying too heavily on noise or idiosyncrasies specific to
the training data of the main task. Besides, learning auxiliary tasks
alongside the main ASR task can improve the model’s generalization
capabilities. For example, learning to identify the speaker or the language
can provide additional contextual clues that help the ASR system better
understand and transcribe ambiguous audio signals.

Additionally, MTL can make more efficient use of available data by leveraging
auxiliary tasks for which more data might be available. In scenarios where
annotated data for ASR is limited, incorporating additional tasks with more
abundant data can help improve the learning process and performance of the ASR
system. Moreover, MTL allows ASR systems to better handle acoustic variability
in speech, such as accents, dialects, or noisy environments, by incorporating
tasks that directly or indirectly encourage the model to learn features that
are invariant to these variations. Last but not least, modern ASR systems
often employ DL architectures that can benefit from end-to-end learning
strategies. MTL fits naturally into this paradigm, allowing for the joint
optimization of multiple objectives within a single model architecture. This
can simplify the training process and reduce the need for separately trained
models or handcrafted features.

###  6.3 Federated multi-task learning and distillation for ASR

FMTL extends the concept of FL by allowing each client to learn a personalized
model that addresses its specific task, while still benefiting from
collaboration with other clients.This approach recognizes the heterogeneity in
clients’ data distributions and tasks. Mathematically and compared with FL
(Equation 3), federated multi-task learning (FMTL) can be formulated as:

$\min_{\theta_{1},\theta_{2},...,\theta_{K}}\sum_{k=1}^{K}F_{k}(\theta_{k})+\lambda
R(\theta_{1},\theta_{2},...,\theta_{K})$ (7)

Different from FL, $R(\theta_{1},\theta_{2},...,\theta_{K})$ is a
regularization term that encourages some form of similarity or sharing among
the model parameters of different tasks, promoting collaboration among
clients. $\lambda$ is a regularization coefficient that balances the trade-off
between fitting the local data well and collaborating with other clients. FMTL
has task-specific model parameters $\theta_{k}$ for each client, where only a
single global model parameter $\theta$ in FL.

In this regard, FMTL offers a promising approach to improving ASR systems
while also enhancing privacy and security measures. This learning paradigm
extends the traditional FL model by enabling the simultaneous training of
multiple tasks across distributed devices or nodes, without the need to share
raw data [129]. FMTL leverages data from a wide range of devices and users,
each potentially offering unique speech data, accents, dialects, and noise
conditions. This diversity helps in training more robust ASR models that can
perform well across various speech patterns and environments [130]. By
learning from many tasks simultaneously, FMTL can personalize ASR models to
individual users or specific groups without compromising the model’s general
performance [131]. This is particularly beneficial for users with unique
speech patterns, such as those with accents or speech impairments. Moreover,
FMTL encourages the development of compact models that can handle multiple
tasks efficiently. For ASR systems, this means that a single model can
potentially perform speech recognition, speaker identification, and even
emotion detection, reducing the computational overhead on client devices
[132].

On the other hand, in FMTL, raw data remains on the user’s device and does not
need to be shared or transferred to a central server. This inherently reduces
the risk of data breaches and unauthorized access, as sensitive speech data is
not centralized [133]. Additionally, FMTL can be combined with differential
privacy techniques to further anonymize the model updates sent from devices to
the central server. This ensures that the shared information does not reveal
sensitive details about the data or the user, enhancing privacy protection
[134]. Moving on, the aggregation process in FMTL can be secured using
cryptographic techniques, ensuring that the aggregated model updates cannot be
traced back to individual users. This secure aggregation process protects user
privacy while allowing the benefits of collective learning [135]. Lastly, by
aggregating model updates from a wide range of tasks and users, FMTL can
improve the system’s robustness to malicious attempts at data poisoning. The
diversity of inputs helps in diluting the impact of any adversarial data
introduced to compromise the model.

End-to-End ASR models Continuing to develop and refine end-to-end deep
learning models that directly map speech inputs to text outputs without the
need for intermediate representations (like phonetic transcriptions)
[mamyrbayev2023hybrid]. Advances in models such as Transformer and Conformer
architectures, which can capture long-range dependencies in speech, are
promising for improving ASR accuracy and efficiency [liu2023sfa].
Specifically, end-to-end models in ASR aim to directly convert speech input
into text output using a single neural network architecture. This simplifies
the ASR pipeline by bypassing traditional stages such as acoustic,
pronunciation, and language modeling [yang2023attention].

An end-to-end ASR model maps an input sequence of speech features
$X=(x_{1},x_{2},\ldots,x_{T})$ to an output sequence of tokens
$Y=(y_{1},y_{2},\ldots,y_{N})$. The model function $f$ with parameters
$\theta$ aims to minimize the difference between the predicted sequence and
the ground truth. Two common loss functions are used [almadhor2023e2e]:

  * •

CTC Loss: Connectionist Temporal Classification (CTC) introduces a ’blank’
token for aligning sequences. The CTC loss is given by:

$\mathcal{L}_{\text{CTC}}(\theta)=-\sum_{(X,Y)\in\mathcal{D}}\log
P(Y|X;\theta)$

where $\mathcal{D}$ is the dataset, and $P(Y|X;\theta)$ is the sequence
probability.

  * •

Attention-based Seq2Seq Loss: This uses an encoder-decoder architecture with
attention to predict each token. The loss is the negative log-likelihood:

$\mathcal{L}_{\text{Seq2Seq}}(\theta)=-\sum_{(X,Y)\in\mathcal{D}}\log
P(Y|X;\theta)$

Besides, model architectures can be described as follows:

  * •

Recurrent Neural Networks (RNNs) including Long Short-Term Memory (LSTM)
networks for temporal dependencies [vander2023using].

  * •

Convolutional Neural Networks (CNNs) for capturing local patterns in speech
features.

  * •

Transformer Models utilize self-attention mechanisms for parallel processing
and superior performance [wang2022optimizing].

Gradient-based methods like stochastic gradient descent (SGD) or Adam are used
to minimize the loss function, optimizing the model parameters $\theta$ to
improve the speech-to-text mapping. However, end-to-end models face challenges
such as data scarcity for low-resource languages, computational resource
demands, and integrating external language models for linguistic improvements
[qu2023emphasizing].

###  6.4 Domain-specific language modeling

DSLM represents a significant advancement in the field of ASR, especially as
systems increasingly cater to specialized fields like healthcare, legal, or
customer service [jia2023deep, li2023prompting]. LM provide probabilities of
sequences of words, crucial for ASR systems to predict the likelihood of
subsequent words in a sentence [38]. A domain-specific LM is trained on text
data from the target domain to capture its unique vocabulary and grammatical
structures [39]. For n-gram models, this involves calculating the conditional
probability of a word given the previous $n-1$ words [40]:

$P(w_{n}|w_{n-1},w_{n-2},\ldots,w_{n-(n-1)})=\frac{C(w_{n-(n-1)},\ldots,w_{n})}{C(w_{n-(n-1)},\ldots,w_{n-1})}$
(8)

Additionally, data augmentation plays a crucial role in ASR-based DA.
Typically, data augmentation in domain-specific ASR involves generating
synthetic training examples by altering existing recordings or using text-to-
speech (TTS) systems to create new audio samples from domain-specific texts.
Mathematically, augmentation techniques can include time stretching, pitch
shifting, adding noise, or simulating room acoustics, each represented as
transformations $T$ applied to the original audio signal $x(t)$:

$x^{\prime}(t)=T(x(t);\phi)$ (9)

where $\phi$ represents the parameters of the transformation. Moving on,
regularization techniques are applied to prevent overfitting to the domain-
specific data, ensuring the model maintains generalization capabilities.
Techniques such as dropout, L1/L2 regularization, or elastic net involve
adding terms to the loss function or modifying the optimization process to
penalize large weights or complex models. For instance, L2 regularization adds
a penalty equal to the square of the magnitude of coefficients:

$L^{\prime}(D;\theta)=L(D;\theta)+\lambda\|\theta\|^{2}$ (10)

where $\lambda$ is a regularization parameter.

While DSLM offers substantial benefits, challenges remain, such as data
scarcity in niche domains, the need for continual model updates to keep pace
with evolving language use, and ensuring privacy and security in sensitive
domains like healthcare and finance. Addressing these challenges through
innovative approaches in model training, data augmentation, and privacy-
preserving technologies will be crucial for the advancement of domain-specific
ASR systems.

Privacy preservation With the advancements in ML, DL, and DTL, ASR systems
have become more practical and scalable. However, these systems also introduce
serious privacy concerns due to the abundance of sensitive acoustic and
textual information in speech data.

While open-source and offline ASR systems can mitigate privacy risks, online
DTL-based systems can amplify these threats. Additionally, the transcription
performance of offline and open-source ASR systems is typically inferior to
cloud-based ASR systems, especially in real-world scenarios [ahmed2020preech].
In this context, the $D_{S}$ data may contain sensitive information that needs
to be protected. Thus, preserving users’ privacy during the knowledge transfer
from the source domain (SD) to the target domain (TD) becomes a critical
issue.

To address this challenge, future research efforts should focus on integrating
effective security and privacy protection strategies. Examples include
decentralized DTL approaches utilizing blockchain technology
[ul2020decentralized, wang2021enabling] and federated DTL methods
[zhang2021federated, maurya2021federated]. These approaches aim to ensure
privacy-preserving knowledge transfer in DTL-based ASR systems.

###  6.5 Interpretation of DTL models

DTL-based ASR models, despite their success, are often considered as ”black
box” systems lacking interpretability. This lack of interpretability raises
doubts about the credibility and repeatability of their decisions, making it
crucial to explain the reasoning behind their predictions. The field of
explainable and interpretable ML/DL has gained increasing interest in various
applications, including speech processing.

Several studies have explored the interpretability of DTL models. In one study
[ramakrishnan2016towards], an agent was designed to explain how it learns a
new task using prior common knowledge, aiming to enhance users’ trust and
acceptance of the system results. Another study [kim2019the] defined
interpretable features in a DTL algorithm and examined the relationship
between the SD and TD in the task, focusing on the interpretability of the
pretrained DTL model.

The work by Lee et al. [lee2021interpretable] introduced a knowledge
distillation approach that generated interpretable embedding procedure (IEP)
knowledge based on PCA and transferred it to the student network using a
message passing neural network. This approach enhanced interpretability while
maintaining accuracy through multi-task learning. Additionally, Carr et al.
proposed an interpretable staged TL (iSTL) scheme for accurate and explainable
classification of optical coherence tomography (OCT) scans with a small sample
size [carr2021interpretable]. iSTL outperformed DTL techniques on unseen data,
utilizing clinical features for predictions with interpretable attention maps.

Delving deeper into techniques for FL distillation (optimizing model size)
within FL frameworks is essential. This exploration involves researching
methods to compress neural ASR models effectively while ensuring their
performance remains intact, especially tailored for edge devices with storage
and computational constraints. It is imperative to investigate the trade-offs
associated with reducing model size while maintaining performance metrics like
WER. Developing strategies to strike a balance between downsizing models and
preserving satisfactory performance levels within federated learning
environments is crucial.

###  6.6 Recent RL techniques for ASR

Exploring incremental DRL approaches [136, 137, 138] in DRL-based ASR systems,
could be very interesting. This approach involves the model continuously
learning from newly acquired data and dynamically adjusting its ASR
functionalities over time. By incrementally updating its knowledge base, the
model can enhance its performance without necessitating full retraining, thus
enabling continual enhancement of ASR systems. This capability not only
fosters greater resilience and adaptability in speech recognition capabilities
but also offers potential applications in scenarios where real-time adaptation
to changing conditions is crucial, such as in noisy environments or with
varying speaker accents. Moreover, incremental RL can potentially lead to more
efficient use of computational resources, as the model only needs to focus on
learning from new data, rather than reprocessing the entire dataset. Further
research in this area could unlock new possibilities for ASR systems to evolve
and improve over time, ultimately enhancing their usability and effectiveness
in diverse real-world settings.

Although some ASR schemes based on RL have been proposed, there remains a
notable scarcity in the application of RL techniques to enhance ASR methods.
While policy gradient and Q-learning are commonly employed, the realm of RL
encompasses various subcategories such as double deep Q-network (DDQN), actor-
critic (AC), deep deterministic policy gradien (DDPG), and more [139], which
hold promise for advancing ASR with innovative approaches. Researchers are
encouraged to delve into these diverse DRL-based methods to further enrich the
field of ASR for both AM and LM fields.

###  6.7 Online DTL

Online DTL combines the principles of TL and online learning with deep neural
networks, enabling models to adapt in real-time to new tasks or data
distributions. This approach is beneficial in dynamic environments where data
arrives sequentially. DL models, specifically Deep Neural Networks (DNNs),
learn through optimizing the weights $\theta$ to minimize a loss function $L$,
which measures the discrepancy between predicted outputs $\hat{y}$ and true
outputs $y$: $\theta^{*}=\arg\min_{\theta}L(D;\theta)$. On the other hand, TL
improves learning in a new target task through the transfer of knowledge from
a related source task, adapting a pre-trained model $\theta_{S}$ on $D_{S}$ to
a $D_{T}$: $\theta_{T}^{*}=\arg\min_{\theta}L(D_{T};\theta_{T})$. In this
regard, online learning updates the model incrementally as new data
$(x_{t},y_{t})$ arrives:

$\theta_{t+1}=\theta_{t}-\alpha_{t}\nabla_{\theta}L(y_{t},f(x_{t};\theta_{t}))$
(11)

Where $\alpha_{t}$ is the learning rate, and $\nabla_{\theta}L$ is the
gradient of the loss with respect to $\theta$. Moving on, online TL integrates
these concepts to continuously adapt a deep learning model to new tasks or
data streams, often involving techniques such as feature extraction, fine-
tuning, model adaptation, and continual learning. The adaptation process at
each time step $t$ can be viewed as
$\theta_{t+1}^{*}=\arg\min_{\theta}L(D_{T_{t}};\theta_{T_{t}})$. Where
$D_{T_{t}}$ represents the data available at time $t$, including new target
domain data.

The approach of online DTL offers a forward-looking solution to this issue.
Particularly, discrepancies in class distributions and the representation of
features between the $D_{S}$ and $D_{T}$ amplify the complexity of online DTL
[140]. To navigate the complexities mentioned, online DTL has been dissected
into two primary methodologies. The first, known as homogeneous online DTL,
operates on the premise of a unified feature space across both domains.
Conversely, heterogeneous online DTL acknowledges the distinct feature spaces
intrinsic to each domain [141]. An exemplary solution to the challenges of
heterogeneous online DTL includes leveraging unlabeled instances of co-
occurrence to forge a connective bridge between the $D_{S}$ and $D_{T}$,
facilitating the precursor to knowledge transfer [142]. Furthering the
discourse, online DTL augmented with extreme learning machines introduces a
novel framework [143]. Addressing the challenge of limited data in the
$D_{T}$, the technique of transfer learning with lag (TLL), rooted in shallow
neural network embeddings, has been applied. This method ensures the
continuity of knowledge transfer, notwithstanding fluctuations in the feature
set.

###  6.8 Transformers and LLMs-based ASR

LLM and transformers represent the forefront of AI, trained on vast datasets
spanning various domains, including text, speech, images, and multi-modal
inputs. Despite extensive research on ASR, existing SOTA approaches often lack
integration of advanced AI techniques like DRL and FL into both AM and LM
domains. For example, LLM based on DTL has demonstrated significant potential
for ASR tasks, particularly for both LM and AM components. The incorporation
of TL techniques into large language model (LLM) facilitates the transfer of
knowledge from extensive pre-training tasks to enhance ASR effectiveness. In
terms of AM, fine-tuning LLM can leverage insights gained from pre-trained
models exposed to sample acoustic data. This enables the AM component to grasp
acoustic features like spectrograms or Mel-frequency cepstral coefficients
(MFCCs) and utilize pre-trained knowledge to improve speech recognition
accuracy. Through fine-tuning, the model can adjust and specialize in specific
datasets or acoustic domains, leading to enhanced ASR performance.

Similarly, the LM aspect of LLM based on DTL can enhance ASR by leveraging TL.
Pre-training LLM on vast text corpora equips it with extensive language
representations, aiding in addressing diverse language challenges. Fine-tuning
enables adaptation to specific language characteristics, improving
transcription accuracy and contextual appropriateness. Both AM and LM fine-
tuning can benefit from DA, incorporating target domain data to tailor models,
reducing domain mismatch, and enhancing effectiveness and generalization.
Utilizing LLM for objective ASR testing and MOS evaluation involves compiling
diverse datasets, fine-tuning LLM, and integrating it into the ASR system.
Evaluation metrics like CER and Pearson’s correlation gauge system
performance, guiding further fine-tuning iterations for improved results. This
iterative process ensures the ASR system’s continual enhancement and accurate
MOS scale generation. Researchers are invited to explore these gaps and
advance the integration of more advanced AI techniques such as DRL and FL into
both AM and LM domains within transformer-based models. Additionally, there is
a need for further investigation into Transformers and DTL-based ASR schemes
specifically tailored for the LM domain. Closing these gaps will contribute to
the development of more robust and effective language models across various
applications.

##  7 Conclusion

In conclusion, recent advancements in deep learning have presented both
challenges and opportunities forASR. Traditional ASR systems require extensive
training datasets, often including confidential information, and consume
significant computational resources. However, the demand for adaptive systems
capable of performing well in dynamic environments has spurred the development
of advanced deep learning techniques such as DTL, FL, and DRL, with all their
variant techniques. These advanced techniques address issues related to DA,
privacy preservation, and dynamic decision-making, thereby enhancing ASR
performance and reducing computational costs.

This survey has provided a comprehensive review of DTL, FL, and RL-based ASR
frameworks, offering insights into the latest developments and helping
researchers and professionals understand current challenges. Additionally, the
integration of transformers, powerful DL models, has been explored for their
ability to capture complex dependencies in ASR sequences. By presenting a
structured taxonomy and conducting critical analyses, this paper has shed
light on the strengths and weaknesses of existing frameworks, as well as
highlighted ongoing challenges. Moving forward, further research is needed to
overcome these challenges and unlock the full potential of advanced DL
techniques in ASR. Future work should focus on refining existing approaches,
addressing privacy concerns in FL, improving RL algorithms for ASR
optimization, and exploring innovative ways to leverage transformers for more
efficient and accurate speech recognition. By continuing to innovate and
collaborate across disciplines, we can push the boundaries of ASR technology
and realize its transformative impact on various fields, including healthcare,
communication, and accessibility.

## References

  * [1] H. Haneche, A. Ouahabi, B. Boudraa, Compressed sensing-speech coding scheme for mobile communications, Circuits, Systems, and Signal Processing (2021) 1–21.
  * [2] D. Michelsanti, Z.-H. Tan, S.-X. Zhang, Y. Xu, M. Yu, D. Yu, J. Jensen, An overview of deep-learning-based audio-visual speech enhancement and separation, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2021).
  * [3] Y. Luo, C. Han, N. Mesgarani, Group communication with context codec for lightweight source separation, IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021) 1752–1761.
  * [4] H. Kheddar, M. Bouzid, D. Megías, Pitch and fourier magnitude based steganography for hiding 2.4 kbps melp bitstream, IET Signal Processing 13 (3) (2019) 396–407.
  * [5] H. Kheddar, A. C. Mazari, G. H. Ilk, Speech steganography based on double approximation of lsfs parameters in amr coding, in: 2022 7th International Conference on Image and Signal Processing and their Applications (ISPA), IEEE, 2022, pp. 1–8.
  * [6] H. Kheddar, D. Megias, M. Bouzid, Fourier magnitude-based steganography for hiding 2.4 kbpsmelp secret speech, in: 2018 International Conference on Applied Smart Systems (ICASS), IEEE, 2018, pp. 1–5.
  * [7] H. Yassine, B. Bachir, K. Aziz, A secure and high robust audio watermarking system for copyright protection, International Journal of Computer Applications 53 (17) (2012) 33–39.
  * [8] M. Yamni, H. Karmouni, M. Sayyouri, H. Qjidaa, Efficient watermarking algorithm for digital audio/speech signal, Digital Signal Processing 120 (2022) 103251.
  * [9] H. Chen, B. D. Rouhani, F. Koushanfar, Specmark: A spectral watermarking framework for ip protection of speech recognition systems., in: INTERSPEECH, 2020, pp. 2312–2316.
  * [10] M. Olivieri, R. Malvermi, M. Pezzoli, M. Zanoni, S. Gonzalez, F. Antonacci, A. Sarti, Audio information retrieval and musical acoustics, IEEE Instrumentation & Measurement Magazine 24 (7) (2021) 10–20.
  * [11] E. Wold, T. Blum, D. Keislar, J. Wheaten, Content-based classification, search, and retrieval of audio, IEEE multimedia 3 (3) (1996) 27–36.
  * [12] W. Boes, et al., Audiovisual transfer learning for audio tagging and sound event detection, Proceedings Interspeech 2021 (2021).
  * [13] Y. Tang, J. Pino, C. Wang, X. Ma, D. Genzel, A general multi-task learning framework to leverage text data for speech to text tasks, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 6209–6213.
  * [14] F. M. Plaza-del Arco, M. D. Molina-González, L. A. Ureña-López, M. T. Martín-Valdivia, Comparing pre-trained language models for spanish hate speech detection, Expert Systems with Applications 166 (2021) 114120.
  * [15] A. C. Mazari, H. Kheddar, Deep learning-based analysis of algerian dialect dataset targeted hate speech, offensive language and cyberbullying, International Journal of Computing and Digital Systems (2023).
  * [16] D. Meghraoui, B. Boudraa, T. Merazi, P. G. Vilda, A novel pre-processing technique in pathologic voice detection: Application to parkinson’s disease phonation, Biomedical Signal Processing and Control 68 (2021) 102604.
  * [17] Y.-Y. Lin, W.-Z. Zheng, W. C. Chu, J.-Y. Han, Y.-H. Hung, G.-M. Ho, C.-Y. Chang, Y.-H. Lai, A speech command control-based recognition system for dysarthric patients based on deep learning technology, Applied Sciences 11 (6) (2021) 2477.
  * [18] Y. Kumar, S. Gupta, W. Singh, A novel deep transfer learning models for recognition of birds sounds in different environment, Soft Computing 26 (3) (2022) 1003–1023.
  * [19] S. Padi, S. O. Sadjadi, R. D. Sriram, D. Manocha, Improved speech emotion recognition using transfer learning and spectrogram augmentation, in: Proceedings of the 2021 International Conference on Multimodal Interaction, 2021, pp. 645–652.
  * [20] Y. Himeur, M. Elnour, F. Fadli, N. Meskin, I. Petri, Y. Rezgui, F. Bensaali, A. Amra, Next-generation energy systems for sustainable smart cities: Roles of transfer learning, Sustainable Cities and Society (2022) 1–35.
  * [21] S. Niu, Y. Liu, J. Wang, H. Song, A decade survey of transfer learning (2010–2020), IEEE Transactions on Artificial Intelligence 1 (2) (2020) 151–166.
  * [22] H. Kheddar, D. Megías, High capacity speech steganography for the g723. 1 coder based on quantised line spectral pairs interpolation and cnn auto-encoding, Applied Intelligence (2022) 1–19.
  * [23] T. A. de Lima, M. Da Costa-Abreu, A survey on automatic speech recognition systems for portuguese language and its variations, Computer Speech & Language 62 (2020) 101055.
  * [24] A. Singh, V. Kadyan, M. Kumar, N. Bassan, Asroil: a comprehensive survey for automatic speech recognition of indian languages, Artificial Intelligence Review 53 (2020) 3673–3704.
  * [25] R. S. Arslan, N. BARIŞÇI, A detailed survey of turkish automatic speech recognition, Turkish journal of electrical engineering and computer sciences 28 (6) (2020) 3253–3269.
  * [26] A. Dhouib, A. Othman, O. El Ghoul, M. K. Khribi, A. Al Sinani, Arabic automatic speech recognition: a systematic literature review, Applied Sciences 12 (17) (2022) 8898.
  * [27] J. Kaur, A. Singh, V. Kadyan, Automatic speech recognition system for tonal languages: State-of-the-art survey, Archives of Computational Methods in Engineering 28 (2021) 1039–1068.
  * [28] A. A. Abushariah, H.-N. Ting, M. B. P. Mustafa, A. S. M. Khairuddin, M. A. Abushariah, T.-P. Tan, Bilingual automatic speech recognition: A review, taxonomy and open challenges, IEEE Access (2022).
  * [29] J. L. K. E. Fendji, D. C. Tala, B. O. Yenke, M. Atemkeng, Automatic speech recognition using limited vocabulary: A survey, Applied Artificial Intelligence 36 (1) (2022) 2095039.
  * [30] V. Bhardwaj, M. T. Ben Othman, V. Kukreja, Y. Belkhier, M. Bajaj, B. S. Goud, A. U. Rehman, M. Shafiq, H. Hamam, Automatic speech recognition (asr) systems for children: A systematic literature review, Applied Sciences 12 (9) (2022) 4419\.
  * [31] R. Errattahi, A. El Hannani, H. Ouahmane, Automatic speech recognition errors detection and correction: A review, Procedia Computer Science 128 (2018) 32–37.
  * [32] H. Aldarmaki, A. Ullah, S. Ram, N. Zaki, Unsupervised automatic speech recognition: A review, Speech Communication 139 (2022) 76–91.
  * [33] A. S. Dhanjal, W. Singh, A comprehensive survey on automatic speech recognition using neural networks, Multimedia Tools and Applications (2023) 1–46.
  * [34] A. B. Nassif, I. Shahin, I. Attili, M. Azzeh, K. Shaalan, Speech recognition using deep neural networks: A systematic review, IEEE access 7 (2019) 19143–19165.
  * [35] M. Malik, M. K. Malik, K. Mehmood, I. Makhdoom, Automatic speech recognition: a survey, Multimedia Tools and Applications 80 (6) (2021) 9411–9457.
  * [36] H. Kheddar, Y. Himeur, S. Al-Maadeed, A. Amira, F. Bensaali, Deep transfer learning for automatic speech recognition: Towards better generalization, Knowledge-Based Systems 277 (2023) 110851.
  * [37] F. Filippidou, L. Moussiades, A benchmarking of ibm, google and wit automatic speech recognition systems, in: IFIP International Conference on Artificial Intelligence Applications and Innovations, Springer, 2020, pp. 73–82.
  * [38] M. Suzuki, H. Sakaji, M. Hirano, K. Izumi, Constructing and analyzing domain-specific language model for financial text mining, Information Processing & Management 60 (2) (2023) 103194.
  * [39] C.-H. H. Yang, Y. Gu, Y.-C. Liu, S. Ghosh, I. Bulyko, A. Stolcke, Generative speech recognition error correction with large language models and task-activating prompting, in: 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2023, pp. 1–8.
  * [40] Z. Dong, Q. Ding, W. Zhai, M. Zhou, A speech recognition method based on domain-specific datasets and confidence decision networks, Sensors 23 (13) (2023) 6036.
  * [41] H. Kheddar, Y. Himeur, A. I. Awad, Deep transfer learning for intrusion detection in industrial control networks: A comprehensive review, Journal of Network and Computer Applications 220 (2023) 103760.
  * [42] Y. Li, X. Zhang, P. Wang, X. Zhang, Y. Liu, Insight into an unsupervised two-step sparse transfer learning algorithm for speech diagnosis of parkinson’s disease, Neural Computing and Applications (2021) 1–18.
  * [43] O. Karaman, H. Çakın, A. Alhudhaif, K. Polat, Robust automated parkinson disease detection based on voice signals with transfer learning, Expert Systems with Applications 178 (2021) 115013.
  * [44] R. A. Ramadan, Detecting adversarial attacks on audio-visual speech recognition using deep learning method, International Journal of Speech Technology (2021) 1–7.
  * [45] Q. Yu, Y. Ma, Y. Li, Enhancing speech recognition for parkinson’s disease patient using transfer learning technique, Journal of Shanghai Jiaotong University (Science) (2021) 1–9.
  * [46] Y. Bai, J. Yi, J. Tao, Z. Tian, Z. Wen, S. Zhang, Fast end-to-end speech recognition via non-autoregressive models and cross-modal knowledge transferring from bert, IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021) 1897–1911.
  * [47] I.-T. Recommendation, Perceptual evaluation of speech quality (pesq): An objective method for end-to-end speech quality assessment of narrow-band telephone networks and speech codecs, Rec. ITU-T P. 862 (2001).
  * [48] S. Zhang, E. Loweimi, P. Bell, S. Renals, On the usefulness of self-attention for automatic speech recognition with transformers, in: 2021 IEEE Spoken Language Technology Workshop (SLT), IEEE, 2021, pp. 89–96.
  * [49] O. Hrinchuk, M. Popova, B. Ginsburg, Correction of automatic speech recognition with transformer sequence-to-sequence model, in: Icassp 2020-2020 ieee international conference on acoustics, speech and signal processing (icassp), IEEE, 2020, pp. 7074–7078.
  * [50] J. Li, R. Su, X. Xie, N. Yan, L. Wang, A multi-level acoustic feature extraction framework for transformer based end-to-end speech recognition, arXiv preprint arXiv:2108.07980 (2021).
  * [51] A. Baade, P. Peng, D. Harwath, Mae-ast: Masked autoencoding audio spectrogram transformer, arXiv preprint arXiv:2203.16691 (2022).
  * [52] J. Bai, J. Chen, M. Wang, M. S. Ayub, Q. Yan, A squeeze-and-excitation and transformer based cross-task model for environmental sound recognition, IEEE Transactions on Cognitive and Developmental Systems (2022).
  * [53] K. Chen, J. Wang, F. Deng, X. Wang, icnn-transformer: An improved cnn-transformer with channel-spatial attention and keyword prediction for automated audio captioning, in: INTERSPEECH, 2022, pp. 4167–4171.
  * [54] K. Deng, S. Cao, Y. Zhang, L. Ma, Improving hybrid ctc/attention end-to-end speech recognition with pretrained acoustic and language models, in: 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, 2021, pp. 76–82.
  * [55] X. Zhou, E. Yılmaz, Y. Long, Y. Li, H. Li, Multi-encoder-decoder transformer for code-switching speech recognition, arXiv preprint arXiv:2006.10414 (2020).
  * [56] G. I. Winata, S. Cahyawijaya, Z. Lin, Z. Liu, P. Fung, Lightweight and efficient end-to-end speech recognition using low-rank transformer, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 6144–6148.
  * [57] M.-H. Lee, S.-E. Lee, J.-S. Seong, J.-H. Chang, H. Kwon, C. Park, Regularizing transformer-based acoustic models by penalizing attention weights for robust speech recognition, in: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, Vol. 2022, International Speech Communication Association, 2022, pp. 56–60.
  * [58] S. R. Shareef, Y. F. Mohammed, Collaborative training of acoustic encoder for recognizing the impaired children speech, in: 2022 Fifth College of Science International Conference of Recent Trends in Information Technology (CSCTIT), IEEE, 2022, pp. 79–85.
  * [59] R. Fan, W. Chu, P. Chang, A. Alwan, A ctc alignment-based non-autoregressive transformer for end-to-end automatic speech recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023) 1436–1448.
  * [60] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding (2019).  arXiv:1810.04805.
  * [61] C.-C. Chiu, T. N. Sainath, Y. Wu, R. Prabhavalkar, P. Nguyen, Z. Chen, A. Kannan, R. J. Weiss, K. Rao, E. Gonina, et al., State-of-the-art speech recognition with sequence-to-sequence models, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018, pp. 4774–4778.
  * [62] Y. Wang, Y. Shi, F. Zhang, C. Wu, J. Chan, C.-F. Yeh, A. Xiao, Transformer in action: a comparative study of transformer-based acoustic models for large scale speech recognition applications, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 6778–6782.
  * [63] A. Aroudi, S. Uhlich, M. F. Font, Trunet: Transformer-recurrent-u network for multi-channel reverberant sound source separation, arXiv preprint arXiv:2110.04047 (2021).
  * [64] L. Wang, W. Wei, Y. Chen, Y. Hu, D 2 net: A denoising and dereverberation network based on two-branch encoder and dual-path transformer, in: 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, 2022, pp. 1649–1654.
  * [65] P. Swietojanski, S. Braun, D. Can, T. F. Da Silva, A. Ghoshal, T. Hori, R. Hsiao, H. Mason, E. McDermott, H. Silovsky, et al., Variable attention masking for configurable transformer transducer speech recognition, in: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2023, pp. 1–5.
  * [66] N. Moritz, G. Wichern, T. Hori, J. Le Roux, All-in-one transformer: Unifying speech recognition, audio tagging, and event detection., in: INTERSPEECH, 2020, pp. 3112–3116.
  * [67] W. Huang, W. Hu, Y. T. Yeung, X. Chen, Conv-transformer transducer: Low latency, low frame rate, streamable end-to-end speech recognition, arXiv preprint arXiv:2008.05750 (2020).
  * [68] R. Fan, W. Chu, P. Chang, J. Xiao, Cass-nat: Ctc alignment-based single step non-autoregressive transformer for speech recognition, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 5889–5893.
  * [69] M. Hadwan, H. A. Alsayadi, S. AL-Hagree, An end-to-end transformer-based automatic speech recognition for qur’an reciters., Computers, Materials & Continua 74 (2) (2023).
  * [70] L. Smietanka, T. Maka, Augmented transformer for speech detection in adverse acoustical conditions, in: 2023 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), IEEE, 2023, pp. 14–18.
  * [71] Y. Li, D. Luo, Adversarial audio detection method based on transformer, in: 2022 International Conference on Machine Learning and Intelligent Systems Engineering (MLISE), IEEE, 2022, pp. 77–82.
  * [72] C. Wang, M. Jia, Y. Zhang, L. Li, Parallel-path transformer network for time-domain monaural speech separation, in: 2023 International Conference on Cyber-Physical Social Intelligence (ICCSI), IEEE, 2023, pp. 509–514.
  * [73] Y. Wang, A. Mohamed, D. Le, C. Liu, A. Xiao, J. Mahadeokar, H. Huang, A. Tjandra, X. Zhang, F. Zhang, et al., Transformer-based acoustic modeling for hybrid speech recognition, in: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2020, pp. 6874–6878.
  * [74] K. Wang, B. He, W.-P. Zhu, Tstnn: Two-stage transformer based neural network for speech enhancement in the time domain, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 7098–7102.
  * [75] Y. Gong, C.-I. Lai, Y.-A. Chung, J. Glass, Ssast: Self-supervised audio spectrogram transformer, in: Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36, 2022, pp. 10699–10709.
  * [76] C. Wu, Y. Wang, Y. Shi, C.-F. Yeh, F. Zhang, Streaming transformer-based acoustic models using self-attention with augmented memory, arXiv preprint arXiv:2005.08042 (2020).
  * [77] V. Nagaraja, Y. Shi, G. Venkatesh, O. Kalinli, M. L. Seltzer, V. Chandra, Collaborative training of acoustic encoders for speech recognition, arXiv preprint arXiv:2106.08960 (2021).
  * [78] G. Ahmed, A. A. Lawaye, T. A. Mir, P. Rana, Toward developing attention-based end-to-end automatic speech recognition, in: International Conference On Innovative Computing And Communication, Springer, 2023, pp. 147–161.
  * [79] Y. Himeur, S. Al-Maadeed, H. Kheddar, N. Al-Maadeed, K. Abualsaud, A. Mohamed, T. Khattab, Video surveillance using deep transfer learning and deep domain adaptation: Towards better generalization, Engineering Applications of Artificial Intelligence 119 (2023) 105698.
  * [80] H. Kheddar, M. Hemis, Y. Himeur, D. Megías, A. Amira, Deep learning for diverse data types steganalysis: A review, arXiv preprint arXiv:2308.04522 (2023).
  * [81] S. Schneider, A. Baevski, R. Collobert, M. Auli, wav2vec: Unsupervised pre-training for speech recognition, arXiv preprint arXiv:1904.05862 (2019).
  * [82] J. Thienpondt, K. Demuynck, Transfer learning for robust low-resource children’s speech asr with transformers and source-filter warping, arXiv preprint arXiv:2206.09396 (2022).
  * [83] Z. Dan, Y. Zhao, X. Bi, L. Wu, Q. Ji, Multi-task transformer with adaptive cross-entropy loss for multi-dialect speech recognition, Entropy 24 (10) (2022) 1429.
  * [84] T. Pellegrini, I. Khalfaoui-Hassani, E. Labbé, T. Masquelier, Adapting a convnext model to audio classification on audioset, arXiv preprint arXiv:2306.00830 (2023).
  * [85] Y. Xin, D. Yang, Y. Zou, Audio pyramid transformer with domain adaption for weakly supervised sound event detection and audio classification, in: Proc. Interspeech 2022, 2022, pp. 1546–1550.
  * [86] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018).
  * [87] Y. Song, D. Jiang, X. Zhao, Q. Xu, R. C.-W. Wong, L. Fan, Q. Yang, L2rs: a learning-to-rescore mechanism for automatic speech recognition, arXiv preprint arXiv:1910.11496 (2019).
  * [88] C.-X. Qin, D. Qu, L.-H. Zhang, Towards end-to-end speech recognition with transfer learning, EURASIP Journal on Audio, Speech, and Music Processing 2018 (1) (2018) 1–9.
  * [89] D. Jiang, C. Tan, J. Peng, C. Chen, X. Wu, W. Zhao, Y. Song, Y. Tong, C. Liu, Q. Xu, et al., A gdpr-compliant ecosystem for speech recognition with transfer, federated, and evolutionary learning, ACM Transactions on Intelligent Systems and Technology (TIST) 12 (3) (2021) 1–19.
  * [90] F. Weninger, J. Andrés-Ferrer, X. Li, P. Zhan, Listen, attend, spell and adapt: Speaker adapted sequence-to-sequence asr, arXiv preprint arXiv:1907.04916 (2019).
  * [91] S. Deena, M. Hasan, M. Doulaty, O. Saz, T. Hain, Recurrent neural network language model adaptation for multi-genre broadcast speech recognition and alignment, IEEE/ACM Transactions on Audio, Speech, and Language Processing 27 (3) (2018) 572–582.
  * [92] S.-I. Ng, W. Liu, Z. Peng, S. Feng, H.-P. Huang, O. Scharenborg, T. Lee, The cuhk-tudelft system for the slt 2021 children speech recognition challenge, arXiv preprint arXiv:2011.06239 (2020).
  * [93] K. Manohar, G. G. Menon, A. Abraham, R. Rajan, A. Jayan, Automatic recognition of continuous malayalam speech using pretrained multilingual transformers, in: 2023 International Conference on Intelligent Systems for Communication, IoT and Security (ICISCoIS), IEEE, 2023, pp. 671–675.
  * [94] J. Cho, M. K. Baskar, R. Li, M. Wiesner, S. H. Mallidi, N. Yalta, M. Karafiat, S. Watanabe, T. Hori, Multilingual sequence-to-sequence speech recognition: architecture, transfer learning, and language modeling, in: 2018 IEEE Spoken Language Technology Workshop (SLT), IEEE, 2018, pp. 521–527.
  * [95] K. Li, Y. Song, I. McLoughlin, L. Liu, J. Li, L.-R. Dai, Fine-tuning audio spectrogram transformer with task-aware adapters for sound event detection.
  * [96] C. Wang, S. Dai, Y. Wang, F. Yang, M. Qiu, K. Chen, W. Zhou, J. Huang, Arobert: An asr robust pre-trained language model for spoken language understanding, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2022).
  * [97] Y. Himeur, I. Varlamis, H. Kheddar, A. Amira, S. Atalla, Y. Singh, F. Bensaali, W. Mansoor, Federated learning for computer vision, arXiv preprint arXiv:2308.13558 (2023).
  * [98] D. Dimitriadis, R. G. Ken’ichi Kumatani, R. Gmyr, Y. Gaur, S. E. Eskimez, A federated approach in training acoustic models., in: Interspeech, 2020, pp. 981–985.
  * [99] D. Guliani, F. Beaufays, G. Motta, Training speech recognition models with federated learning: A quality/cost framework, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 3080–3084.
  * [100] H. Zhu, J. Wang, G. Cheng, P. Zhang, Y. Yan, Decoupled federated learning for asr with non-iid data, arXiv preprint arXiv:2206.09102 (2022).
  * [101] X. Cui, S. Lu, B. Kingsbury, Federated acoustic modeling for automatic speech recognition, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 6748–6752.
  * [102] T. Nguyen, S. Mdhaffar, N. Tomashenko, J.-F. Bonastre, Y. Estève, Federated learning for asr based on wav2vec 2.0, in: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2023, pp. 1–5.
  * [103] C.-H. H. Yang, J. Qi, S. Y.-C. Chen, P.-Y. Chen, S. M. Siniscalchi, X. Ma, C.-H. Lee, Decentralizing feature extraction with quantum convolutional neural network for automatic speech recognition, in: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2021, pp. 6523–6527.
  * [104] Y. Gao, T. Parcollet, S. Zaiem, J. Fernandez-Marques, P. P. de Gusmao, D. J. Beutel, N. D. Lane, End-to-end speech recognition from federated acoustic models, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 7227–7231.
  * [105] H. Mehmood, A. Dobrowolska, K. Saravanan, M. Ozay, Fednst: Federated noisy student training for automatic speech recognition, arXiv preprint arXiv:2206.02797 (2022).
  * [106] J. C. Vásquez-Correa, A. Álvarez Muniain, Novel speech recognition systems applied to forensics within child exploitation: Wav2vec2. 0 vs. whisper, Sensors 23 (4) (2023) 1843.
  * [107] C. Tan, D. Jiang, H. Mo, J. Peng, Y. Tong, W. Zhao, C. Chen, R. Lian, Y. Song, Q. Xu, Federated acoustic model optimization for automatic speech recognition, in: Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24–27, 2020, Proceedings, Part III 25, Springer, 2020, pp. 771–774.
  * [108] N. Tomashenko, S. Mdhaffar, M. Tommasi, Y. Estève, J.-F. Bonastre, Privacy attacks for automatic speech recognition acoustic models in a federated learning framework, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 6972–6976.
  * [109] D. Guliani, L. Zhou, C. Ryu, T.-J. Yang, H. Zhang, Y. Xiao, F. Beaufays, G. Motta, Enabling on-device training of speech recognition models with federated dropout, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 8757–8761.
  * [110] C. Chen, Y. Hu, N. Hou, X. Qi, H. Zou, E. S. Chng, Self-critical sequence training for automatic speech recognition, in: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2022, pp. 3688–3692.
  * [111] T. Kala, T. Shinozaki, Reinforcement learning of speech recognition system based on policy gradient and hypothesis selection, in: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2018, pp. 5759–5763.
  * [112] A. Tjandra, S. Sakti, S. Nakamura, Sequence-to-sequence asr optimization via reinforcement learning, in: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2018, pp. 5829–5833.
  * [113] A. Tjandra, S. Sakti, S. Nakamura, End-to-end speech recognition sequence training with reinforcement learning, IEEE Access 7 (2019) 79758–79769.
  * [114] Ł. Dudziak, M. S. Abdelfattah, R. Vipperla, S. Laskaridis, N. D. Lane, Shrinkml: End-to-end asr model compression using reinforcement learning, arXiv preprint arXiv:1907.03540 (2019).
  * [115] A. Mehrotra, Ł. Dudziak, J. Yeo, Y.-y. Lee, R. Vipperla, M. S. Abdelfattah, S. Bhattacharya, S. Ishtiaq, A. G. C. Ramos, S. Lee, et al., Iterative compression of end-to-end asr model using automl, arXiv preprint arXiv:2008.02897 (2020).
  * [116] Y.-L. Shen, C.-Y. Huang, S.-S. Wang, Y. Tsao, H.-M. Wang, T.-S. Chi, Reinforcement learning based speech enhancement for robust speech recognition, in: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 6750–6754.
  * [117] R. T.-H. Tsai, C.-H. Chen, C.-K. Wu, Y.-C. Hsiao, H.-y. Lee, Using deep-q network to select candidates from n-best speech recognition hypotheses for enhancing dialogue state tracking, in: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, 2019, pp. 7375–7379.
  * [118] H. Chung, H.-B. Jeon, J. G. Park, Semi-supervised training for sequence-to-sequence speech recognition using reinforcement learning, in: 2020 international joint conference on neural networks (IJCNN), IEEE, 2020, pp. 1–6.
  * [119] Z. Chen, W. Zhang, End-to-end speech recognition with reinforcement learning, in: Eighth International Conference on Electronic Technology and Information Science (ICETIS 2023), Vol. 12715, SPIE, 2023, pp. 392–398.
  * [120] A. Hamza, D. Addou, H. Kheddar, Machine learning approaches for automated detection and classification of dysarthria severity, in: 2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM), Vol. 1, IEEE, 2023, pp. 1–6.
  * [121] S. Feng, B. M. Halpern, O. Kudina, O. Scharenborg, Towards inclusive automatic speech recognition, Computer Speech & Language 84 (2024) 101567.
  * [122] Z. Zhou, J. Chen, N. Wang, L. Li, D. Wang, Adversarial data augmentation for robust speaker verification, arXiv preprint arXiv:2402.02699 (2024).
  * [123] Adastreamlite: Environment-adaptive streaming speech recognition on mobile devices, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7 (4) (2024) 1–29.
  * [124] J. H. Yeo, M. Kim, J. Choi, D. H. Kim, Y. M. Ro, Akvsr: Audio knowledge empowered visual speech recognition by compressing audio knowledge of a pretrained model, IEEE Transactions on Multimedia (2024).
  * [125] N. Djeffal, D. Addou, H. Kheddar, S. A. Selouani, Noise-robust speech recognition: A comparative analysis of lstm and cnn approaches, in: 2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM), Vol. 1, IEEE, 2023, pp. 1–6.
  * [126] Q. Zhao, L. Yang, N. Lyu, A driver stress detection model via data augmentation based on deep convolutional recurrent neural network, Expert Systems with Applications 238 (2024) 122056.
  * [127] Z. Jin, M. Geng, J. Deng, T. Wang, S. Hu, G. Li, X. Liu, Personalized adversarial data augmentation for dysarthric and elderly speech recognition, IEEE/ACM Transactions on Audio, Speech, and Language Processing (2023).
  * [128] A. Brack, E. Entrup, M. Stamatakis, P. Buschermöhle, A. Hoppe, R. Ewerth, Sequential sentence classification in research papers using cross-domain multi-task learning, International Journal on Digital Libraries (2024) 1–24.
  * [129] H. Zhang, M. Tao, Y. Shi, X. Bi, K. B. Letaief, Federated multi-task learning with non-stationary and heterogeneous data in wireless networks, IEEE Transactions on Wireless Communications (2023).
  * [130] A. Singh, S. Chandrasekar, T. Sen, S. Saha, Federated multi-task learning for complaint identification using graph attention network, IEEE Transactions on Artificial Intelligence (2023).
  * [131] X. Jiang, J. Zhang, L. Zhang, Fedradar: Federated multi-task transfer learning for radar-based internet of medical things, IEEE Transactions on Network and Service Management (2023).
  * [132] B. Azadi, M. Haslgrübler, B. Anzengruber-Tanase, G. Sopidis, A. Ferscha, Robust feature representation using multi-task learning for human activity recognition, Sensors 24 (2) (2024) 681.
  * [133] J. Ji, Z. Shu, H. Li, K. X. Lai, M. Lu, G. Jiang, W. Wang, Y. Zheng, X. Jiang, Edge-computing based knowledge distillation and multi-task learning for partial discharge recognition, IEEE Transactions on Instrumentation and Measurement (2024).
  * [134] R. Šajina, N. Tanković, I. Ipšić, Multi-task peer-to-peer learning using an encoder-only transformer model, Future Generation Computer Systems 152 (2024) 170–178.
  * [135] C. Ye, H. Zheng, Z. Hu, M. Zheng, Pfedsa: Personalized federated multi-task learning via similarity awareness, in: 2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS), IEEE, 2023, pp. 480–488.
  * [136] Z. Wang, C. Chen, D. Dong, Lifelong incremental reinforcement learning with online bayesian inference, IEEE Transactions on Neural Networks and Learning Systems 33 (8) (2021) 4003–4016.
  * [137] J. Wang, J. Cao, S. Wang, Z. Yao, W. Li, Irda: Incremental reinforcement learning for dynamic resource allocation, IEEE Transactions on Big Data 8 (3) (2020) 770–783.
  * [138] Z. Wang, C. Chen, H.-X. Li, D. Dong, T.-J. Tarn, Incremental reinforcement learning with prioritized sweeping for dynamic environments, IEEE/ASME Transactions on Mechatronics 24 (2) (2019) 621–632.
  * [139] A. Gueriani, H. Kheddar, A. C. Mazari, Deep reinforcement learning for intrusion detection in iot: A survey, in: 2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM), Vol. 1, IEEE, 2023, pp. 1–7.
  * [140] P. Zhao, S. C. Hoi, J. Wang, B. Li, Online transfer learning, Artificial intelligence 216 (2014) 76–102.
  * [141] Q. Wu, H. Wu, X. Zhou, M. Tan, Y. Xu, Y. Yan, T. Hao, Online transfer learning with multiple homogeneous or heterogeneous sources, IEEE Transactions on Knowledge and Data Engineering 29 (7) (2017) 1494–1507.
  * [142] H. Wu, Y. Yan, Y. Ye, H. Min, M. K. Ng, Q. Wu, Online heterogeneous transfer learning by knowledge transition, ACM Transactions on Intelligent Systems and Technology (TIST) 10 (3) (2019) 1–19.
  * [143] R. Alasbahi, X. Zheng, An online transfer learning framework with extreme learning machine for automated credit scoring, IEEE Access 10 (2022) 46697–46716.  doi:10.1109/ACCESS.2022.3171569.

  *[SOTA]: state-of-the-art
  *[AI]: artificial intelligence
  *[HMI]: human-machine interaction
  *[ASR]: automatic speech recognition
  *[AM]: acoustic model
  *[SM]: semantic model
  *[LM]: language model
  *[NLP]: natural language processing
  *[ED]: event detection
  *[DL]: deep learning
  *[S2S]: sequence-to-sequence
  *[DTL]: deep transfer learning
  *[DRL]: deep reinforcement learning
  *[FL]: federated learning
  *[SS]: speech security
  *[LPC]: linear predictive coding
  *[TPR]: true positive rate
  *[TNR]: true negative rate
  *[WER]: word error rate
  *[CER]: character error rate
  *[PESQ]: perceptual evaluation of speech quality
  *[RTF]: real-time factor
  *[RER]: relative error rate
  *[LMs]: language model
  *[CTC]: connectionist temporal classification
  *[DA]: domain adaptation
  *[CV]: computer vision
  *[MHSA]: multi-head self-attention
  *[CNN]: convolutional neural network
  *[FCF]: feature correlation-based fusion
  *[SSAST]: self-supervised audio spectrogram transformer
  *[ML]: machine learning
  *[AMs]: acoustic model
  *[AST]: audio spectrogram transformer
  *[AT]: audio tagging
  *[WSJ]: wall street journal
  *[SWBD]: switchboard
  *[WERs]: word error rate
  *[CS]: code-switching
  *[RNN]: recurrent neural network
  *[LSTM]: long short term memory
  *[CNNs]: convolutional neural network
  *[RNNs]: recurrent neural network
  *[mAP]: mean average precision
  *[APT]: audio pyramid transformer
  *[CAFT]: client adaptive federated training
  *[FedNST]: federated noisy student training
  *[RL]: reinforcement learning
  *[FMTL]: federated multi-task learning
  *[SD]: source domain
  *[TD]: target domain
  *[DDQN]: double deep Q-network
  *[AC]: actor-critic
  *[DDPG]: deep deterministic policy gradien
  *[LLM]: large language model