# Domain Adapting Deep Reinforcement Learning for Real-world Speech Emotion
Recognition

Thejan Rajapakshe, Rajib Rana, Sara Khalifa,
and Björn W. Schuller T. Rajapakshe is affiliated with University of Southern
Queensland, Australia.
e-mail<EMAIL_ADDRESS>R. Rana is affiliated with University of
Southern Queensland, Australia. S. Khalifa is affiliated with Distributed
Sensing Systems Group, Data61, CSIRO Australia. B. Schuller is Chair of
Embedded Intelligence for Health Care & Wellbeing, University of Augsburg,
Germany and further affiliated with GLAM – Group on Language, Audio, & Music,
Imperial College London, UK

###### Abstract

Computers can understand and then engage with people in an emotionally
intelligent way thanks to speech-emotion recognition (SER). However, the
performance of SER in cross-corpus and real-world live data feed scenarios can
be significantly improved. The inability to adapt an existing model to a new
domain is one of the shortcomings of SER methods. To address this challenge,
researchers have developed domain adaptation techniques that transfer
knowledge learnt by a model across the domain. Although existing domain
adaptation techniques have improved performances across domains, they can be
improved to adapt to a real-world live data feed situation where a model can
self-tune while deployed. In this paper, we present a deep reinforcement
learning-based strategy (RL-DA) for adapting a pre-trained model to a real-
world live data feed setting while interacting with the environment and
collecting continual feedback. RL-DA is evaluated on SER tasks, including
cross-corpus and cross-language domain adaption schema. Evaluation results
show that in a live data feed setting, RL-DA outperforms a baseline strategy
by 11 % and 14 % in cross-corpus and cross-language scenarios, respectively.

###### Index Terms:

Reinforcement Learning, Speech Emotion Recognition, Domain Adaptation

##  1 Introduction

Emotion-aware interaction can significantly improve human-computer
interaction. Consequently, there has been much research on automatic speech
emotion recognition (SER). While SER within the same corpus is possible with
reasonable accuracy, cross-corpus SER performance needs significant
improvement [1, 2, 3]. Cross-corpus SER is crucial to achieving emotion-aware
interactions in real-life applications, as speech signals obtained from
different devices, recording background, spoken language, and acoustic signal
conditions can be different with the training dataset and real-world
implementation [1].

Researchers have explored domain adaptation techniques to improve the cross-
corpus SER performance [4, 5, 6, 7]. Domain adaptation is a transfer learning
approach where a trained model undergoes training aiming to optimise for a
different distribution other than the previously trained distribution. The key
limitation of the existing domain adaptation methods is that they cannot be
adapted to the dynamic nature of the real-life setting. In such a setting, it
will be beneficial if an intelligent agent interacting with a customer/user
could dynamically update itself when it wrongly categorises speech emotion. In
this paper, we address this challenge. We develop a domain adaptation
technique based on reinforcement learning (RL), where we exploit the
dynamically updating feature of RL to develop a cross-corpus SER technique
that can be dynamically updated.

Reinforcement Learning has been used to optimise real-world interactions with
robots and machines after training in a simulated environment [8, 9], however,
it has not been used in the field of SER. In a typical RL scenario, an agent
takes actions in an environment, which is interpreted into a reward and a
representation of the state, which are fed back into the agent. This does not
trivially match a domain adaptation scenario of SER. To the best of our
knowledge, we are the first to develop and evaluate techniques for using RL
for this task.

A possible use case of the proposed technique is shown in Figure 1. A base
deep learning model can be pre-trained with a labelled source dataset before
starting Reinforcement Learning based Domain Adaptation (RL-DA). Then, the
pre-trained model is optimised to the target domain with RL. During this
optimisation, the data feed can come from a live data source, where the model
gets feedback from the customer regarding its interpretation of emotion in the
customer’s voice. Feedback can, for example, be implemented by a thumbs up or
down button on the customer console. An example application demonstrating the
above use case can be found online111https://rlemotion.cloud.edu.au. This
application consumes recorded audio and infers the emotion from that audio
utterance while users are able to provide feedback on the inferred emotion.
This feedback is used for RL optimisation.

Figure 1: Overall architecture of using reinforcement learning in domain
adaptation for the speech emotion recognition task

Using widely used speech corpora, we evaluate the model in cross-corpus and
cross-language scenarios and show that we achieve better SER performance than
fully supervised benchmark models. Moreover, we simulate a real-life (live)
data feed scenario and show that our proposed RL model significantly
outperforms fully supervised benchmark models.

##  2 Literature Review

Deep Reinforcement Learning (Deep RL), a combination of RL and deep neural
networks (DNN), emerged as deep learning has advanced. Deep RL is mostly
applied in gaming [10, 11, 12], recommendation systems [13], and robotics
[14]. However, the use of RL for speech-based applications is slowly getting
traction[15].

In this section, we discuss the existing literature in two groups: one group
combines literature on Domain Adaptation in SER, and the other group
consolidates the previous work done with RL for domain adaptation.

###  2.1 Domain Adaptation

Domain adaptation was introduced to overcome the “corpus bias” issue. Since
deep learning methods are powerful in extracting non-linear features in the
input, domain adaptation can easily be implemented on deep learning-based
platforms, yielding more robust and performing models [16, 17].

TABLE I: Summary and focus on the literature on RL for domain adaptation and speech emotion classification Paper | Focus
---|---
Reinforcement Learning |  Domain Adaptation |  Speech Emotion Recognition
Leonetti et al, 2016[18] | ✓ | ✓ | ✗
Mao et al, 2016[4] | ✗ | ✓ | ✓
Carr et al, 2018[12] | ✓ | ✓ | ✗
Lakomkin et al, 2018[19] | ✓ | ✗ | ✓
Hossain and Muhammad, 2019[20] | ✗ | ✗ | ✓
Koo et al, 2019[21] | ✓ | ✓ | ✗
Arndt et al, 2020[22] | ✓ | ✓ | ✗
Ahn et al, 2021[23] | ✗ | ✓ | ✓
This paper | ✓ | ✓ | ✓

Mao et al. experimented on a two-class task for domain adaptation by sharing
priors between source and target domains. They pre-trained a two-layer neural
network with unsupervised learning and then shared the common classifier
parameters between source and target [4]. Our approach follows the RL paradigm
for domain adaptation instead of unsupervised learning, and we pre-train the
source model with a labelled source dataset.

Gharib et al. proposed unsupervised adversarial domain adaptation and pre-
trained the model with two conditional sets achieving nearly 10 % increase in
accuracy [24].

Ahn et al. proposed a Few-shot learning methodology in unsupervised domain
adaptation for cross-corpus SER tasks[23]. They used multiple corpora to
optimise the emotion recognition robustness to unseen samples.

Abdelwahab and Busso studied the best adaptation technique for speech emotion
recognition models out of supervised domain adaptation and online training.
They also studied how the size of the labelled data affects the performance of
the resulting model [25].

One stream of research on domain adaptation is on Adversarial Domain
Adaptation [26, 27, 28]. Latif et al. proposed using adversarial training
techniques to improve cross-lingual domain adaption [29]. This is an
unsupervised learning method that utilises two distinct auto-encoders for
source and target, as well as a single discriminator, to determine whether an
audio utterance is fake or real. Although we focus on cross-lingual domain
adaptation, our main target is enabling domain adaptation in real-world
applications.

###  2.2 Reinforcement Learning for Domain Adaptation

The RL agent gains knowledge via experiences, and these experiences are
obtained by exploring and exploiting. The RL agent uses the policies to choose
the best action given the present state – the method is called exploitation.
Since the RL agent does not have enough experience to update the policy in the
initial phase, it explores the environment. Exploration is done by randomly
selecting actions without considering the policy, and the experience is stored
in the memory buffer with a reward. The main challenge in this exploration is
the unsafe choice of actions in some states [22].

Prior knowledge can be fed to the RL agent before releasing it to the real
world in many ways. Demonstration based learning [30, 31], and pre-training
and domain adaptation [22] are some of the methods to nourish the RL agent
with knowledge. This reduces the amount of exploration time and the risk of
unsafe exploration. Previous studies also have shown that pre-training has
reduced the training time to achieve higher performance[32, 33, 34].

Koo et al. consider the feature extractor for the target domain as a generator
where the parameters are similar to the latent feature set in the source and
are trained with adversarial training loss. They proposed this to enhance the
coherent training of the target domain feature extractor [21]. Though their
research proposes an RL-based approach in domain adaptation, they have not
studied speech emotion recognition.

Leonetti et al. have introduced a new RL algorithm, “DARLING”, which can adapt
to the new environment and improve the reliability of the decisions [18].

Hazara and Kyrki proposed a transfer approach that captures a simulated
system’s core features and rationalises the dynamics combined with incremental
learning. They demonstrated their approach by applying it to a robot with a
basketball task and showed the target model has improved task generalisation
capability over direct usage[35].

In summary:

  1. 1.

Literature on domain adaptation for SER is very rich, many approaches involve
deep learning techniques.

  2. 2.

Very few studies involve RL for speech emotion recognition tasks such as EmoRL
[19]. But there is lack of research in the literature that uses RL for domain
adaptation for SER.

  3. 3.

Table I provides a brief overview of the literature on RL for domain
adaptation and speech emotion classification, which attests that there has
been a lack of literature that combines RL with DA for the problem of SER,
which is the focus of this paper.

##  3 Methodology

###  3.1 Background

Reinforcement Learning follows the principle of behaviourist psychology and
learns similar to a child who learns to perform a new task. There are two main
components in an RL problem: the Environment and the Agent. The agent performs
actions on an environment, and the environment returns a reward regarding the
action performed along with the next state of the environment. Figure 2 shows
the concise architecture of general RL methodology.

Figure 2: Deep Q network in Reinforcement Learning architecture

A Markov Decision Process can interpret an RL problem, which has the
parameters state space $S$, action space $A$, and reward $R$. A state $s\in S$
is the representation of the current environment, action $a\in A$ is a single
task that can be performed on the environment, and reward $r\in R$ is the
feedback value returned from the environment after executing the task $a$ for
the state $s$. RL agent learns a policy $\pi(a,s)$ that represents a
probability of taking action $a$ for a specific state $s$. The Q-value or
quality value represents the expected reward when action $a$ is performed on
an environment with state $s$. The optimum Q-value $Q^{*}(a,s)$ can be written
as in Equation 1;

$Q^{*}_{\pi}(s,a)=max\mathbb{E}_{\pi}[R_{t+1}+\gamma
q_{\pi}(S_{t+1},A_{t+1})].$ (1)

Q-values are estimated using Q-tables in q-learning, while a deep neural
network called Deep Q Network (DQN) is used to estimate the Q values in deep
q-learning. The loss function in Equation 2 is used to calculate the loss and
$Q_{target}$ can be as of Equation 3.

$L=\Sigma{(Q_{target}-Q)^{2}},$ (2) $Q_{target}=R(s_{t+1},a_{t+1})+\gamma
Q(s_{t+1},a_{t+1}).$ (3)

Combining the two equations 2 and 3, the loss function can be re-written as in
Equation 4.

$L=\Sigma{(R(s_{t+1},a_{t+1})+\gamma Q(s_{t+1},a_{t+1})-Q(s_{t},a_{t}))^{2}}.$
(4)

The loss $L$ should be minimised in the training phase of RL.

###  3.2 Domain Adapting Reinforcement Learning for Speech Emotion Recognition

In this study, RL is used as an emotion recognising game where an RL agent
attempts to recognise the correct emotion (action $a$) for a given audio
utterance (state $s$). A reward $r$ is returned by the environment. The reward
value is obtained either by the feedback from the user in a real-world
scenario or by comparing the ground truth and inferred emotion by the RL agent
when training with labelled datasets. A policy $\pi$ is learnt by the RL agent
to maximise the reward gained at each episode. The state space $S$ is defined
as the distribution of speech audio, the Action space $A$ is defined as the
discrete emotion classes, and action selection is the inference of an emotion
to a given audio utterance. According to Equation 4 to minimise $L$,
$Q(s_{t},a_{t})$ should be maximised whereas the Reward $R_{t+1}$ should be
maximal at $L^{*}$.

Pre-training the DQN model prior to utilising an RL agent has proven to
increase the performance of the model [33, 34]. Here, RL is used to optimise
the pre-trained model to the target domain. The RL agent with a pre-trained
DQN model interacts with the environment and tries to optimise the DQN to
adapt to the target domain.

###  3.3 Baseline

Though RL methodologies do not report performance based on accuracy since
there is no labeled data associated with RL experiments, we are using a
supervised learning (SL) approach as the baseline to compare the performance
in our methodology. We use a DNN architecture similar to the DNN used in the
RL agent (Deep Q Network in Figure 2). We first train the model with the
Source Dataset to be aligned with the pre-training in the RL approach and
obtain a Base Model as in Figure 1. Then, the Base Model is trained with the
Target dataset to match with the RL-DA. This allows us to compare the two
approaches RL-DA and Supervised Learning-based Domain Adaptation, with similar
parameters. We use four labelled datasets, and section 4.1 describes the
datasets used in this study. A fraction of 20 % of each dataset is reserved as
a testing dataset, and the remaining subsection is used for either pre-
training or RL domain optimisation. Section 4 describes the different kinds of
experiments carried out within this study to see the effectiveness of our
approach.

##  4 Experimental Setup

###  4.1 Datasets

To measure the performance of the proposed RL based Domain Adaptation (RL-DA)
approach, we use four widely used publicly available datasets for our
experiments: MSP-IMPROV [36], IEMOCAP [37], ESD [38] and EmoDB [39]. The
selection of datasets covers the ability to use them for Cross-Corpus ($CC$)
experiments as well as Cross-Language ($CL$) experiments.

Since the datasets do not contain a balanced number of utterances under each
emotion class, we use an audio augmentation technique to generate audio
utterances. We use Vocal Tract Length Perturbation (VLTP) [40] to augment
audio utterances balancing utterances under each emotion class when preparing
the testing subsets. All the four speech emotion datasets we use contain
categorical emotional labels, and we use four emotions widely used in the
literature: ‘happiness’, ‘sadness’, ‘anger’, and ‘neutral’.

####  4.1.1 IEMOCAP (IEM)

IEM is a popular dataset containing 12 hours of multi-speaker acted audio-
visual data. The IEMOCAP dataset contains dyadic sessions with both improvised
and scripted scenarios. It contains categorical emotion labels happiness,
sadness, anger, and neutrality as well as dimensional labels valence,
dominance, and activation [37]. This study uses the audio data modality from
improvised scenarios and categorical labels.

####  4.1.2 MSP-IMPROV (MSP)

MSP is an acted audio-visual database that is popular in multi-modal speech
emotion recognition research. The database was initially developed for a study
in audio-visual emotional perception, but this dataset has also been used in
speech emotion recognition studies [41, 42, 23, 43] which makes it a candidate
for our study as well. The dataset was recorded in a controlled environment
and consists of $20$ determined scripts covering the primary four emotions:
happiness, sadness, anger, and neutral [36].

####  4.1.3 Emotional Speech Dataset (ESD)

ESD is a public database of speech data that was initially developed for
speech synthesis and voice conversion. It contains utterances from $20$
speakers in two languages (10 Mandarin and 10 English) categorically labelled
into five emotion classes: happy, surprise, neutral, angry, and sad [38]. We
use utterances from $10$ English speakers under the four emotions happiness,
sadness, anger, and neutral.

####  4.1.4 Berlin EmoDB (EmoDB)

EmoDB is an actor recorded database of emotional speech utterances in the
German language. $10$ actors ($5$ male and $5$ female) in the age range
between $21$ years and $35$ years speak $10$ scripted texts in $7$ different
emotions such as anger, boredom, disgust, fear, happiness, sadness, and
neutral [39]. We use EmoDB as the target dataset for cross-language
experiments. We choose the utterances of the four emotion classes: anger,
happiness, sadness, and neutral.

####  4.1.5 Diverse Environments Multichannel Acoustic Noise Database (DEMAND)

The DEMAND dataset is a popular real-world background noise dataset. The
database contains background noises from $18$ different environments under six
categories. An array of $16$ microphones is used to record the audio in each
environment stored in $16$ channels. The DEMAND dataset contains versions of
both $16$ kHz and $48$ kHz sampling rates, and we use $48$ kHz and a down-
sampled to $22$ kHz version to match with the other datasets used in this
study.

A summary of the datasets used in this study is shown in Table II

TABLE II: Summary of the datasets used in this study. The ‘# Classes‘ column contains the total number of classes in the original datasets irrespective of the four emotions (‘happiness’, ‘sadness’, ‘anger’, and ‘neutral’) we used in our study. Dataset | # Training | # Testing | # Classes | Language
---|---|---|---|---
IEMOCAP | 2780 | 852 | 4 | English
MSP-Improv | 432 | 148 | 4 | English
ESD | 7744 | 2428 | 5 | English
EmoDB | 344 | 100 | 7 | German

###  4.2 Input feature

We use Mel Frequency Cepstral Coefficients (MFCC) as the input features [44,
45]. We use $2,048$ as the frame length and the hop length as $512$ and
extract $40$ MFCCs using the Librosa python library for audio and music
analysis [46].

###  4.3 Model configuration

The functionality of the DNN model is to classify the emotion embedded in
speech utterances, and the study uses MFCCs of speech utterances as the input
for the model. We use a Convolutional Neural Network (CNN) and Long Short-Term
Memory (LSTM) combined model since it motivates the capability to learn both
frequency and temporal components in the speech signal [47, 34]. We stack CNN,
LSTM, and fully connected layers, respectively, for the model to learn the
discriminative features [45].

As shown in Figure 2, the feature matrix is fed into two layers of 2D
convolution layers of filter sizes $5$ and $3$, and the output of the first 2D
convolution layer is batch normalised. Hyperparameter batch size is set to
$128$. The output from the second 2D convolution layer is passed to the LSTM
layer of $16$ cells after flattening and then to a fully connected layer of
$256$ units. A dropout rate of $0.3$ is applied before the last Dense output
layer. The number of units in the output layer is four, equal to the number of
emotions to be classified. The linear activation function of the output layer
is used when the model is used in the RL agent since it outputs Q-values for a
specific state, and they should not be normalised. The input shape of the
model is $40\times 87$, where $40$ MFCCs are used in the input and $87$ is the
number of MFCC frames.

The model is optimised using the Adam Optimiser with a learning rate of
$2.5\times 10^{-4}$. We use the popular Deep Learning API Keras [48] with
Tensorflow [49] (version 2.1.0) as the back-end for modelling and training
purposes in this study.

###  4.4 Experiments

We design and run several experiments to evaluate the proposed RL based domain
adaptation approach. For evaluation, we develop three scenarios. (1) Pre-
training with a source dataset and domain optimisation with a target dataset
separately; (2) Pre-training with a subset of the source dataset and domain
optimisation with a mix of the remaining source and target datasets; (3)
Domain adaptation in a recreated real-world scenario. Details of each scenario
and results are explained in Sections 5.1, 5.2, and 5.3 respectively.

The Unweighted Average Recall rate (UAR %) is one of the popular metrics used
by researchers to measure the performance of speech based machine learning
models. The recall is calculated for each label of the classification problem,
and the UAR is calculated by taking the unweighted average of the recall
values of each label

The Python code repository of the experiments and data preparation is hosted
in public GitHub repository “RL-
DomainAdaptation”222https://github.com/jayaneetha/RL-DomainAdaptation

##  5 Evaluations

This section reports the results and description obtained through the
experiments in section 4. We use the IEMOCAP and MSP-Improv datasets as source
datasets, while the ESD dataset is used as the target dataset in the cross-
corpus schema, and EmoDB is used as the target dataset in the cross-language
schema. We use Max Boltzman Policy (Max.B)[50] as the RL policy in the RL
agent.

###  5.1 Experiment with separate datasets for source and target

Here, we evaluate the proposed RL-DA approach for separate source and target
datasets. We do this for both the cross-corpus and cross-language schemas.
Table III and Figure 3 (a) show the datasets used for each schema.

TABLE III: Summary of the datasets used in the cross-corpus (CC) and cross-language (CL) experiments. (The Target dataset is not mixed with Source data) | Training Dataset | Testing Dataset
---|---|---
| Source | Target
CC | IEM | ESD | IEM + ESD
MSP | ESD | MSP + ESD
CL | IEM | EmoDB | IEM + EmoDB
MSP | EmoDB | MSP + EmoDB
Figure 3: Composition of datasets used in Target and Source: (a) the Target
dataset is not mixed with Source data and (b) the Target dataset is mixed with
Source data. The source dataset is used to pre-train the base model while the
target dataset is used for domain adaptation.

First, the model is pre-trained using the training subset of the source
dataset, obtaining the Base model. Then,  the pre-trained base model
parameters are transferred onto the DQN of the RL agent. Afterwards, the RL
deep Q learning algorithm is executed to optimise the DQN model applying the
target dataset in the environment which outputs the domain optimised DNN
model. To measure the performance of the domain optimised model, we use a
Supervised Learning approach. We take the testing subset of the Target dataset
and infer the emotion of each testing utterance and record the inference
emotion with the labelled emotion for the utterance. The UAR value for each
experiment is calculated using the recorded data about inference emotions and
ground truth.

Supervised Learning approach based performance is also measured to compare the
performance of our methodology as the baseline (SL-DA - sep). We create a DNN
model that is similar in architecture as used in DQN (the Deep Q Network in
Figure 2) and is pre-trained by the Source dataset. Then, the same model is
trained by the training subset of the Target dataset. Subsequently, the
resulting model is tested with the testing subset of the Target dataset to
obtain the testing accuracy.

The UAR of the baseline (SL-DA - sep) model after training with supervised
learning and the UAR of the model after training with the RL-DA approach (RL-
DA) are reported in Table IV.

TABLE IV: Comparison of performance between the Supervised Learning based (SL-DA - sep) and the Reinforcement Learning based Domain Adaptation approach (RL-DA) for the cross-corpus (CC) and cross-language (CL) experiments, where the Target dataset was not mixed with Source data | Dataset | UAR(%)
---|---|---
| Source | Target | SL-DA - sep | RL-DA
CC | IEM | ESD | 63.54 $\pm$ 2.11 | 63.99 $\pm$ 1.95
MSP | ESD | 63.18 $\pm$ 3.34 | 66.10 $\pm$ 0.84
CL | IEM | EmoDB | 74.67 $\pm$ 1.03 | 73.17 $\pm$ 0.62
MSP | EmoDB | 56.67 $\pm$ 8.26 | 65.50 $\pm$ 0.71

We find that the accuracy of RL-DA surpasses the baseline UAR by 0.45 %, 2.92
%, and 8.83 %, respectively, except for in the IEM-EmoDB combination.
Additionally, while comparing the SL-DA method and the RL-DA method, the
cross-language schema’s standard deviation has decreased from 8.26 to 0.71,
respectively, demonstrating that the model performances are more consistent.

This outcome is noteworthy because, unlike SL approaches, which feed labelled
data to train the model, the RL methods only provide feedback rewards that
show the accuracy of inference made during the RL optimisation phase. So,
achieving on par SL accuracy also means that the method is successful in
training the model to the SL standard.

We calculate UAR to compare the performance of our models. We use the model
architecture, source, and the target dataset as the parameters of similarity
in the literature on cross-corpus speech emotion recognition to compare our
results. The results are represented in Table V.

TABLE V: Comparison of UAR (%) between the Reinforcement Learning based Domain Adaptation approach (RL-DA) and UAR (%) presented in the SER Literature in a similar setup (same source and target datasets) for the cross-corpus (CC) and cross-language (CL) experiment where the Target dataset was not mixed with Source data | Dataset | UAR (%)
---|---|---
| Source | Target | RL-DA | Literature
CC | IEM | ESD | $63.99\pm 1.95$ | -
MSP | ESD | $66.10\pm 0.84$ | -
CL | IEM | EmoDB | $73.17\pm 0.62$ | 62.91[51]
MSP | EmoDB | $65.50\pm 0.71$ | -

###  5.2 Experiment mixing the source and target datasets

This experiment is carried out to evaluate our proposed methodology while
including a subset of the source dataset in the RL optimisation phase. The aim
of such mixing is to preserve generalisable features of the source dataset.

This experiment is performed similar to the previous experiment mentioned in
5.1, but with the following modifications: Only a subset of the training set
of the source dataset is used in pre-training, and a mix of the remaining
training set of the source dataset and the training set of the target dataset
is used in the RL optimisation. Table VI and Figure 3 (b) show how the
datasets are used in this experiment. We measure the performance by using SL
as a baseline (SL-DA - mix) to compare with our proposed approach.

TABLE VI: Composition of datasets used in Target and Source when experimenting by mixing datasets in the cross-corpus (CC) and cross-language (CL) schemata | Training Dataset | Testing Dataset
---|---|---
| Source | Target
CC | 50% IEM | 50% IEM + ESD | IEM + ESD
50% MSP | 50% MSP + ESD | MSP + ESD
CL | 50% IEM | 50% IEM + EmoDB | IEM + EmoDB
50% MSP | 50% MSP + EmoDB | MSP + EmoDB

Results of the experiment by mixing source data into the target dataset as
mentioned above are presented in Table VII. The accuracy of the RL-DA approach
is compared with the Supervised Learning approach – SL-DA - mix.

TABLE VII: Comparison of performance between Supervised Learning based (SL-DA - mix) and Reinforcement Learning based Domain Adaptation approach (RL-DA) in the experiment with mixed target datasets for cross-corpus (CC) and cross-language (CL) schemata. | Dataset | UAR(%)
---|---|---
| Source | Target | SL-DA - mix | RL-DA
CC | IEM | IEM + ESD | 56.22 $\pm$ 1.15 | 77.86 $\pm$ 0.43
MSP | MSP + ESD | 55.52 $\pm$ 0.97 | 63.51 $\pm$ 2.40
CL | IEM | IEM + EmoDB | 61.33 $\pm$ 5.44 | 85.67 $\pm$ 2.72
MSP | MSP + EmoDB | 59.17 $\pm$ 1.43 | 77.33 $\pm$ 1.43

The RL-DA approach outperforms SL-DA - mix at least by 8 % in each experiment.
Since the target dataset includes components from the source dataset, the
representations of the model learnt during the pre-training stage are kept
while the model adapts to the new domain.

We compare the performance of RL-DA in two scenarios where the target dataset
is separated (S) with the source dataset (Section 5.1), and the target dataset
is mixed (M) with utterances from the source dataset (Section 5.2) and
visualised in Figure 4.

Figure 4: Comparison of accuracy of each RL-DA experiment with separate
datasets vs mixed datasets

From the above visualisation, we note that the mixing of source and target
datasets yields better performance than experiments with separate datasets
except for the experiment for MSP-ESD combination. The mixing of datasets
increases the diversity of the training utterances which leads the model to
learn generalised feature representations when training.

TABLE VIII: Testing Accuracy of SL-DA - rw and the proposed RL-DA approach of each experiment mimicking a real-world scenario for each cross-corpus (CC) and cross-language (CL) schema with (w/) and without (w/o) background noise. Background Noise is added to simulate real-world conditions. |  | Dataset | UAR(%)
---|---|---|---
|  | Source | Target | SL-DA - rw | RL-DA | $\Delta$
w/o noise | CC | IEM | ESD | 53.11 $\pm$ 1.45 | 62.89 $\pm$ 0.51 | 9.78
MSP | ESD | 47.07 $\pm$ 2.56 | 63.18 $\pm$ 2.15 | 16.11
CL | IEM | EmoDB | 60.67 $\pm$ 1.70 | 73.83 $\pm$ 1.55 | 13.16
MSP | EmoDB | 46.50 $\pm$ 1.08 | 66.17 $\pm$ 0.24 | 19.67
w/ noise | CC | IEM | ESD + noise | 53.11 $\pm$ 1.45 | 63.62 $\pm$ 1.65 | 10.51
MSP | ESD + noise | 47.07 $\pm$ 2.56 | 57.77 $\pm$ 1.46 | 10.70
CL | IEM | EmoDB + noise | 60.67 $\pm$ 1.70 | 68.50 $\pm$ 3.63 | 7.83
MSP | EmoDB + noise | 46.50 $\pm$ 1.08 | 63.67 $\pm$ 1.65 | 17.17

###  5.3 Experiments simulating a live data feed scenario

We experiment to quantify the performance difference between SL and RL
optimisation in a simulated live data feed scenario. Testing in a real live
data feed scenario would require integrating the system with a call centre or
similar setting, which is problematic. However, we closely simulate the live
scenario by continually pushing the data from the target dataset. RL-DA uses
this data to continually update itself, whereas the SL-DA method uses the
static model, which is trained using the source dataset. The methodology of
the RL optimisation is similar to the experiment described in 5.1. The
baseline model (SL-DA - live) is first trained with a training subset of the
source dataset, and then, the testing accuracy is measured with testing
subsets of the target dataset.

We also measure the performance of RL-DA with real-world audio. We mix
background audio with our target datasets to create a new variation of the
dataset with background sound. Audio from the kitchen environment of the
DEMAND dataset is used as the background sound to mix with target datasets
(ESD and EmoDB). We use -5 db as the Signal-to-Noise ratio when mixing the
background audio to the speech utterances.

The testing accuracy of the models trained in three different ways (SL-DA -
live, RL-DA - target dataset without background noise (RL-DA w/o noise), and
RL-DA - target dataset with background noise (RL-DA w/ noise)) in a real-world
scenario mentioned in 5.3 are tabulated in Table VIII. The $\Delta$ column
shows the difference in accuracy between our approach and the baseline, where
one has to use a model pre-trained only in the source dataset.

Observing the $\Delta$ column in Table VIII, it is clear that RL-based domain
adaptation (RL-DA) outperforms the baseline SL strategy in the live data feed
situation (SL-DA - live). Supervised learning methodologies do not have the
possibility to adapt to the deviation of a domain distribution without a
labelled dataset to retrain during inference time. On the other hand, RL based
learning methodologies constantly receive feedback from the environment and
optimise the agent model for the domain deviations on the go. One can argue
that for the supervised method, one can retrain the model. But this requires
manual intervention, whereas applying RL, this can be reached without any
manual intervention. Therefore, we suggest that RL is more suitable in domain
adaptation to be used in real-world scenarios because of the adaptability
during the optimisation phase.

Figure 5: Comparison of UAR of SL-DA - live, the proposed RL-DA approach
without (w/o) background noise, and with (w/) background noise of each cross-
corpus and cross-language schema simulating live data feed scenario.

Figure 5 shows the comparison between the UAR of the Baseline, RL-DA without
noise, and RL-DA with noise after executing the ‘simulated live data feed’
experiments. We observe that both RL-DA experiments (with and without
background audio) show better results than that of the baseline approach.

##  6 Conclusion

This study investigated the possibility of using RL for domain adaptation in
SER. While several strategies have been used for domain adaptation for SER in
the past, we were the first to propose RL-technique for domain adaptation
herein. Evaluations of this research suggest that using RL can improve domain
adaptability in a real-world scenario compared to a SL approach. An accuracy
increment of 11 % in cross-corpus and 14 % in cross-language schema was
observable in the results when using the proposed RL-DA approach compared to
the baseline strategy.

We discussed that the main reason for the improved domain adaptability in a
real-world setting is the ability of the RL agent to continually incorporate
feedback. On the other hand, supervised learning does not have such a feedback
mechanism unless the model is retrained using a manual process.

The outcome of this study is likely to lead the community in this new
direction and help eventuate real-world emotion-aware applications that can
adapt to the users’ emotional expression under a new domain.

## References

  * [1] B. Schuller, B. Vlasenko, F. Eyben, M. Wollmer, A. Stuhlsatz, A. Wendemuth, and G. Rigoll, “Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies,” _IEEE Transactions on Affective Computing_ , vol. 1, no. 2, pp. 119–131, 7 2010.
  * [2] R. Milner, M. A. Jalal, R. W. Ng, and T. Hain, “A Cross-Corpus Study on Speech Emotion Recognition,” _2019 IEEE Automatic Speech Recognition and Understanding Workshop, ASRU 2019 - Proceedings_ , pp. 304–311, 12 2019.
  * [3] J. Gideon, M. G. McInnis, and E. M. Provost, “Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG),” _IEEE Transactions on Affective Computing_ , vol. 12, no. 4, pp. 1055–1068, 2021.
  * [4] Q. Mao, W. Xue, Q. Rao, F. Zhang, and Y. Zhan, “Domain adaptation for speech emotion recognition by sharing priors between related source and target classes,” in _2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 3 2016, pp. 2608–2612.
  * [5] N. Liu, Y. Zong, B. Zhang, L. Liu, J. Chen, G. Zhao, and J. Zhu, “Unsupervised Cross-Corpus Speech Emotion Recognition Using Domain-Adaptive Subspace Learning,” _ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings_ , vol. 2018-April, pp. 5144–5148, 9 2018.
  * [6] Y. Xi, P. Li, Y. Song, Y. Jiang, and L. Dai, “Speaker to emotion: Domain adaptation for speech emotion recognition with residual adapters,” _2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019_ , pp. 513–518, 11 2019.
  * [7] E. N. Ocquaye, Q. Mao, Y. Xue, and H. Song, “Cross lingual speech emotion recognition via triple attentive asymmetric convolutional neural network,” _International Journal of Intelligent Systems_ , vol. 36, no. 1, pp. 53–71, 1 2021.
  * [8] J. Tan, T. Zhang, E. Coumans, A. Iscen, Y. Bai, D. Hafner, S. Bohez, and V. Vanhoucke, “Sim-to-Real: Learning Agile Locomotion For Quadruped Robots,” in _Robotics: Science and Systems_. MIT Press Journals, 2018.
  * [9] B. Balaji, S. Mallya, S. Genc, S. Gupta, L. Dirac, V. Khare, G. Roy, T. Sun, Y. Tao, B. Townsend, E. Calleja, S. Muralidhara, and D. Karuppasamy, “DeepRacer: Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 2746–2754, 5 2020.
  * [10] I. Szita, “Reinforcement learning in games,” _Adaptation, Learning, and Optimization_ , vol. 12, pp. 539–577, 2012.
  * [11] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with Deep Reinforcement Learning,” 12 2013.
  * [12] T. Carr, M. Chli, and G. Vogiatzis, “Domain Adaptation for Reinforcement Learning on the Atari,” _Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS_ , vol. 4, pp. 1859–1861, 12 2018.
  * [13] I. Munemasa, Y. Tomomatsu, K. Hayashi, and T. Takagi, “Deep reinforcement learning for recommender systems,” _2018 International Conference on Information and Communications Technology, ICOIACT 2018_ , vol. 2018-January, pp. 226–233, 4 2018.
  * [14] W. Zhao, J. P. Queralta, and T. Westerlund, “Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey,” _2020 IEEE Symposium Series on Computational Intelligence, SSCI 2020_ , pp. 737–744, 12 2020.
  * [15] S. Latif, H. Cuayáhuitl, F. Pervez, F. Shamshad, H. Shehbaz Ali, E. Cambria, and S. Latif, “A survey on deep reinforcement learning for audio-based applications,” _Artificial Intelligence Review 2022_ , pp. 1–48, 7 2022.
  * [16] D. Yu, M. L. Seltzer, J. Li, J.-T. Huang, and F. Seide, “Feature Learning in Deep Neural Networks - Studies on Speech Recognition Tasks,” 1 2013.
  * [17] O. E. Nii Noi, M. Qirong, G. Xu, and Y. Xue, “Coupled Unsupervised Deep Convolutional Domain Adaptation for Speech Emotion Recognition,” in _2018 IEEE Fourth International Conference on Multimedia Big Data (BigMM)_. IEEE, 9 2018, pp. 1–5.
  * [18] M. Leonetti, L. Iocchi, and P. Stone, “A synthesis of automated planning and reinforcement learning for efficient, robust decision-making,” _Artificial Intelligence_ , vol. 241, pp. 103–130, 12 2016.
  * [19] E. Lakomkin, M. A. Zamani, C. Weber, S. Magg, and S. Wermter, “EmoRL: Continuous Acoustic Emotion Classification using Deep Reinforcement Learning,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 4445–4450, 4 2018.
  * [20] M. S. Hossain and G. Muhammad, “Emotion recognition using deep learning approach from audio–visual emotional big data,” _Information Fusion_ , vol. 49, pp. 69–78, 9 2019.
  * [21] S. Koo, H. Yu, and G. G. Lee, “Adversarial approach to domain adaptation for reinforcement learning on dialog systems,” _Pattern Recognition Letters_ , vol. 128, pp. 467–473, 12 2019.
  * [22] K. Arndt, M. Hazara, A. Ghadirzadeh, and V. Kyrki, “Meta Reinforcement Learning for Sim-to-real Domain Adaptation,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 2725–2731, 5 2020.
  * [23] Y. Ahn, S. J. Lee, and J. W. Shin, “Cross-Corpus Speech Emotion Recognition Based on Few-Shot Learning and Domain Adaptation,” _IEEE Signal Processing Letters_ , vol. 28, pp. 1190–1194, 2021.
  * [24] S. Gharib, K. Drossos, C. Akir, D. Serdyuk, and T. Virtanen, “Detection and Classification of Acoustic Scenes and Events,” in _Detection and Classification of Acoustic Scenes and Events_ , Surrey, UK, 2018.
  * [25] M. Abdelwahab and C. Busso, “Supervised domain adaptation for emotion recognition from speech,” _ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings_ , vol. 2015-August, pp. 5058–5062, 8 2015.
  * [26] Y. Yin, B. Huang, Y. Wu, and M. Soleymani, “Speaker-Invariant Adversarial Domain Adaptation for Emotion Recognition,” _ICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction_ , vol. 20, pp. 481–490, 10 2020.
  * [27] A. Mathur, N. Berthouze, and N. D. Lane, “Unsupervised domain adaptation under label space mismatch for speech classification,” _Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH_ , vol. 2020-October, pp. 1271–1275, 2020.
  * [28] C. Lu, Y. Zong, W. Zheng, Y. Li, C. Tang, and B. W. Schuller, “Domain Invariant Feature Learning for Speaker-Independent Speech Emotion Recognition,” _IEEE/ACM Transactions on Audio Speech and Language Processing_ , vol. 30, pp. 2217–2230, 2022.
  * [29] S. Latif, J. Qadir, and M. Bilal, “Unsupervised Adversarial Domain Adaptation for Cross-Lingual Speech Emotion Recognition,” _2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019_ , 9 2019\.
  * [30] A. Nair, B. McGrew, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Overcoming Exploration in Reinforcement Learning with Demonstrations,” _Proceedings - IEEE International Conference on Robotics and Automation_ , pp. 6292–6299, 9 2018.
  * [31] Z. Zhu, K. Lin, and J. Zhou, “Transfer Learning in Deep Reinforcement Learning: A Survey,” 9 2020.
  * [32] T. Rajapakshe, R. Rana, S. Latif, S. Khalifa, and B. W. Schuller, “Pre-training in Deep Reinforcement Learning for Automatic Speech Recognition,” _arXiv:1910.11256 [cs]_ , 2019.
  * [33] T. Rajapakshe, S. Latif, R. Rana, S. Khalifa, and B. W. Schuller, “Deep Reinforcement Learning with Pre-training for Time-efficient Training of Automatic Speech Recognition,” _arXiv:2005.11172 [cs, eess]_ , 2020.
  * [34] T. Rajapakshe, R. Rana, S. Khalifa, J. Liu, and B. Schuller, “A Novel Policy for Pre-trained Deep Reinforcement Learning for Speech Emotion Recognition,” in _Australasian Computer Science Week 2022_ , vol. 2022\. New York, NY, USA: ACM, 2 2022, pp. 96–105.
  * [35] M. Hazara and V. Kyrki, “Transferring Generalizable Motor Primitives From Simulation to Real World,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 2172–2179, 2 2019.
  * [36] C. Busso, S. Parthasarathy, A. Burmania, M. AbdelWahab, N. Sadoughi, and E. M. Provost, “MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception,” _IEEE Transactions on Affective Computing_ , vol. 8, no. 1, pp. 67–80, 2017.
  * [37] C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, “IEMOCAP: interactive emotional dyadic motion capture database,” _Language Resources and Evaluation_ , vol. 42, no. 4, p. 335, 2008.
  * [38] K. Zhou, B. Sisman, R. Liu, and H. Li, “Emotional voice conversion: Theory, databases and ESD,” _Speech Communication_ , vol. 137, pp. 1–18, 2 2022\.
  * [39] F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, “A Database of German Emotional Speech,” in _Interspeech_ , Lisbona, 2005, pp. 1517–1520.
  * [40] N. Jaitly and E. Hinton, “Vocal Tract Length Perturbation (VTLP) improves speech recognition,” in _30th International Conference on Machine Learning_ , 2013.
  * [41] Z. Peng, X. Li, Z. Zhu, M. Unoki, J. Dang, and M. Akagi, “Speech Emotion Recognition Using 3D Convolutions and Attention-Based Sliding Recurrent Networks with Auditory Front-Ends,” _IEEE Access_ , vol. 8, pp. 16 560–16 572, 2020.
  * [42] X. Xie, J. Lou, and L. Zhang, “Speech Emotion Recognition Model with Time-Scale-Invariance MFCCs as Input,” _2021 International Wireless Communications and Mobile Computing, IWCMC 2021_ , pp. 537–542, 2021.
  * [43] Y. Gao, S. Okada, L. Wang, J. Liu, and J. Dang, “Domain-Invariant Feature Learning for Cross Corpus Speech Emotion Recognition,” pp. 6427–6431, 4 2022\.
  * [44] S. Davis and P. Mermelstein, “Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences,” _IEEE transactions on acoustics, speech, and signal processing_ , vol. 28, no. 4, pp. 357–366, 1980.
  * [45] S. Latif, R. Rana, S. Khalifa, R. Jurdak, and J. Epps, “Direct Modelling of Speech Emotion from Raw Speech,” in _Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH_ , 2019, pp. 3920–3924.
  * [46] B. McFee, C. Raffel, D. Liang, D. P. W. Ellis, M. McVicar, E. Battenberg, and O. Nieto, “librosa: Audio and music signal analysis in python,” vol. 8, 2015\.
  * [47] S. Latif, R. Rana, S. Khalifa, R. Jurdak, J. Qadir, and B. W. Schuller, “Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends,” _arXiv:2001.00378 [cs, eess]_ , 2020.
  * [48] F. Chollet and others, _Keras_ , 2015. [Online]. Available: https://keras.io
  * [49] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, _TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems_ , 2015. [Online]. Available: https://www.tensorflow.org/
  * [50] M. Wiering, “Explorations in Efficient Reinforcement Learning,” Ph.D. dissertation, University of Amsterdam, 1999.
  * [51] N. Liu, B. Zhang, B. Liu, J. Shi, L. Yang, Z. Li, and J. Zhu, “Transfer Subspace Learning for Unsupervised Cross-Corpus Speech Emotion Recognition,” _IEEE Access_ , vol. 9, pp. 95 925–95 937, 2021.

|  Thejan Rajapakshe received the bachelor degree in Applied Sciences from
Rajarata University of Sri Lanka and the Bachelor of Information Technology
from the University of Colombo, School of Computing, in 2016. He is currently
a research scholar at the University of Southern Queensland (USQ). His
research interests include reinforcement learning, speech processing, and deep
learning.
---|---
|  Rajib Rana is an experimental computer scientist, Advance Queensland Senior
Research Fellow and an Associate Professor of Computer Science in the
University of Southern Queensland. He is also the Director of the IoT Health
research program at the University of Southern Queensland. He is recipient of
the prestigious Young Tall Poppy QLD Award 2018 as one of Queensland’s most
outstanding scientists for achievements in the area of scientific research and
communication. Rana’s research work aims to capitalise on advancements in
technology along with sophisticated information and data processing to better
understand disease progression in chronic health conditions and develop
predictive algorithms for chronic diseases, such as mental illness and cancer.
His current research focus is on Unsupervised Representation Learning,
Reinforcement Learning, Adversarial Machine Learning and Emotional Speech
Generation. He received his B. Sc. degree in Computer Science and Engineering
from Khulna University, Bangladesh, with the Prime Minister and President’s
Gold Medal for outstanding achievements and his Ph. D. in Computer Science and
Engineering from the University of New South Wales, Sydney, Australia, in
2011. He received his postdoctoral training at Autonomous Systems Laboratory,
CSIRO, before joining the University of Southern Queensland in 2015.
---|---
|  Sara Khalifa is currently a senior research scientist at the Distributed
Sensing Systems research group, Data61—CSIRO. She is also an honorary adjunct
lecturer at the University of Queensland and conjoint lecturer at the
University of New South Wales. Her research interests rotate around the broad
aspects of mobile and ubiquitous computing, mobile sensing, and Internet of
Things (IoT). She obtained a PhD in Computer Science and Engineering from UNSW
(Sydney, Australia). Her PhD dissertation received the 2017 John Makepeace
Bennett Award which is awarded by CORE (Computing Research and Education
Association of Australasia) to the best PhD dissertation of the year within
Australia and New Zealand in the field of Computer Science. Her research has
been recognised by multiple iAwards including 2017 NSW Mobility Innovation of
the year, 2017 NSW R&D Innovation of the year, National Merit R&D Innovation
of the year, and the Merit R&D award at the Asia Pacific ICT Alliance (APICTA)
Awards, commonly known as the ‘Oscar’ of the ICT industry in the Asia Pacific,
among others.
---|---
|  Björn W. Schuller received his diploma in 1999, his doctoral degree for his
study on Automatic Speech and Emotion Recognition in 2006, and his
habilitation and Adjunct Teaching Professorship in the subject area of Signal
Processing and Machine Intelligence in 2012, all in electrical engineering and
information technology from TUM in Munich/Germany. He is Professor of
Artificial Intelligence in the Department of Computing at the Imperial College
London/UK, where he heads GLAM — the Group on Language, Audio, & Music, Full
Professor and head of the Chair of Embedded Intelligence for Health Care and
Wellbeing at the University of Augsburg/Germany, and CEO/CSO of audEERING. He
was previously full professor and head of the Chair of Complex and Intelligent
Systems at the University of Passau/Germany. Professor Schuller is Fellow of
the IEEE, Golden Core Member of the IEEE Computer Society, Fellow of the ISCA,
Fellow of the BCS, Senior Member of the ACM, and Fellow and President-emeritus
of the Association for the Advancement of Affective Computing (AAAC). He
(co-)authored 5 books and 1200+ publications in peer-reviewed books, journals,
and conference proceedings leading to more than overall 48 000 citations
(h-index = 99). Schuller was general chair of ACII 2019, Program Chair of
Interspeech 2019 and ICMI 2019, repeated Area Chair of ICASSP, and former
Editor in Chief of the IEEE Transactions on Affective Computing, and is Field
Chief Editor of the Frontiers in Digital Health.
---|---


# Single-particle properties of the near-threshold proton-emitting resonance
in 11B

Nguyen Le Anh<EMAIL_ADDRESS>Department of Theoretical Physics, Faculty
of Physics and Engineering Physics, University of Science, Ho Chi Minh City,
Vietnam  Vietnam National University, Ho Chi Minh City, Vietnam  Department of
Physics, Ho Chi Minh City University of Education, 280 An Duong Vuong,
District 5, Ho Chi Minh City, Vietnam  Bui Minh Loc<EMAIL_ADDRESS>Center for Exotic Nuclear Studies, Institute for Basic Science (IBS), Daejeon
34126, Korea  Naftali Auerbach<EMAIL_ADDRESS>School of Physics and
Astronomy, Tel Aviv University, Tel Aviv 69978, Israel  Vladimir Zelevinsky
<EMAIL_ADDRESS>Facility for Rare Isotope Beams, Michigan State
University, East Lansing, Michigan 48824, USA  National Superconducting
Cyclotron Laboratory, Michigan State University, East Lansing, Michigan 48824,
USA  Department of Physics and Astronomy, Michigan State University, East
Lansing, Michigan 48824, USA

###### Abstract

The excitation function of proton elastic scattering from 10Be at keV energy
is calculated using the self-consistent Skyrme Hartree-Fock in the continuum
method. The calculation successfully reproduces the narrow near-threshold
proton-emitting resonance ($E_{x}=11.4$ MeV, $\Gamma=6$ keV, and quantum
number $J^{\pi}=1/2^{+}$) in 11B relevant to the $\beta$-delayed proton
emission of 11Be. This supports the recent experimental result of Y. Ayyad et
al. at the ReA3 re-accelerator facility of the National Superconducting
Cyclotron Laboratory (NSCL) at the Michigan State University. The resonance is
interpreted as the $s_{1/2}$ single-proton resonance state in the Skyrme
Hartree-Fock mean-field theory.

Introduction. The study of loosely bound states of atomic nuclei close to the
drip lines and beyond is one of the major current directions of nuclear
science [1]. At limits of nuclear stability, rare decay processes were
discovered for weakly bound nuclei, especially for nuclei with the halo
structure (see [2] for a review).

A very particular case is the $\beta$-delayed proton emission ($\beta p$) that
was theoretically predicted for the halo nucleus 11Be in Refs. [3, 4] with the
unexpectedly high intensity [4]. The decay was indirectly observed in Refs.
[5, 6, 7]. The existence of a narrow resonance in 11B was suggested to explain
this observation [5, 8]. However, there was no suitable level in 11B known at
that time. There have been several attempts of theoretical calculations to
confirm the resonance in 11B and to estimate the $\beta^{-}p$ decay branching
ratio in 11Be [9, 10, 11, 12]. These studies are also related to the search
for the so-called neutron dark decay mode proposed in Ref. [13] in order to
explain the existing discrepancy between two different methods of neutron
lifetime measurements (see [14] for the review). If the dark neutron decay is
possible for the unbound nucleon, then it should occur also for the quasi-
bound neutron in a nucleus with sufficiently low neutron binding energy, and
11Be is the most promising nucleus for such studies [15].

Recently, the $\beta^{-}p$ decay was directly observed for the first time in
the 11Be $\rightarrow{}^{10}$Be + $\beta^{-}+p$ [16]. The result showed that
the $\beta^{-}p$ decay is the sequence of the disintegration proceeded via an
intermediate near-threshold narrow resonance in the $\beta^{-}$ decay product
11B∗ at an energy $E_{r}=196(20)$ keV above the proton separation energy, with
a total width of $\Gamma_{p}=12(5)$ keV, and spin-parity quantum numbers
$J^{\pi}=(1/2^{+},3/2^{+})$. Moreover, the dedicated experiment employing the
10Be($p,p$) reaction was performed later by the same collaboration at the ReA3
re-accelerator NSCL facility [17]. The experimental result indicated a very
narrow resonance ($\Gamma_{p}=4.4$ keV) with $J^{\pi}=1/2^{+}$ located by 182
keV above the proton separation energy.

Using an appropriate optical potential for the keV-energy scattering, it is
straightforward to calculate the excitation function of the elastic scattering
10Be($p,p$). The detailed calculation for the nuclear structure can be
directly linked to the problem of scattering without introducing additional
approximations, especially in the very low-energy region where the coupling to
the inelastic levels can be neglected. In the pioneering work Ref. [18], the
Hartree-Fock (HF) single-particle potential was expressed as an equivalent
optical potential. The version of the optical potential obtained from the HF
calculations using the Skyrme effective interaction was demonstrated to be an
appropriate formalism for the keV-energy region [19, 20]. Not only the
background cross section but also the positions and widths of single-particle
resonances could be predicted. Recently, the method was successfully applied
to the study of keV-nucleon radiative capture reactions in nuclear
astrophysics [21, 22]. In this work, the Skyrme HF in the continuum method is
applied to reproduce the near-threshold resonance in 11B recently found in
Ref. [17].

Method. The solution starts with the Skyrme HF calculation for the ground
state of 10Be using the skyrme_rpa computer program provided in Ref. [23]. The
versions of Skyrme interactions used in this study include SkM∗ [24], SGII
[25], SLy4 [26], and SAMi [27]. The local equivalent optical potential within
the Skyrme HF formalism (called the Skyrme HF optical potential) $V(E,r)$, is
obtained following the method of Ref. [19]:

$\displaystyle V(E,r)=\dfrac{m^{*}(r)}{m}\left\\{V_{\rm
HF}(r)+\dfrac{1}{2}\dfrac{d^{2}}{dr^{2}}\left(\dfrac{\hbar^{2}}{2m^{*}(r)}\right)\right.$
$\displaystyle\left.-\dfrac{m^{*}(r)}{2\hbar^{2}}\left[\dfrac{d}{dr}\left(\dfrac{\hbar^{2}}{2m^{*}(r)}\right)\right]^{2}\right\\}+\left[1-\dfrac{m^{*}(r)}{m}\right]E,$
(1)

where $m$ is the nucleon mass, $V_{\rm HF}(r)$ and $m^{*}(r)$ are the mean-
field potential and the effective proton mass within the Skyrme HF approach,
respectively [28]. At the keV energy, the nuclear central potential $V(E,r)$
is real and depends weakly on the incident energy. The center-of-mass
correction, the rearrangement term, and the non-local effect are included in
the Skyrme HF optical potential [19, 20]. Note that the one-body Coulomb
potential that plays an important role in the low-energy scattering is
obtained self-consistently in the calculation.

The partial scattering equations

$\displaystyle\left\\{\dfrac{\hbar^{2}}{2m^{\prime}}\left[-\dfrac{d^{2}}{dr^{2}}+\dfrac{\ell(\ell+1)}{r^{2}}\right]+V(E,r)-E\right\\}$
$\displaystyle\chi_{\ell}(E,r)=0,$ (2)

is solved with the Skyrme HF optical potential in Eq. (Single-particle
properties of the near-threshold proton-emitting resonance in 11B) to obtain
the scattering phase shifts and the excitation function using a computer
program such as ECIS06 [29]. The use of $m^{\prime}=mA/(A-1)$ takes into
account a large part of the center-of-mass correction important in the case of
light nuclei. The $s$-wave scattering ($\ell=0$) dominates at the energy of
our interest. The width of the resonance $\Gamma(E_{r})$ is calculated from
the derivative of the phase shift $\delta_{\ell=0}(E)$ with respect to energy
at the position of the resonance $E_{r}$,

$\Gamma(E_{r})=2\left[\dfrac{d}{dE}\delta_{\ell=0}(E)\right]^{-1}.$ (3)

Discussion. Without any adjustment, the Skyrme HF calculation provides a good
description of the background cross section and the resonance with the single-
particle properties. The resonance recently found by Y. Ayyad et al. [17] is
located at 182 keV that is close to the $s_{1/2}$ single-particle resonance
with $J^{\pi}=1/2^{+}$.

Figure 1: Calculated excitation function (a) and $s$-wave phase-shift (b) in
the center-of-mass frame for the 10Be($p,p$) elastic scattering. The
calculation with SAMi interaction is shown as an example. The experimental
data and the $R$-matrix fitting (the blue-dashed line) are taken from Y. Ayyad
et al. [17].

Our calculations with the SkM∗ and SAMi interactions show the resonance in the
appropriate energy region. Outside the resonance, the calculation agrees well
with experimental data. In general, the Skyrme HF calculation cannot predict
exactly the location of the unknown resonance in 10Be($p,p$). Therefore, the
nuclear central part of the Skyrme HF optical potential is slightly scaled by
the factor $\lambda$ that is close to unity (Table 1) in order to correct the
resonance position to 182 keV. In the case of SLy4 and SGII interactions, the
calculations with $\lambda=1$ do not show the resonance in the energy region.
As the resonance is just a few hundred keV above the threshold, it is possibly
shifted to the sub-threshold energy with using SLy4 and SGII versions.
However, with the slight correction (Table 1), the resonance at 182 keV
appears for all selected variants of the Skyrme interactions.

Table 1: The scaling factor $\lambda$ in the calculations with different Skyrme interactions. The values of $\lambda$ are very close to unity. The calculated resonance width is about 5.5-6.0 keV. Skyrme interaction | $\lambda$ |  $\Gamma$ [keV]
---|---|---
SkM∗ [24] | $1.004$ | $5.43$
SGII [25] | $1.028$ | $5.97$
SLy4 [26] | $0.984$ | $6.09$
SAMi [27] | $0.998$ | $5.99$

It is expected that the width of a near-threshold resonance is narrow. Indeed,
the calculated resonance widths are 5-6 keV with different Skyrme interactions
as seen in Table 1. Fig. 1 shows the calculation with and without the
adjustment for the case of SAMi interaction as an example. Only the phase
shift of the $s$-wave is shown in Fig. 1(b) as the contributions of other
waves are negligible. The resonance energy, $E_{r}=166$ keV, and the width,
$\Gamma=4.05$ keV (the dotted line in Fig. 1), are obtained with $\lambda=1$.
The calculations with different Skyrme interactions using the values of
$\lambda$ in Table 1 cannot be distinguished in Fig. 1.

As seen from Fig. 1 and Table 1, the position of the resonance is very
sensitive to slight changes of the Skyrme HF potential. It can be understood
by observing the accumulation of the volume integral. The value of this
intergal, $4\pi\int_{\infty}^{r}V(r)r^{2}\,dr$, with $V(r)$ as the sum of the
nuclear central and Coulomb potential, as a function of the relative distance
$r$ is shown in Fig. 2. As the potential only weakly depends on energy, the
illustration is done at zero energy. Note that the accumulation starts at the
large distance where the total potential almost vanishes ($r=15$ fm in this
case). At the distances from 5 to 15 fm, only the effect of the Coulomb
potential survives. At a distance where the nuclear potential plays a
noticeable role (at about 5 fm, if $\lambda$ is unity), the difference in the
calculations applying different Skyrme interactions can be observed clearly as
shown in Fig. 2(a). Therefore, the excitation functions for different Skyrme
interactions vary.

Figure 2: Illustration of the sensitivity of the resonance position to the
accumulation of the volume integral of the total potential from the outer to
the inner region. The calculations with different Skyrme interactions are
convergent after the adjustment of the potential depth to correct the
resonance position at 182 keV.

Figs. 2(a) and (b) show that the small variation of $\lambda$ makes a large
change in the accumulation of the volume integral. Consequently, the result of
the calculation is changed. When the value of $\lambda$ is adjusted to correct
the resonance location, all computations are getting very close. The
calculations with different Skyrme interactions lead to the same properties of
the studied $s_{1/2}$ single-particle resonance. Finally, we have to emphasize
that slight changes of the HF potential caused by the scaling factor $\lambda$
do not affect the calculated single-particle structure and nuclear properties.

Conclusion. The experimental result [17] of the near-threshold resonance in
11B relevant to the $\beta$-delayed proton emission of 11Be is interpreted as
the $s_{1/2}$ single-particle resonance predicted by the Skyrme HF
calculation. Its simple structure is given by [10Be($0_{\rm g.s.}^{+}$) +
$p(s_{1/2})$]. The calculated width of this resonance is $\Gamma=6$ keV and
$J^{\pi}=1/2^{+}$. The experiment in Ref. [17] is crucial for localizing the
exact position of this resonance. The final result of calculations does not
depend on the choice of the version of the Skyrme interactions. The Skyrme HF
calculation is an excellent tool for describing the scattering process
including low-lying single-particle resonances in the keV-energy region.

The great current interest in experiments at rare isotope beam facilities
around the world and in corresponding reaction theory is important for the
progress in understanding unusual features of exotic nuclei in relation to
nuclear structure, astrophysics, and the standard model of particle physics.
Another branch of this physics is related to the role of clusters in physics
of light and medium nuclei; this will be touched in the prepared FRIB
experiment on the decay of 11Be into 7 Li and alpha-particle.

Acknowledgment. B. M. L. was supported by the Institute for Basic Science
(IBS-R031-D1).

## References

  * Johnson  _et al._ [2020] C. W. Johnson, K. D. Launey, N. Auerbach, S. Bacca, B. R. Barrett, C. R. Brune, M. A. Caprio, P. Descouvemont, W. H. Dickhoff, C. Elster, P. J. Fasano, K. Fossez, H. Hergert, M. Hjorth-Jensen, L. Hlophe, B. Hu, R. M. I. Betan, A. Idini, S. König, K. Kravvaris, D. Lee, J. Lei, A. Mercenne, R. N. Perez, W. Nazarewicz, F. M. Nunes, M. Płoszajczak, J. Rotureau, G. Rupak, A. M. Shirokov, I. Thompson, J. P. Vary, A. Volya, F. Xu, R. G. T. Zegers, V. Zelevinsky, and X. Zhang, White paper: from bound states to the continuum, J. Phys. G: Nucl. Part. Phys. 47, 123001 (2020).
  * Pfützner  _et al._ [2012] M. Pfützner, M. Karny, L. Grigorenko, and K. Riisager, Radioactive decays at limits of nuclear stability, Rev. Mod. Phys. 84, 567 (2012).
  * Horoi and Zelevinsky [2003] M. Horoi and V. Zelevinsky, in: April Meeting, American Physical Society, Philadelphia, PA, 2003, abstract U10.001 (2003).
  * Baye and Tursunov [2011] D. Baye and E. M. Tursunov, $\beta$ delayed emission of a proton by a one-neutron halo nucleus, Phys. Lett. B 696, 464 (2011).
  * Riisager  _et al._ [2014] K. Riisager, O. Forstner, M. Borge, J. Briz, M. Carmona-Gallardo, L. Fraile, H. Fynbo, T. Giles, A. Gottberg, A. Heinz, J. Johansen, B. Jonson, J. Kurcewicz, M. Lund, T. Nilsson, G. Nyman, E. Rapisarda, P. Steier, O. Tengblad, R. Thies, and S. Winkler, 11Be($\beta p$), a quasi-free neutron decay?, Phys. Lett. B 732, 305 (2014).
  * Borge  _et al._ [2013] M. J. G. Borge, L. M. Fraile, H. O. U. Fynbo, B. Jonson, O. S. Kirsebom, T. Nilsson, G. Nyman, G. Possnert, K. Riisager, and O. Tengblad, Rare $\beta$p decays in light nuclei, J. Phys. G: Nucl. Part. Phys. 40, 035109 (2013).
  * Riisager  _et al._ [2020] K. Riisager, M. J. G. Borge, J. A. Briz, M. Carmona-Gallardo, O. Forstner, L. M. Fraile, H. O. U. Fynbo, A. G. Camacho, J. G. Johansen, B. Jonson, M. V. Lund, J. Lachner, M. Madurga, S. Merchel, E. Nacher, T. Nilsson, P. Steier, O. Tengblad, and V. Vedia, Search for beta-delayed proton emission from 11Be, Eur. Phys. J. A 56, 100 (2020).
  * Riisager [2014] K. Riisager, The strength of beta-decays to the continuum, Nucl. Phys. A 925, 112 (2014).
  * Volya [2020] A. Volya, Assessment of the beta-delayed proton decay rate of 11Be, Europhys Lett. 130, 12001 (2020).
  * Okołowicz  _et al._ [2020] J. Okołowicz, M. Płoszajczak, and W. Nazarewicz, Convenient Location of a Near-Threshold Proton-Emitting Resonance in ${}^{11}\mathrm{B}$, Phys. Rev. Lett. 124, 042502 (2020).
  * Okołowicz  _et al._ [2021] J. Okołowicz, M. Płoszajczak, and W. Nazarewicz, $\beta^{-}{\rm p}$ and $\beta^{-}\alpha$ decay of the 11Be neutron halo ground state (2021).
  * Elkamhawy  _et al._ [2021] W. Elkamhawy, Z. Yang, H.-W. Hammer, and L. Platter, $\beta$-delayed proton emission from 11Be in effective field theory, Phys. Lett. B 821, 136610 (2021).
  * Fornal and Grinstein [2018] B. Fornal and B. Grinstein, Dark Matter Interpretation of the Neutron Decay Anomaly, Phys. Rev. Lett. 120, 191801 (2018).
  * Wietfeldt and Greene [2011] F. E. Wietfeldt and G. L. Greene, Colloquium: The neutron lifetime, Rev. Mod. Phys. 83, 1173 (2011).
  * Pfützner and Riisager [2018] M. Pfützner and K. Riisager, Examining the possibility to observe neutron dark decay in nuclei, Phys. Rev. C 97, 042501 (2018).
  * Ayyad  _et al._ [2019] Y. Ayyad, B. Olaizola, W. Mittig, G. Potel, V. Zelevinsky, M. Horoi, S. Beceiro-Novo, M. Alcorta, C. Andreoiu, T. Ahn, M. Anholm, L. Atar, A. Babu, D. Bazin, N. Bernier, S. S. Bhattacharjee, M. Bowry, R. Caballero-Folch, M. Cortesi, C. Dalitz, E. Dunling, A. B. Garnsworthy, M. Holl, B. Kootte, K. G. Leach, J. S. Randhawa, Y. Saito, C. Santamaria, P. Šiurytė, C. E. Svensson, R. Umashankar, N. Watwood, and D. Yates, Direct observation of proton emission in ${}^{11}\mathrm{Be}$, Phys. Rev. Lett. 123, 082501 (2019).
  * Ayyad  _et al._ [2022] Y. Ayyad, W. Mittig, T. Tang, B. Olaizola, G. Potel, N. Rijal, N. Watwood, H. Alvarez-Pol, D. Bazin, M. Caamaño, J. Chen, M. Cortesi, B. Fernández-Domínguez, S. Giraud, P. Gueye, S. Heinitz, R. Jain, B. P. Kay, E. A. Maugeri, B. Monteagudo, F. Ndayisabye, S. N. Paneru, J. Pereira, E. Rubino, C. Santamaria, D. Schumann, J. Surbrook, L. Wagner, J. C. Zamora, and V. Zelevinsky, Evidence of a near-threshold resonance in 11B relevant to the $\beta$-delayed proton emission of 11Be, arxiv.2205.04973 (2022).
  * Vautherin and Vénéroni [1968] D. Vautherin and M. Vénéroni, Potential scattering by a nuclear hartree-fock field, Physics Letters B 26, 552 (1968).
  * Dover and Giai [1971] C. B. Dover and N. V. Giai, Low-energy neutron scattering by a Hartree-Fock field, Nucl. Phys. A 177, 559 (1971).
  * Dover and Giai [1972] C. B. Dover and N. V. Giai, The nucleon-nucleus potential in the Hartree-Fock approximation with Skyrme’s interaction, Nucl. Phys. A 190, 373 (1972).
  * Anh and Loc [2021] N. L. Anh and B. M. Loc, Bound-to-continuum potential model for the ($p,\gamma$) reactions of the cno nucleosynthesis cycle, Phys. Rev. C 103, 035812 (2021).
  * Anh  _et al._ [2021] N. L. Anh, P. N. Huan, and B. M. Loc, Potential model within a bound-to-continuum approach for low-energy nucleon radiative capture by ${}^{12}\mathrm{C}$ and ${}^{16}\mathrm{O}$, Phys. Rev. C 104, 034622 (2021).
  * Colò  _et al._ [2013] G. Colò, L. Cao, N. Van Giai, and L. Capelli, Self-consistent RPA calculations with Skyrme-type interactions: The skyrme_rpa program, Comput. Phys. Commun. 184, 142 (2013).
  * Bartel  _et al._ [1982] J. Bartel, P. Quentin, M. Brack, C. Guet, and H.-B. Håkansson, Towards a better parametrisation of Skyrme-like effective forces: A critical study of the SkM force, Nucl. Phys. A 386, 79 (1982).
  * Giai and Sagawa [1981] N. V. Giai and H. Sagawa, Spin-isospin and pairing properties of modified Skyrme interactions, Phys. Lett. B 106, 379 (1981).
  * Chabanat  _et al._ [1998] E. Chabanat, P. Bonche, P. Haensel, J. Meyer, and R. Schaeffer, A Skyrme parametrization from subnuclear to neutron star densities Part II. Nuclei far from stabilities, Nucl. Phys. A 635, 231 (1998).
  * Roca-Maza  _et al._ [2012] X. Roca-Maza, G. Colò, and H. Sagawa, New Skyrme interaction with improved spin-isospin properties, Phys. Rev. C 86, 031306 (2012).
  * Vautherin and Brink [1972] D. Vautherin and D. M. Brink, Hartree-Fock Calculations with Skyrme’s Interaction. I. Spherical Nuclei, Phys. Rev. C 5, 626 (1972).
  * Raynal [1971] J. Raynal, ECIS06 code is distributed by the NEA DATA Bank, Paris, France; ECIS72 described in “Optical model and coupled-channels calculations in nuclear physics,”, in  _Computing as a language of physics. ICTP International Seminar Course, Trieste, Italy_ (1971) p. 281.