# Data Augmentation for Improving Tail-traffic Robustness
in Skill-routing for Dialogue Systems

Ting-Wei Wu1,2, Fatemeh Sheikholeslami2, Mohammad Kachuee2, Jaeyoung Do2,
Sungjin Lee2
1Georgia Institute of Technology, 2Amazon Alexa AI, Seattle, WA
<EMAIL_ADDRESS>
{shfateme, kachum, domjae<EMAIL_ADDRESS>

###### Abstract

Large-scale conversational systems typically rely on a skill-routing component
to route a user request to an appropriate skill and interpretation to serve
the request. In such system, the agent is responsible for serving thousands of
skills and interpretations which create a long-tail distribution due to the
natural frequency of requests. For example, the samples related to play music
might be a thousand times more frequent than those asking for theatre show
times. Moreover, inputs used for ML-based skill routing are often a
heterogeneous mix of strings, embedding vectors, categorical and scalar
features which makes employing augmentation-based long-tail learning
approaches challenging. To improve the skill-routing robustness, we propose an
augmentation of heterogeneous skill-routing data and training targeted for
robust operation in long-tail data regimes. We explore a variety of
conditional encoder-decoder generative frameworks to perturb original data
fields and create synthetic training data. To demonstrate the effectiveness of
the proposed method, we conduct extensive experiments using real-world data
from a commercial conversational system. Based on the experiment results, the
proposed approach improves more than 80% (51 out of 63) of intents with less
than 10K of traffic instances in the skill-routing replication task.

##  1 Introduction

Recent large-scale conversational systems such as Amazon Alexa, Apple Siri,
Google Assistant, and Microsoft Cortana have shown great promise toward
natural human-machine interactions (Sarikaya, 2017). Such systems often
involve multiple ML-based components to fulfill user requests. Components such
as Automated Speech Recognition (ASR) to transcribe the request, Natural
Language Understanding (NLU) to assign a user’s utterance to a set of
potential interpretations i.e. domains, intent, and parse sentence entities.
Then, based on the NLU interpretations and other contextual signals (e.g.
device type), a skill routing component is to select the best NLU
interpretation and route the request to an appropriate skill.

Self-learning based on customer satisfaction metrics is the state-of-the-art
method for the skill routing problem. Typically, the skill routing problem is
cast as a contextual bandit to optimize a reward signal generated by ML-based
customer satisfaction estimators. While such self-learning approach is
promising in terms of scalability, in a commercial system with thousands of
skills/intents creating a long-tail distribution and considering the
disparities in estimation quality for the customer satisfaction signals for
different traffic segments, it is often challenging to maintain routing
quality for entire traffic by solely relying on bandit learning objective. To
address such issues, current self-learning methods rely on replication
objectives to ensure policy robustness across off-policy bandit updates
(Kachuee et al., 2022; Kachuee and Lee, 2022).

In this work, we attempt to enhance the robustness of skill routing systems by
augmenting the low-appearance domain and intent data subsets. Typically, for a
certain request, we are given a set of routing candidates represented as
hypotheses each composed of an ASR transcribed text as well as other
categorical data fields such as NLU interpretations, device type, device
status, and the proposed skill. A model-based skill routing system is trained
by replicating the ideal skill-routing decisions represented through a (large)
training set of such requests and their contextual signals coupled with the
correct corresponding skill for routing. However, in practice such datasets
exhibit an imbalanced traffic between common requests and tail requests,
leading to low replication accuracy and robustness in these domains.
Therefore, we are interested in augmenting training data for such low-count
segments.

However augmenting such _heterogeneous_ data is non-trivial. Most natural
language processing tasks with only text inputs focus on pure text
augmentation by perturbing original texts at token spans (Louvan and Magnini,
2020; Ye et al., 2021; Wu et al., 2018) or entire sentences (Einolghozati et
al., 2019; Chen and Yu, 2021). Several works also leverage paraphrasing
techniques (Einolghozati et al., 2019; Cho et al., 2019; Gao et al., 2020) to
enhance the robustness of task-oriented dialog systems. However, manual
paraphrasing dataset preparation with intense laboring is required especially
for tail requests. Conditional generative approaches (Yoo et al., 2018; Peng
et al., 2020; Duan et al., 2020; Xia et al., 2020; Malandrakis et al., 2019;
Qiu et al., 2020) instead provide flexible solutions of modeling text
distribution that introduces variability yet preserves top-level semantics,
which is ideal for labor-free data augmentation.

Nevertheless, modeling such distributions remains challenging and mostly
unexplored in the research community, especially in the context of dialogue
systems. In this paper, we explore the idea of generative data augmentation
based on variational autoencoders (VAE) and transformer architectures to
generate samples from the conditional distribution of skill routing
hypothesis. The main contributions of this paper are as follows:

  1. 1.

Introducing a data augmentation framework for generating heterogeneous
features available in conversational assistant systems.

  2. 2.

Enriching the training set and enhancing the robustness of skill routing
models by leveraging the data augmenters applied on perturbed samples.

  3. 3.

Conducting extensive experiments using data from a real-world conversational
system to demonstrate the impact of the proposed methods on various metrics
for routing robustness and generation quality.

Figure 1: An overview of the base skill-routing system.

##  2 Problem Formulation

###  2.1 Skill routing

We consider the general skill routing problem in conversational systems. Given
training dataset $\mathcal{D}=\\{d_{i}\\}$, each instance is represented as a
pair $d_{i}=(H_{i},a_{i})$ with $H_{i}=\\{h_{1},h_{2},...,h_{N}\\}$ denoting a
set of $N$ hypotheses and $a_{i}\in\\{1,...,N\\}$ denoting the logged action,
i.e., the skill routing decision in the historical data, i.e., by the rule-
based systems and/or previous ML model in production (Li et al., 2021). The
intermediary task to maintain policy robustness which is of focus here is to
replicate the past actions $a_{i}$ of the deployed system by learning function
$f_{\theta}(H_{i})$ (Kachuee et al., 2021).

Let us define the replication accuracy for evaluating the robustness of
$f_{\theta}$:

$\displaystyle Acc(f_{\theta},D_{test})=\sum_{(H_{i},a_{i})\in
D_{test}}\frac{\mathbbm{1}[f_{\theta}(H_{i})=a_{i}]}{|D_{test}|}$

Thus, the offline evaluation metric $Acc(f_{\theta},D_{test})$ captures how
much $f_{\theta}$ matches the current stable routing behavior, which is
considered a crucial element in hybrid policy setup (Kachuee et al., 2022).

###  2.2 Heterogeneous data

A given hypothesis $h_{j}=(h^{u}_{j},h^{v}_{j})$ in the skill routing model
input set $H_{i}$ contains utterance-level $h^{u}_{j}$ and hypothesis-level
$h^{v}_{j}$ information, each of which has a collection of various data fields
of different formats, leading to a heterogeneous dataset. Utterance-level
information $h^{u}_{j}$ are shared across all hypotheses in $H_{i}$ and
includes the following data fields: (a) ASR-transcribed user request text $x$,
(b) device type $c^{d}$, (c) device status $c^{s}$. Hypothesis-level
information $h^{v}_{j}$ is instead unique to each hypothesis which has (d) NLU
interpretation consisting of (domain,intent,slot) pairs $n_{j}$, (e) NLU
confidence $c^{a}_{j}$, and (f) the proposed skill to serve the request
$s_{j}$.

##  3 Methodology

###  3.1 Base Model Architecture

Figure 2: The schematic of the conditional VAE with prior network for text-
only data augmentation. Figure 3: Two of main proposed generative frameworks
for heterogeneous data augmentation. (a) Conditional Variational BERT
structure by inputting a masked template and approximating the posterior
distribution with variational inference. $h_{1}\sim h_{7}$ are the hidden
states at the middle of layers of BERT. (b) Joint T5 model structure with a
sophisticated input-output sequence design, trained with mask contrastive
learning and frequency-aware loss. The encoder states will be sent into the
decoder with (1) true sequences and (2) false noises.

Figure 1 shows our base model architecture, which for a given a list of $N$
hypotheses $H=\\{h_{1},...,h_{N}\\}$, with $h_{j}=(h^{u}_{j},h^{v}_{j})$
defined as in Section 2.2, proceeds as follows. First, the ASR-transcribed
text $x$ is encoded using word vectors and a BiLSTM module. Words in other
fields such as NLU interpretation entities are also converted into fixed-size
embedding vectors via the embedding modules, while all other categorical data
fields $(c^{d},c^{s},n_{j},s_{j},c^{a}_{j})$ represented as indices are
encoded via trainable embedding layers (matrices). These feature vectors are
then concatenated to get the final representation $e_{j}\in\mathcal{R}^{d}$
for hypothesis $h_{j}$. Embedded routing hypotheses
$\\{e_{1},e_{2},...,e_{N}\\}$ are then sorted based on the NLU interpretation
confidence and are processed by a BiLSTM module followed by a MLP, finally
yielding output action probabilities $[p_{1},...,p_{N}]$.

###  3.2 Heterogeneous Data Augmentation

As discussed in the Section 2.1, given the training set
$\mathcal{D}=[\mathcal{D}_{0};\mathcal{D}_{T}]$ comprised of frequent requests
$\mathcal{D}_{0}$ and the tail requests $\mathcal{D}_{T}$, we are interested
in creating an augmentation set $\mathcal{D}_{A}=g_{\phi}(\mathcal{D}_{T})$,
which will then be incorporated during training
$\mathcal{D^{\prime}}=[\mathcal{D}_{0};\mathcal{D}_{T};\mathcal{D}_{A}]$ to
boost the replication accuracy of the low-traffic segments. Our proposition is
to replace a subset of data fields described in Section 2.2 from
$\mathcal{D}_{T}$ with the generated data from the proposed generative
framework $g_{\phi}$, through implicitly modeling the joint data distribution
of the heterogeneous feature sets $(h^{u}_{j},h^{v}_{j})$.

We focus on augmenting the shared utterance-level fields $h^{u}$, including
the ASR-transcribed text $x$, device type $c^{d}$ and device status signals
$c^{s}$, with artificially generated ${h^{u}}^{\prime}$. To do so, and in
order to preserve the original semantics and prevent unrealistic deviation
from natural data, We aim at modeling the joint data distribution
$P({h^{u}}^{\prime}|n_{1},s_{1})$, conditioned on $n_{1}$ the NLU
interpretation (domain, intent, slot pairs) and the proposed skill $s_{1}$
from the top-1 hypothesis $h_{1}$ in a parametric way. The premise is that
augmentation with different text and context variations can help boosting the
routing robustness for such traffic segments.

In the following sections, we propose three generative frameworks to generate
such samples ${h^{u}}^{\prime}$ to replace the original data fields $h^{u}$:
(1) conditional VAEs with prior networks (pcVAE) (2) conditional variational
BERT with masked language modeling (MLM) (CV-BERT MLM), and (3) Joint T5
Seq2Seq modeling. The first two models are adaptations of the conditional
variational auto-encoders, which the latter follows the BERT MLM task by
modifying tokens in the original text. Since the T5 model is better capable of
handling the heterogeneous nature of skill-routing data and shows higher
quality of the generated text, and due to space limitation, the first two
methods are discussed in Appendix B, and the third model is discussed in
section 3.3; see Figure 2 and 3 for an overview.

###  3.3 Joint T5 Seq2seq Model

Conditional VAEs cannot produce high-quality texts with limited in-domain data
and CV-BERT MLM aims to perturb the original text at the token level which
preserves the structures yet sacrifices the diversity. Instead, we propose a
transformer-based framework using T5 (Raffel et al., 2020) that can more
flexibly generate categorical fields and the utterance text from scratch. T5
is an encoder-decoder model pre-trained on a multi-task mixture of
unsupervised and supervised tasks and each task is converted into a text-to-
text format. During pretraining, the spans of the input sequence are masked by
the so-called sentinel tokens (a.k.a unique mask tokens) and the output
sequence is formed as a concatenation of the same sentinel tokens and the real
masked tokens. Herein, we also concatenate the categorical fields and top-1
NLU interpretation/skill $n_{1},s_{1}$ and modify the inputs in Figure 3. The
intuition is to ask the T5 model to predict each categorical field one after
another and accurately generate the utterance given $n_{1},s_{1}$.

Masked contrastive learning: Vanilla T5 seq2seq models may suffer from
exposure bias problem where they are trained with teacher forcing and are
never exposed to incorrectly generated tokens. To ensure the fluency of
generated sequences, we introduce a contrastive learning loss that explicitly
forces the model to differentiate valid output sequences for a given input
sentence. We train the T5 model by maximizing the cosine similarity of
positive representation pairs $(z_{x}^{(i)},z_{y}^{(i)})$ and minimizing that
of the negative pairs $(z_{x}^{(i)},z_{y}^{(j)})$, which is randomly sampled
from other non-target output sequence in the same batch.

$\displaystyle\mathcal{L}_{cont}(\theta)=\sum_{i=1}^{N}\log\frac{\exp(\langle
z_{x}^{(i)},z_{y}^{(i)}\rangle/\tau)}{\sum_{z_{y}^{(j)}\in S}\exp(\langle
z_{x}^{(i)},z_{y}^{(j)}\rangle/\tau)}$

In contrast with text summarization or translation tasks, we may encounter
samples with different output sequences but the same input condition sequence
$z_{x}^{(j)}=z_{x}^{(i)}$. It is not reasonable to consider these
$z_{y}^{(j)}$ as negative samples to be contrasted. Instead, we apply a mask
to only consider the negative samples in the same batch that have different
inputs $z_{x}^{(j)}\neq z_{x}^{(i)}\in S$.

Frequency-aware cross entropy loss: Maximum a-Posterior (MAP) objective may
prompt the model to sample frequent and saver tokens, which is the last thing
data augmentation would favor. To avert the low diversity problem, we follow
Jiang et al. (2019) to introduce a Frequency-Aware Cross Entropy (FACE) loss
by incorporating a weighting mechanism conditioned on token frequency. FACE
loss function of predicted output token $y_{t}$ at time step $t$ could be
expressed as:

$\displaystyle
FACE(y_{t})=-\sum^{K}_{i=1}w_{i}\delta_{i}(y_{t})\log(P(c_{i}|y_{<t},X))$

where $K$ is the vocabulary size, $c_{i}$ is the candidate token,
$\delta_{i}(y_{t})=1$ if $y_{t}=c_{i}$ and $w_{i}$ will be calculated based on
Pre-weight and Post-weight conditions. For Pre-weight,
$w_{i}=1-\frac{f_{i}}{\max_{j}f_{j}},\forall j\in\\{1...K\\}$ where $f_{i}$ is
the relative frequency of the candidate token $c_{i}$. This function will
penalize high frequency tokens to have lower weights. For Post-weight, the
function will try to penalize the model’s conservativeness: if the predicted
token $y_{t}$ has a higher frequency than the ground truth $c_{i}$, which
indicates the model conservatively picked a “safe” token, then its loss will
be scaled up by $w_{i}$ > 1.

$\displaystyle
w_{i}=1+\frac{ReLU(freq(y_{t})-freq(c_{i}))}{\sum_{j}^{K}freq(c_{j})}$

Model | Fluency | Diversity | Field KL
---|---|---|---
Base | MCL | PreW | PostW | Perplexity | Unique rate | Dist-1 | Dist-2 | Ent-4 | Device | Source | Target
cVAE | - | - | - | 4.866 | 0.9027 | 0.086 | 0.553 | 9.946 | - | - | -
pcVAE | - | - | - | 3.354 | 0.8984 | 0.078 | 0.492 | 9.878 | - | - | -
CV-BERT MLM | - | - | - | - | 0.6734 | 0.079 | 0.324 | 9.481 | 3.74E-02 | 4.80E-03 | 4.70E-03
Joint T5 |  |  |  | 2.675 | 0.7893 | 0.057 | 0.280 | 9.538 | 5.85E-02 | 3.95E-03 | 4.22E-03
Joint T5 | ✓ |  |  | 2.653 | 0.8046 | 0.059 | 0.280 | 9.613 | 8.80E-02 | 1.72E-03 | 2.13E-03
Joint T5 |  | ✓ |  | 2.999 | 0.7375 | 0.050 | 0.262 | 9.393 | 7.63E-01 | 4.10E-03 | 4.44E-03
Joint T5 |  |  | ✓ | 2.531 | 0.8065 | 0.063 | 0.293 | 9.608 | 4.63E-02 | 1.56E-03 | 1.77E-03
Joint T5 | ✓ |  | ✓ | 2.553 | 0.8007 | 0.060 | 0.291 | 9.641 | 4.28E-02 | 1.51E-03 | 1.76E-03
Test on low-count intents only
cVAE | - | - | - | 8.396 | 0.9591 | 0.259 | 0.795 | 7.739 | - | - | -
pcVAE | - | - | - | 4.009 | 0.9332 | 0.275 | 0.843 | 6.897 | - | - | -
CV-BERT MLM | - | - | - | - | 0.5886 | 0.213 | 0.638 | 7.218 | 1.26E-01 | 6.26E-03 | 5.78E-03
Joint T5 |  |  |  | 2.543 | 0.6410 | 0.195 | 0.648 | 7.210 | 1.0454 | 3.84E-03 | 3.93E-03
Joint T5 | ✓ |  |  | 2.841 | 0.6894 | 0.187 | 0.647 | 7.404 | 3.1899 | 3.10E-01 | 3.16E-03
Joint T5 |  | ✓ |  | 2.935 | 0.6336 | 0.178 | 0.632 | 7.060 | 6.0623 | 5.54E-02 | 5.50E+02
Joint T5 |  |  | ✓ | 2.307 | 0.6520 | 0.211 | 0.642 | 7.127 | 2.26E-01 | 2.47E-03 | 2.92E-03
Joint T5 | ✓ |  | ✓ | 2.325 | 0.6340 | 0.218 | 0.646 | 6.971 | 2.45E-01 | 5.52E-03 | 5.64E-03

Table 1: Intrinsic evaluation for all generation methods. We break down
different versions of Joint T5 for comparison including: MCL with masked
contrastive learning, frequency-aware loss with pre/post weight calculation
(PreW/PostW). Second half of the table indicates the results of the test set
on samples with low-count intents.

##  4 Experiment Setting

###  4.1 Dataset

We use data from a commercial conversational system in our experiments. We
select 4 critical domains: Knowledge, Shopping, Video, Books that have lower
replication accuracy on the current skill routing production system. All data
have been processed so that users are not identifiable (de-identified). For
training and intrinsic evaluation of the proposed generative models, we use a
dataset of size 4M in domains of interest for training and 10K for validation.
For testing, we prepare two test sets with one as 10k samples covering all 4
interested domains and another as 1k samples of selected low-count intents
from such domains. For the base skill routing model, the dataset contains
another $\sim$ 80M traffic covering all 37 domains. We first filter the data
with only 4 interested domains yielding 5M data. We use the proposed data
augmentation approaches to augment the original 5M data with 1:1 and 1:5
ratio. That is, each data point in this set is use to create 1 or 5
artificially generated samples by using the proposed models to re-generate one
or more of the data fields while keeping the remaining fields intact. These 1X
or 5X augmentation datasets are used with the original 80M dataset to train
the downstream skill-routing model; See App. C for setup details and network
architecture.

###  4.2 Evaluation metrics

Intrinsic evaluation: These metrics evaluate the quality of the generated
texts, irrespective of the influence on the downstream tasks: (a)
Reconstruction accuracy: we use test reconstruction loss, BLEU-1,2 between
generated and original texts. (b) Fluency: uses language model perplexity to
evaluate how fluent the generated texts are. (c) Diversity: uses the unique
rate as the proportion of unique sentences across all sentences. (d) Dist-1
and Dist-2: for a more granular evaluation of diversity, it calculates the
percentage of the number of unique 1-grams/2-grams over all 1-grams/2-grams of
tested sentence. (e) Ent-4: measures how evenly the empirical 4-gram
distribution is for a given sentence. (f) Field KL: calculates the KL
divergence between the original and the reconstructed categorical fields; see
Appendix D for more details.

Extrinsic evaluation: We train skill routing models with the original training
data and augmented data from different data augmentation methods, and evaluate
the replication accuracy for each intent while focusing the report on the four
domains of interest. Normal training (without any augmentation) serves as the
baseline while results for training with 5X oversampling of the 4-domains is
also report for benchmarking. In models trained with data augmentation, models
are trained to jointly model the three fields $(x,c^{d},c^{s})$ defined in
2.2, and in models denoted by "nlubin", we additionally change the nlu
confidence field in each hypothesis with a probability of $0.8$, over values
(HIGH, MEDIUM, LOW). The trained augmentation models are cVAE, pcVAE,
conditional variational BERT with MLM training, and joint T5 seq2seq model.

##  5 Main Results

###  5.1 Intrinsic Evaluation

Table 1 presents different model comparison for the intrinsic metrics (see
Appendix E for additional comparisons). VAE-based methods generate samples
with higher diversity but with higher perplexity, while CV-BERT MLM- which
only modifies few tokens- shows a lower unique rate but better text quality.
Overall, Joint T5 models exhibit better text quality and fluency than VAE-
based models, but with lower diversity since variations most derive from the
token sampling at the output. We found T5 with the masking contrastive loss
has better diversity and T5 with post-weight frequency loss has the best
performance both on perplexity and field generation.

Qualitative analysis: Table 2 gives some generated text samples for the
proposed generative models. It is observed that cVAE generates shorter
colloquial texts while pcVAE introduces more interesting utterances yet with
some grammatical errors; whereas T5 model generates more realistic text
examples. Finally, CV-BERT MLM is able to replace some missing tokens with
intriguing word fragments that preserve the original NLU interpretation.
However, there are also cases when the generated words are repeated with lower
quality and may have syntax errors, e.g. “what is the temperature in the
center of the center of the earth.”

Figure 4: Percentage of intents that have higher accuracy than the baseline
vs. intent count threshold. We show the total intent counts falling in each
threshold category in the parentheses, and group similar models in the same
color family for better illustration. Figure 5: Percentage of intent count
difference that exceeds 98% replication accuracy vs. intent count threshold.
Positive bars indicate the model has more intents with accuracy exceed 98%
than the baseline.

NLU Interpretations | Domain: Knowledge, Intent: QA Intent, slotKey: Question
---|---
Model | Generated texts
cVAE | when is easter
is all puma large
what’s super bowl i goes
spell onomatopoeia
pcVAE | what is sixty four minus twenty
how do you pronounce a word cable
how many hours till the philadelphia world kansas basketball game tonight
how do you probably is new york and fiftieth day
Joint T5 | how old is mary gomez
are there any more games tomorrow
what’s the current price of gold in the united states
is it still good to plant a salmon
what’s the score of the nets play today
Joint T5 | what is the song called with the lyrics that she’s not really gonna win the world
what’s the current time in Phoenix Arizona
what’s the most famous song in the world
how long does it take a cat to eat
(MCL & PostW) | tell me about this product of ninja turtles
| what’s the movie with Kevin Durant in it
CV-BERT | who is this spy $\leftarrow$ who is Sharon Carmichael
how many syllables does April have $\leftarrow$ how many provinces does Canada
have
what’s twenty hundred divided by two $\leftarrow$ what’s one fourteen divided
by two
MLM | do you know about thefire discovery in Sacramento California
|  $\leftarrow$ do you have information about the dinosaur discovery in
Jamestown California
| what is the temperature in the center of the center of the earth
$\leftarrow$ what is the
| hole in the center of the iris of the eye

Table 2: Qualitative examples of generated texts from different generative
models. Texts on the right of $\leftarrow$ in CV-BERT MLM are the original
text.

###  5.2 Extrinsic Evaluation

To study the impact of different data augmentation techniques, we utilize the
augmentation sets in training the downstream skill-routing system and evaluate
replication accuracy. In Figure 4, we calculate the percentage of intents that
improves on replication accuracy, compared with baseline system (trained
without augmentation). The figure is obtained by setting an intent count
threshold $\tau$, keeping the intents whose count falls below $\tau$, and
reporting the percentage of intent improvements by varying $\tau$. All
proposed data augmentation approaches show improvements across different
thresholds, where lower-count intents have larger improvement as that
augmented data better helps these tail segments, while simply duplicating data
(oversampling) helps with the performance to a lesser degree. Surprisingly,
some VAE models with higher diversity yet poorer quality lead to better
improvements, however increasing augmentation size is helpful in high-quality
texts by T5 model.

In Figure 5, for each method, we calculate the percentage of intent with
replication accuracy exceeding 98%. After subtracting the percentage of
intents in the baseline system, the positive bars indicate the percentage of
intents (over those with count < $\tau$) that exceed 98% than the system
without augmentation. Models pcVAE and T5_5x have the best performance, while
cVAE deteriorates the original replication accuracy performance. See Appendix
F for additional results.

Overall we make the following key observations:

  1. 1.

Text diversity seems to weigh more than text quality to boost replication
accuracy, while simply oversampling does not provide much gain.

  2. 2.

Increasing augmentation size is more useful when generated data quality is
high intrinsically.

  3. 3.

Data augmentation is more useful for intents that have lower counts.

##  6 Conclusion

In this paper, we investigated the use of heterogeneous data augmentation of
enhancing the skill routing system robustness for tail data. We empirically
explored VAE-based, conditional BERT and T5-based structures to jointly model
the routing hypothesis distribution with heterogeneous fields. In addition to
intrinsic evaluation, we also we showed that applying such data augmentation
can be an effective way to increase replication performance on tail requests
on a real-world dataset.

## Limitations

This work proposes augmentation of heterogeneous data through various
generative models in order to improve skill routing accuracy over tail traffic
segments. The main challenge of such scenaria is two fold, namely generating
meaningful high-quality text, as well as maintaining the joint distribution of
the natural dataset across all present heterogeneous fields in the artificial
set. In our experiments, we have carried out tests over real data, and
multiple fields with varying types have been modeled here. In practice there
could be many more fields that are present in the data, and expanding this
framework over more fields could be beneficial. However, as the dimension and
heterogeneity of the input fields increase, the task of learning such high-
dimensional joint distribution grows harder, which is necessary in order to
generate sound data where various realization of these fields in a given
sample are meaningful. More tests on such expanded setup may reveal that extra
measures for promoting such behaviour is required during training.

## Ethics Statement

This paper is base on utilizing generative language models in order to
increase the size of the training data through artificially-generate real-
looking data samples towards improving replication of the successful requests
in tail traffic segments. Although we do not see direct ethical risks
regarding this approach, one possible aspect could be that there is no direct
evaluation of the generated data with respect to containing any sensitive
samples such as profanity or discrimination. Nevertheless, given clean
training data for training such generative models, samples generated via these
models are also expected to be free of such cases. We did not observe any
inappropriate samples in the artificially generated texts, and performing data
cleansing for the training set is out of scope for this work.

## References

  * Berthelot et al. (2019) David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. 2019.  Mixmatch: A holistic approach to semi-supervised learning.
  * Chen and Yu (2021) Derek Chen and Zhou Yu. 2021.  Gold: Improving out-of-scope detection in dialogues using data augmentation.  _ArXiv_ , abs/2109.03079.
  * Chen et al. (2021) Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, and Diyi Yang. 2021.  An empirical survey of data augmentation for limited data learning in nlp.
  * Cho et al. (2019) Eunah Cho, He Xie, and William M. Campbell. 2019.  Paraphrase generation for semi-supervised learning in NLU.  In _Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation_ , pages 45–54, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Duan et al. (2020) Yu Duan, Jiaxin Pei, Canwen Xu, and Chenliang Li. 2020.  Pre-train and plug-in: Flexible conditional text generation with variational auto-encoders.  In _ACL_.
  * Einolghozati et al. (2019) Arash Einolghozati, Sonal Gupta, Mrinal Mohit, and Rushin Shah. 2019.  Improving robustness of task oriented dialog systems.
  * Gao et al. (2020) Silin Gao, Yichi Zhang, Zhijian Ou, and Zhou Yu. 2020.  Paraphrase augmented task-oriented dialog generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 639–649, Online. Association for Computational Linguistics.
  * Golovneva and Peris (2020) O. Yu. Golovneva and Charith S. Peris. 2020.  Generative adversarial networks for annotated data augmentation in data sparse nlu.  In _ICON_.
  * Hou et al. (2018) Yutai Hou, Yijia Liu, Wanxiang Che, and Ting Liu. 2018.  Sequence-to-sequence data augmentation for dialogue language understanding.  In _COLING_.
  * Jiang et al. (2019) Shaojie Jiang, Pengjie Ren, Christof Monz, and Maarten de Rijke. 2019.  Improving neural response diversity with frequency-aware cross-entropy loss.  In _The World Wide Web Conference on - WWW '19_. ACM Press.
  * Kachuee and Lee (2022) Mohammad Kachuee and Sungjin Lee. 2022.  Constrained policy optimization for controlled self-learning in conversational ai systems.  _arXiv preprint arXiv:2209.08429_.
  * Kachuee et al. (2022) Mohammad Kachuee, Jinseok Nam, Sarthak Ahuja, Jin-Myung Won, and Sungjin Lee. 2022\.  Scalable and robust self-learning for skill routing in large-scale conversational AI systems.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Track_ , pages 1–8, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  * Kachuee et al. (2021) Mohammad Kachuee, Hao Yuan, Young-Bum Kim, and Sungjin Lee. 2021.  Self-supervised contrastive learning for efficient user satisfaction prediction in conversational agents.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4053–4064.
  * Kurata et al. (2016) Gakuto Kurata, Bing Xiang, and Bowen Zhou. 2016.  Labeled Data Generation with Encoder-Decoder LSTM for Semantic Slot Filling.  In _Proc. Interspeech 2016_ , pages 725–729.
  * Lee et al. (2021) Seanie Lee, Dong Bok Lee, and Sung Ju Hwang. 2021.  Contrastive learning with adversarial perturbations for conditional text generation.  _ArXiv_ , abs/2012.07280.
  * Li et al. (2021) Han Li, Sunghyun Park, Aswarth Dara, Jinseok Nam, Sungjin Lee, Young-Bum Kim, Spyros Matsoukas, and Ruhi Sarikaya. 2021.  Neural model robustness for skill routing in large-scale conversational ai systems: A design choice exploration.
  * Louvan and Magnini (2020) Samuel Louvan and Bernardo Magnini. 2020.  Simple data augmentation for multilingual nlu in task oriented dialogue systems.  In _CLiC-it_.
  * Malandrakis et al. (2019) Nikolaos Malandrakis, Minmin Shen, Anuj Goyal, Shuyang Gao, Abhishek Sethi, and Angeliki Metallinou. 2019.  Controlled text generation for data augmentation in intelligent artificial agents.  In _Proceedings of the 3rd Workshop on Neural Generation and Translation_ , pages 90–98, Hong Kong. Association for Computational Linguistics.
  * Marek et al. (2021) Petr Marek, Vishal Ishwar Naik, Vincent Auvray, and Anuj Goyal. 2021.  Oodgan: Generative adversarial network for out-of-domain data generation.
  * Peng et al. (2020) Baolin Peng, Chenguang Zhu, Michael Zeng, and Jianfeng Gao. 2020.  Data augmentation for spoken language understanding via pretrained language models.
  * Qiu et al. (2020) Liang Qiu, Yizhou Zhao, Weiyan Shi, Yuan Liang, Feng Shi, Tao Yuan, Zhou Yu, and Song-Chun Zhu. 2020.  Structured attention for unsupervised dialogue structure induction.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1889–1899, Online. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(1).
  * Sarikaya (2017) Ruhi Sarikaya. 2017.  The technology behind personal digital assistants: An overview of the system architecture and key components.  _IEEE Signal Processing Magazine_ , 34(1):67–81.
  * Wang et al. (2021) Cheng Wang, Sun Kim, Taiwoo Park, Sajal Choudhary, Sunghyun Park, Young-Bum Kim, Ruhi Sarikaya, and Sungjin Lee. 2021.  Handling long-tail queries with slice-aware conversational systems.
  * Wu et al. (2018) Xing Wu, Shangwen Lv, Liangjun Zang, Jizhong Han, and Songlin Hu. 2018.  Conditional bert contextual augmentation.
  * Xia et al. (2020) Congying Xia, Chenwei Zhang, Hoang Nguyen, Jiawei Zhang, and Philip S. Yu. 2020\.  Cg-bert: Conditional text generation with bert for generalized few-shot intent detection.  _ArXiv_ , abs/2004.01881.
  * Ye et al. (2021) Seonghyeon Ye, Jiseon Kim, and Alice Oh. 2021.  Efficient contrastive learning via novel data augmentation and curriculum learning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 1832–1838, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Yoo et al. (2018) Kang Min Yoo, Youhyun Shin, and Sang-goo Lee. 2018.  Data augmentation for spoken language understanding via joint variational generation.
  * Zhang et al. (2018) Yizhe Zhang, Michel Galley, Jianfeng Gao, Zhe Gan, Xiujun Li, Chris Brockett, and Bill Dolan. 2018.  Generating informative and diverse conversational responses via adversarial information maximization.
  * Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.
  * Zhou and Peng (2019) Xinyu Zhou and Yang Peng. 2019.  Short-spoken language intent classification with conditional sequence generative adversarial network.  In _2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI)_ , pages 1753–1756.

##  Appendix A A closer review on relevant prior work

Data Augmentation in Task-oriented Dialogs. To improve the model robustness in
task-oriented dialogs (Chen et al., 2021), surface modification on texts like
paraphrasing (Gao et al., 2020; Cho et al., 2019; Einolghozati et al., 2019)
or token replacement (Louvan and Magnini, 2020; Wu et al., 2018) is popular to
introduce additional synthetic data for training and evaluating. Latent space
modification like introducing noises (Kurata et al., 2016), adversarial
training (Lee et al., 2021) and data mixing (Berthelot et al., 2019) is
another research line to enhance the smoothness of predicted data distribution
yet with lower interpretability. Many systems also augment out-of-scope
queries to cope with few-shot or zero-shot NLU scenarios (Chen and Yu, 2021;
Marek et al., 2021). In skill routing, slice-aware structure (Wang et al.,
2021) and random noise injection (Li et al., 2021) are two closest works to
improve model robustness. We follow the research heuristics to propose more
advanced data augmentation frameworks to directly apply large-scale
modification on routing data.

Conditional Text Generation. Besides modification on original data, directly
modeling conditional text distribution is another promising category for
advanced data augmentation. Seq2seq models (Hou et al., 2018; Peng et al.,
2020) are widely used to directly model output distribution. GAN-based
approaches (Golovneva and Peris, 2020; Zhou and Peng, 2019) implicitly train a
text generator that simulates real utterances, which may somehow suffer from
the mode collapse issue. Conditional Variational Auto-Encoder (cVAE) models
like modeling conditional samples (Yoo et al., 2018; Duan et al., 2020;
Malandrakis et al., 2019), introducing structure attention (Qiu et al., 2020)
or leveraging pretrained models (Xia et al., 2020) are also an important
family to improve over seq2seq models for generating more diverse and relevant
text. However, these works pay much attention on pure text generation where we
extend such framework to incorporate augmentation possibility in the
heterogeneous form of routing data.

##  Appendix B Details of Methods

In this section we provide a detailed discussion on our generative model
adaptations based on conditional VAEs and BERT.

###  B.1 Conditional VAE (cVAE) with prior network (pcVAE)

We first consider an adaptation of Variational Autoencoder (VAE) for
conditional text generation in Figure 2, which is one of the most popular
frameworks for text generation. The basic idea of VAE is to encode the input
text $x$ into a latent code $z$. To control the generation, a decoder network
then digests the concatenation of $z$ and condition embeddings $Emb(c)$ from
the desired NLU interpretations $n_{1}$ and the skill $s_{1}$ to reconstruct
the original input using samples from $z$. During inference, we only sample
$z$ from the prior distribution $\mathcal{N}(0,I)$ for decoding.

However, such model only learns an invariant representation $z$ that is
independent of the desired conditions. To better constrain the behavior of
conditional randomness, we follow Zhao et al. (2017) to introduce a prior
network that models the conditional prior distribution $P(z|c)$ and use
another recognition network $q(z|x,c)$ to approximate the true posterior
distribution $P(z|x,c)$. The variational lower bound can be written as:

$\displaystyle\mathcal{L}(x,c)$ $\displaystyle=E_{q(z|x,c)}[logP(x|z,c)]$
$\displaystyle-D_{KL}(q(z|x,c)||P(z|c))$ $\displaystyle\leq logP(x|c)$ (1)

During inference, we first sample a latent variable $z$ from the conditional
prior distribution $\mathcal{N}(z|\mu^{\prime},{\sigma^{\prime}}^{2}I)$ from
the prior network and generate texts through the decoder $P(x|z,c)$. We use
GRU networks for both text encoder and decoder.

###  B.2 Conditional Variational BERT with Mask Language Modeling (CV-BERT
MLM)

Without much training power, the generated texts from the above models could
be rather arbitrary and not grammatically sound. Moreover, we would like to
generate other data fields besides texts, such as device type and device
status signal represented as categorical ids. The most obvious way is to
randomly perturb them, which nevertheless renders undesirable samples that may
not follow the original joint distribution. To ensure the quality and
diversity of generated samples, we propose a new conditional augmentation
framework in Figure 3 by incorporating cVAE in the backbone of BERT.

Formally, we dissect BERT into two parts, where the first 6 layers serve as
the encoder and the last 6 become the decoder. The input sequence is the
concatenation of categorical fields and the utterance text. Suppose we have 3
categorical fields: $(C_{1},C_{2},C_{3})$ e.g. (Device_id, Source_id,
Target_id) and a utterance $x=(W_{1},...,W_{T})$, giving the following input
serquence:

$\displaystyle(C_{1},C_{2},C_{3},\texttt{[CLS]},W_{1},...,W_{T},\texttt{[SEP]})$
(2)

We then follow the mask language modeling (MLM) task in BERT to mask a few
random tokens in the input sequence and aim to recover these masked tokens at
the BERT output during training.

However, our goal is not to replicate the same tokens, instead replacing them
with reasonably sound counterparts to augment the original data. Therefore, we
incorporate cVAE idea mentioned in Section B.1 inside BERT. We use the encoder
to model the true posterior distribution
$P(z|x,c)\sim\mathcal{N}(z|\mu,\sigma^{2}I)$ and minimize its KL divergence
with another conditional prior distribution
$P(z|c)\sim\mathcal{N}(z|\mu^{\prime},\sigma^{\prime 2}I)$. Here we replace
the embedding of [CLS] token from the encoder output with the sampled $z$ to
create the variation. The decoder will digest the perturbed encoder outputs to
reconstruct the missing tokens. To preserve the semantics of the original
sentence, we replace the segment embeddings in BERT with the embeddings
$Emb(c)$ from the desired NLU interpretation and skill and attach it to each
time step to control the text generation.

After training, we first perform an additional part-of-speech tagging step on
the input sequence to be perturbed and determine only verbs and nouns for
masking, which are more meaningful to alter. Then we sample a latent variable
$z$ from the trained prior network $P(z|c)$ to insert in BERT and generate a
new perturbed sequence.

##  Appendix C Experimental setup

For cVAE and pcVAE, we set the word embedding size as 512 and the utterance
encoder has the hidden size of 1024 for each direction. The context encoder
has a hidden size of 128 and the response decoder has a hidden size of 1024.
The prior network consists of two fully connected networks with bottleneck
hidden size as 100 and tanh non-linearity. The latent variable $z$ has a size
of 128. The batch size is set as 256 and the models are trained end-to-end
using the Adam optimizer with a learning rate of 0.001 and 30 epochs. We set
the fixed KL loss weight as 0.1. For CV-BERT MLM, we adopt the pretrained
$\mathbf{BERT}_{base}$ as the main backbone model and include the same prior
network with $z$ size as 768. We set the mask probability of 90% for
categorical fields and 30% of utterances. We train BERT models for 15 epochs
with a learning rate of 5e-5. For T5, we set the batch size as 16, learning
rate as 1e-4 and training epochs as 5. We run each experiments on 6 Tesla V100
GPUs.

##  Appendix D Intrinsic Evaluation Metrics

We use an ensemble of automatic metrics to evaluate the performance of
generated samples without any expert-defined labels. For VAE-based methods, we
evaluate the text performance in terms of reconstruction accuracy, fluency and
diversity:

  1. 1.

Reconstruction accuracy: we use test reconstruction loss, BLEU-1,2 between
generated and original texts.

  2. 2.

Fluency: we use language model perplexity to evaluate how fluent the generated
texts are.

  3. 3.

Diversity: we use the unique rate as the proportion of unique sentences across
all sentences.

For all other methods, since there is no direct comparison in terms of
reconstruction. We focus on the metrics of fluency and diversity. We also
introduce additional diversity metrics following Zhang et al. (2018) to
increase the granularity of diversity evaluation. We use Dist-1 and Dist-2 to
calculate the percentage of the number of unique 1-grams/2-grams over all
1-grams/2-grams of tested sentence. Further, we calculate Ent-4 as how evenly
the empirical 4-gram distribution is for a given sentence ($F(w)$ is the
frequency of word $w$).

$Ent=-\frac{1}{\sum_{w}F(w)}\sum_{w\in V}F(w)log\frac{F(w)}{\sum_{w}F(w)}$ (3)

To evaluate the quality of categorical field generation whether it follows yet
not reconstructs the original distribution, we perform the monte carlo
sampling on predicted distribution from generative models and calculate the KL
divergence (Field KL) with the ground truth distribution.

##  Appendix E Intrinsic Evaluation Results

Table 3 shows a comparison of results for two VAE methods in different target
domains. First, we observe that larger training data will render better
generalization power, which gives better reconstruction accuracy and lower
perplexity in all four domains. Individual domains have rather disparate
statistics, where the knowledge domain has the poorest performance with a high
variance of utterance distribution. By introducing a prior network, it
provides a better perplexity and diversity which conditions on top-1 NLU
interpretations more explicitly.

Metrics | Reconstruction accuracy | Fluency | Diversity
---|---|---|---
|  | Test loss | BLEU-1 | BLEU-2 | Perplexity | Unique rate
Model | Domain | 1M | 4M | 1M | 4M | 1M | 4M | 1M | 4M | 1M | 4M
cVAE | Knowledge | 3.261 | 1.701 | 0.902 | 0.947 | 0.813 | 0.886 | 25.39 | 5.307 | 0.9110 | 0.8928
cVAE | Shopping | 2.019 | 1.184 | 0.928 | 0.956 | 0.666 | 0.710 | 7.385 | 3.190 | 0.8557 | 0.8680
cVAE | Video | 1.789 | 1.059 | 0.933 | 0.957 | 0.767 | 0.809 | 5.843 | 2.804 | 0.8891 | 0.8984
cVAE | Books | 2.016 | 1.299 | 0.943 | 0.958 | 0.853 | 0.881 | 7.365 | 3.579 | 0.8241 | 0.8508
pcVAE | Knowledge | 1.775 | 0.675 | 0.941 | 0.974 | 0.875 | 0.931 | 5.686 | 1.968 | 0.9596 | 0.9188
pcVAE | Shopping | 1.021 | 0.643 | 0.960 | 0.970 | 0.719 | 0.737 | 2.704 | 1.905 | 0.6767 | 0.7195
pcVAE | Video | 0.981 | 0.625 | 0.960 | 0.968 | 0.817 | 0.827 | 2.587 | 1.869 | 0.8321 | 0.8727
pcVAE | Books | 1.185 | 0.749 | 0.963 | 0.969 | 0.890 | 0.899 | 3.186 | 2.120 | 0.8302 | 0.9021

Table 3: Intrinsic evaluation for VAE-based methods. 1M and 4M indicate the
number of data for training the models.

##  Appendix F Extrinsic Evaluation Results

Sorting the intents in terms of accuracy: To see the improvement/regression
from each intent pair, in Figure 6 we first sort the raw replication accuracy
with the descending order and calculate the difference of the accuracy between
each method and the baseline.

Furthermore, Figure 7 shows the sampling predicted and true distribution of
device id from the best T5 model. We can see the model nicely follows the
original distribution with its strong representation power.

Figure 6: Replication accuracy difference from the baseline. We sort the
intent based on their accuracy in the descending order and show the accuracy
difference from the baseline. Figure 7: True and Predicted field distribution
of device id comparison.


# Wearable and Continuous Prediction of Passage of Time Perception for
Monitoring Mental Health

Lara Orlandic, Adriana Arza Valdes, David Atienza  Embedded Systems Laboratory
(ESL), Swiss Federal Institute of Technology Lausanne (EPFL), Switzerland
Email: [lara.orlandic, adrana.arza<EMAIL_ADDRESS>
###### Abstract

A person’s passage of time perception (POTP) is strongly linked to their
mental state and stress response, and can therefore provide an easily
quantifiable means of continuous mental health monitoring. In this work, we
develop a custom experiment and Machine Learning (ML) models for predicting
POTP from biomarkers acquired from wearable biosensors. We first confirm that
individuals experience time passing slower than usual during fear or sadness
(p = $0.046$) and faster than usual during cognitive tasks (p = $2\times
10^{-5}$). Then, we group together the experimental segments associated with
fast, slow, and normal POTP, and train a ML model to classify between these
states based on a person’s biomarkers. The classifier had a weighted average
F-1 score of 79%, with the fast-passing time class having the highest F-1
score of 93%. Next, we classify each individual’s POTP regardless of the task
at hand, achieving an F-1 score of 77.1% when distinguishing time passing
faster rather than slower than usual. In the two classifiers, biomarkers
derived from the respiration, electrocardiogram, skin conductance, and skin
temperature signals contributed most to the classifier output, thus enabling
real-time POTP monitoring using noninvasive, wearable biosensors.

###### Index Terms:

passage of time perception, wearable sensors, mental health monitoring,
machine learning, biomarkers

##  I INTRODUCTION

The way in which humans perceive the passage of time is a psychological and
neurological phenomenon linked to emotions, memory, attention, and the body’s
response to stress [1, 2]. When people are busy, amused or excited, they
experience time passing faster than it truly does. Conversely, when people are
afraid or under stress, time seems to slow down [2, 3, 1]. Furthermore, during
the recent COVID-19 pandemic lockdown, it was found that the experience of
time passing slowly was associated with increased stress, decreased task load,
and decreased satisfaction with one’s amount of social interactions [4].
Moreover, a person’s passage of time perception (POTP) is a quantifiable
measure that is intricately linked to their mental state [3].

Along with distortions in POTP, various emotions also induce changes in
physiological processes such as heart rate, blood pressure, muscular
contraction and respiration [5, 3]. This phenomenon illustrates the notion
that our perception of time is related to our homeostatic state, and thus, to
our physiological stress response that our body triggers to deal with a
disturbance in homeostatic balance [6].

Physiological monitoring through noninvasive biosensors has previously been
used as a means of monitoring mental health conditions including depression,
anxiety, bipolar disorders, and many more [7]. These sensors enable continuous
measurements and online estimations of a person’s emotions [8], thereby
providing real-time insights into their mental state and facilitating timely
interventions in the case of deteriorating mental health. Biomarkers computed
from such sensors have been previously used to classify a person’s stress
response [9, 8], cognitive load [10], and emotions on the arousal-valence
scale [11]. However, no study has thus far has used unobtrusively measured
biosignals to predict individuals’ POTP, which provides additional insights
into their mental state.

Figure 1: Overall methodology of the POTP statistical analysis and
classification

In this study, we assess the hypothesis that a person’s POTP relates to their
stress-related physiological processes, as these relate to their mental state.
Thus, we aim to use noninvasive biosignal monitoring and Machine Learning (ML)
tools to predict a person’s POTP, both directly through binary classification,
and indirectly through multi-class prediction of activities that elicit
significant changes in POTP. We first investigate the correlations between
different induced emotions and their effects on subjects’ POTP. Then, we
propose a ML model training and optimization procedure for predicting the
experimental segment – and its corresponding POTP – a subject experienced
given their physiological features. Next, we use this ML technique to
distinguish periods of fast time perception from those of slow time perception
across all experimental segments. Finally, we investigate the influence of
individual biomarkers in the POTP and emotion classification outcome to
determine which features affect each model.

##  II METHODS

The set of contributions of this work, namely, the quantification and
classification of POTP, is presented in Fig. 1. First, various emotional
states are induced in the subject through a custom-designed experimental
protocol. Since the physiological stress response has been associated with
different levels of cognitive workload, job performance, emotions, and optimal
physical states [9, 8], we investigate multiple biosignals and the correlation
of their changes with the perception of passage of time during a variety of
stimuli that elicit different emotions, cognitive loads, and stress levels.
Thus, we induce emotions and cognitive states on a set of participants as they
perform specific tasks and watch emotional short films while we measure their
response to these stimuli, as well as their sense of the passage of time.

Throughout the experiment, five biosignals are measured – electrocardiogram
(ECG), skin temperature (SKT), electrodermal activity (EDA), respiration
(RSP), and photoplethysmography (PPG) – for their proven contribution to
emotion and psychological stress monitoring [8, 9]. Next, we develop a ML
optimization procedure to select the ML model and hyperparameters that perform
the best classification of emotional states and time perceptions based on the
subjects’ physiological features. Finally, the models are tested to predict
subjects’ emotional states and time perceptions.

###  II-A Induced Emotional States: Experimental Protocol Design

In order to investigate the relationship between subjects’ emotional state and
their POTP, we designed an experiment to elicit various emotions in healthy
volunteers using films, cognitive tasks, and intermittent rest states. The
exact experimental protocol and the groupings of segments for statistical
analysis are displayed in Table I:

TABLE I: Experimental protocol No | Segment | Duration | Class
---|---|---|---
1 | Relaxation audio | 3 min | Rest
2 | Neutral Clip | 2 min | Neutral
3 | Rest | 2 min | Neutral
4 | Fear Clip | 2 min | Emotional
5 | Mathematics Task | 3 min | Cognitive
6 | Rest | 1.5 min | Rest
7 | Stroop Color Test | 1.5 min | Cognitive
8 | Sadness Clip | 1.5 min | Emotional
9 | Rest | 3 min | Rest

The video clips were selected from the Emotional Movie [12] and the FilmStim
Databases [13], which are two validated databases containing footage to induce
specific emotions. The mathematics activity consists of solving arithmetic
tasks given time constraints with startling negative feedback [14]. The Stroop
Color Test is a color-word reading exercise used to measure cognitive
flexibility and working memory [15].

Between experimental segments, the subject completes a questionnaire including
a visual analogue scale for stress (VASS), in which they select their stress
level at that moment from 0 to 100 [16]. Also, the subjects estimate the
duration of the past experimental segment using a time scale ranging from 0 to
5 minutes in increments of 30 seconds. Rest phases are included between
experimental phases to give the subject time to reset their emotional state to
its baseline. During the experiment, the Shimmer Node3 ECG [17] and Empatica
E4 wristband [18], which are two lightweight and unobtrusive sensing devices,
are used to measure the 5 biosignals.

There were 18 participants recruited for this study: 13 males and 5 females
between the ages of 22 and 31. Each experiment was conducted in one sitting
using a custom-made Android application to display the instructions and tasks.
The ethical approval for this study was obtained from the Cantonal Ethics
Commissions for Human Research Vaud (ID 2019-00321).

Figure 2: Pipeline for machine learning model selection and testing

###  II-B Biosignal Measurement and Analysis

The body’s physiological response to emotions and stress is characterized by
the combination of several biometric variables [9]. These variables can be
monitored through parameters derived from biosignals that are unobtrusively
measured and continuously monitored. After preprocessing each signal (i.e.,
filtering and delineation), various parameters are extracted. The
preprocessing algorithms and primary parameters obtained from each time series
signal are computed as in [8], [10], and [19]. Next, 80 physiological features
in the time and frequency domains are extracted from the parameters in
segmentation windows of 45s. Previous works have used windows of length 60s
[8, 20, 9], but learning curves indicated that this produced insufficient data
for training. Therefore, we reduced the window length for feature extraction
to 45s as a data augmentation technique. These features capture the subject’s
physiological response during each emotional state. Several key biosignal
parameters and features from were the biomarkers are selected, are described
as follows:

####  II-B1 SKT

For the SKT signal, we compute the gradient ($SKT\\_gradient$) and total
power, ($SKT\\_power$) of its power spectral density (PSD).

####  II-B2 EDA

The EDA signal is divided into two main components: the skin conductance level
(SCL) and the skin conductance response (SCR) as the driver phasic signal
($SCR\\_power$). The gradient and mean of the SCL ($SCL\\_gradient$,
$SCL\\_mean$) are obtained.

####  II-B3 RSP

We compute respiration rate and period ($RSP\\_Rate$,$RSP\\_Prd$), duration of
air inhaled ($InspTime$) and exhaled ($ExpTime$) and compute statistics based
on these parameters. In the frequency domain, we compute the PSD in four
different bands of equal bandwidth between 0-1 Hz ($RSP\\_PSD_{1-4}$).
Additionally, we consider the normalized band power in these four bands
($RSP\\_nPSD_{1-4}$), as well as in 5 fine-grained bands in 0.08-0.6 Hz
($RSP\\_pBF_{1-5}$). Then, we extract $RSP\\_F1pond$, which is the mean
frequency of a Gaussian distribution used to fit the PSD estimated in the HF
band ($0.15-0.5Hz$). Moreover, we applied the method proposed in [21] to
compute the estimated respiratory frequency, as the largest peak power
($RSP\\_Pk$) of the Lomb-Scargle PSD of respiration using a Welch periodogram.
Finally, we compute the average signal power across all windows
($RSP\\_power$).

####  II-B4 ECG

From the ECG, the RR intervals are obtained as in [22], and then features are
extracted based on the Heart Rate Variability (HRV) analysis [23], such as its
mean ($ECG\\_RR\\_mean$), median ($ECG\\_RR\\_median$) and standard deviation
($ECG\\_RR\\_SDNN$). Additionally, we compute its normalized bandpower in the
very low frequency ($ECG\\_RR\\_nVLF$), low frequency ($ECG\\_RR\\_nLF$), and
high frequency ($ECG\\_RR\\_nHF$) bands centered at 0.04 Hz, 0.15 Hz, and 0.4
Hz, respectively. Non-linear features are also extracted from Poincaré plot
indicating vagal and sympathetic function. They are the following: the length
of the transverse axis ($ECG\\_RR\\_T$), the length of the longitudinal axis
($ECG\\_RR\\_L$), and their ratio, called Cardiac Sympathetic Index
($ECG\\_RR\\_CSI$), as well as the modified CSI ($ECG\\_RR\\_CSI\\_modified$)
[20].

####  II-B5 PPG

We compute several PPG parameters, including the pulse period ($PPG\\_PP$),
pulse wave rising time ($PPG\\_PRT$), pulse wave decreasing time
($PPG\\_PDT$), pulse width until reflected wave ($PPG\\_PW$). We then extract
ensemble statistics from each parameter, as well as the same frequency
analysis as for the RR intervals.

###  II-C Passage of Time Perception Assessment

In order to quantify subjects’ POTP, we define the relative time estimation
error metric $t_{rel}$. This metric is calculated based on the correct segment
time $t_{correct}$ and the subjects’ estimation of the passed time
$t_{perceived}$, as shown in Equation 1. A positive $t_{rel}$ means that the
person experienced time as passing faster than it truly did, whereas a
negative $t_{rel}$ corresponds to the perception of time passing slower.

$t_{rel}=\frac{t_{correct}-t_{perceived}}{t_{correct}}*100$ (1)

The segments are grouped into three categories, as displayed in Table I:
emotional, cognitive, and neutral. Neutral segments are placed after the
initial rest period to avoid bias by any previous experimental segment or
sensor placement. We then test the statistical significance in the difference
of means of $t_{rel}$ for each category to confirm previous hypotheses that
each segment corresponds to a given POTP.

###  II-D ML Model of Emotional State and Time Passage

Next, we train ML models to predict a person’s POTP based on their
physiological features. Hence, we perform two classification tasks: a binary
classification to determine when each person determines that time is passing
faster rather than slower than usual, as well as a multiclass classification
of the experimental state of the user (emotional, cognitive, neutral), as
these states each correspond to a different POTP.

We compare 8 state-of-the art ML classification algorithms to perform the
inference: Logistic Regression (LogReg), Decision Tree Classifier (DTC), k
Nearest Neighbor (KNN), Linear Discriminant Analysis (LDA), Gaussian Naive
Bayes (GNB), Support Vector Machines (SVM), Random Forest (RF) and eXtreme
Gradient Boosting (XGB). A ML model development pipeline is implemented to
ensure generalizability of the chosen model across all subjects, as displayed
in Fig. 2.

First, every biosignal for each experimental segment is divided into 45 s non-
overlapping segments. The signals are then preprocessed and the aforementioned
features are extracted. Features with multiple NaN values are removed as they
may be unstable. Then, all features are scaled by subtracting the mean and
dividing by the standard deviation of each feature using the training dataset.
In both classification tasks, the same randomly-selected 22% of subjects is
designated as the testing set to assess the generalizability of the trained
model. No samples belonging to the same subject appear in both the testing and
training sets, nor the training and validation sets of each cross-validation
(CV) fold.

Next, we train all eight ML models and perform 10-fold Leave-n-Subjects-Out
CV, thereby ensuring that signal segments belonging to the same subject are
not used for training and validation at a given CV fold. 20% of the subjects
are used for validation in each fold. The utilized metric is the F-1 score in
the case of binary classification, and weighted average F-1 score in the
multi-label case. The selected model is the one with the highest mean F-1
score across the training folds. In the case that multiple algorithms produce
similar mean F-1 scores, the learning curves of the algorithms are analyzed to
examine the bias-variance trade-off. The model with the higher variance than
bias is selected, as the subsequent hyperparameter optimization and feature
elimination steps intend to reduce overfitting [24].

Once the algorithm is selected, its hyperparameters are tuned using Tree-
structured Parzen Estimators (TPE) [25] with the objective of maximizing the
mean CV F-1 score. Next, Recursive Feature Elimination with Cross-Validation
(RFECV) is performed to remove features that do not contribute to the
classification outcome [24]. Following RFECV, the aforementioned TPE procedure
is performed to re-optimize the model to its new feature set.

##  III RESULTS

First, we analyze the results of the statistical analysis of relative time
errors in the feature set to determine which experimental segments truly
produce significant POTP distortions. We use this information, along with the
distribution of $t_{rel}$ values, to set up the emotion classification
procedure and determine thresholds for the POTP classification. ML models are
trained to perform each classification task based on the extracted biomarkers
and subsequently analyze each model’s generalization capabilities.

Figure 3: Average $t_{rel}$ (blue), with standard deviations (black), for each
experimental segment

###  III-A Passage of Time Perception

The average $t_{rel}$ across all subjects for each segment are displayed in
Fig. 3. We can see that the most positive $t_{rel}$ occurs for the mathematics
test, whereas the most negative one occurs for the fear clip.

Next, we performed one-sample, one-tailed t-tests to determine whether or not
these time errors deviated significantly from zero during each experimental
segment. Tasks corresponding to the same hypothesized POTP direction were
grouped together, as described in Section I. The null hypothesis for neutral
(rest and neutral clip) and emotional (fear and sadness) tasks is that the
time error is less than zero, since people typically perceive time as passing
slower when they are bored or afraid. Conversely, for the cognitive
(mathematics and Stroop) tasks, we ran a right-tailed t-test, since people
normally perceive time as passing faster when they are busy. This grouping
leads to a nearly balanced sample of neutral, emotional, and cognitive
segments.

The results of the t-tests are summarized in Table II. The neutral segments
showed no significant deviation from zero (p = 0.942). Conversely, the
cognitive tasks and the emotional tasks were significantly higher than zero (p
= $2.01\times 10^{-5}$) and lower than zero (p = 0.0456), respectively. The
sadness clip showed significant variance in $t_{rel}$ and no significant
deviation from zero by itself, perhaps due to the short duration of the clip
and subjectivity in individuals’ perceptions of sadness.

TABLE II: Statistical Analysis Results Class | Average $t_{rel}$ | P-Value | Pass. of Time
---|---|---|---
Emotional | -16.1% | 0.0456 | Slower
Neutral | 6.94% | 0.942 | No change
Cognitive | 23.6% | $2\times 10^{-5}$ | Faster

The reported VASS stress levels were highest during the mathematics task
($63\pm 26$) and fear clips ($38\pm 20$). There was no significant correlation
($\alpha=0.05$) between the stress levels and either the signed or absolute
value of $t_{rel}$.

Finally, to facilitate classification based directly on POTP, we define
thresholds on $t_{rel}$ to identify biosignal segments during which the
individual subjects experience time passing significantly fast and slowly.
Fig. 4 shows a histogram of all $t_{rel}$ values in the training dataset. We
notice a bi-modal distribution of positive and negative time errors, so we fit
Gaussian curves to the positive and negative $t_{rel}$ values. The upper
threshold is located two standard deviations to the right of the mean of
negative $t_{rel}$ values, while the lower threshold is two standard
deviations to the left of the positive $t_{rel}$ mean. This process provides
an empirical estimation of statistically significant low and high values of
$t_{rel}$.

Figure 4: Classification thresholds for $t_{rel}$

Therefore, our final classification labels for the passage of time are as
follows:

  * •

Time passes faster (1) if $t_{rel}>10$

  * •

Time passes slower (-1) if $t_{rel}<-19$

This grouping leads to an imbalanced sample of the two classes, since there
are about twice as many segments in which time passes faster rather than
slower.

###  III-B POTP Classification

Once we determined which experimental states corresponded to a faster, slower,
or normal POTP, we built a ML model to classify these states. We first use the
procedure described in Fig. 2 to train a ML model to determine whether a
person was in the emotional, cognitive, or neutral phases of the experiment
based on their physiological features. The algorithm selection procedure is
shown in Fig. 5, which displays the CV F-1 scores of each model.

Figure 5: CV mean and st. dev. F-1 scores for each ML model Figure 6: Learning
curves of the SVM and RF algorithms

We note from Fig. 5 that SVM exhibits the highest mean F-1 score, but its
standard deviation is much larger than that of RF, which has a similar mean.
To finalize our model selection, we examine the learning curves, as shown in
Fig. 6.

We can see that the training and CV accuracies of the SVM model converge
quickly to around 0.7, implying low variance and high bias, and in the RF
there remains a large gap between the training and validation accuracies,
implying high variance. We therefore select the RF model for further analysis
because its variance can be reduced using TPE and RFECV. Following TPE, the
average CV score of the RF model increased by 7.2%, as shown in Table III.
Then, RFECV revealed that the optimal number of features was 45.

TABLE III: ML Optimization Step-by-Step Results ML Step | Mean CV F-1 Score (%)
---|---
3-Class RF | 2-Class XGB
First training | 56.7% | 69.0%
Hyperparameter Opt. | 63.3% | 70.6%
RFECV | 63.3% (45 feat.) | 70.8% (18 feat.)
Hyperparameter Re-Opt. | – | 72.4%
Test on Unseen Data | 79.0% | 77.1%

The final model is then tested on new, unseen data from four subjects. The
results are displayed in the confusion matrix in Fig. 7, as well as the F-1
scores in Table IV. We can see that the “Fast POTP” class is the easiest for
the classifier to distinguish, with a 100% precision and highest F-1 score of
93%. Most of the misclassifications are due to the “No Change in POTP” signals
being classified as “Slow POTP” signals. The weighted average of the F-1
scores of all classes with respect to the number of data points per class is
79%.

TABLE IV: Emotion Classifier Results on Unseen data Class | Precision | Recall | F-1 Score
---|---|---|---
Emotional (Slow POTP) | 64% | 88% | 74%
Neutral (No change POTP) | 83% | 62% | 71%
Cognitive (Fast POTP) | 100% | 87% | 93%
Weighted Avg. | 82% | 79% | 79%
Figure 7: Confusion matrix for the emotion classifier

Finally, we compute the Shapley (SHAP) values of the model, which are measures
of the relative importance of the features in the model’s classification
decision [26]. The features and their relative importances to each class are
displayed in Fig. 8. We can see that by far, the most important features are
the $SCL\\_gradient$ and $SKT\\_power$. The remaining features relate to the
ECG, RSP, SKT, SCR, and SCL signals.

The SHAP values of the passage of time perception classifier are displayed in
Fig. 9. In this case, the two most important features are PSD features of the
ECG and RSP inspiration time. Other important features are derived from the
ECG and RSP signals. All of the important RSP features relate to the time-
domain inspiration time parameter, whereas the important ECG features are
computed using the time and frequency domain of the R-R interval signal.

Figure 8: SHAP values indicating the feature importance to the output of the
emotion classifier Figure 9: SHAP values of the time perception classifier

##  IV DISCUSSION AND FUTURE WORK

The way a person perceives the passage of time is a quantifiable metric that
indicates their mental state. Therefore, continuous monitoring of subjects’
POTP may provide insights into their mental well-being. In this work, we have
developed an experiment to induce certain emotions that have known effects on
the passage of time. Our results have confirmed our hypotheses that people
consistently interpret time as passing slower during fear or sadness (p =
0.046), and faster during mentally taxing tasks (p = $2\times 10^{-5}$). Then,
we have developed a classifier to predict which experimental state, which is
correlated to a change or lack thereof in POTP, a subject experienced based on
physiological features extracted from unobtrusively measured biosignals. This
classifier obtained a weighted average F-1 score of 79%, with fast POTP tasks
being the easiest to distinguish with an F-1 score of 93%.

Next, we classify the POTP directly by identifying segments with significantly
high and low $t_{rel}$ values regardless of the task at hand. We obtain a
77.1% F-1 score in distinguishing time passing fast rather than slow, meaning
that is possible to determine a person’s POTP based solely on their
physiological signals. When we analyzed the feature importance of the two
models, the emotion classifier used more diverse biosignals than the POTP
classifier, the latter of which did not heavily weigh features derived from
SCL or SKT. Both classifiers place heavy importance on the
$ECG\\_RR\\_median$, $ECG\\_RR\\_mean$, $ECG\\_RR\\_SDNN$, and
$RSP\\_InspTime\\_mean$ biomarkers. These results indicate that by monitoring
a few biosignals with simple, wearable sensors, it is possible to
unobtrusively monitor POTP and mental state on a continuous basis.

In the future, these preliminary results may be enhanced by using a larger,
more diverse sample of subjects for testing and training, as well as longer
experimental segments. These factors may help overcome the high variance seen
in $t_{rel}$ of the sadness clip, as well as the class imbalance in the binary
classification task. With more individuals and longer biosignal durations,
Deep Learning analysis may be employed.

## ACKNOWLEDGMENT

This work has been partially supported by the Innosuisse Innocheque (No.
38484.1), the ML-edge Swiss National Science Foundation (NSF) Research project
(GA No. 200020182009/ 1), and the MyPreHealth research project (Hasler
Foundation project No. 16073). We would also like to thank Giulio Masinelli
for his help with the experimental setup.

## References

  * [1] S. Droit-Volet and W. H. Meck, “How emotions colour our perception of time,” _Trends in Cognitive Sciences_ , vol. 11, no. 12, pp. 504–513, 12 2007.
  * [2] K. van Hedger _et al._ , “The influence of social stress on time perception and psychophysiological reactivity,” _Psychophysiology_ , vol. 54, no. 5, pp. 706–712, 5 2017.
  * [3] S. Droit-Volet and S. Gil, “The emotional body and time perception,” _Cognition and Emotion_ , vol. 30, no. 4, pp. 687–699, 5 2016.
  * [4] R. S. Ogden, “The passage of time during the UK Covid-19 lockdown,” _PLoS ONE_ , vol. 15, no. 7, 7 2020.
  * [5] M. T. Valderas _et al._ , “Human emotion recognition using heart rate variability analysis with spectral bands based on respiration,” in _Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS_ , 2015, pp. 6134–6137.
  * [6] R. S. Lazarus, “From psychological stress to the emotions: a history of changing outlooks.” _Annual review of psychology_ , vol. 44, pp. 1–21, 1 1993.
  * [7] E. Garcia-Ceja, V. Osmani, and O. Mayora, “Automatic stress detection in working environments from smartphones’ accelerometer data: A first step,” _IEEE Journal of Biomedical and Health Informatics_ , vol. 20, no. 4, pp. 1053–1060, 7 2016.
  * [8] V. Montesinos _et al._ , “Multi-modal acute stress recognition using off-the-shelf wearable devices,” in _Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS_ , 2019, pp. 2196–2201.
  * [9] A. Arza _et al._ , “Measuring acute stress response through physiological signals: towards a quantitative assessment of stress,” _Medical and Biological Engineering and Computing_ , pp. 1–17, 8 2018.
  * [10] G. Masinelli _et al._ , “Self-aware machine learning for multimodal workload monitoring during manual labor on edge wearable sensors,” _IEEE Design and Test_ , vol. 2356, no. c, pp. 1–7, 2020.
  * [11] L. Santamaria-Granados _et al._ , “Using deep convolutional neural network for emotion detection on a physiological signals dataset (AMIGOS),” _IEEE Access_ , vol. 7, pp. 57–67, 2019.
  * [12] S. Carvalho _et al._ , “The emotional movie database (EMDB): A self-report and psychophysiological study,” _Applied Psychophysiology Biofeedback_ , vol. 37, no. 4, pp. 279–294, 12 2012.
  * [13] A. Schaefer _et al._ , “Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers,” _Cognition and Emotion_ , vol. 24, no. 7, pp. 1153–1172, 2010.
  * [14] R. Baumeister and K. Vohs, _Encyclopedia of Social Psychology_ , E. Harmon-Jones and P. Winkielman, Eds. California: SAGE Publications, Inc., 2007.
  * [15] F. Scarpina and S. Tagini, “The Stroop color and word test,” _Frontiers in Psychology_ , vol. 8, no. APR, p. 557, 4 2017.
  * [16] F. X. Lesage, S. Berjot, and F. Deschamps, “Clinical stress assessment using a visual analogue scale,” _Occupational Medicine_ , vol. 62, no. 8, pp. 600–605, 12 2012.
  * [17] Shimmer, “3 ECG Unit,” http://www.shimmersensing.com/products.
  * [18] Empatica, “E4 Wristband,” https://www.empatica.com/research/e4/.
  * [19] G. Masinelli _et al._ , “SPARE: A spectral peak recovery algorithm for PPG signals pulsewave reconstruction in multimodal wearable devices,” _Sensors_ , vol. 21, no. 8, p. 2725, 4 2021.
  * [20] F. Dell’Agnola _et al._ , “Cognitive workload monitoring in virtual reality based rescue missions with drones,” in _12th International Conference on Virtual, Augmented and Mixed Reality, Copenhagen, Denmark_ , 7 2020\.
  * [21] A. Hernando _et al._ , “Inclusion of respiratory frequency information in heart rate variability analysis for stress assessment,” _IEEE Journal of Biomedical and Health Informatics_ , vol. 20, no. 4, pp. 1016–1025, 7 2016\.
  * [22] L. Orlandic _et al._ , “Reward: Design, optimization, and evaluation of a real-time relative-energy wearable r-peak detection algorithm,” in _41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)_ , 2019, pp. 3341–3347.
  * [23] Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology., “Heart rate variability: standards of measurement, physiological interpretation and clinical use.” _Circulation_ , vol. 93, no. 5, pp. 1043–1065, 3 1996.
  * [24] M. A. Munson and R. Caruana, “On feature selection, bias-variance, and bagging,” in _Lecture Notes in Computer Science_ , vol. 5782 LNAI, no. PART 2, 2009, pp. 144–159.
  * [25] J. Bergstra _et al._ , “Algorithms for hyper-parameter optimization,” _Neural Information Processing Systems (NIPS 2011)_ , vol. 24, dec 2011.
  * [26] E. Štrumbelj and I. Kononenko, “An efficient explanation of individual classifications using game theory,” _Journal of Machine Learning Research_ , vol. 11, pp. 1–18, 2010.