# Unveiling Black-boxes: Explainable Deep Learning Models for Patent
Classification

Md Shajalal1,2, Sebastian Denef3, Md. Rezaul Karim4, Alexander Boden1,5,
Gunnar Stevens2
1Fraunhofer-Institute for Applied Information Technology FIT, Germany
2University of Siegen, Germany
3AGENTS.inc, Berlin, Germany
4RWTH Aachen University, Aachen, Germany
5Bonn-Rhein-Sieg University of Applied Sciences, Germany

Abstract. Recent technological advancements have led to a large number of
patents in a diverse range of domains, making it challenging for human experts
to analyze and manage. State-of-the-art methods for multi-label patent
classification rely on deep neural networks (DNNs), which are complex and
often considered black-boxes due to their opaque decision-making processes. In
this paper, we propose a novel deep explainable patent classification
framework by introducing layer-wise relevance propagation (LRP) to provide
human-understandable explanations for predictions. We train several DNN
models, including Bi-LSTM, CNN, and CNN-BiLSTM, and propagate the predictions
backward from the output layer up to the input layer of the model to identify
the relevance of words for individual predictions. Considering the relevance
score, we then generate explanations by visualizing relevant words for the
predicted patent class. Experimental results on two datasets comprising two-
million patent texts demonstrate high performance in terms of various
evaluation measures. The explanations generated for each prediction highlight
important relevant words that align with the predicted class, making the
prediction more understandable. Explainable systems have the potential to
facilitate the adoption of complex AI-enabled methods for patent
classification in real-world applications.††This is the “Submitted Manuscript”
to the 1st World Conference on eXplainable Artificial Intelligence (xAI2023),
Lisbon, Portugal. The published manuscript by Springer can be found here
https://doi.org/10.1007/978-3-031-44067-0_24

_Keywords_ Patent Classification $\cdot$ Explainability $\cdot$ Layer-wise
relevance propagation $\cdot$ Deep Learning $\cdot$ Interpretability

##  1 Introduction

Patent classification is an important task in the field of intellectual
property management, involving the categorization of patents into different
categories based on their technical contents [1]. Traditional approaches to
patent classification have relied on manual categorization by experts, which
can be time-consuming and subjective [2]. However, due to the exponential
growth of patent applications in recent times, it has become increasingly
challenging for human experts to classify patents. The international patent
classification (IPC) system, which consists of 645 labels for the general
classes and over 67,000 labels for the sub-groups, reflects the magnitude of
challenges in multi-level patent classification tasks [1]. Furthermore, patent
texts are generally lengthy and contain irregular scientific terms, making
them a challenging field of application for text classification approaches, as
patents often include highly technical and scientific terms that are not
commonly used in everyday language, and authors often use jargon to make their
patents unique and innovative [3]. These factors contribute to the significant
challenges associated with patent classification, making it a formidable task.

However, recent advancements in machine learning (ML) and deep neural network
(DNN) have made significant progress in automating the patent classification
process. In the past, classical ML models, such as support vector machine
(SVM), K-nearest neighbour, and naive bayes, have been widely used to
automatically classify patent texts [4]. However, more recently, several DNN
models have been proposed to address the challenges associated with patent
classification. Generally, these models represent patent text using word
embedding and transformer-based pre-trained models [5, 6, 1, 2, 7]. The DNN
models, including recurrent neural networks (RNN) and their variants such as
convolutional neural networks (CNN), long short-term memory networks (LSTM),
bidirectional LSTM (Bi-LSTM), and gated recurrent unit (GRU), can learn to
classify patents based on their textual content [5, 7, 2, 8, 9]. Hence, these
enable faster and more reliable categorization of patents and scientific
articles.

Mathematically, DNN-based classification approaches are often complex in their
architecture, and the decision-making procedures can be opaque [10, 11]. While
these approaches may exhibit efficient performance in classifying patents, the
decisions they make are often not understandable to patent experts, or even to
practitioners of artificial intelligence (AI). As a result, it is crucial to
ensure that the methods and decision-making procedures used in patent
classification are transparent and trustworthy, with clear explanations
provided for the reasons behind each prediction. This is particularly
important because patents are legal documents, and it is essential to
comprehend the reasoning behind the classification decisions made by the
model. Therefore, patent classification models should be designed to be
explainable, allowing the reasons and priorities behind each prediction to be
presented to users. This will help build trust in the predictive models and
promote transparency among users and stakeholders.

For text-based uni-modal patent classification tasks, explanations can be
provided by highlighting relevant words and their relevance to the prediction,
thus increasing trust of users in the accuracy of predictions. In recent
years, there has been a growing interest in developing explainable artificial
intelligence (XAI) to unveil the black-box decision-making process of DNN
models in diverse fields, including image processing [12], text processing,
finance [13, 14], and health applications [15, 16]. These XAI models can
provide insights into the decision-making process, explaining the reasoning
behind specific predictions, the overall model’s priorities in decision
making, and thereby enhancing the transparency and trustworthiness of the
application [11, 17, 10, 18, 12].

In this paper, our goal is to develop a patent classification framework that
not only predicts the classes of patents but also provides explanations for
the predicted classes. To achieve this, we propose a new explainable method
for patent classification based on layer-wise relevance propagation (LRP).
This method can break down the contribution of patent terms that are crucial
in classifying a given patent into a certain class. We start by representing
the patent terms using a high-dimensional distributed semantic feature vector
obtained from pre-trained word-embedding models. Next, we proceed to train
several DNN-based models, including Bi-LSTM, CNN, and CNN-BiLSTM, which are
capable of predicting the patent class. Finally, the LRP-enabled explanations
interface highlights relevant words that contributed to the final prediction,
providing an explanation for the model’s decision.

We conducted experiments using two benchmark patent classification datasets,
and the experimental results demonstrated the effectiveness of our approach in
both classifying patent documents and providing explanations for the
predictions. Our contributions in this paper are twofold:

  1. 1.

We propose an LRP-based explainability method that generates explanations for
predictions by highlighting relevant patent terms that support the predicted
class.

  2. 2.

Our developed DNN models show effective performance in terms of multiple
evaluation metrics on two different benchmark datasets, and performance
comparison with existing works confirms their consistency and effectiveness.

Overall, explainable DNN models offer promising solutions for patent
classification, enabling faster and more accurate categorization while
providing insights into the decision-making process. With the increasing
volume of patent applications, the development of such explainable models
could be beneficial in automatically categorizing patents with efficiency and
transparency.

The rest of the paper is structured as follows: section 2 presents the summary
of existing research on patent classification. Our proposed explainable deep
patent classification framework is presented in section 3. We demonstrate the
effectiveness of our methods in classifying patents and explaining the
predictions in detail in section 4. Finally, section 5 concluded our findings
with some future directions in explainable patent classification research.

##  2 Related Work

In recent years, the patent classification task has gained significant
attention in the field of natural language processing (NLP) research, as
evidenced by several notable studies [19, 2, 3]. Various methods have been
employed for classifying and analyzing patent data, and the methods can be
categorized based on different factors such as the techniques utilized, the
tasks’ objectives (e.g., multi-class or multi-level classification), and the
type of resources used to represent the patent data (i.e., uni-modal vs multi-
modal) [20, 7, 9]. However, traditional approaches have relied on classical ML
and bag-of-words (BoW)-based text representation, which have limitations in
capturing semantic and contextual information of the text, as they can only
capture lexical information. With the advent of different word-embedding
techniques such as _word2vec_ by Mikolov et al. [21, 22], _Glove_ by
Pennington et al. [23], and _FastText_ by Bojanowski et al. [24], the NLP
research has been revolutionized with the ability to represent text using
high-dimensional semantic vector representations [25, 26, 27]. More recently,
there has been a growing trend in employing transformer-based pre-trained
models, including deep bidirectional transformer (BERT) [28], robust optimized
BERT (RoBERTa) [29], distilled BERT (DistilBERT) [30], and XLNet [31], for
text representation in NLP tasks.

Shaobo et al. [2] introduced a deep patent classification framework that
utilized convolutional neural networks (CNNs). They started by representing
the text of patents, which was extracted from the title and abstract of the
USPTO-2 patent collection, using a skip-gram-based word-embedding model [2].
They then used the resulting high-dimensional semantic representations to
train CNN model. Similarly, Lee et al. [3] also employed a CNN-based neural
network model, however, they fine-tuned a pre-trained BERT model for text
representations. A DNN-based framework employing Bi-LSTM-CRF and Bi-GRU-HAN
models has been introduced to extract semantic information from patents’ texts
[7].

A multi-level classification framework [9] has been proposed utilizing fine-
tuned transformer-based pre-trained models, such as BERT, XLNet, RoBERTa, and
ELECTRA[32]. Their findings revealed that XLNet outperformed the baseline
models in terms of classification accuracy. In another study, Roudsari et al.
[20] addressed multi-level (sub-group level) patent classification tasks by
fine-tuning a DistilBERT model for representing patent texts. Jiang et al. [6]
presented a multi-modal technical document classification technique called
TechDoc, which incorporated NLP techniques, such as word-embedding, for
extracting textual features and descriptive images to capture information for
technical documents. They modelled the classification task using CNNs, RNNs,
and Graph neural networks (GNNs). Additionally, Kang et al. [33] employed a
multi-modal embedding approach for searching patent documents.

A patent classification method called Patent2vec has been introduced, which
leverages multi-view patent graph analysis to capture low-dimensional
representations of patent texts [8]. Pujari et al. [34] proposed a
transformer-based multi-task model (TMM) for hierarchical patent
classification, and their experimental results showed higher precision and
recall compared to existing non-neural and neural methods. They also proposed
a method to evaluate neural multi-field document representations for patent
text classification. Similarly, Aroyehun et al. [35] introduced a hierarchical
transfer and multi-task learning approach for patent classification, following
a similar methodology. Roudsari et al. [36] compared different word-embedding
methods for patent classification performance. Li et al. [37] proposed a
contrastive learning framework called CoPatE for patent embedding, aimed at
capturing high-level semantics for very large-scale patents to be classified.
An automated ensemble learning-based framework for single-level patent
classification is introduced by Kamateri et al. [38] .

However, to the best of our knowledge, none of the existing patent
classification methods are explainable. Given the complexity of the multi-
level classification task, it is crucial for users and patent experts to
understand the reasoning behind the AI-enabled method’s predictions, as it
classifies patents into one of more than 67,000 classes (including sub-group
classes). Therefore, the aim of this paper is to generate explanations that
highlight relevant words, helping users understand the rationale behind the
model’s predictions. Taking inspiration from the effectiveness and
interpretability of layer-wise relevance propagation (LRP) in other short-text
classification tasks [39, 40, 41], we have adopted LRP [12] as the method for
explaining the complex neural networks-based patent classification model.

Figure 1: A conceptual overview diagram of our explainable patent
classification framework.

##  3 Explainable Patent Classification

Our proposed explainable patent classification framework consists of two major
components, i) training DNN-based classification model using the semantic
representation of patent text, and ii) explanation generation component
leveraging layer-wise relevance propagation (LRP). The conceptual diagram with
major components is depicted in Fig 1. Our method first represents
preprocessed patent texts semantically by high-dimensional vector leveraging
pre-trained word embedding models. Then, the semantic representations for
patent text are fed to train multiple DNN-based classification models
including Bi-LSTM, CNN, and CNN-BiLSTM. For a particular deep patent
classification model, our introduced LRP algorithm computes the relevance
score towards a certain class for a given patent by redistributing the
relevance score with backward propagation from the output layer to the input
layer. Eventually, we get the score for patent terms that highlight the
relevancy related to the predicted class of a given input patent.

###  3.1 Training deep neural models

Before training any specific DNN-based patent classification model, we employ
_FastText_ word-embedding model to represent each word of patent text with a
high-dimensional feature vector and the element of each vector carries
semantic and contextual information of that word. _FastText_ is a character
n-gram-based embedding technique. Unlike, _Glove_ and _Word2Vec_ , it can
provide a word vector for out-of-vocabulary (OOV) words. Patents’ text
contains less used scientific terms and some words that are higly context
specific. For example, patent in the field of chemistry has a lot of reagents
and chemical names, even for some new patents the reagents’ names might be
completely new, proposed by the inventors. Considering this intuition, we
chose _FastText_ embedding instead of _Glove_ and _word2vec_. We make a
sequence of embedding of the words for each patent and then fed it into the
deep-learning model. Our trained different neural network models includes
bidirectional LSTM (Bi-LSTM), convolutional neural networks (CNN), CNN-BiLSTM,
a combination of CNN and Bi-LSTM.

###  3.2 Explaining predictions with LRP

Let $c$ denotes the predicted class for the input patent $p$. The LRP
algorithm applies the layer-wise conservation principle to calculate the
relevance score for features. The computation starts from the output layer and
then redistributes the relevance weight, eventually back-propagating it to the
input layers [40, 39]. In other words, the relevance score is computed at each
layer of the DNN model. Following a specific rule, the relevance score is
attributed from lower-layer neurons to higher-layer neurons, and each
immediate-layer neuron is assigned a relevance score up to the input layers,
based on this rule. The flow of propagation for computing the relevance is
depicted by the red arrow that goes from the output towards the input layers
in Fig. 1.

The prediction score, $f_{c}(p)$ by our deep patent classification model,
which is a scalar value corresponding to the patent class $c$. Using LRP, our
aim is to identify the relevance score for each dimension $d$ of a given
patent vector $p$ for the target patent class $c$. Our objective is to compute
the relevance score of each input feature (i.e., words) that illustrates how
positively (or negatively) contributes to classifying the patent as class $c$
(or another class). Let $z_{j}$ be the neuron of the upper layer and the
computation of the neuron is calculated as

$z_{j}=\sum_{i}z_{i}\cdot w_{ij}+b_{j},$ (1)

where $w_{ij}$ be the weight matrix and $b_{j}$ denotes the bias [40]. Given
that the relevance score for upper-layer neurons $z_{j}$ is $R_{j}$ and we
move towards lower-layer neurons to distribute that relevance. In the final
layer, there is only one neuron (i.e., the prediction score) and in that case,
$R_{j}$ is the prediction score by the function $f_{c}(p)$. The redistribution
of the relevance to the lower layers is done by following two major steps. We
need to compute relevance messages to go from upper-layer to lower-layer
neurons [40].

Let $i$ be the immediate lower layer and its neurons are denoted by $z_{i}$.
Computationally, the relevance massages $R_{i\leftarrow j}$ can be computed as
followings [40].

$R_{i\leftarrow j}=\frac{z_{i}\cdot w_{ij}+\frac{\epsilon\cdot
sign(z_{j})+\delta\cdot b_{j}}{N}}{z_{j}+\epsilon\cdot sign(z_{j})}\cdot
R_{j}.$ (2)

The total number of neurons in the layer $i$ is denoted as $N$ and $\epsilon$
is the stabilizer, a small positive real number (i.e., 0.001). By summing up
all the relevance scores of the neuron in $z_{i}$ in layer $i$, we can obtain
the relevance in layer $i$, $R_{i}=\sum_{i}R_{i\leftarrow j}$. $\delta$ can be
either 0 or 1 (we use $\delta=1$) [40, 41]. With the relevance messages, we
can calculate the amount of relevance that circulates from one layer’s neuron
to the next layer’s neuron. However, the computation for relevance
distribution in the fully connected layers is computed as $R_{j\rightarrow
k}=\frac{z_{jk}}{\sum_{j}z_{jk}}R_{k}$ [39]. The value of the relevance score
for each relevant term lies in [0,1]. The higher the score represents higher
the relevancy of the terms towards the predicted class.

##  4 Experiments

This section presents the details about the datasets, experiment results, and
discussion of generated explanation with LRP.

###  4.1 Dataset

AI-Growth-Lab patent dataset: We conducted experiments on a dataset containing
1.5 million patent claims annotated with patent class111Dataset:
https://huggingface.co/AI-Growth-Lab [42]. According to the CPC patent system,
the classification is hierarchical with multiple levels including section,
class, subclass, and group. For example, there are 667 labels in the subclass
level [42]. However, for a better understanding of the generated explanations
and the reasons behind a prediction for a given patent, we modeled the patent
classification task with 9 general classes including Human necessities,
Performing operations; transporting, Chemistry; metallurgy, Textiles; paper,
Fixed constructions, Mechanical engineering; lighting; heating; weapons;
blasting engines or pumps, Physics, Electricity and General.

BigPatent dataset:  BigPatent222Dataset:
https://huggingface.co/datasets/ccdv/patent-classification/tree/main dataset
is prepared by processing 1.3 million patent texts [43]. However, the
classification dataset contains in total of 35k patent texts with 9 above-
mentioned classes as labels. They provided the dataset by splitting it into
training, validation, and testing set, the number of samples are 25K, 5K, and
5K, respectively. There are two different texts for each patent, one is a raw
text from patent claims and another version is the human-generated abstract
summarized from the patent claims.

Figure 2: The distribution of the patents for different class on AI-growth-Lab
data Figure 3: The distribution of the patents for different class on
BigPatent data

However, the number of samples per patent class is varied widely for both both
datasets, which means both are imbalanced dataset. The horizontal bar chart in
Fig. 2 and 3 show the level of imbalance for both datasets. This imbalance
distribution of samples per class poses an additional challenge in this multi-
level classification task.

###  4.2 Experimental setup

We conducted experiments using three different DNN models, namely Bi-LSTM,
CNN, and CNN-BiLSTM, utilizing the _FastText_ pre-trained word-embedding model
for text representation in the embedding layers. The Bi-LSTM model consists of
a layer of Bi-LSTM with 64 units after embedding layer, followed by another
Bi-LSTM layer with 32 units, and then two fully-connected layers with 64 and 9
units, respectively. We applied the rectified linear units (ReLU) activation
function in the hidden dense layer, and the softmax activation function in the
output layer. For the CNN model, after the embedding layer, we have a
1-dimensional convolutional layer followed by a global average pooling layer,
and finally, the output layer is a fully-connected layer with 9 units. The
CNN-BiLSTM model has a convolutional layer followed by a global average
pooling layer, and then the Bi-LSTM part is similar to the above-mentioned Bi-
LSTM model. The activation functions in the fully connected hidden and output
layers are ReLU and softmax, respectively. We implemented our methods using
_scikit-learn_ and _Keras_ , and represented the patent text using the
_FastText_ pre-trained word-embedding
model333https://fasttext.cc/docs/en/crawl-vectors.html. For implementing LRP
for the Bi-LSTM network, we followed the method described in
[40]444https://github.com/ArrasL/LRP_for_LSTM. For the BigPatent dataset, the
training, testing, and validation sets are already split. For the AI-Growth-
Lab data, the ratio for the training and testing set is 80% and 20%,
respectively.

Table 1: The performance of different deep patent classification models on two datasets in terms of precision, recall and f1-score. The best result is in bold. Dataset | Method | Precision | Recall | F1-Score
---|---|---|---|---
AI-Growth-Lab | Bi-LSTM | 0.69 | 0.70 | 0.69
CNN | 0.62 | 0.63 | 0.62
CNN-BiLSTM | 0.69 | 0.68 | 0.69
BigPatent | Bi-LSTM | 0.79 | 0.78 | 0.78
CNN | 0.75 | 0.76 | 0.76
CNN-BiLSTM | 0.77 | 0.76 | 0.76

###  4.3 Performance analysis

The performance of the proposed classification models was evaluated using
three evaluation metrics, including Precision, Recall, and F1-Score, on two
datasets, as shown in Table 1. The results demonstrate consistent performance
across most of the deep classification models. Among them, the Bi-LSTM model
exhibited better performance in terms of all evaluation metrics on both
datasets. However, the performance of the other two models, CNN and CNN-
BiLSTM, was also consistent and effective, though slightly lower than the Bi-
LSTM model. Specifically, for the first dataset, CNN-BiLSTM performed equally
well in terms of Precision (0.69) and F1-Score (0.69), while the performance
of the CNN-based model was comparatively lower for the AI-Growth-Lab dataset,
with a Precision of 62%, which was 7% lower than the best-performing Bi-LSTM
model. However, for the BigPatent dataset, the CNN model exhibited
considerably better performance, with a Precision of 75%, which was only 4%
lower than the Bi-LSTM model. The performance difference between the models
for the other two metrics was even lower, at 2%.

Table 2: Class-wise performance of Bi-LSTM model on BigPatent Dataset Patent Class | label | Precision | Recall | F1-score
---|---|---|---|---
Human necessities | 1 | 0.79 | 0.91 | 0.85
Performing_operations | 2 | 0.74 | 0.66 | 0.70
Chemistry | 3 | 0.75 | 0.88 | 0.81
Textiles | 4 | 0.71 | 0.74 | 0.73
Fixed_constructions | 5 | 0.65 | 0.70 | 0.67
Mechanical_engineering | 6 | 0.60 | 0.84 | 0.70
Physics | 7 | 0.75 | 0.82 | 0.78
Electricity | 8 | 0.78 | 0.86 | 0.82
General | 9 | 0.71 | 0.46 | 0.41

The performance of all DNN-based classifiers on the BigPatent dataset is
significantly superior compared to the first dataset. This may be attributed
to the fact that the BigPatent dataset includes finely-grained abstracts of
patents which are generated by human assessors, taking into consideration the
patent texts. As a result, the semantic representation of the fine-tuned text
in the BigPatent dataset is enriched compared to the raw patent claims in
other dataset. We present the performance of Bi-LSTM model by showcasing the
class-wise performance on the BigPatent dataset. Table 2 displays the
performance across nine different patent classes. The Bi-LSTM model
demonstrates favorable and consistent performance across most patent classes,
with the exception of the general category. It is hypothesized that the
patents in the _“general”_ category may contain more commonly used terms
compared to patents in other area-specific categories. Consequently, the
captured semantic information may not be sufficient, potentially resulting in
lower performance in terms of recall and F1-Score for the _“general”_ class
compared to other classes.

Table 3: Performance comparison with related works Method | Precision | Recall | F1-Score
---|---|---|---
Out Method | 0.79 | 0.78 | 0.78
Roudsari et al. [9] (Bi-LSTM) | 0.7825 | 0.6421 | 0.6842
Roudsari et al. [9] (CNN-BiLSTM) | 0.7930 | 0.6513 | 0.6938
Shaobo et al. [2] (DeepPatent) | 0.7977 | 0.6552 | 0.6979

We compared the performance of our models with similar models that used
_FastText_ embedding for patent text representation. Compared to existing
works by Roudsari et al. [9] and Shaobo et al. [2], the performance of our
trained models is effective. Roudsari et al. also trained similar models with
semantic text representation with a pre-trained _FastText_ word-embedding
model. They also develop similar DNN models including Bi-LSTM and CNN-BiLSTM.
Shaobo et al. [2] introduced CNN-based deep patent modelling employing
_FastText_ word-embedding model. The performance of our methods on BigPatent
data is higher than their models for all evaluation metrics except Precision.
The comparison shows the effectiveness of our methods in classifying patents.

###  4.4 Generated explanation for prediction

We attempted to unbox the black-box nature of the deep patent classification
model by adopting a layer-wise relevance propagation technique to compute the
relevance score for each term by back-propagating the prediction score from
the output layer to input layers. To represent the explanation per predicted
class for a given patent text, we highlighted the related words that
contributed to the classifier’s prediction. As an example explanation, a
patent is classified as _Chemistry_ , and the related words that contributed
to the prediction are highlighted in red color in Fig 4. The figure shows the
explanation highlighting relevant words for the patent that classified as
chemistry. The intensity of the color represents the contributions of a
particular word. The higher the intensity of the color (red), the better the
relevancy the word is.

Figure 4: An example explanation for a patent classified as _Chemistry_
patents highlighting relevant words. The higher the intensity of the color,
the better the relevancy of the words contributing to the prediction. Figure
5: An example explanation for a patent classified as _Chemistry_ patents
highlighting relevant words. The higher the intensity of the color, the better
the relevancy of the words contributing to the prediction.

We can see that from the figure, the most relevant words include, _alkali,
alkyl, monomer, acid, acrylate, acrylonitrile, acetate, polymer, ether_. We
can observed that the highlighted words are completely related to terms used
in organic chemistry and the explanation makes sense why this patent has been
classified as a chemical patent. The next relevant list of words is soluble,
water, stiffness, enhanching, etc. These words are directly related to
chemistry except _stiffness_ and _enhancing_. Since _enhancing_ the
_stiffness_ of the paper or paperboard is the objective of this patent, these
words are selected as relevant.

Figure 6: An example explanation for a patent classified as Electricity
patents highlighting relevant words. The higher the intensity of the color,
the better the relevancy of the words contributing to the prediction.

For another example patent in the field of Electricity, Fig. 6 illustrates the
explanations highlighting relevant words that contributed to the classifier to
decide that the patent is from _electricity_ field. The most relevant words,
in this case, include _power, channel, modem, device, bonded, bandwidth,
data,_ etc. We can see that all identified related words are used in
_electricity_ literature. The word _device_ is used for common use in some
other fields also, but this word also can be used to mention any electrical
instrument in electricity-related explanation. However, there are some words
selected as relevant for both examples which are not relevant to the specific
fields but can be used in literature for any field. One plausible reason is
that those also might carry considerable importance in describing the any
scientific object (i.e., explaining chemical reaction) and capture good
contextual and semantic information in _FastText_ embedding.

###  4.5 Limitations

Our model can explain the prediction for multi-label classification. Since the
patents are classified in different levels and the patent classification
system has a huge set of classes to classify in different levels, it should be
explainable for multi-level classification also. This will be more challenging
to explain the prediction for different subgroups-level classes. Another
limitation is that our utilized pre-trained word-embedding model is not
trained on the patent corpus. The local word-embedding model trained with
patent corpus might capture better contextual and semantic information for
scientific terms and jargon. Hence, the performance might be better than the
current approach.

##  5 Conclusion and Future Direction

This paper aimed at explaining the predictions from DNN-based patent
classification models with layer-wise relevance propagation technique to
identify the relevance of different words in the patent texts for a certain
predicted class. Layer-wise relevance propagation technique can capture
context-specific explanatory and relevant words to explain the predictions
behind certain predicted classes. The experimental results demonstrated the
effectiveness of classifying patent documents with promising performance
compared to existing works. We observed that the explanations generated by the
LRP technique make it easier to understand why a certain patent is classified
as a specific patent class. Most of the captured words have high relevancy
with the patent domain, even though a few words marked as related are not that
relevant (which, however, should also provide useful information to human
expert in assessing the predictions). Even though our approach would still
need to be evaluation with users, we can observe that the explanations are
helpful to understand the question why a certain patent was classified into a
specific class, and to assess the results of deep-learning-based complex
artificial intelligence-enabled models.

Since patents have a lot of scientific and uncommon words and phrases (i.e.,
jargon) that are not often used in other texts, we plan to train a local word-
embedding model with patent texts to have better representation in our future
work. It would be interesting to apply a transformer-based approach for the
same purpose. The explanations for sub-group level prediction and capturing
the sub-group context will be even more explanatory. However, the generated
explanations will need to be evaluated by human experts in the patent
industry. Therefore, we plan to have a user-centric evaluation for the
generated explanations and elicit more human-centric requirements to be
addressed in the future for better adoption real-word applications.

## Acknowledgment

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under the Marie Skłodowska-Curie grant
agreement No 955422.

## References

  * Kucer et al. [2022] Michal Kucer, Diane Oyen, Juan Castorena, and Jian Wu.  Deeppatent: Large scale patent drawing recognition and retrieval.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 2309–2318, 2022.
  * Li et al. [2018] Shaobo Li, Jie Hu, Yuxin Cui, and Jianjun Hu.  Deeppatent: patent classification with convolutional neural networks and word embedding.  _Scientometrics_ , 117:721–744, 2018.
  * Lee and Hsiang [2020] Jieh-Sheng Lee and Jieh Hsiang.  Patent classification by fine-tuning bert language model.  _World Patent Information_ , 61:101965, 2020.
  * D’hondt et al. [2013] Eva D’hondt, Suzan Verberne, Cornelis Koster, and Lou Boves.  Text representations for patent classification.  _Computational Linguistics_ , 39(3):755–775, 2013\.
  * Luo et al. [2020] Mengzhen Luo, Xiaoyu Shi, Qianqian Ji, Mingsheng Shang, Xianbo He, and Weiguo Tao.  A deep self-learning classification framework for incomplete medical patents with multi-label.  In _Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery: Volume 2_ , pages 566–573. Springer, 2020.
  * Jiang et al. [2022] Shuo Jiang, Jie Hu, Christopher L Magee, and Jianxi Luo.  Deep learning for technical document classification.  _IEEE Transactions on Engineering Management_ , 2022.
  * Chen et al. [2020] Liang Chen, Shuo Xu, Lijun Zhu, Jing Zhang, Xiaoping Lei, and Guancan Yang.  A deep learning based method for extracting semantic information from patent documents.  _Scientometrics_ , 125:289–312, 2020.
  * Fang et al. [2021] Lintao Fang, Le Zhang, Han Wu, Tong Xu, Ding Zhou, and Enhong Chen.  Patent2vec: Multi-view representation learning on patent-graphs for patent classification.  _World Wide Web_ , 24(5):1791–1812, 2021.
  * Haghighian Roudsari et al. [2022] Arousha Haghighian Roudsari, Jafar Afshar, Wookey Lee, and Suan Lee.  Patentnet: multi-label classification of patent documents using deep learning based language understanding.  _Scientometrics_ , pages 1–25, 2022.
  * Shrikumar et al. [2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje.  Learning important features through propagating activation differences.  In _International conference on machine learning_ , pages 3145–3153. PMLR, 2017.
  * Lundberg and Lee [2017] Scott M Lundberg and Su-In Lee.  A unified approach to interpreting model predictions.  _Advances in neural information processing systems_ , 30, 2017.
  * Bach et al. [2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, Klaus-Robert Müller, and Wojciech Samek.  On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation.  _PloS one_ , 10(7):e0130140, 2015.
  * Kute et al. [2021] Dattatray Vishnu Kute, Biswajeet Pradhan, Nagesh Shukla, and Abdullah Alamri.  Deep learning and explainable artificial intelligence techniques applied for detecting money laundering–a critical review.  _IEEE Access_ , 9:82300–82317, 2021.
  * Shajalal et al. [2022] Md Shajalal, Alexander Boden, and Gunnar Stevens.  Explainable product backorder prediction exploiting cnn: Introducing explainable models in businesses.  _Electronic Markets_ , pages 1–16, 2022.
  * Yang et al. [2022] Guang Yang, Qinghao Ye, and Jun Xia.  Unbox the black-box for the medical explainable ai via multi-modal and multi-centre data fusion: A mini-review, two showcases and beyond.  _Information Fusion_ , 77:29–52, 2022.
  * Adadi and Berrada [2020] Amina Adadi and Mohammed Berrada.  Explainable ai for healthcare: from black box to interpretable models.  In _Embedded Systems and Artificial Intelligence: Proceedings of ESAI 2019, Fez, Morocco_ , pages 327–337. Springer, 2020.
  * Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  " why should i trust you?" explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pages 1135–1144, 2016.
  * Binder et al. [2016] Alexander Binder, Grégoire Montavon, Sebastian Lapuschkin, Klaus-Robert Müller, and Wojciech Samek.  Layer-wise relevance propagation for neural networks with local renormalization layers.  In _Artificial Neural Networks and Machine Learning–ICANN 2016: 25th International Conference on Artificial Neural Networks, Barcelona, Spain, September 6-9, 2016, Proceedings, Part II 25_ , pages 63–71. Springer, 2016\.
  * Shalaby et al. [2018] Marawan Shalaby, Jan Stutzki, Matthias Schubert, and Stephan Günnemann.  An lstm approach to patent classification based on fixed hierarchy vectors.  In _Proceedings of the 2018 SIAM International Conference on Data Mining_ , pages 495–503. SIAM, 2018.
  * Roudsari et al. [2020] Arousha Haghighian Roudsari, Jafar Afshar, Charles Cheolgi Lee, and Wookey Lee.  Multi-label patent classification using attention-aware deep learning model.  In _2020 IEEE International Conference on Big Data and Smart Computing (BigComp)_ , pages 558–559. IEEE, 2020.
  * Le and Mikolov [2014] Quoc Le and Tomas Mikolov.  Distributed representations of sentences and documents.  In _International conference on machine learning_ , pages 1188–1196. PMLR, 2014.
  * Mikolov et al. [2013] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean.  Distributed representations of words and phrases and their compositionality.  _Advances in neural information processing systems_ , 26, 2013.
  * Pennington et al. [2014] Jeffrey Pennington, Richard Socher, and Christopher D Manning.  Glove: Global vectors for word representation.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP)_ , pages 1532–1543, 2014.
  * Bojanowski et al. [2017] Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov.  Enriching word vectors with subword information.  _Transactions of the association for computational linguistics_ , 5:135–146, 2017.
  * Shajalal and Aono [2018] Md Shajalal and Masaki Aono.  Sentence-level semantic textual similarity using word-level semantics.  In _2018 10th International Conference on Electrical and Computer Engineering (ICECE)_ , pages 113–116. IEEE, 2018.
  * Shajalal and Aono [2019] Md Shajalal and Masaki Aono.  Semantic textual similarity between sentences using bilingual word semantics.  _Progress in Artificial Intelligence_ , 8:263–272, 2019\.
  * Shajalal and Aono [2020] Md Shajalal and Masaki Aono.  Coverage-based query subtopic diversification leveraging semantic relevance.  _Knowledge and Information Systems_ , 62:2873–2891, 2020\.
  * Devlin et al. [2018] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * Liu et al. [2019] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ , 2019.
  * Sanh et al. [2019] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf.  Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter.  _arXiv preprint arXiv:1910.01108_ , 2019.
  * Yang et al. [2019] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32, 2019.
  * Clark et al. [2020] Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning.  Electra: Pre-training text encoders as discriminators rather than generators.  _arXiv preprint arXiv:2003.10555_ , 2020.
  * Kang et al. [2020] Myungchul Kang, Suan Lee, and Wookey Lee.  Prior art search using multi-modal embedding of patent documents.  In _2020 IEEE International Conference on Big Data and Smart Computing (BigComp)_ , pages 548–550. IEEE, 2020.
  * Pujari et al. [2021] Subhash Chandra Pujari, Annemarie Friedrich, and Jannik Strötgen.  A multi-task approach to neural multi-label hierarchical patent classification using transformers.  In _Advances in Information Retrieval: 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28–April 1, 2021, Proceedings, Part I 43_ , pages 513–528. Springer, 2021.
  * Aroyehun et al. [2021] Segun Taofeek Aroyehun, Jason Angel, Navonil Majumder, Alexander Gelbukh, and Amir Hussain.  Leveraging label hierarchy using transfer and multi-task learning: A case study on patent classification.  _Neurocomputing_ , 464:421–431, 2021.
  * Roudsari et al. [2021] Arousha Haghighian Roudsari, Jafar Afshar, Suan Lee, and Wookey Lee.  Comparison and analysis of embedding methods for patent documents.  In _2021 IEEE International Conference on Big Data and Smart Computing (BigComp)_ , pages 152–155. IEEE, 2021.
  * Li et al. [2022] Huahang Li, Shuangyin Li, Yuncheng Jiang, and Gansen Zhao.  Copate: A novel contrastive learning framework for patent embeddings.  In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_, pages 1104–1113, 2022.
  * Kamateri et al. [2022] Eleni Kamateri, Vasileios Stamatis, Konstantinos Diamantaras, and Michail Salampasis.  Automated single-label patent classification using ensemble classifiers.  In _2022 14th International Conference on Machine Learning and Computing (ICMLC)_ , pages 324–330, 2022.
  * Arras et al. [2017a] Leila Arras, Franziska Horn, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek.  " what is relevant in a text document?": An interpretable machine learning approach.  _PloS one_ , 12(8):e0181142, 2017a.
  * Arras et al. [2017b] Leila Arras, Grégoire Montavon, Klaus-Robert Müller, and Wojciech Samek.  Explaining recurrent neural network predictions in sentiment analysis.  _arXiv preprint arXiv:1706.07206_ , 2017b.
  * Karim et al. [2021] Md Rezaul Karim, Sumon Kanti Dey, Tanhim Islam, Sagor Sarker, Mehadi Hasan Menon, Kabir Hossain, Md Azam Hossain, and Stefan Decker.  Deephateexplainer: Explainable hate speech detection in under-resourced bengali language.  In _2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA)_ , pages 1–10. IEEE, 2021.
  * Bekamiri et al. [2021] Hamid Bekamiri, Daniel S Hain, and Roman Jurowetzki.  Patentsberta: a deep nlp based hybrid model for patent distance and classification using augmented sbert.  _arXiv preprint arXiv:2103.11933_ , 2021.
  * Sharma et al. [2019] Eva Sharma, Chen Li, and Lu Wang.  Bigpatent: A large-scale dataset for abstractive and coherent summarization.  _arXiv preprint arXiv:1906.03741_ , 2019.


#  $\Lambda_{c}$ decays at BESIII

Y. Gao
on behalf of the BESIII  University of Science and Technology of China, Hefei
230026, People’s Republic of China
State Key Laboratory of Particle Detection and Electronics, Beijing 100049,
Hefei 230026, People’s Republic of China

###### Abstract

BESIII has made great progress in taking data which is the largest data
samples near the $\Lambda_{c}^{+}\bar{\Lambda}_{c}^{-}$ threshold. We have
measured the branching fraction of $\Lambda_{c}^{+}\to n\pi^{+}$ to be
$(6.6\pm 1.2_{\mathrm{stat.}}\pm 0.4_{\mathrm{syst.}})\times 10^{-4}$ with the
statistical significance of $7.3~{}\sigma$ firstly using 3.9$\mathrm{fb}^{-1}$
$e^{+}e^{-}$ collision collected with BESIII detector at six center-of-mass
energies between 4.612 and 4.699 GeV. With the result of the branching
fraction of $\Lambda_{c}^{+}\to p\pi^{0}$ from Belle, the ratio of the
branching fractions between $\Lambda_{c}^{+}\to n\pi^{+}$ and
$\Lambda_{c}^{+}\to p\pi^{0}$ is measured to be larger than 7.2 at 90%
confidence level. The branching fractions of
$\Lambda_{c}^{+}\to\Lambda\pi^{+}$ and $\Lambda_{c}^{+}\to\Sigma^{0}\pi^{+}$
are measured to be $(1.31\pm 0.08_{\mathrm{stat.}}\pm
0.05_{\mathrm{syst.}})\times 10^{-2}$ and $(1.22\pm 0.08_{\mathrm{stat.}}\pm
0.07_{\mathrm{syst.}})\times 10^{-2}$, respectively, which are consistent with
previous BESIII results. More results of $\Lambda_{c}$ decays will be
published with better precision this year using 6.4$\mathrm{fb}^{-1}$
$e^{+}e^{-}$ collision data samples between 4.600 and 4.946 GeV.

##  I Introduction

$\Lambda_{c}^{+}$ is the ground states of singly charmed baryon and most of
the charmed baryons will eventually decay to $\Lambda_{c}^{+}$ but the
knowledge of $\Lambda_{c}^{+}$ decays is still very limited in comparison with
charmed mesons. Firstly, only a single semileptonic (SL) decay mode
$\Lambda_{c}^{+}\to\Lambda l^{+}\nu_{l}$ has been observed. Secondly, the
doubly Cabibbo-suppressed (DCS) modes have not been systematically studied.
Thirdly, the known exclusive decays of $\Lambda_{c}^{+}$ account for only
about 60% of the total branching fraction. In particular, information about
the decays that involve a neutron is minimal, considering that such decays
should account for nearly half of the $\Lambda_{c}$ decay rate. Fourth, the
precision of singly Cabibbo-suppressed (SCS) modes is very limit and some two-
body SCS modes still need to be searched. Improved knowledge of the
$\Lambda_{c}^{+}$ decays is essential for the studies of the whole charmed
baryon family WhitePaper .

Several different phenomenological models, e.g. current algebra Current1 ;
Current2  and SU(3) flavor symmetry SU(3)1 ; SU(3)2 ; SU(3)3 , have been
employed as tools to reveal the dynamics of charmed baryon decays. The
hadronic decay amplitudes of $\Lambda_{c}^{+}$ consist of factorizable and
nonfactorizable components. The non-factorizable contributions are important
in these decays which can be constrained by measurements. So it is important
to get comprehensive and precise experimental inputs to understand the
validity of different phenomenological models.

The two-body SCS decay $\Lambda_{c}^{+}\to n\pi^{+}$, together with
$\Lambda_{c}^{+}\to p\pi^{0}$ are of great interest and have been studied
extensively in the context of various phenomenological models SU(3)1 ; SU(3)2
; Uppal1994 ; Chen2003 ; Cheng2018 ; Geng2018 ; Geng2019 ; Geng2020 ; Zou2020
; Zhao2020 . They predict quite different decay rates for $\Lambda_{c}^{+}\to
p\pi^{0}$ and $\Lambda_{c}^{+}\to n\pi^{+}$ and distinguishing between these
models with experimental results is highly desirable. This ratio is predicted
to be 2 by the SU(3) flavor symmetry model SU(3)1 ; SU(3)2 ; Geng2018 , 4.5 or
8.0 by the constituent quark model Uppal1994 , 3.5 by a dynamical calculation
based on pole model and current-algebra Cheng2018 , 4.7 by the SU(3) flavor
symmetry including the contributions from $\mathcal{O}(\overline{15})$
Geng2019 , and 9.6 by the topological-diagram approach Zhao2020 .

To date, the branching fractions of only a few SCS decay modes have been
measured, and all with limited precision PDG2020 , and those involving a
neutron in the final state have never been measured. So it is also important
to search the decay $\Lambda_{c}^{+}\to n\pi^{+}$ and other SCS decay modes of
$\Lambda_{c}^{+}$.

##  II The BESIII detector

The BESIII detector BESIIIDector2010  records symmetric $e^{+}e^{-}$
collisions provided by the BEPCII storage ring BESIIIDector2016 , which
operates in the center-of-mass energy range from 2.0 GeV to 4.946 GeV. BESIII
has collected large data samples in this energy region BESIIIDector2020 . The
cylindrical core of the BESIII detector covers 93% of the full solid angle and
consists of a helium-based multilayer drift chamber (MDC), a plastic
scintillator time-of-flight system (TOF), and a CsI(Tl) electromagnetic
calorimeter (EMC), which are all enclosed in a superconducting solenoidal
magnet providing a 1.0 T magnetic field. The solenoid is supported by an
octagonal flux-return yoke with resistive plate counter muon identification
modules interleaved with steel. The charged-particle momentum resolution at 1
$\mathrm{GeV}/c$ is 0.5%, and the $dE/dx$ resolution is 6% for electrons from
Bhabha scattering. The EMC measures photon energies with a resolution of 2.5%
(5%) at 1 GeV in the barrel (end cap) region. The time resolution in the TOF
barrel region is 68 ps, while that in the end cap region is 60 ps
BESIIIDector20202 .

##  III Recent Results

###  III.1 Observation of the Singly Cabibbo-Suppressed Decay
$\Lambda_{c}^{+}\to n\pi^{+}$

The first observation of the SCS decay $\Lambda_{c}^{+}\to n\pi^{+}$ is
reported using $3.9\mathrm{fb}^{-1}$ $e^{+}e^{-}$ collision data collected
with the BESIII detector at six center-of-mass energies between 4.612 and
4.699 GeV. The integrated luminosities of the data samples at 4.612, 4.628,
4.641, 4.682, and 4.699 GeV are 103.5, 519.9, 548.2, 527.6, 1664.3, and 534.4
$\mathrm{pb}^{-1}$ Lum , respectively. Charge-conjugate modes are implicitly
included.

Simulated samples are produced with a GEANT4-based Monte Carlo (MC) package
Geant4 . The detector response are used to determine detection efficiencies
and to estimate backgrounds. A detailed description of the MC of this work can
be found in npi .

A double-tag (DT) approach DTMethod  is implemented to search for
$\Lambda_{c}^{+}\to n\pi^{+}$. A data sample of $\bar{\Lambda}_{c}^{-}$
baryons, referred to as the single-tag (ST) sample, is reconstructed with ten
exclusive hadronic decay modes shown in FIG. 1, where the intermediate
particles $K_{S}^{0}$, $\bar{\Lambda}$, $\bar{\Sigma}^{0}$,
$\bar{\Sigma}^{-}$, and $\pi^{0}$ are reconstructed with the decays
$K_{S}^{0}\to\pi^{+}\pi^{-}$, $\bar{\Lambda}\to\bar{p}\pi^{+}$,
$\bar{\Sigma}^{0}\to\gamma\bar{\Lambda}$, $\bar{\Sigma}^{-}\to\bar{p}\pi^{0}$,
and $\pi^{0}\to\gamma\gamma$, respectively. Those events in which the signal
decay $\Lambda_{c}^{+}\to n\pi^{+}$ is reconstructed in the system recoiling
against the $\bar{\Lambda}_{c}^{-}$ candidates of the ST sample are denoted as
DT candidates.

A detailed description of the selection of the ST sample is shown in npi . The
decay $\Lambda_{c}^{+}\to n\pi^{+}$ is searched for among the remaining tracks
recoiling against the ST $\bar{\Lambda}_{c}^{-}$ candidates. Only one tight
charged track is allowed, which is then assigned to be the $\pi^{+}$ from the
signal decay. To suppress contamination from long-lifetime particles in the
final state, the candidate events are further required to be without any loose
tracks.

The $\Lambda_{c}^{+}$ decay branching fractions ($\mathcal{B}$) are determined
as

$\mathcal{B}=\frac{N_{\mathrm{obs}}}{\sum_{ij}N_{ij}^{\mathrm{ST}}\cdot(\epsilon_{ij}^{\mathrm{DT}}/\epsilon_{ij}^{\mathrm{ST}})},$
(1)

where the subscripts $i$ and $j$ represent the ST modes and the data samples
at different c.m. energies, respectively. The parameters
$N_{ij}^{\mathrm{ST}}$, $\epsilon_{ij}^{\mathrm{ST}}$ and
$\epsilon_{ij}^{\mathrm{DT}}$ are the ST yields, ST and DT efficiencies,
respectively. The detection efficiencies $\epsilon_{ij}^{\mathrm{ST}}$ and
$\epsilon_{ij}^{\mathrm{DT}}$ are estimated from MC samples.

The ST $\bar{\Lambda}_{c}^{-}$ candidates are identified using the variables
of beam-constrained invariant mass
$M_{\mathrm{BC}}=\sqrt{E_{\mathrm{beam}}^{2}/c^{4}-|\vec{p}_{\bar{\Lambda}_{c}^{-}}|^{2}/c^{2}}$,
where $E_{\mathrm{beam}}$ is the beam energy,
$\vec{p}_{\bar{\Lambda}_{c}^{-}}$ is the momentum of the
$\bar{\Lambda}_{c}^{-}$ candidate. The $M_{\mathrm{BC}}$ distributions of
surviving candidates for the ten ST modes are illustrated in FIG. 1 for the
data sample at $\sqrt{s}=4.682~{}\mathrm{GeV}$, where clear
$\bar{\Lambda}_{c}^{-}$ signals are observed in each sample. No peaking
backgrounds are found with the investigation of the inclusive MC sample. The
sum of the ST yields for all the six data samples is $90,692\pm 359$, where
the uncertainty is statistical.

Figure 1:  The $M_{\mathrm{BC}}$ distributions of the ST modes for data sample
at $\sqrt{s}=4.682~{}\mathrm{GeV}$. The points with error bars represent data.
The (red) solid curves indicate the fit results and the (blue) dashed curves
describe the background shapes.

To improve detection efficiency, the neutron is selected through the recoiling
mass ($M_{\rm rec}$) against the ST $\bar{\Lambda}_{c}^{-}$ and $\pi^{+}$:

$M^{2}_{\rm
rec}=(E_{\mathrm{beam}}-E_{\pi^{+}})^{2}/c^{4}-|\rho\cdot\vec{p}_{0}-\vec{p}_{\pi^{+}}|^{2}/c^{2},$
(2)

where $E_{\pi^{+}}$ and $\vec{p}_{\pi^{+}}$ are the energy and momentum of the
$\pi^{+}$ candidate,
$\rho=\sqrt{E_{\mathrm{beam}}^{2}/c^{2}-m_{\Lambda_{c}^{+}}^{2}c^{2}}$, and
$\vec{p}_{0}=-\vec{p}_{\bar{\Lambda}_{c}^{-}}/|\vec{p}_{\bar{\Lambda}_{c}^{-}}|$
is the unit direction opposite to the ST $\bar{\Lambda}_{c}^{-}$. After
imposing all selection conditions, the distribution of $M_{\rm rec}$ of the
accepted DT candidate events from the combined six data samples at different
c.m. energies is shown in FIG. 2, where a peak at the neutron mass is
observed, representing the $\Lambda_{c}^{+}\to n\pi^{+}$ signal. Additionally,
there are two prominent structures peaking at the $\Lambda$ and $\Sigma^{0}$
mass regions, which represent the Cabibbo-favored (CF) processes
$\Lambda_{c}^{+}\to\Lambda\pi^{+}$ and $\Lambda_{c}^{+}\to\Sigma^{0}\pi^{+}$,
respectively. The potential backgrounds can be classified into two categories:
those directly originating from continuum hadron production in the
$e^{+}e^{-}$ annihilation (referred to as $q\bar{q}$ background hereafter) and
those from $e^{+}e^{-}\to\Lambda_{c}^{+}\bar{\Lambda}_{c}^{-}$ events
(referred to as $\Lambda_{c}^{+}\bar{\Lambda}_{c}^{-}$ background hereafter),
excluding contributions from $\Lambda_{c}^{+}\to n\pi^{+}$, $\Lambda\pi^{+}$,
and $\Sigma^{0}\pi^{+}$ signals.

Figure 2:  The $M_{\mathrm{rec}}$ distribution of the accepted DT candidate
events from the combined six data samples. The black points with error bars
are data. The red, blue, and green dashed lines indicate the curves for the
neutron, $\Lambda$, and $\Sigma$ peaks, respectively. The brown and gray
shaded histograms for the two background components are from the inclusive MC
sample, and the dark brown dashed line indicates the curve that describes the
two background components from fitting. The black line is the sum over all the
components in the fit. The inset shows the $M_{\mathrm{rec}}$ distribution in
the ST $M_{\mathrm{BC}}$ sideband region, and the gray dashed line indicates
the curve that describes the $q\bar{q}$ background.

The fit distributions are depicted in FIG. 2, and correspond to signal yields
of $50\pm 9$, $376\pm 22$, and $343\pm 22$ for the decays $\Lambda_{c}^{+}\to
n\pi^{+}$, $\Lambda\pi^{+}$ and $\Sigma^{0}\pi^{+}$, respectively, where the
uncertainties are statistical and the statistical significance of
$\Lambda_{c}^{+}\to n\pi^{+}$ is $7.3\sigma$.

The branching fraction of $\Lambda_{c}^{+}\to n\pi^{+}$ is measured to be
$(6.6\pm 1.2_{\rm stat}\pm 0.4_{\rm syst})\times 10^{-4}$, this is a first-
time measurement. Meanwhile, the branching fractions of the Cabibbo-favored
decays $\Lambda_{c}^{+}\to\Lambda\pi^{+}$ and
$\Lambda_{c}^{+}\to\Sigma^{0}\pi^{+}$ are measured to be $(1.31\pm 0.08_{\rm
stat}\pm 0.05_{\rm syst})\times 10^{-2}$ and $(1.22\pm 0.08_{\rm stat}\pm
0.07_{\rm syst})\times 10^{-2}$, respectively. Taking the upper limit of the
branching fraction of $\Lambda_{c}^{+}\to p\pi^{0}$ from the Belle experiment,
$\mathcal{B}(\Lambda_{c}^{+}\to p\pi^{0})<8.0\times 10^{-5}$ at the 90%
confidence level Belle , the ratio of branching fractions between
$\Lambda_{c}^{+}\to n\pi^{+}$ and $\Lambda_{c}^{+}\to p\pi^{0}$ is calculated
to be larger than 7.2 at the 90% confidence level, which disagrees with most
predictions of the phenomenological models SU(3)1 ; SU(3)2 ; Geng2018 ;
Uppal1994 ; Cheng2018 ; Geng2019 ; Zhao2020 .

###  III.2 The results in the future

There are a lot of new results will be published using the data samples from
4.600 to 4.946 GeV. We list some of them in the TABLE 1. Most of them use the
DT Method and the statistical precision can improve a lot from MC simulation.
More new decay modes of $\Lambda_{c}^{+}$ will also be searched using these
data samples and will be published very soon. With the largest data samples
near $\Lambda_{c}^{+}\bar{\Lambda}_{c}^{-}$ threshold in BESIII, the knowledge
about $\Lambda_{c}^{+}$ decays can improve a lot. There is an interest in the
$\Lambda_{c}^{+}$ decay and have a more detailed explanation of studying the
properties of $\Lambda_{c}^{+}$ decays in WhitePaper .

Table 1: The results in the future. Decay | Method |  Advantage from MC Simulation
---|---|---
Hadronic Decays
$\Lambda_{c}^{+}\to p\pi^{0}$ | DT Method | about $3\sigma$ significance
$\Lambda_{c}^{+}\to p\eta$ | ST Method |  $\sigma_{\mathrm{stat.}}$: 22.6% $\to$ 8.1%
$\Lambda_{c}^{+}\to p\eta^{\prime}$ | DT Method | about $4\sigma$ significance
Semileptonic Decays
$\Lambda_{c}^{+}\to pK^{-}e^{+}\nu_{e}$ | DT Method |  $>5\sigma$ significance
$\Lambda_{c}^{+}\to\Lambda e^{+}\nu_{e}$ | DT Method |  $\sigma_{\mathrm{stat.}}$: 25% $\to$ 12%
Inclusive Decays
$\bar{\Lambda}_{c}^{-}\to\bar{n}+X$ | DT Method |  $\sigma_{\mathrm{stat.}}$: 32% $\to$ 3%
$\Lambda_{c}^{+}\to X+e^{+}\nu_{e}$ | DT Method |  $\sigma_{\mathrm{stat.}}$: 9% $\to$ 3%

##  IV Summary

BESIII has made great progress in taking data from 4.612 GeV to 4.946 GeV
which are the large data samples near the
$\Lambda_{c}^{+}\bar{\Lambda}_{c}^{-}$ threshold Lum . We have measured the
branching fraction of $\Lambda_{c}^{+}\to n\pi^{+}$,
$\Lambda_{c}^{+}\to\Lambda\pi^{+}$, and $\Lambda_{c}^{+}\to\Sigma^{0}\pi^{+}$
to be $(6.6\pm 1.2_{\mathrm{stat}}\pm 0.4_{\mathrm{syst}})\times 10^{-4}$,
$(1.31\pm 0.08_{\mathrm{stat}}\pm 0.05_{\mathrm{syst}})\times 10^{-2}$, and
$(1.22\pm 0.08_{\mathrm{stat}}\pm 0.07_{\mathrm{syst}})\times 10^{-2}$,
respectively by using data samples corresponding to a total integrated
luminosity of 3.9 $\mathrm{fb}^{-1}$ collected between 4.612 and 4.699 GeV
with the BESIII detector. Taking the upper limit of the branching fraction of
$\Lambda_{c}^{+}\to p\pi^{+}$ from the Belle experiment,
$\mathcal{B}(\Lambda_{c}^{+}\to p\pi^{0})<8.0\times 10^{-5}$ at the 90%
confidence level, the ratio of branching fractions between $\Lambda_{c}^{+}\to
n\pi^{+}$ and $\Lambda_{c}^{+}\to p\pi^{0}$ is calculated to be larger than
7.2 at the 90% confidence level. In order to obtain an improved understanding
it is desirable to perform improved studies of $\Lambda_{c}^{+}\to p\pi^{0}$
branching fraction npi . More studies about the $\Lambda_{c}^{+}$ decays are
proceeding and the results will be published very soon.

###### Acknowledgements.

We thank our BEPCII colleagues for the excellent luminosity and our BESIII
collaborators for their many efforts in advancing the knowledge of
$\Lambda_{c}^{+}$ decays. We are grateful to the FPCP2022 committee for the
organization of this nice conference.

## References

  * (1) M. Ablikim et al. (BESIII Collaboration), Chin. Phys. C 44, 040001 (2020).
  * (2) H. Y. Cheng and B. Tseng, Phys. Rev. D 48, 4188 (1993).
  * (3) K. K. Sharma and R. C. Verma, Eur. Phys. J. C 7, 217 (1999).
  * (4) K. K. Sharma and R. C. Verma, Phys. Rev. D 55, 7067 (1997).
  * (5) C. D. L$\ddot{\mathrm{u}}$, W. Wang, and F. S. Yu, Phys. Rev. D 93, 056008 (2016).
  * (6) C. Q. Geng, Y. K. Hsiao, Y. H. Lin, and L. L. Liu, Phys. Lett. B 776, 265 (2018).
  * (7) T. Uppal, R. C. Verma, and M. P. Khanna, Phys. Rev. D 49, 3417 (1994).
  * (8) S. L. Chen, X. H. Guo, X. Q. Li, and G. L. Wang, Commun. Theor. Phys. 40, 563 (2003).
  * (9) H. Y. Cheng, X. W. Kang, and F. R. Xu, Phys. Rev. D 97, 074028 (2018).
  * (10) C. Q. Geng, Y. K. Hsiao, C. W. Liu, and T.-H. Tsai, Phys. Rev. D 97, 073006 (2018).
  * (11) C. Q. Geng, C. W. Liu, and T.-H. Tsai, Phys. Lett. B 790, 225 (2019).
  * (12) C. Q. Geng, C. W. Liu, and T.-H. Tsai, Phys. Rev. D 101, 053002 (2020).
  * (13) J. Zou, F. Xu, G. Meng, and H. Y. Cheng, Phys. Rev. D 101, 014011 (2020).
  * (14) H. J. Zhao, Y. L. Wang, Y. K. Hsiao, and Y. Yu, J. High Energy Phys. 02 (2020) 165.
  * (15) M. Ablikim et al. (BESIII Collaboration), Nucl. Instrum. Meth. A 614, 345-399 (2010).
  * (16) C. H. Yu et al., Proceedings of IPAC2016, Busan, Korea, 2016, doi:10.18429/JACoW-IPAC2016-TUYA01.
  * (17) M. Ablikim et al. (BESIII Collaboration), Chin. Phys. C 44, 040001 (2020).
  * (18) X. Li et al., Radiat. Detect. Technol. Methods 1, 13 (2017); Y. X. Guo et al., Radiat. Detect. Technol. Methods 1, 15 (2017); P. Cao et al., Nucl. Instrum. Meth. A 953, 163053 (2020).
  * (19) https://doi.org/10.48550/arXiv.2205.04809.
  * (20) S. Agostinelli et al. (GEANT4 Collaboration), Nucl. Instrum. Method Phys. Res., Sect. A 506, 250 (2003).
  * (21) M. Ablikim et al. (BESIII Collaboration), Phys. Rev. Lett. 128, 142001 (2022).
  * (22) J. Adler et al. (The Mark III Collaboration), Phys. Rev. Lett. 62, 1821 (1989).
  * (23) S. X. Li et al. (Belle Collaboration), Phys. Rev. D 103, 072004 (2021).
  * (24) P. A. Zyla et al. (Particle Data Group), PTEP 2020, 083C01 (2020).


# Acoustical Analysis of Speech Under Physical Stress in Relation to
Physical Activities and Physical Literacy

###### Abstract

Human speech production encompasses physiological processes that naturally
react to physic stress. Stress caused by physical activity (PA), e.g.,
running, may lead to significant changes in a person's speech. The major
changes are related to the aspects of pitch level, speaking rate, pause
pattern, and breathiness. The extent of change depends presumably on physical
fitness and well-being of the person, as well as intensity of PA. The general
wellness of a person is further related to his/her physical literacy (PL),
which refers to a holistic description of engagement in PA. This paper
presents the development of a Cantonese speech database that contains audio
recordings of speech before and after physical exercises of different
intensity levels. The corpus design and data collection process are described.
Preliminary results of acoustical analysis are presented to illustrate the
impact of PA on pitch level, pitch range, speaking and articulation rate, and
time duration of pauses. It is also noted that the effect of PA is correlated
to some of the PA and PL measures.

Index Terms: acoustical analysis of speech, physical activity, physical
literacy, speech corpus

##  1 Introduction

Speech, as one of the most natural and common media of human communication,
carries a rich array of information that extends far beyond the verbal message
being expressed. From an utterance, the speaker's gender, age, dialectal
background, emotional status and personality could be identified by human
listeners. Part of the paralinguistic information in speech carries
physiological and healthy condition of a speaker. It is feasible to capture
the natural speech sound in the form of acoustic signals, and extract the
speaker's information via signal processing techniques and statistical
modeling [1, 2]. Acoustic and linguistic analysis of speech signals are
effective means of detecting and quantifying disorders, diseases, and other
changes in human body.

Speech production is an effortful activity that involves the brain and the
respiratory system. It is closely coupled with breathing. While talking,
inhalations are shortened and accelerated, while exhalations are prolonged and
slowed down [3]. The timing of inhalations depends on the control of habitual
respiratory cycles [4], is mostly determined in accordance with the linguistic
structure of speech [5]. If the functioning of a person's respiratory system
does not function properly, evident changes in terms of perceptual and
acoustic properties of the produced speech can be observed. In our daily
lives, the pattern of breathing is often altered by physical activity (PA). PA
refers to any body movement that increases the energy expenditure from resting
state and reaches exercise state. It may take place during running, swimming
or weight lifting, etc. [6]. In exercise states, PAs of different intensities
pose different levels of physical stress to human body. Respiratory rate
increases accordingly with the stress to provide adequate oxygen supply to
human body. The effect of PAs on breathing is expected to be manifested in the
produced speech. In [7], it was shown that perceivable prosodic characteristic
of speech, such as pause pattern, shifting of fundamental frequency (F0) and
formants might be consistent indicators of physical stress. Detection of
exercise intensity in PA was successfully performed using paralinguisic
features of speech [8].

The well-being of a person is related to participation in PAs. For older
adults, regular physical exercises was found to improve subjective well-being
measures, i.e. happiness, satisfaction with life [9]. Adolescents who reported
greater engagement in PAs tend to experience less stress and depression [10],
and have higher self-perceived physical competence [11], which is one’s
ability to move with competence in a wide variety of activities [12].
Recently, lights are shed on the concept of physical literacy (PL), which
extends beyond the participation of PAs [13]. A physically literate individual
should understand the value and responsibility, and is confident, motivated,
skilful in participating in PA throughout the lifetime. Numerous instruments
for assessing PL were proposed and validated [14]. Positive correlation
between perceived PL and PA was reported in a large population study [15].

PL is correlated with PA, while PA determines the well-being and competence of
a person, and affects the characteristics of breathing and speaking.
Intuitively, a physically literate person with strong competence tends to have
good respiratory control. This hypothetically implies less noticeable effect
of physical stress, in terms of the difference in speech characteristics
between resting and exercise states. The present study starts with the
collection of speech data in PA of different intensities. The speech database
is intended to support investigation on the relationship between PA/PL and
speech; facilitate statistical analysis of speech features and physiological
measures; provide training data for classification of PA intensity experienced
by the subjects. The design of PAs follows the Canadian Assessment of Physical
Literacy-2 (CAPL-2) test protocol [16]. For general population, speech
production may become laborious and difficult during and after PAs. This
results in breathy, intermittent and unstable voice as compared to that
produced in resting state. A fundamental hypothesis of this research is that
the influence of PA on speech production can be revealed by characteristic
changes of acoustic speech signals. F0 and temporal characteristics are
extracted from speech signal to reflect the difference of speech between
resting and exercise states. The correlation between the difference, and the
measures of PAs and PL, is investigated.

##  2 Design of Speech Corpus

###  2.1 About Cantonese

Cantonese is a major Chinese dialect widely spoken in Hong Kong, Macau,
Guangdong and Guangxi Provinces of Mainland China as well as overseas Chinese
communities. Each Chinese character is pronounced as a single syllable
carrying a lexical tone. The lexical tone (T) is determined by the temporal
pattern of fundamental frequency (F0) across the voiced portion of a syllable.
Cantonese has six tones, including T1 (high-level), T2 (high-rising), T3 (mid-
level), T4 (low-falling), T5 (low-rising) and T6 (low-level) [17]. As shown in
Figure 1, a legitimate Cantonese syllable can be divided into an onset and a
rime. Onset is a consonant, while rime can contain either a nucleus or a
nucleus followed by a coda. Nucleus can be a vowel or a diphthong, and coda
can be a final consonant. Both onset and coda are optional in a Cantonese
syllable. In the inventory of Cantonese phonemes, there are a total of $19$
initial consonants, $11$ vowels, $11$ diphthongs and $6$ final consonants [17,
18].

Figure 1: Structure of Cantonese syllable. '*' denotes optional.

###  2.2 Design of speech material

The speech materials consist of three parts. The first part contains two text
passages. Passage A is the story of `the North Wind and the Sun'. It comprises
142 syllables. The story content was originated from [19], and used in the
studies of Cantonese perceptual evaluation of voice [20] and speech rhythm
[21]. Passage B was composed by following the corpus designed in [8] to elicit
designated types of syntactic breaks and investigate how placing of pauses
interacts with breathing. The second part of speech materials contains three
sustained vowels, namely the low-central vowel /a:/, high-front vowel /i:/ and
mid-back vowel /O:/. Sustained vowels are most useful to reflect voice
characteristics in stable articulation states. The third part contains digit
sequences ``3947'', ``3948'', and ``0526''. The digits cover all of the 6
Cantonese tones. The digit sequences are intended for reliably measuring F0
level and F0 range of a Cantonese speaker.

###  2.3 Recording

Speech recording took place in an outdoor sports ground of basketball court
size. Under the instruction of qualified fitness trainers, each subject was
required to go through three different parts of PAs from CAPL-2. The first
part was the Canadian Agility and Movement Skill Assessment (CAMSA) [22]. It
measured the skill execution and completion time for a series of dynamic
movement to assess the subject's movement capabilities. The second exercise
was plank, a reliable measure of isometric core muscular endurance [23]. The
third part was the Progressive Aerobic Cardiovascular Endurance Run (PACER).
It assessed aerobic endurance and determined the maximal oxygen consumption
($\text{VO}_{2}$ max) of the subjects [24]. The physical intensity was
presumably increasing from CAMSA to PACER. Each subject was required to wear
heart rate monitors. The average heart rate during speaking was regarded as an
objective indicator of the activity intensity [25].

Speech recordings were made before and in-between the physical exercises.
Since the recordings shared the same linguistic content, they were most
suitable for contrastive investigation on the influence of PA on
paralinguistic factors of speech. A digital recorder (TASCOM DR-44WL) was
located at $20$-$50$ centimeters in front of the participant’s mouth. The gain
of the recorder was adjusted to maintain the background noise level below
-$30$ dB (relative to the maximum input level). Two-channel stereo recording
was made at sampling rate of 44.1 kHz. Each recording contains the four
designated text paragraphs. Four sets of speech recordings were obtained from
each subject.

The recorded audio signals were down-sampled to 16 kHz with mono-channel. Each
recording was divided into three sections corresponding to the three parts of
text materials.

###  2.4 Subjects and measurement of PA and PL

30 native Cantonese speakers (23 males, 7 females), aged between 18 to 21,
were recruited by convenience sampling. All speakers were university students
majored in sports science. The Physical Activity Readiness Questionnaires
(PAR-Q) [26] were distributed to all subjects. They all indicated answer
``NO'' to all questions, and became eligible to participate in the physical
excises. Informed consents were obtained prior to the recording session. The
subjects were required to complete the questionnaire of the 9-item Perceived
Physical Literacy Instrument (PPLI) [27]. The responses were rated on 5-point
Likert scale ranging from “strongly disagree” to “strongly agree”. The 9
items, denoted as PL1-9, considered the “knowledge and understanding”, “self-
expression and communication with others”, and “sense of self and self-
confidence” of a personnel. The subjects were also asked to report their daily
routine on PAs in sedentary, light, moderate and vigorous intensities.
Metabolic equivalent of task (MET), which represents the amount of PAs in
daily life, was measured based on the reported routine [28]. One MET is
defined as an oxygen uptake of 3.5 ml per kilogram per minute while a person
is sitting at rest. MET minutes per week was calculated for each level of PA
intensity. The subject's performance in terms of time and skill scores in
CAMSA, total elapsed time in plank, and number of round finished in PACER, was
measured.

##  3 Speech Feature Extraction

###  3.1 Extraction of F0

F0 in a person's speech carries linguistic, paralinguistic and physiological
information. F0 was extracted using the auto-correlation (AC) function via
Praat [29, 30]. Reliability of F0 estimation is sensitive to the parameter
settings, specifically, the floor ($F0_{floor}$) and ceiling values
($F0_{floor}$). Extracting the F0 for different speakers within the same floor
and ceiling values may not be optimal. Male voice typically has lower F0 than
female. F0 range is also related to language and emotional status of the
speaker [31]. We expect that physical stress would cause large alteration of
F0. A two-stage strategy of F0 extraction was adapted to improve the accuracy
of F0 estimation [32]. In the first-pass extraction, the floor and ceiling
values were initialized to 75 Hz and 600 Hz. Subsequently the 35-th quantile
($q_{35}$) and the 65-th quantile ($q_{65}$) of the F0 values were obtained.
Before the second-pass processing, the floor and ceiling values were updated
as $q_{35}*0.72-10$ and $q_{65}*1.9+10$. These equations were manually tuned
to better correlate between automatic and manual estimation of F0 [33]. The F0
contour was re-estimated as the final output. The utterance-level F0 was
obtained by averaging the F0 values of the vowel segments. Average F0 values
of tone 1 (T1), tone 3 (T3) and tone 6 (T6) in each utterance were calculated.
They represented the high, medium and low level of F0 in the utterance.

We are interested in how the F0 level changes resting state to exercise
states. The magnitude of F0 change is measured in terms of cents (¢), defined
as,

$\text{\textcent}=1200*\log_{2}{\frac{F0_{e}}{F0_{r}}}$

The variation of the F0 contour within an utterance is represented by the
dynamism quotient (DQ), which is given by dividing the standard deviation by
mean [31, 34]. Higher DQ indicates larger variation of F0. The change of DQ
from resting state to exercise states was measured as
$\text{DQ}_{e}/\text{DQ}_{r}$.

Table 1: Mean (standard deviation) of speech features.

Feature | Measure | Rest | CAMSA | Plank | PACER
---|---|---|---|---|---
F0 (utterance) | Hz | 143 (38) | 147 (41) | 144 (40) | 159 (41)
DQ (%) | 15.58 (2.90) | 16.12 (3.10) | 15.95 (3.34) | 16.48 (2.65)
F0 (Tone 1) | Hz | 162 (45) | 166 (48) | 163 (46) | 178 (46)
DQ (%) | 14.29 (3.05) | 15.01 (3.15) | 15.22 (3.35) | 15.63 (3.06)
F0 (Tone 3) | Hz | 142 (39) | 144 (41) | 143 (40) | 157 (40)
DQ (%) | 11.45 (2.73) | 12.41 (2.90) | 11.79 (2.70) | 12.69 (2.3)
F0 (Tone 6) | Hz | 133 (37) | 137 (40) | 135 (39) | 147 (39)
DQ (%) | 10.95 (2.55) | 11.30 (2.31) | 12.18 (3.28) | 12.60 (2.47)
| Speaking
---
rate
spm | 246 (34) | 267 (38) | 267 (34) | 252 (37)
| Articulation
---
rate
306 (36) | 341 (41) | 340 (42) | 366 (36)
| Speech to
---
non-speech
ratio
Ratio | 4.33 (1.25) | 3.90 (1.25) | 4.02 (1.35) | 2.35 (0.73)

###  3.2 Timing characteristics

Increased respiratory rate is a typical response to physical stress. To
understand how speech properties change with intensity level of physical
stress, temporal characteristics of speech including speaking rate,
articulation rate and speech-to-pause time ratio were examined. Speaking rate
is computed as the number of elicited syllables per minute (spm). Articulation
rate has the same definition as speaking rate, except that the time intervals
occupied by non-speech segments are excluded. Speech-to-pause time ratio is
the total time of speech divided by the total duration of silence. Besides
syntactic pauses, breathing pauses are frequently found in speech under
physical stress. The speech-to-pause ratio is expected to carry the
information of breathing strategy in speech under physical stress. The change
of speech-to-pause time ratio (tr) from resting state to exercise states is
expressed by $\text{tr}_{e}/\text{tr}_{r}$. For speaking rate (sr) and
articulation rate (ar), the difference between the two states are calculated
as by $\text{sr}_{e}-\text{sr}_{r}$, and $\text{ar}_{e}-\text{ar}_{r}$.

To automatically locate the time boundaries of phones and syllables, forced
alignment was applied to the utterances with the Gaussian Mixture Model-Hidden
Markov Model (GMM-HMM) tri-phone acoustic model, using the text transcription
of ``The North Wind and The Sun". The acoustic model was trained on a large-
vocabulary database of adult speech called CUSENT. The corpus contained about
20 hours of speech recorded from $76$ adult speakers [18]. The acoustic
features for GMM-HMM training consist of $13$-dimensional Mel-frequency
cepstral coefficients (MFCC) and their first- and second-order derivatives
extracted at every $0.010$ second. Feature space Maximum Likelihood Linear
Regression (fMLLR) was applied to the acoustic features[35]. Forced alignment
and acoustic model training were implemented by the Kaldi speech recognition
toolkit [36].

Table 2: Difference of speech features between resting and exercise states.

Feature |  Measure of Difference | Rest-CAMSA | Rest-Plank | Rest-PACER
---|---|---|---|---
F0 (utterance) | Cents | 39 (84) | 11 (81) | 181 (141)
DQ ratio (%) | 104.30 (13.40) | 102 (12.37) | 107.15 (14.98)
F0 (Tone 1) | Cents | 37 (99) | 0 (103) | 168 (153)
DQ ratio (%) | 105.96 (13.35) | 107.10 (13.55) | 110.99 (17.44)
F0 (Tone 3) | Cents | 29 (95) | 10 (86) | 181 (159)
DQ ratio (%) | 112.55 (30.89) | 105.63 (21.49) | 115.12 (26.88)
F0 (Tone 6) | Cents | 33 (80) | 16 (77) | 172 (128)
DQ ratio (%) | 105.52 (20.13) | 112.46 (21.77) | 119.23 (29.09)
| Speaking
---
rate
spm difference | 21 (29) | 21 (33) | 6 (37)
| Articulation
---
rate
35 (27) | 34 (33) | 61 (34)
| Speech to
---
non-speech
ratio
Ratio (%) | 91.76 (24.12) | 94.59 (24.40) | 56.79 (19.49)

##  4 Results and Discussion

Table 3: Correlation between the acoustic changes in exercise states and the
PA/PL measures.

Feature | Measure | Rest-CAMSA | Rest-Plank | Rest-PACER
---|---|---|---|---
PA/PL Measures | PCC | PA/PL Measures | PCC | PA/PL Measures | PCC
F0 (utterance) | Cents | - | - | - | - | - | -
DQ ratio | PL1 | -0.456 | PL3 | -0.377 | MET (total) | 0.420
Pacer round | -0.399 | PL9 | -0.377
CAMSA skill score | -0.501 | MET (total) | 0.379
Speaking rate |  spm difference | PL1 | -0.410 | PL1 | -0.560 | PL1 | -0.540
CAMSA time score | -0.376
| Articulation
---
rate
PL1 | -0.403 | PL1 | -0.466 | PL1 | -0.465
Speech-to-pause time ratio | Ratio | PL2 | -0.388 | PL1 | -0.513 | PPLI | -0.376
Pacer round | 0.374 | PL2 | -0.448 | CAMSA time score | -0.370

###  4.1 Analysis of speech features

Table 1 reports the mean and the standard deviation of F0, speaking rate,
articulation rate and speech-to-pause time ratio in the resting and exercise
states. The difference of feature values between resting and exercise states
are shown in Table 2. Given limited amount of speech recordings, the gender
effect is not considered in the analysis. ANOVA and the two-tailed t-test are
applied. A significant threshold of 0.05 is assumed.

The results of ANOVA indicate that the change of F0 levels in different
exercises are significantly different ($p<0.001$). An increase of over 150
cents is observed in PACER. The DQs in CAMSA, plank and PACER do not differ
significantly. For each type of exercise, significant difference is found in
the DQs of T1, T3, and T6 ($p<0.001$). Physical stress exhibits different
extents of influence to different tones.

The two-tailed t-test shows that the speaking rate, articulation rate and
speech-to-pause time ratio of exercise states are significantly different from
that in resting state ($p<0.01$). ANOVA indicates the increase of speaking
rate and articulation rate, and the decrease of speech-to-pause time ratio,
are significantly different among CAMSA, plank and PACER ($p<0.001$). The
temporal characteristics of speech in PACER are further compared to resting
state via the t-test. A notable increase in articulation rate is observed
($p<0.001$), but not in speaking rate. Subjects generally requires more
frequent and deeper breathing in high-intensity exercise. The speech-to-pause
time ratio is lower in the exercise states ($p<0.001$). These observations
suggest that the subjects tend to talk faster and use longer time to breath in
PACER.

###  4.2 Correlation with PA and PL measures

We attempt to correlate the measures of PA/PL with the changes of
acoustical/temporal characteristics of speech in physical stress. The daily
exercise routine of the subjects and their performance in different exercises
are provided as the PA/PL measures. The correlation is measured by the
Pearson's correlation coefficient (PCC). PA/PL measures that are significantly
correlated to the changes of speech features are reported in Table 3.

The changes of speaking rate and articulation rate in exercise states are
correlated mostly to PL1 and PL2. PL1 and PL2 correspond to `I am physically
fit in accordance with my age' and 'I have a positive attitude and interest in
sports'. As subjects are confident with their physical fitness and
enthusiastic in sports, the temporal characteristics of speech are less
altered during exercises. The self-perceived physical fitness in youngsters is
a reliable predictor of actual fitness [37]. Subjects who rated themselves
physically fit could be in fact in good shape, and better control their
breathing during exercises. This leads to similar speaking and articulation
rates between resting and exercise states. For speech-to-pause time ratio,
subjects who perform well in CAMSA, i.e. high CAMSA time score, have less
changes in the time ratio in PACER. Subjects who endured more rounds in PACER
enlarges the speech-to-pause time ratio more in CAMSA. A negative correlation
is found in between the PPLI score and the speech-to-time ratio in PACER. The
overall descriptor of a person's PL is connected to characteristic changes of
speech.

PA/PL measures do not significantly correlate to the shift of F0 in exercise
states ($p>0.05$). Changes of DQ have negative correlation with PL1, PL3 and
PL9. The descriptions of PL3 and PL 9 are `I appreciate myself or others doing
sports' and `I am aware of the benefits of sports related to health'. Both PL3
and PL9 concern the knowledge and understanding of participating in PAs. A
better understanding towards PA, and better performance in PA, i.e. higher
CAMSA skill score and more PACER rounds, lead to less deviation of F0 contour
during exercises. The frequency of engagement in PA also plays a significant
role in the changes of acoustic features in exercise states. Higher total MET
score corresponds to larger DQs.

##  5 Conclusion

We presented the collection of speech data for investigating the relation
between physical stress and speech. Statistical analysis has shown that there
are different extent of changes in fundamental frequency, speaking and
articulation rate, and speech-to-pause time ratio in different physical
activities. Variations of F0 in different Cantonese tones are different in the
resting and exercise states. Correlation analysis indicates that part of the
measures of physical activities and physical literacy, and daily habits of
participating in physical activities, are associated with the changes in
acoustic/ temporal characteristics of speech. To extend the present study,
more speech data will be collected from subjects of diverse background. In-
depth analysis of the 6 Cantonese tones under physical stress, investigation
on the interaction between pauses and breathing, classification of physical
stress, and automatic assessment of physical literacy will be among the future
works.

##  6 Acknowledgements

This research is partially supported by a GRF project grant (Ref: CUHK
14208020) from Hong Kong Research Grants Council.

## References

  * [1] N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, ``Front-end factor analysis for speaker verification,'' _IEEE Transactions on Audio, Speech, and Language Processing_ , vol. 19, no. 4, pp. 788–798, 2010.
  * [2] G. Mohammadi, A. Vinciarelli, and M. Mortillaro, ``The voice of personality: Mapping nonverbal vocal behavior into trait attributions,'' in _Proc. of SSPW_ , 2010, pp. 17–20.
  * [3] B. Conrad and P. Schoenle, ``Speech and respiration,'' _Archiv für Psychiatrie und Nervenkrankheiten_ , vol. 226, no. 4, pp. 251–268, 1979.
  * [4] J. Trouvain and K. P. Truong, ``Prosodic characteristics of read speech before and after treadmill running,'' in _Proc. of Interspeech_ , 2015.
  * [5] A. Rochet-Capellan and S. Fuchs, ``The interplay of linguistic structure and breathing in german spontaneous speech,'' in _Proc. of Interspeech_ , 2013, pp. 2014–2018.
  * [6] B. Kumar, R. Robinson, and S. Till, ``Physical activity and health in adolescence,'' _Clinical Medicine_ , vol. 15, no. 3, p. 267, 2015.
  * [7] K. W. Godin and J. H. Hansen, ``Physical task stress and speaker variability in voice quality,'' _EURASIP Journal on Audio, Speech, and Music Processing_ , vol. 2015, no. 1, pp. 1–13, 2015.
  * [8] K. P. Truong, A. Nieuwenhuys, P. Beek, and V. Evers, ``A database for analysis of speech under physical stress: detection of exercise intensity while running and talking,'' in _Proc. of Interspeech_ , 2015, pp. 3705–3709.
  * [9] E. McAuley, B. Blissmer, D. X. Marquez, G. J. Jerome, A. F. Kramer, and J. Katula, ``Social relations, physical activity, and well-being in older adults,'' _Preventive medicine_ , vol. 31, no. 5, pp. 608–617, 2000.
  * [10] R. Norris, D. Carroll, and R. Cochrane, ``The effects of physical activity and exercise training on psychological stress and well-being in an adolescent population,'' _Journal of psychosomatic research_ , vol. 36, no. 1, pp. 55–65, 1992.
  * [11] J. Timo, Y.-P. Sami, W. Anthony, and L. Jarmo, ``Perceived physical competence towards physical activity, and motivation and enjoyment in physical education as longitudinal predictors of adolescents’ self-reported physical activity,'' _Journal of science and medicine in sport_ , vol. 19, no. 9, pp. 750–754, 2016.
  * [12] L. C. Edwards, A. S. Bryant, R. J. Keegan, K. Morgan, and A. M. Jones, ``Definitions, foundations and associations of physical literacy: a systematic review,'' _Sports medicine_ , vol. 47, no. 1, pp. 113–126, 2017\.
  * [13] M. Whitehead, _Physical literacy: Throughout the lifecourse_. Routledge, 2010.
  * [14] M. Tremblay and M. Lloyd, ``Physical literacy measurement: The missing piece,'' _Physical and health education journal_ , vol. 76, no. 1, pp. 26–30, 2010\.
  * [15] S. M. Choi, R. K. W. Sum, E. F. L. Leung, and R. S. K. Ng, ``Relationship between perceived physical literacy and physical activity levels among hong kong adolescents,'' _PLoS One_ , vol. 13, no. 8, p. e0203105, 2018.
  * [16] P. E. Longmuir, K. E. Gunnell, J. D. Barnes, K. Belanger, G. Leduc, S. J. Woodruff, and M. S. Tremblay, ``Canadian assessment of physical literacy second edition: a streamlined assessment of the capacity for physical activity among children 8 to 12 years of age,'' _BMC public health_ , vol. 18, no. 2, pp. 1–12, 2018.
  * [17] R. S. Bauer and P. K. Benedict, _Modern cantonese phonology_. Walter de Gruyter, 2011, vol. 102.
  * [18] T. Lee, W. K. Lo, P. Ching, and H. Meng, ``Spoken language resources for cantonese speech processing,'' _Speech Communication_ , vol. 36, no. 3-4, pp. 327–342, 2002.
  * [19] E. Zee, ``Chinese (hong kong cantonese),'' _Journal of the International Phonetic Association_ , vol. 21, no. 1, pp. 46–48, 1991.
  * [20] T. Law, K. Lee, J. Lam, A. C. van Hasselt, and M. C.-F. Tong, ``The construction of the cantonese perceptual evaluation of voice (canpev): the content validation process,'' in _Proc. of VOICE_ , 2010, p. 159.
  * [21] P. P. K. Mok and V. Dellwo, ``Comparing native and non-native speech rhythm using acoustic rhythmic measures: Cantonese, Beijing Mandarin and English,'' in _Proc. Speech Prosody 2008_ , 2008, pp. 423–426.
  * [22] P. E. Longmuir, C. Boyer, M. Lloyd, M. M. Borghese, E. Knight, T. J. Saunders, E. Boiarskaia, W. Zhu, and M. S. Tremblay, ``Canadian agility and movement skill assessment (camsa): Validity, objectivity, and reliability evidence for children 8–12 years of age,'' _Journal of sport and health science_ , vol. 6, no. 2, pp. 231–240, 2017.
  * [23] K. L. Schellenberg, J. M. Lang, K. M. Chan, and R. S. Burnham, ``A clinical tool for office assessment of lumbar spine stabilization endurance: prone and supine bridge maneuvers,'' _American journal of physical medicine & rehabilitation_, vol. 86, no. 5, pp. 380–386, 2007.
  * [24] S. N. Scott, D. L. Thompson, and D. P. Coe, ``The ability of the pacer to elicit peak exercise response in youth [corrected].'' _Medicine and science in sports and exercise_ , vol. 45, no. 6, pp. 1139–1143, 2013.
  * [25] J. Karvonen and T. Vuorimaa, ``Heart rate and exercise intensity during sports activities,'' _Sports medicine_ , vol. 5, no. 5, pp. 303–311, 1988.
  * [26] R. J. Shephard, ``Par-q, canadian home fitness test and exercise screening alternatives,'' _Sports medicine_ , vol. 5, no. 3, pp. 185–195, 1988.
  * [27] R. K. Sum, C.-F. Cheng, T. Wallhead, C.-C. Kuo, F.-J. Wang, and S.-M. Choi, ``Perceived physical literacy instrument for adolescents: A further validation of ppli,'' _Journal of Exercise Science & Fitness_, vol. 16, no. 1, pp. 26–31, 2018.
  * [28] M. Fan, J. Lyu, and P. He, ``Chinese guidelines for data processing and analysis concerning the international physical activity questionnaire,'' _Chinese journal of Epidemiology_ , vol. 35, no. 8, pp. 961–964, 2014.
  * [29] Y. Jadoul, B. Thompson, and B. De Boer, ``Introducing parselmouth: A python interface to praat,'' _Journal of Phonetics_ , vol. 71, pp. 1–15, 2018.
  * [30] P. Boersma, ``Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound,'' in _Proceedings of the institute of phonetic sciences_ , vol. 17, no. 1193, 1993, pp. 97–110.
  * [31] H. Traunmüller and A. Eriksson, ``The frequency range of the voice fundamental in the speech of male and female adults,'' _Manuscript, Department of Linguistics, University of Stockholm_ , 1995.
  * [32] K. Evanini and C. Lai, ``The importance of optimal parameter setting for pitch extraction.'' _The Journal of the Acoustical Society of America_ , vol. 128, no. 4, pp. 2291–2291, 2010.
  * [33] C. De Looze and S. Rauzy, ``Automatic detection and prediction of topic changes through automatic detection of register variations and pause duration,'' in _Proc. of Interspeech_ , 2009.
  * [34] R. Hincks, ``Processing the prosody of oral presentations,'' in _InSTIL/ICALL Symposium 2004_ , 2004.
  * [35] M. J. Gales, ``Maximum likelihood linear transformations for hmm-based speech recognition,'' _Computer speech & language_, vol. 12, no. 2, pp. 75–98, 1998.
  * [36] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz _et al._ , ``The kaldi speech recognition toolkit,'' in _Proc. of ASRU_ , no. EPFL-CONF-192584, 2011\.
  * [37] J. L. Germain and H. A. Hausenblas, ``The relationship between perceived and actual physical fitness: a meta-analysis,'' _Journal of Applied Sport Psychology_ , vol. 18, no. 4, pp. 283–296, 2006.