# SGG: Learning to Select, Guide, and Generate
for Keyphrase Generation

Jing Zhao, Junwei Bao, Yifan Wang, Youzheng Wu, Xiaodong He, Bowen Zhou
JD AI Research
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

Keyphrases, that concisely summarize the high-level topics discussed in a
document, can be categorized into present keyphrase which explicitly appears
in the source text, and absent keyphrase which does not match any contiguous
subsequence but is highly semantically related to the source. Most existing
keyphrase generation approaches synchronously generate present and absent
keyphrases without explicitly distinguishing these two categories. In this
paper, a Select-Guide-Generate (SGG) approach is proposed to deal with present
and absent keyphrase generation separately with different mechanisms.
Specifically, SGG is a hierarchical neural network which consists of a
pointing-based selector at low layer concentrated on present keyphrase
generation, a selection-guided generator at high layer dedicated to absent
keyphrase generation, and a guider in the middle to transfer information from
selector to generator. Experimental results on four keyphrase generation
benchmarks demonstrate the effectiveness of our model, which significantly
outperforms the strong baselines for both present and absent keyphrases
generation. Furthermore, we extend SGG to a title generation task which
indicates its extensibility in natural language generation tasks.111Our code
is released in https://github.com/JD-AI-Research-NLP/SGG.

##  1 Introduction

Automatic keyphrase prediction recommends a set of representative phrases that
are related to the main topics discussed in a document Liu et al. (2009).
Since keyphrases can provide a high-level topic description of a document,
they are beneficial for a wide range of natural language processing (NLP)
tasks, such as information extraction Wan and Xiao (2008), text summarization
Wang and Cardie (2013) and question generation Subramanian et al. (2018).

Existing methods for keyphrase prediction can be categorized into extraction
and generation approaches. Specifically, keyphrase extraction methods identify
important consecutive words from a given document as keyphrases, which means
that the extracted keyphrases (denoted as present keyphrases) must exactly
come from the given document. However, some keyphrases (denoted as absent
keyphrases) of a given document do not match any contiguous subsequence but
are highly semantically related to the source text. The extraction methods
fail to predict these absent keyphrases. Therefore, generation methods have
been proposed to produce a keyphrase verbatim from a predefined vocabulary, no
matter whether the generated keyphrase appears in the source text. Compared
with conventional extraction methods, generation methods have the ability of
generating absent keyphrases as well as present keyphrases.

Figure 1: An example of keyphrase prediction by SGG.

CopyRNN Meng et al. (2017) is the first to employ the sequence-to-sequence
(Seq2Seq) framework Sutskever et al. (2014) with the copying mechanism Gu et
al. (2016) to generate keyphrases for the given documents. Following the
CopyRNN, several Seq2Seq-based keyphrase generation approaches have been
proposed to improve the generation performance Chen et al. (2018); Ye and Wang
(2018); Chen et al. (2019); Zhao and Zhang (2019); Wang et al. (2019); Yuan et
al. (2020). All these existing methods generate present and absent keyphrases
synchronously without explicitly distinguishing these two different categories
of keyphrases, which leads to two problems: (1) They complicate the
identification of present keyphrases. Specifically, they search for words over
the entire predefined vocabulary containing a vast amount of words (e.g.,
50,000 words) to generate a present keyphrase verbatim, which is
overparameterized since a present keyphrase can be simply selected from a
continuous subsequence of the source text containing limited words (e.g., less
than 400 words). (2) They weaken the generation of absent keyphrases. Existing
models for absent keyphrase generation are usually trained on datasets mixed
with a large proportion of present keyphrases. Table 1 shows that nearly half
of the training data are present keyphrases, which leads to the extremely low
proportions of absent keyphrases generated by such a model, i.e., CopyRNN. The
above observation demonstrates that these methods are biased towards
replicating words from source text for present keyphrase generation, which
will inevitably affect the performance on generating absent keyphrases.

Training(%) | Test(%)
---|---
|  Inspec |  Krapivin |  NUS |  SemEval
49.79 |  13.12 |  11.74 |  11.30 |  11.25
Table 1: Proportions of absent keyphrases in training set and predictions of
CopyRNN on four commonly used datasets, where top-10 predictions are
considered.

To address the aforementioned problems, we propose a Select-Guide-Generate
(SGG) approach, which deals with present and absent keyphrase generation
separately with different stages based on different mechanisms. Figure 1
illustrates an example of keyphrase prediction by SGG. The motivation behind
is to solve keyphrase generation problem from selecting to generating, and use
the selected results to guide the generation. Specifically, our SGG is
implemented with a hierarchical neural network which performs Seq2Seq learning
by applying a multi-task learning strategy. This network consists of a
selector at low layer, a generator at high layer, and a guider at middle layer
for information transfer. The selector generates present keyphrases through a
pointing mechanism Vinyals et al. (2015), which adopts attention distributions
to select a sequence of words from the source text as output. The generator
further generates the absent keyphrases through a pointing-generating (PG)
mechanism See et al. (2017). Since present keyphrases have already been
generated by the selector, they should not be generated again by the
generator. Therefore, a guider is designed to memorize the generated present
keyphrases from the selector, and then fed into the attention module of the
generator to constrain it to focus on generating absent keyphrases. We
summarize our main contributions as follows:

  * •

We propose a SGG approach which models present and absent keyphrase generation
separately in different stages, i.e., select, guide, and generate, without
sacrificing the end-to-end training through back-propagation.

  * •

Extensive experiments are conducted to verify the effectiveness of our model,
which not only improves present keyphrase generation but also dramatically
boosts the performance of absent keyphrase generation.

  * •

Furthermore, we adopt SGG to a title generation task, and the experiment
results indicate the extensibility and effectiveness of our SGG approach on
generation tasks.

##  2 Related Work

As mentioned in Section 1, the extraction and generation methods are two
different research directions in the field of keyphrase prediction. The
existing extraction methods can be broadly classified into supervised and
unsupervised approaches. The supervised approaches treat keyphrase extraction
as a binary classification task, which train the models with the features of
labeled keyphrases to determine whether a candidate phrase is a keyphrase
Witten et al. (1999); Medelyan et al. (2009); Gollapalli et al. (2017). In
contrast, the unsupervised approaches treat keyphrase extraction as a ranking
task, scoring each candidate using some different ranking metrics, such as
clustering Liu et al. (2009), or graph-based ranking Mihalcea and Tarau
(2004); Wang et al. (2014); Gollapalli and Caragea (2014); Zhang et al.
(2017).

This work is mainly related to keyphrase generation approaches which have
demonstrated good performance on keyphrase prediction task. Following CopyRNN
Meng et al. (2017), several extensions have been proposed to boost the
generation capability. In CopyRNN, model training heavily relies on large
amount of labeled data, which is often unavailable especially for the new
domains. To address this problem, Ye and Wang (2018) proposed a semi-
supervised keyphrase generation model that utilizes both abundant unlabeled
data and limited labeled data. CopyRNN uses the concatenation of article title
and abstract as input, ignoring the leading role of the title. To address this
deficiency, Chen et al. (2019) proposed a title-guided Seq2Seq network to
sufficiently utilize the already summarized information in title. In addition,
some research attempts to introduce external knowledge into keyphrase
generation, such as syntactic constraints Zhao and Zhang (2019) and latent
topics Wang et al. (2019).

These approaches do not consider the one-to-many relationship between the
input text and target keyphrases, and thus fail to model the correlation among
the multiple target keyphrases. To overcome this drawback, Chen et al. (2018)
incorporated the review mechanism into keyphrase generation and proposed a
model CorrRNN with correlation constraints. Similarly, SGG separately models
one-to-many relationship between the input text and present keyphrases and
absent keyphrases. To avoid generating duplicate keyphrases, Chen et al.
(2020) proposed an exclusive hierarchical decoding framework that includes a
hierarchical decoding process and either a soft or a hard exclusion mechanism.
For the same purpose, our method deploys a guider to avoid the generator
generating duplicate present keyphrases. Last but most important, all these
methods do not consider the difference between present and absent keyphrases.
We are the first to discriminately treat present and absent keyphrases in
keyphrase generation task.

##  3 Methodology

###  3.1 Problem Definition

Given a dataset including $K$ data samples, where the $j$-th data item
$\langle x^{(j)},y^{(j,p)},y^{(j,a)}\rangle$ consists of a source text
$x^{(j)}$, a set of present keyphrases $y^{(j,p)}$ and a set of absent
keyphrases $y^{(j,a)}$. Different from CopyRNN Meng et al. (2017) splitting
each data item into multiple training examples, each of which contains only
one keyphrase as target, we regard each data item as one training example by
concatenating its present keyphrases as one target and absent keyphrases as
another one. Specifically, assume that the $j$-th data item consists of $m$
present keyphrases $\\{y^{(j,p)}_{1},...,y^{(j,p)}_{m}\\}$ and $n$ absent
keyphrases $\\{y^{(j,a)}_{1},...,y^{(j,a)}_{n}\\}$, the target present
keyphrases $y^{(j,p)}$ and target absent keyphrases $y^{(j,a)}$ are
represented as:

$y^{(j,p)}=y^{(j,p)}_{1}\ ||\ y^{(j,p)}_{2}\ ||\ ...\ ||\ y^{(j,p)}_{m}$
$y^{(j,a)}=y^{(j,a)}_{1}\ ||\ y^{(j,a)}_{2}\ ||\ ...\ ||\ y^{(j,a)}_{n}$

where $||$ is a special splitter to separate the keyphrases. We then get the
source text $x^{(j)}$, the present keyphrases $y^{(j,p)}$ and the absent
keyphrases $y^{(j,a)}$ all as word sequences. Under this setting, our model is
capable of generating multiple keyphrases in one sequence as well as capturing
the mutual relations between these keyphrases. A keyphrase generation model is
to learn the mapping from the source text $x^{(j)}$ to the target keyphrases
$(y^{(j,p)},y^{(j,a)})$. For simplicity, $(x,y^{p},y^{a})$ is used to denote
each item in the rest of this paper, where $x$ denotes a source text sequence,
$y^{p}$ denotes its present keyphrase sequence and $y^{a}$ denotes its absent
keyphrase sequence.

###  3.2 Model Overview

The architecture of our proposed Select-Guide-Generate (SGG) approach is
illustrated in Figure 2. Our model is the extension of Seq2Seq framework which
consists of a text encoder, a selector, a guider, and a generator. The text
encoder converts the source text $x$ into a set of hidden representation
vectors $\\{\mathbf{h}_{i}\\}^{L}_{i=1}$ with a bi-directional Long Short-term
Memory Network (bi-LSTM) Hochreiter and Schmidhuber (1997), where $L$ is the
length of source text sequence. The selector is a uni-directional LSTM, which
predicts the present keyphrase sequence $y^{p}$ based on the attention
distribution over source words. After selecting present keyphrases, a guider
is produced by a guider to memorize the prediction information of the
selector, and then fed to the attention module of a generator to adjust the
information it pays attention to. The selection-guided generator is also
implemented as a uni-directional LSTM, which produces the absent keyphrase
sequence $y^{a}$ based on two distributions over predefined-vocabulary and
source words, respectively. At the same time, a soft switch gate $p_{gen}$ is
employed as a trade-off between the above two distributions.

Figure 2: The architecture of the proposed SGG which is implemented with a
hierarchical neural network.

###  3.3 Text Encoder

The goal of a text encoder is to provide a series of dense representations
$\\{\mathbf{h}_{i}\\}^{L}_{i=1}$ of the source text. In our model, the text
encoder is implemented as a bi-LSTM Hochreiter and Schmidhuber (1997) which
reads an input sequence $x$ = $\\{x_{i}$}${}^{L}_{i=1}$ from two directions
and outputs a sequence of forward hidden states
$\\{\overrightarrow{\mathbf{h}_{i}}$}${}^{L}_{i=1}$ and backward hidden states
$\\{\overleftarrow{\mathbf{h}_{i}}$}${}^{L}_{i=1}$ by iterating the following
equations:

$\displaystyle\overrightarrow{\mathbf{h}_{i}}=\mathtt{LSTM}(x_{i},{\mathbf{h}}_{i-1})$
(1)
$\displaystyle\overleftarrow{\mathbf{h}_{i}}=\mathtt{LSTM}(x_{i},{\mathbf{h}}_{i+1})$
(2)

The final hidden representation $\mathbf{h}_{i}$ of the $i$-th source word is
the concatenation of forward and backward hidden states, i.e.,
$\mathbf{h}_{i}=[\overrightarrow{\mathbf{h}_{i}};\overleftarrow{\mathbf{h}_{i}}]$.

###  3.4 Selector

A selector is designed to generate present keyphrase sequences through the
pointer mechanism Vinyals et al. (2015), which adopts the attention
distribution as a pointer to select words from the source text as output.
Specifically, given source text sequence $x$ and previously generated words
$\\{y^{p}_{1},...,y^{p}_{t-1}\\}$, the probability distribution of predicting
next word $y^{p}_{t}$ in present keyphrases is:

$\displaystyle\mathcal{P}(y^{p}_{t}\ |\
y^{p}_{<t},x)=\mathcal{\mathbf{missing}}{\alpha}^{p,t}=\mathtt{softmax}(\mathbf{u}^{p,t})$
(3) $\displaystyle
u^{p,t}_{i}={\mathbf{V}}^{T}_{p}\mathtt{tanh}({\mathbf{W}}_{p}[\mathbf{s}^{p}_{t};\mathbf{h}_{i}]+{\mathbf{b}}_{p})$
(4)

where $\alpha^{p,t}$ is the attention Bahdanau et al. (2015) distribution at
decoding time step $t$, $i\in(1,...,L)$, and ${\mathbf{V}}_{p}$,
${\mathbf{W}}_{p}$ and ${\mathbf{b}}_{p}$ are trainable parameters of the
model. $\mathbf{u}^{p,t}$ can be viewed as the degree of matching between
input at position $i$ and output at position $t$. $\mathbf{s}^{p}_{t}$
represents the hidden state at deciding time step $t$, and is updated by
equation:

$\mathbf{s}^{p}_{t}=\mathtt{LSTM}(y^{p}_{t-1},\mathbf{s}^{p}_{t-1},\mathbf{c}^{p}_{t-1})$
(5)

where context vector
$\mathbf{c}^{p}_{t-1}=\sum_{i=1}^{L}\alpha^{p,t-1}_{i}\mathbf{h}_{i}$ is the
weighted sum of source hidden states.

###  3.5 Guider

A guider is designed to fully utilize the attention information of the
selector to guide the generator on absent keyphrase generation. The idea
behind is to utilize a guider $\mathbf{r}$ to softly indicate which words in
source text have been generated by the selector. This is important for helping
the generator to focus on generating the absent keyphrases. Specifically,
$\mathbf{r}$ is constructed through the accumulation of the attention
distributions over all decoding time steps of the selector, computed as:

$\mathbf{r}=\sum_{t=1}^{M}\mathbf{\alpha}^{p,t}$ (6)

where $M$ is the length of present keyphrase sequence. $\mathbf{r}$ is an
unnormalized distribution over the source words. As the attention distribution
of selector is equal to the probability distribution over the source words,
$\mathbf{r}$ represents the possibility that these words have been generated
by the selector. The calculation of guider is inspired by the coverage vector
Tu et al. (2016) that is sequentially updated during the decoding process. In
contrast to this, the guider here is a static vector which is capable of
memorizing a global information.

###  3.6 Selection-Guided Generator

A generator aims to predict an absent keyphrase sequence based on the guidance
of the selection information from the guider. Unlike present keyphrases, most
words in absent keyphrases do not appear in source text. Therefore, the
generator generates absent keyphrases by picking up words from both a
predefined large scale vocabulary and the source text See et al. (2017); Gu et
al. (2016). The probability distribution of predicting next word $y^{a}_{t}$
in absent keyphrases is defined as:

$\displaystyle\mathcal{P}(y^{a}_{t}\ |\ y^{a}_{<t},x)$ (7)
$\displaystyle=p_{gen}\mathcal{P}_{vocab}(y^{a}_{t})+(1-p_{gen})\\!\\!\\!\sum_{i:y^{a}_{t}=x_{i}}\alpha^{a,t}_{i}$

where $\mathcal{P}_{vocab}$ is the probability distribution over the
predefined vocabulary, which is zero if $y^{a}_{t}$ is an out-of-vocabulary
(OOV) word. Similarly, if $y^{a}_{t}$ does not appear in the source text, then
$\sum_{i:y^{a}_{t}=x_{i}}\alpha^{a,t}_{i}$ is zero. $\mathcal{P}_{vocab}$ is
computed as:

$\mathcal{P}_{vocab}(y^{a}_{t})=\mathtt{softmax}({\mathbf{W}}[\mathbf{s}^{a}_{t};\mathbf{c}^{a}_{t}]+{\mathbf{b}})$
(8)

where ${\mathbf{W}}$ and ${\mathbf{b}}$ are learnable parameters,
$\mathbf{s}^{a}_{t}$ is the hidden state of generator, and
$\mathbf{c}^{a}_{t}$ is the context vector for generating absent keyphrase
sequence, computed by the following equations:

$\displaystyle\mathbf{c}^{a}_{t}$
$\displaystyle=\sum_{i=1}^{L}\mathbf{\alpha}^{a,t}_{i}\mathbf{h}_{i}$ (9)
$\displaystyle\mathcal{\alpha}^{a,t}$
$\displaystyle=\mathtt{softmax}(\mathbf{u}^{a,t})$ (10) $\displaystyle
u^{a,t}_{i}$
$\displaystyle={\mathbf{V}}^{T}_{a}\mathtt{tanh}({\mathbf{W}}_{a}[\mathbf{s}^{a}_{t};\mathbf{h}_{i};\mathbf{r}]+{\mathbf{b}}_{a})$
(11)

where ${\mathbf{V}}_{a}$, ${\mathbf{W}}_{a}$ and ${\mathbf{b}}_{a}$ are
learnable parameters. $\mathbf{r}$ is a vector produced by the guider. The
generation probability $p_{gen}$ at time step $t$ is computed as:

$p_{gen}={\mathtt{\sigma}}({\mathbf{W}}_{gen}[\mathbf{c}^{a}_{t};\mathbf{s}^{a}_{t};\mathtt{emb}(y^{a}_{t-1})]+{\mathbf{b}}_{gen})$
(12)

where ${\mathbf{W}}_{gen}$ and ${\mathbf{b}}_{gen}$ are learnable parameters,
$\mathtt{\sigma(\cdot)}$ represents a sigmoid function and
$\mathtt{emb}(y^{a}_{t-1})$ is the embedding of $y^{a}_{t-1}$. In addition,
$p_{gen}$ in formula (LABEL:equ_8) is used as a soft switch to choose either
generating words over vocabulary or copying words from source text based on
distribution $\alpha^{a,t}$.

###  3.7 Training

Given the set of data pairs $\\{x^{(j)},y^{(j,p)},y^{(j,a)}\\}^{K}_{j=1}$, the
loss function of the keyphrase generation consists of two parts of cross
entropy losses:

$\displaystyle\mathcal{L}_{p}(\theta)=-\sum_{j=1}^{K}\sum_{i=1}^{M}log(\mathcal{P}(y^{(j,p)}_{i}{\bf|}{x^{(j)}};\theta))$
(13)
$\displaystyle\mathcal{L}_{a}(\theta)=-\sum_{j=1}^{K}\sum_{i=1}^{N}log(\mathcal{P}(y^{(j,a)}_{i}{\bf|}{x^{(j)}};\theta))$
(14)

where $\mathcal{L}_{p}$ and $\mathcal{L}_{a}$ are the losses of generating
present and absent keyphrases, respectively. $N$ is the word sequence length
of absent keyphrases, and $\theta$ are the parameters in our model. The
training objective is to jointly minimize the two losses:

$\mathcal{L}=\mathcal{L}_{p}+\mathcal{L}_{a}.$ (15)

##  4 Experiment

###  4.1 Dataset

We use the dataset collected by Meng et al. (2017) from various online digital
libraries, which contains approximately 570K samples, each of which contains a
title and an abstract of a scientific publication as source text, and author-
assigned keywords as target keyphrases. We randomly select the example which
contains at least one present keyphrase to construct the training set. Then, a
validation set containing 500 samples will be selected from the remaining
examples. In order to evaluate our proposed model comprehensively, we test
models on four widely used public datasets from the scientific domain, namely
Inspec Hulth and Megyesi (2006), Krapivin Krapivin et al. (2009), SemEval-2010
Kim et al. (2010) and NUS Nguyen and Kan (2007), the statistic information of
which are summarized in Table 2.

Dataset |  #Abs |  #PKPs |  #AKPs
---|---|---|---
Test |  Inspec |  500 |  3,654 |  1,349
|  Krapivin |  400 |  1,299 |  1,040
|  NUS |  211 |  1,333 |  1,128
|  SemEval |  100 |  625 |  841
Validation |  500 |  1,158 |  1,418
Training |  453,757 |  1,082,285 |  1,073,404
Table 2: Statistics of the dataset. #Abs, #PKPs, #AKPs denote the number of abstracts, present keyphrases, and absent keyphrases, respectively. Method | Inspec | Krapivin | NUS | SemEval
---|---|---|---|---
F1@5 |  F1@10 |  F1@5 |  F1@10 |  F1@5 |  F1@10 |  F1@5 |  F1@10
TF-IDF | 22.1 | 31.3 | 12.9 | 16.0 | 13.6 | 18.4 | 12.8 | 19.4
TextRank | 22.3 | 28.1 | 18.9 | 16.2 | 19.5 | 19.6 | 17.6 | 18.7
KEA | 9.8 | 12.6 | 11.0 | 15.2 | 6.9 | 8.4 | 2.5 | 2.6
CopyRNN | 27.8 | 34.2 | 31.1 | 26.6 | 33.4 | 32.6 | 29.3 | 30.4
CopyTrans† | 21.1 | 16.2 | 26.4 | 20.5 | 35.1 | 28.2 | 29.5 | 26.3
CorrRNN | – | – | 31.8 | 27.8 | 35.8 | 33.0 | 32.0 | 32.0
CatSeq | 29.0 | 30.0 | 30.7 | 27.4 | 35.9 | 34.9 | 30.2 | 30.6
SGG | 30.6 | 35.9 | 28.8 | 25.3 | 36.3 | 35.8 | 33.8 | 33.6
Table 3: F1@5/10 results of predicting present keyphrases of different models on four datasets. The best and second best performance in each column are highlighted with bold and underline respectively. † indicates that the model is reimplemented. Method | Inspec | Krapivin | NUS | SemEval
---|---|---|---|---
CopyRNN | 10.0 | 20.2 | 11.6 | 6.7
CopyTrans† | 5.6 | 16.9 | 8.9 | 4.1
CorrRNN† | 8.5 | 15.2 | 8.0 | 3.5
CatSeq | 2.9 | 7.4 | 3.1 | 2.5
SGG | 11.0 | 23.5 | 12.4 | 4.9
Table 4: Recall@50 results of predicting absent keyphrases of different models
on four datasets. The CorrRNN is retrained following the implementation
details in Chen et al. (2018) as they did not report the Recall@50 results.

###  4.2 Baselines and Evaluation Metrics

For present keyphrase prediction, we compare our model with both extraction
and generation approaches. Extraction approaches include two unsupervised
extraction methods: TF-IDF, TextRank Mihalcea and Tarau (2004) and one classic
supervised extraction method KEA Witten et al. (1999). For the generation
baselines, some models, such as CopyRNN, split each data item into multiple
training examples, each of which only contains one keyphrase, while the other
models concatenate all keyphrases as target. To simplicity, the pattern of
training model only with one keyphrase is denoted as one-to-one and with the
concatenation of all keyphrases as one-to-many. The generation baselines are
the following state-of-the-art encoder-decoder models:

  * •

CopyRNN(one-to-one) Meng et al. (2017) represents a RNN-based encoder-decoder
model incorporating the copying mechanism.

  * •

CopyTrans(one-to-many) is a transformer-based Vaswani et al. (2017) encoder-
decoder model incorporating the copying mechanism.

  * •

CorrRNN(one-to-many) Chen et al. (2018) is an extension of CopyRNN
incorporating the coverage mechanism Tu et al. (2016).

  * •

CatSeq(one-to-many) Yuan et al. (2020) has the same model structure as
CopyRNN. The difference is CatSeq is trained by one-to-many.

The baseline CopyTrans has not been reported in existing papers and thus is
retrained. The implementation of Transformer is base on open source tool
OpenNMT 222 https://github.com/OpenNMT/OpenNMT-py. For our experiments of
absent keyphrase generation, only generation methods are chosen as baselines.
The copying mechanism used in all reimplemented generation models is based on
the version See et al. (2017), which is slightly different from the
implementations by version Meng et al. (2017); Gu et al. (2016). SGG indicates
the full version of our proposed model, which contains a selector, a guider,
and a generator. Note that SGG is also trained under one-to-many pattern.

Same as CopyRNN, we adopt top-$N$ macro-averaged F-measure (F1) and recall as
our evaluation metrics for the present and absent keyphrases respectively. The
choice of larger $N$ (i.e., 50 v.s. 5 and 10) for absent keyphrase is due to
the fact that absent keyphrases are more difficult to be generated than
present keyphrases. For present keyphrase evaluation, exact match is used for
determining whether the predictions are correct. For absent keyphrase
evaluation, Porter Stemmer is used to stem all the words in order to remove
words’ suffix before comparisons.

###  4.3 Implementation Details

We set maximal length of source sequence as 400, 25 for target sequence of
selector and generator, and 50 for the decoders of all generation baselines.
We choose the top 50,000 frequently-occurred words as our vocabulary. The
dimension of the word embedding is 128. The dimension of hidden state in
encoder, selector and generator is 512. The word embedding is randomly
initialized and learned during training. We initialize the parameters of
models with uniform distribution in [-0.2,0.2]. The model is optimized using
Adagrad Duchi et al. (2011) with learning rate = 0.15, initial accumulator =
0.1 and maximal gradient normalization = 2. In the inference process, we use
beam search to generate diverse keyphrases and the beam size is 200 same as
baselines. All the models are trained on a single Tesla P40.

###  4.4 Results and Analysis

In this section, we present the results of present and absent keyphrase
generation separately. The results of predicting present keyphrases are shown
in Table 3, in which the F1 at top-5 and top-10 predictions are given. We
first compare our proposed model with the conventional keyphrase extraction
methods. The results show that our model performs better than extraction
methods with a large margin, demonstrating the potential of the Seq2Seq-based
generation models in automatic keyphrase extraction task. We then compare our
model with the generation baselines, and the results indicate that our model
still outperforms these baselines significantly. The better performance of SGG
illustrates the pointing based selector is sufficient and more effective to
generate present keyphrase.

We further analyze the experimental results of absent keyphrase generation.
Table 4 presents the recall results of the generation baselines and our model
on four datasets. It can be observed that our model significantly improves the
performance of absent keyphrase generation, compared to the generation
baselines. This is because SGG is equipped with a generator that is not biased
to generate present keyphrases and the designed guider in SGG further guides
the generator to focus on generating absent keyphrases. Table 5 shows the
proportion of absent keyphrases generated by SGG. The comparison of Table 1
and 5 demonstrates that our model have the ability to generate large portions
of absent keyphrases rather than tending to generate present keyphrases.

Method |  Inspec |  Krapivin |  NUS |  SemEval
---|---|---|---|---
CopyRNN |  13.12 |  11.74 |  11.30 |  11.25
SGG |  79.16 |  79.28 |  76.02 |  79.20
Table 5: Proportion of absent keyphrases in the predictions of CopyRNN and
generator. The proportion of CopyRNN is same as Table 1.

In addition, an interesting phenomenon can be found from the results of
CopyRNN and CatSeq that one-to-one pattern generally performs better than one-
to-many if under the same model structure in absent keyphrase generation. To
explore this phenomenon, we use the same code, same training set to retrain
CopyRNN under one-to-one and one-to-many patterns, and the test results show
that one-to-one could boost the performance in absent keyphrase generation.
However, SGG cannot be trained under one-to-one pattern as the core of guider
in SGG is to memory all present keyphrases. Even so, SGG still has better
performance than CopyRNN. The results of SGG achieve 1.6% average gain than
CopyRNN and 31.8% average gain than the best-performing results of one-to-many
baselines over four test sets.

###  4.5 SGG for Title Generation

In this section, we explore the extensibility of SGG in other natural language
generation (NLG) tasks, i.e., title generation. We adopt the same dataset
described in Section 4.1 for title generation, which contains abstracts,
present keyphrases, absent keyphrases, and titles. Specifically, a title
generation model takes an abstract as input and generates a title as output.
To train SGG model for title generation, present keyphrases appearing in the
titles are used as labels to train the selectors333The present keyphrase
information used for training SGG is not used during inference. Datasets
without given present keyphrases should consider to conduct labeling., and the
titles are used to train the generators. The idea behind is to utilize the
present keyphrase generation as an auxiliary task to help the main title
generation task. In order to evaluate SGG on title generation, we choose
models CopyTrans and pointer-generator (PG-Net) See et al. (2017) as
baselines. We use ROUGE-1 (unigram), ROUGE-2 (bi-gram), ROUGE-L (LCS) and
human evaluation as evaluation metrics. For human evaluation, we randomly
selects 100 abstracts for each test set, then distribute them to four people
on average. The evaluation standard is the fluency of generated title and
whether it correctly provides the core topics of an abstract.

Inspec |  RG-1 |  RG-2 |  RG-L |  Human
---|---|---|---|---
CopyTrans |  83.58 |  43.81 |  45.25 |  74/100
PG-Net |  83.03 |  43.44 |  45.20 |  77/100
SGG |  84.25 |  44.98 |  46.87 |  83/100
Krapivin |  RG-1 |  RG-2 |  RG-L |  Human
CopyTrans |  84.23 |  50.01 |  50.63 |  89/100
PG-Net |  84.75 |  50.82 |  51.48 |  87/100
SGG |  84.96 |  51.35 |  52.34 |  90/100
NUS |  RG-1 |  RG-2 |  RG-L |  Human
CopyTrans |  86.76 |  54.90 |  52.49 |  82/100
PG-Net |  86.59 |  52.59 |  50.61 |  79/100
SGG |  87.01 |  54.90 |  52.57 |  89/100
SemEval |  RG-1 |  RG-2 |  RG-L |  Human
CopyTrans |  86.92 |  55.10 |  53.05 |  82/100
PG-Net |  86.68 |  50.16 |  51.31 |  78/100
SGG |  87.54 |  53.38 |  53.55 |  84/100
Table 6: Results of title generation of various models on four datasets. Dataset | Absent keyphrase generation | Title generation
---|---|---
Recall@50 | RG-1 | RG-2 | RG-L | BLEU-4
Inspec | 8.6(-2.4) | 83.51(-0.74) | 44.40(-0.58) | 45.80(-1.07) | 11.02(+0.41)
Krapivin | 23.2(-0.3) | 84.56(-0.40) | 50.56(-0.79) | 50.87(-0.48) | 11.46(-1.38)
Table 7: Results of SG on absent keyphrase generation and title generation
tasks. ($\pm$) indicates the comparison of the results of SG and SGG. The
results of SGG please refer to Table 4 and Table 6.

The results of title generation are shown in Table 6, from which we observe
that our proposed model SGG achieves better performance than the strong
baselines on all datasets, proving that SGG could be directly applied to title
generation task and still keep highly effective.

Figure 3: Proportions of test examples that the predictions of generator
overlap with the predictions of selector. Here only the top-1 predictions of
generator and selector are used.

###  4.6 Ablation Study on Guider

In this section, we further study the effectiveness of our proposed guider
module. Table 7 displays the results of SG (only a selector, a generator, no
guider) and its comparison with SGG on the two largest test sets Inspec and
Krapivin, which illustrates that the guider has a remarkable effect on absent
keyphrase and title generation tasks.

In more detail, we analyze that the function of guiders on these two tasks is
different, which depends on the correlation between the targets of selector
and generator. For example, in the task of keyphrase generation, the words
predicted from selector should not be repeatedly generated by generator
because the present keyphrases and absent keyphrases in a given text usually
do not have overlapping words. However, in the task of title generation, the
selected words by selector should be paid more attention on by generator since
they are usually part of the target titles. To verify the above analysis, we
visualize two examples of the attention scores in generators for the two tasks
in Figure 4. For keyphrase generation, SG repeatedly generates “implicit
surfaces” that has already been generated by its selector. In contrast, SGG
successfully avoids this situation and it correctly generates the absent
keyphrase “particle constraint”. For title generation, the guider helps SGG to
assign higher attention scores to the words in “seat reservation” that has
been generated by selector.

Figure 4: Visualization of attention score in generator for keyphrase
generation and title generation. The words marked in  red have already been
generated by the selector. The words marked in  blue are the generation of the
generator. In these two examples, phrase “particle constraint” is the correct
absent keyphrase for keyphrase generation and “seat reservation problem” is
part of the correct title for title generation.

Figure 3 gives the proportion of test examples that the predictions of
generator overlap with the predictions of selector. We observe that SG is more
likely to generate the words that have been generated by selector than SGG in
keyphrase generation. In contrast, the results on title generation indicate
that SGG is more likely to generate previously selected words than SG for this
task. Through the analysis above, we conjecture that the guider is able to
correctly guide the behaviour of generator in different tasks, i.e., learn to
encourage or discourage generating previously selected words.

##  5 Conclusion

In this paper, a Select-Guide-Generate (SGG) approach is proposed and
implemented with a hierarchical neural model for keyphrase generation, which
separately deals with the generation of present and absent keyphrases.
Comprehensive empirical studies demonstrate the effectiveness of SGG.
Furthermore, a title generation task indicates the extensibility of SGG in
other generation tasks.

##  6 Acknowledgments

This work is supported by the National Key Research and Development Program of
China under Grant No. 2018YFB2100802.

## References

  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  _Proceedings of ICLR_.
  * Chen et al. (2018) Jun Chen, Xiaoming Zhang, Yu Wu, Zhao Yan, and Zhoujun Li. 2018.  Keyphrase generation with correlation constraints.  In _Proceedings of EMNLP_.
  * Chen et al. (2020) Wang Chen, Hou Pong Chan, Piji Li, and Irwin King. 2020.  Exclusive hierarchical decoding for deep keyphrase generation.  _arXiv preprint arXiv:2004.08511_.
  * Chen et al. (2019) Wang Chen, Yifan Gao, Jiani Zhang, Irwin King, and Michael R Lyu. 2019.  Title-guided encoding for keyphrase generation.  In _Proceedings of AAAI_.
  * Duchi et al. (2011) John Duchi, Elad Hazan, and Yoram Singer. 2011.  Adaptive subgradient methods for online learning and stochastic optimization.  _Journal of Machine Learning Research_ , 12(Jul):2121–2159.
  * Gollapalli and Caragea (2014) Sujatha Das Gollapalli and Cornelia Caragea. 2014.  Extracting keyphrases from research papers using citation networks.  In _Proceedings of AAAI_.
  * Gollapalli et al. (2017) Sujatha Das Gollapalli, Xiao-Li Li, and Peng Yang. 2017.  Incorporating expert knowledge into keyphrase extraction.  In _Proceedings of AAAI_.
  * Gu et al. (2016) Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016.  Incorporating copying mechanism in sequence-to-sequence learning.  In _Proceedings of ACL_.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ , 9(8):1735–1780.
  * Hulth and Megyesi (2006) Anette Hulth and Beáta B Megyesi. 2006.  A study on automatically extracted keywords in text categorization.  In _Proceedings of ACL_.
  * Kim et al. (2010) Su Nam Kim, Olena Medelyan, Min-Yen Kan, and Timothy Baldwin. 2010.  Semeval-2010 task 5 : Automatic keyphrase extraction from scientific articles.  In _Proceedings of the 5th International Workshop on Semantic Evaluation_.
  * Krapivin et al. (2009) Mikalai Krapivin, Aliaksandr Autaeu, and Maurizio Marchese. 2009.  Large dataset for keyphrases extraction.  Technical report, University of Trento.
  * Liu et al. (2009) Zhiyuan Liu, Peng Li, Yabin Zheng, and Maosong Sun. 2009.  Clustering to find exemplar terms for keyphrase extraction.  In _Proceedings of EMNLP_.
  * Medelyan et al. (2009) Olena Medelyan, Eibe Frank, and Ian H Witten. 2009.  Human-competitive tagging using automatic keyphrase extraction.  In _Proceedings of EMNLP_.
  * Meng et al. (2017) Rui Meng, Sanqiang Zhao, Shuguang Han, Daqing He, Peter Brusilovsky, and Yu Chi. 2017.  Deep keyphrase generation.  In _Proceedings of ACL_.
  * Mihalcea and Tarau (2004) Rada Mihalcea and Paul Tarau. 2004.  Textrank: Bringing order into text.  In _Proceedings of EMNLP_.
  * Nguyen and Kan (2007) Thuy Dung Nguyen and Min-Yen Kan. 2007.  Keyphrase extraction in scientific publications.  In _Proceedings of International Conference on Asian Digital Libraries_.
  * See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning. 2017.  Get to the point: Summarization with pointer-generator networks.  In _Proceedings of ACL_.
  * Subramanian et al. (2018) Sandeep Subramanian, Tong Wang, Xingdi Yuan, Saizheng Zhang, Adam Trischler, and Yoshua Bengio. 2018.  Neural models for key phrase extraction and question generation.  In _Proceedings of ACL_.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014.  Sequence to sequence learning with neural networks.  In _Proceedings of NIPS_.
  * Tu et al. (2016) Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016.  Modeling coverage for neural machine translation.  In _Proceedings of ACL_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Proceedings of NIPS_.
  * Vinyals et al. (2015) Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015.  Pointer networks.  In _Proceedings of NIPS_.
  * Wan and Xiao (2008) Xiaojun Wan and Jianguo Xiao. 2008.  Single document keyphrase extraction using neighborhood knowledge.  In _Proceedings of AAAI_.
  * Wang et al. (2014) Fang Wang, Zhongyuan Wang, Senzhang Wang, and Zhoujun Li. 2014.  Exploiting description knowledge for keyphrase extraction.  In _Proceedings of PRICAI_.
  * Wang and Cardie (2013) Lu Wang and Claire Cardie. 2013.  Domain-independent abstract generation for focused meeting summarization.  In _Proceedings of ACL_.
  * Wang et al. (2019) Yue Wang, Jing Li, Hou Pong Chan, Irwin King, Michael R. Lyu, and Shuming Shi. 2019\.  Topic-aware neural keyphrase generation for social media language.  In _Proceedings of ACL_.
  * Witten et al. (1999) Ian H. Witten, Gordon W. Paynter, Eibe Frank, Carl Gutwin, and Craig G. Nevillmanning. 1999.  Kea: Practical automatic keyphrase extraction.  In _Proceedings of ACM Conference on Digital Libraries_.
  * Ye and Wang (2018) Hai Ye and Lu Wang. 2018.  Semi-supervised learning for neural keyphrase generation.  In _Proceedings of EMNLP_.
  * Yuan et al. (2020) Xingdi Yuan, Tong Wang, Rui Meng, Khushboo Thaker, Peter Brusilovsky, Daqing He, and Adam Trischler. 2020.  One size does not fit all: Generating and evaluating variable number of keyphrases.  In _Proceedings of ACL_.
  * Zhang et al. (2017) Yuxiang Zhang, Yaocheng Chang, Xiaoqing Liu, Sujatha Das Gollapalli, Xiaoli Li, and Chunjing Xiao. 2017.  Mike: keyphrase extraction by integrating multidimensional information.  In _Proceedings of CIKM_.
  * Zhao and Zhang (2019) Jing Zhao and Yuxiang Zhang. 2019.  Incorporating linguistic constraints into keyphrase generation.  In _Proceedings of ACL_.


# BERTops: Studying BERT Representations under a Topological Lens

Jatin Chauhan, Manohar Kaul  Dept. of Computer Science & Engineering
Indian Institute of Technology Hyderabad
Email: {chauhanjatin100<EMAIL_ADDRESS>
###### Abstract

Proposing scoring functions to effectively understand, analyze and learn
various properties of high dimensional hidden representations of large-scale
transformer models like BERT can be a challenging task. In this work, we
explore a new direction by studying the topological features of BERT hidden
representations using persistent homology (PH). We propose a novel scoring
function named “persistence scoring function (PSF)” which: (i) accurately
captures the homology of the high-dimensional hidden representations and
correlates well with the test set accuracy of a wide range of datasets and
outperforms existing scoring metrics, (ii) captures interesting post fine-
tuning “per-class“ level properties from both qualitative and quantitative
viewpoints, (iii) is more stable to perturbations as compared to the baseline
functions, which makes it a very robust proxy, and (iv) finally, also serves
as a predictor of the attack success rates for a wide category of black-box
and white-box adversarial attack methods. Our extensive correlation
experiments demonstrate the practical utility of PSF on various NLP tasks
relevant to BERT 111Code is available here. .

###### Index Terms:

Machine Learning, Neural Networks, Persistent Homology, BERT

##  I Introduction

Extensive research is being conducted to comprehend the functionality of the
_transformer_ [1] model and its variants [2] by studying them from various
perspectives such as: _language modeling_[3], _generalization_ [4],
_robustness_ and _domain adaptation_ standpoints. Many recent works have
focused on proposing a wide range of scoring functions and metrics based on
the attention mechanism of these models [5, 6] as well as their contextualized
embeddings [7], which can be used to assess the aforementioned properties [8]
and discover newer lines of work such as predicting their performance without
explicitly training and/or testing [9].

Scoring functions which can estimate and/or predict properties based on the
hidden state representations of these models have diverse use cases. One such
use case is the estimation of testing accuracies of the trained NLP models,
which is helpful in the scenarios where held out sets are not available at
all, some examples being online leaderboards [10, 11]. One can argue that
these held out test sets can be created by sampling from the training data
itself or using techniques like cross-validation. However, this reduction in
the size of the training set usually hurts the performance of deep learning
models [12, 13]. Another very interesting and highly practical use case is the
estimation of the adversarial vulnerability of these fine-tuned BERT models,
where we use the attack success rate of any given attacker (black or white-
box) as the proxy for this experiment, without actually performing the attack.
This benefits us greatly from a computational perspective, since performing an
attack by developing a held-out sample set can be extremely resource-intensive
as well as time-consuming. For example, we empirically observed that some
black-box attacks can take nearly $15$ hours to generate adversarial examples
for merely $1000$ sentences on a single dataset. We note that these
estimations, however, also need to be reliable as well as invariant to slight
input transformations, which can otherwise result in spurious and non-
generalizable conclusions. One of the best ways to show it is by studying the
stability of these scoring functions to perturbations in the space over which
they are computed.

In this work, we provide a novel scoring function based on the hidden state
representations of the transformer variant BERT, using persistent homology
(PH)[14, 15]. We show that our scoring function, named persistence scoring
function (PSF), captures the $0$-th and $1$-st dimensional homology features
which are essentially the connected components and holes (or tunnels) in the
hidden representation space, and thus more accurately defines the spread of
data points via their persistence values, which is otherwise difficult for
functions based solely on Euclidean distance or dimensionality reduction
techniques [16].

Through our empirical investigations, we first show that the scores generated
by our proposed PSF correlate well to the test set accuracy of fine-tuned BERT
models on a diverse category of datasets. We also verify PSF’s superior
scoring performance over strong existing baseline methods. More interestingly,
we observed that PSF scores also shows high correlations to the f1-score per
class label on the datasets and that the qualitative observations from this
experiment can help in understanding some properties of the fine-tuned models
which are retained from the pre-training phase. We then show that PSF exhibits
stability against noisy perturbations in the input space and thus serves as a
robust scoring function. Finally, based on the aforementioned preliminary
experiments, we show that PSF can also serve as an estimator of vulnerability
against both black-box and white-box adversarial attacks and significantly
outperforms the state-of-the-art vulnerability scoring baselines.

We also note the parallel between PSF and the problem of predicting the
generalization gap (GP) [17], which is the difference between a model’s
performance on the training data and its performance on the unseen data drawn
from the same distribution, for which various bounds based on _complexity
measures_ such as the VC dimension and Rademacher complexity have been
proposed to “explain and interpret” how sensitive a neural network’s output is
to changes in the input space (e.g., added noise, distribution shifts etc).
The rationale behind PSF follows similar lines, but more importantly, it is
_invariant to slight input transformations_ and also simultaneously serves as
a strong indicator of a network’s likelihood of being successfully attacked,
for which no prior work in NLP exists. It is worth noting that in the absence
of such complexity measures and estimators, one has to resort to generating
numerous _augmented training sets_ along with their test sets for all possible
input transformations (including adversarial examples), which can be
computationally infeasible.

Our formulation of PSF also has a very direct connection to the compactness of
Euclidean embeddings of the data points (the work of [18] provides relevant
bounds for the same). Intuitively speaking, lower the PSF score for a given
cluster of points in Euclidean space, more compact the cluster is, which
ultimately accounts for well separated classes and improved performance
(section V-B). Subsequently, higher PSF correlates to higher adversarial
vulnerability, since the intra-cluster spread is larger, making the model
prone to attacks, as discussed in more detail in section V-C.

The summary of our contributions is as follows:

  1. 1.

To the best of our knowledge, we are the first to propose a scoring function
(i.e., PSF) using topological features of BERT’s hidden representation space.

  2. 2.

Extensive experiments demonstrate that PSF can be used as an estimator of
important properties of fine-tuned BERT models and is stable against
perturbations.

  3. 3.

Our method requires only a few lines of post-processing code and can be
utilized with any representation learning method, in and even outside the
realm of NLP models.

##  II Related Work

Using scoring functions as estimators and/or predictors of some property of
deep learning models and datasets has gained widespread interest in recent
years. [19] used optimal transport distance between datasets to predict the
performance of BERT on _domain adaptation_ from one dataset to another, [20]
worked on word embedding matrix compression algorithms and introduced an eigen
overlap score to identify the best performing compression method, without
actually training downstream models on them. [9] tried to predict the
evaluation score of an NLP experiment for machine translation, while the work
of [8] tries to predict the performance drop of modern NLP models under
domain-shift.

Complementary to other mathematical sub-fields, _persistent homology_ (PH) has
also been used for such works. [21] proposed a complexity measure, named
_neural persistence_ , based on PH to characterize the structural complexity
of MLPs, whereas[22, 23] proposed PH-based algorithms to predict the ability
of models to generalize and detect adversarial samples. Both these PH-based
methods employ methods that are fundamentally different than ours, namely (i)
their filtrations are constructed on binary graphs as opposed to our
filtration construction which are built on BERT hidden representations, and
(ii) their methods mainly focus on computer vision related tasks.

PH has also been previously used in NLP by [24, 25, 26] for document analysis,
standard classification tasks and studying BERT’s attention matrix, however,
none of these prior works have studied the topological features of the space
of BERT hidden representations via PH.

(a) $X=X_{0}$

(b) $X_{0.2}$

(c) $X_{0.35}$

(d) Persistence diagram

Figure 1:  (a) Original data points $X$ with $r=0$. (b,c) “Fattened points” by
placing balls of radius $r>0$ centered on each point in $X$ and generating
spaces $X_{0.2}$ and $X_{0.35}$. As we increase $r$, one of the topological
features (tunnels) born at $r=0.2$, dies at $r=0.35$. (d) The persistence
diagram summarizes the lifetimes of the topological features of $X$. The green
dots represent the connected components and the red dots represent the tunnels
formed.

##  III Background

Persistent Homology (PH): In this section, we briefly review _persistent
homology_ (PH). For a more complete coverage of algebraic topology and PH, we
refer the reader to[15].

Let $X=\\{x_{1},\cdots,x_{n}\\}$ be a compact subset in a metric space
$(M,d)$, where $d_{M}:M\rightarrow\mathbb{R}$ is the underlying distance
function. Consider a ball $B_{r}(x)=\\{y\in M\mid d_{M}(x,y)\leq r\\}$ of
radius $r$ centered on $x\in X$. Imagine placing such a ball of radius $r$ on
every point in $X$ to get a model $X_{r}=\cup_{i=1}^{n}B_{r}(x_{i})$; note
that $X_{0}=X$.

As we vary the radius $r$ from $0$ to $\infty$, (as shown in Figures 1(a)–1(c)
), we observe changes in $X_{r}$. Persistent homology summarizes the change in
topology of $X_{r}$, as $r$ is increased. Here, $X_{r}$ is considered as a
topological space and its $j$-th homology group $H_{j}(X_{r})$
($j=0,1,\cdots)$, is a vector space, and its dimension $\dim H_{j}(X_{r})$
depicts the _number of connected components_ ($j=0$), _tunnels/holes_ ($j=1$),
_voids_ ($j=2$), and so on. For example, $X_{0.2}$ in Figure 1(b) comprises of
one connected component (i.e., $\dim H_{0}(X_{0.2})=1$) and two tunnels (i.e.,
$\dim H_{1}(X_{0.2})=2$).

Notice that these topological features can _be born_ (appear) and _die_
(disappear) as the radius $r$ is increased. For example, one of the tunnels in
$X_{0.2}$ dies in $X_{0.35}$. By gathering the _birth-death_ pairs
$P_{l}(X)=\\{\mskip-5.0mu\\{(b_{i},d_{i})\in\mathbb{R}^{2}\mid i\in
I\\}\mskip-5.0mu\\}$ , we obtain a multi-set. The collection $P_{l}(X)$ refers
to the $l$-th persistence diagram, where $l$ is the dimension of the homology
group. Figure 1(d) shows the birth-death pairs corresponding to the number of
connected components (in green) and the tunnels (in red) in the same
persistence diagram. Points close to the diagonal are topological features
with short lifespans and are considered _topological noise_. We are interested
in features that _persist_ over larger intervals of $r$.

##  IV Persistence Scoring Function (PSF)

Here, we propose a novel scoring function called _persistent scoring function
(PSF)_. Let $X$ denote the finite and compact subset of BERT hidden state
vectors. As explained in Section III, we can study the topological features,
expressed at various spatial scales, of the BERT hidden representation space
(i.e., $X$). $X$ is studied as a collection of $l$ persistence diagrams and
denoted by $P_{l}(X)$.

Inspired by[27], we propose our PSF. First, we concatenate the collection of
persistence diagrams, denoted by $P(X)$, as

$P(X)=\bigoplus_{l=0}^{l_{m}}P_{l}(X)$

where $\bigoplus$ denotes concatenation of persistence diagrams represented as
multi-sets of birth-death pairs into a single multi-set $P(X)$ and $l_{m}$ is
the maximum dimension of the computed homology group. This concatenation
operation is different from the _union of multi-sets_ and preserves the total
multiplicity of birth-death pairs across all persistence diagrams.

Our PSF based on the concatenated persistence diagrams $P(X)$, denoted by
$\mathcal{L}(p,q;P(X))$, is then computed as

$\displaystyle\frac{1}{|P(X)|}\sum_{i=1}^{|P(X)|}\left(\frac{|d_{i}-b_{i}|}{\hat{d}}\right)^{p}\left(\frac{|d_{i}+b_{i}|}{2\hat{d}}\right)^{q}$
(1)

where $|P(X)|$ is the cardinality of the concatenated multiset and $\hat{d}$
is computed as: $max\\{d_{i}|(b_{i},d_{i})\in P(X)\\}$. In (1), the term
$|d_{i}-b_{i}|$ (difference between the death and birth of a topological
feature) denotes the _lifetime_ or _persistence_ of the topological feature.
Parameter $p$ is varied to ignore or emphasize the persistent features,
whereas the parameter $q$ is used to weight topological features that are born
later at higher values of radius $r$.
Multi-Valued PSF: Following a similar reasoning about multi-headed subnetworks
as provided in the works of [1, 28], we too define the PSF via multiple pairs
of values for $p$ and $q$, which are finally composed together. For this work,
we have averaged over the PSF computed for different pairs of $p$ and $q$ in
(1) as the composition function. This has twofold benefits: (i) multiple pairs
for the tuple $(p,q)$ can provide different levels of penalty and emphasis on
the topological features and (ii) we can bypass the manual effort of tuning
PSF for a specific pair of $p$ and $q$. Henceforth, we refer to _multi-valued
PSF_ as PSF for the remainder of this work.

Figure 2: Sample TSNE and PD visualizations for SST-2 dataset

(a) Accuracy vs PSF Value

(b) Accuracy vs Silhouette Coefficient

(c) Accuracy vs Davies Bouldin Index

Figure 3: Accuracy scores of the datasets against the estimator functions:
PSF, Silhouette Coefficient and Davies Bouldin Index.

##  V Empirical Study

###  V-A Setup

We validate the utility of PSF empirically using the BERT [2] model consisting
of 12 layers and 12 attention heads per layer from [29]. We consider ten
widely used datasets, spanning small and large datasets alike, to show that
PSF is agnostic to dataset size. Six of these are standard multi-class
classification datasets: AG News(AG), DBPedia 14(DB), Sogou News(SG), Yelp
Review Full(YR), Yahoo Answers Topics(YA) and TREC(TC), while the rest are
sentiment polarity classification datasets: SST-2(SS), IMDB(IM), Yelp
Polarity(YP) and Rotten Tomatoes(RT). The datasets are taken from the
_huggingface_ 222https://huggingface.co/docs/datasets/datasets-library. The
experiments were performed on a single NVIDIA Titan-X GPU with $12$ GB CUDA
memory.

For the PSF score outlined in (1), we select multiple pairs of values for $p$
and $q$ from a very small set as:
$\\{p,q\\}\in\mathcal{S}=\\{2,3\\}\times\\{2,3\\}$, rather than performing a
naive grid search over a large set. We exclude values less than $2$ from the
set because they provide weaker penalization and pairs with small persistence
values can be considered as noise and ignored, while values greater than $3$
provide very strong penalization and diminish many persistence pairs.

The construction of persistence diagrams is done using the well-known
Vietoris-Rips filtration from _Ripser_ [30] library. To obtain the persistence
diagrams $P_{0}(X)$ and $P_{1}(X)$, the training samples from each class of a
given dataset are passed through the fine-tuned BERT and the corresponding
vector representations of [CLS] token from the final layer are passed through
the filtration mechanism. We compute the persistence diagrams upto homology
group 1, ie, the holes/tunnels in the compact space.

Downsampling the Points: Since the number of samples per class can be large
depending upon the dataset used, we first use Kernel Density Estimation (KDE)
[31] over the vector representations of the [CLS] tokens and then sample a
desired number of points from KDE, which are finally passed through the
filtration mechanism to compute the PDs. All the experiments have been
performed over the points (of each class) sampled from KDE. Finally, the PSF
score for a given dataset is computed by averaging the class-wise PSF scores.

In Figure 2, we juxtapose a t-SNE plot of BERT hidden states and the
corresponding persistence diagram (only dim-1 topological features, i.e., the
”holes” in the space) for the SST-2 dataset. Observe that for class label $0$
(blue points), there is a wider spread of birth-death pairs, where later birth
indicates the formation of very large holes due to points covering a wider
expanse. In comparison, points for class label $1$ (orange points) are more
densely packed and have no points in the other cluster, therefore they cover a
shorter range of birth in the persistence diagram.

###  V-B Results

####  V-B1 Estimating Test Accuracy via PSF

In this section, we report the results of PSF as an estimator of the test
performance of fine-tuned BERT models. This is done by calculating the
_Spearman rank correlation_ $\rho$ of PSF to the test set accuracy of the
corresponding datasets. It is noteworthy that in this experiment, we desire a
negative correlation because of the fact that the model performances are
highly associated with the cluster quality of the vector representations in
the latent space, i.e., more compact clusters result in lower PSF values (due
to lower persistence in topology features) but result in higher accuracy due
to improved class separation. Improved model performance with more compact
clusters of data points has been previously shown by exhaustive experiments of
[32].

As baselines, we compare to the popular Silhouette coefficient (SC) [33],
where we subtract the value of SC on each dataset from 1 to ensure consistency
in experiments, and the Davies Bouldin index (DBI) [34]. Both these methods
have been widely used in the literature to assess the quality of the learned
representations of the data by machine learning models. For reference, we
point the reader to the works by[35, 36]. Figure 3 shows the dataset
accuracies versus the PSF values of the different estimator functions. We can
observe that PSF, with a stronger negative correlation of $-0.515$,
outperforms both SC and DBI with a correlation of $-0.442$, thus affirming our
claims.

We mentioned the practical utility of such estimators in section I. Apart from
comparing the performance across datasets, we expect that PSF can also be
quite useful in the scenarios where we want to analyze and compare the
performance of variants of a single model trained on different random seeds to
select the best performing ones, which we will explore in depth in our future
works.

Figure 4: Classwise F1-scores vs the PSF values.

####  V-B2 Fine-Grained Analysis

We further conduct experiments to show that PSF can also be used for a more
fine-grained analysis at a class-wise level and capture interesting properties
about the same class labels across multiple datasets. This is done by
calculating the Spearman correlation ($\rho$) of the f1-scores of the class
labels to the corresponding class-wise PSF values. Similar to the previous
experiment, we expect a strong negative correlation. Note that here we
concatenate the PSF of the class labels across all the datasets in a single
array (as opposed to averaging over an entire dataset). As shown in figure 4,
PSF achieves a strong negative correlation of $-0.685$.

From the qualitative viewpoint, we observed that the PSF value of the
“Business and Finance” class was approximately the highest and its f1-score
was approximately the lowest compared to other class labels across the
datasets, while the PSF of “Sports“ class was comparatively lower and its
f1-score was higher than various other labels. We also observed that variance
in the PSF values of the negative polarity class(label $0$) in sentiment
datasets was high, whereas the positive polarity class (label $1$) had
comparatively marginal variance, implying a distinct spread of the polarity
classes across the datasets. We believe that such findings can help shed some
light on the properties that the fine-tuned BERT models retain from the self-
supervised pre-training phase pertaining to both data and tasks [37].

Figure 5: Stability analysis of PSF against SC and DBI.

####  V-B3 Stability of PSF

Robustness against minor changes/perturbations in the inputs used to compute
such estimators is one of the most important and desirable properties. Recent
works such as [38] in NLP and [17] in vision have focused on providing stable
estimator and predictor metrics. Here, we show that PSF is robust to
perturbations that are generated by adding random Gaussian noise to the word
embedding vectors of a subset of the input samples. Following the techniques
outlined in[39], we add noise
$\epsilon_{h}^{i}\sim\mathcal{N}(0,\,\sigma_{h}^{2})$ to each dimension $h$ of
the $768$ dimensional embedding vectors of the words in input sample $i$
chosen for perturbation (here $\sigma_{h}^{2}$ is the variance of the
corresponding dimension (column) of BERT’s embedding matrix). We then
recompute the PSF scores (KDE step is also repeated) for the test accuracy
estimation experiment in section V-B1. It is evident from the results shown in
figure 5 that the reduction in the correlation of PSF (reduction of $0.014$)
is significantly lower than the reduction of $0.042$ and $0.052$ for baselines
SC and DBI respectively, thus verifying the stability of PSF.

###  V-C PSF as an Estimator of Adversarial Vulnerability of trained Models

Considering the experiments of the previous section as our basis, we present
the core analysis results of our work in this section.

It is well-known that deep natural language models are highly susceptible to
adversarial attacks, as shown by various attack methods, elaborately described
in [40]. Usually, the adversarial vulnerability is quantitatively evaluated by
attacking the NLP model via multiple text samples and calculating the success
rate of the attacker, which, however, has various computational issues as
described in section I. Thus, the question - “Can we estimate the success rate
without actually performing the attack?” becomes extremely pertinent and
interesting. We explicitly point out that here we are trying to estimate the
value of the success rate as an expectation over a large number of samples,
thus providing a _global perspective_ of the attacker, rather than evaluating
it per sample as done by [23], which requires an actual input sample for
evaluation, a bottleneck we try to overcome here. The following experiments
show that PSF can serve as a good estimator of the success rates of both the
black-box and white-box categories of attackers and substantially outperform
the baselines as well.

####  V-C1 Setup

The experimental setup for computing PSF as well as the datasets are retained
from section V-A. Here, we first describe the baselines used for comparison,
then briefly mention the adversarial attack methods and lastly the setup for
performing the attack.

Baseline 1: We call the first baseline as Adjusted Local Intrinsic
Dimensionality (ALID), a modified version of Local Intrinsic Dimensionality
(LID) proposed by [41], originally used for the characterization of
adversarial regions of neural networks. For our work, we use it for the
characterization of the compactness of representations of the samples for each
class label. For a given vector $\mathbf{x^{c}_{j}}$ in class $c$, we first
calculate its ALID as follows:

$ALID(\mathbf{x^{c}_{j}})=r_{k}(\mathbf{x^{c}_{j}})\left(-\frac{1}{k}\sum_{i=1}^{k}log\frac{r_{i}(\mathbf{x^{c}_{j}})}{r_{k}(\mathbf{x^{c}_{j}})}\right)$
(2)

where $r_{i}(\mathbf{x^{c}_{j}})$ is the distance between $\mathbf{x^{c}_{j}}$
and its $i^{th}$ nearest neighbor within the samples of points of class $c$
and $r_{k}(\mathbf{x^{c}_{j}})$ is the maximum distance. Here, $k$ is the
nearest neighbor hyperparameter (tuned manually). The ALID score for class $c$
is then obtained as

$ALID^{c}(.)=\frac{\frac{1}{N_{c}}\sum_{j=1}^{N_{c}}ALID(\mathbf{x^{c}_{j}})}{\max\\{ALID(\mathbf{x^{c}_{j}})\mid
j\in\\{1,...,N_{c}\\}\\}}$ (3)

where $N_{c}$ is the number of samples in class $c$. Finally, the ALID score
for a given dataset is the average of class-wise ALID scores.

Baseline 2: We refer to the second baseline as the Adjusted Mahalanobis Score
(AMS), which is a modification of the Mahalanobis distance-based confidence
score proposed by [42], originally used for detection of out-of-distribution
and adversarial samples (generated by attack methods), both done
simultaneously. Here, we use it as a Gaussian density-based scoring mechanism
to measure the compactness of representations of the samples for each class
label. For a given class $c$, we first calculate the empirical mean
($\hat{\mu}$) and covariance ($\hat{\mathbf{V}}$) as follows

$\hat{\mu}=\frac{1}{N_{c}}\sum_{i=1}^{N_{c}}\mathbf{x^{c}_{i}},\\\
\hat{\mathbf{V}}=\frac{1}{N_{c}}\sum_{i=1}^{N_{c}}(\mathbf{x^{c}_{i}}-\hat{\mu})(\mathbf{x^{c}_{i}}-\hat{\mu})^{T}$
(4)

where $N_{c}$ is the number of samples in class $c$. The AMS score for vector
$\mathbf{x_{j}}$ in class $c$ is then calculated as

$AMS(\mathbf{x^{c}_{j}})=(\mathbf{x^{c}_{j}}-\hat{\mu})^{T}\hat{\mathbf{V}}^{-1}(\mathbf{x^{c}_{j}}-\hat{\mu})$
(5)

Finally, the AMS score for class $c$ is obtained as:

$AMS^{c}(.)=\frac{\frac{1}{N_{c}}\sum_{j=1}^{N_{c}}AMS(\mathbf{x^{c}_{j}})}{\max\\{AMS(\mathbf{x^{c}_{j}})\mid
j\in\\{1,...,N_{c}\\}\\}}$ (6)

The AMS score for a given dataset is the average of class-wise AMS scores.

It is important to note that there do not exist any baselines for direct
comparison in this experiment, thus we select the above two state-of-the-art
methods from the literature on adversarial attacks on models.

Attack Methods: We consider six state-of-the-art attack methods, including
both black and white box attacks. These are: Probability Weighted Word
Saliency (PWWS) [43], Genetic Algorithm (GA) [44], TextFooler (TF) [45],
Textbugger Black-box (TB-B) as well as Textbugger White-box (TB-W) [46] and
lastly, Universal Adversarial Triggers (UAT) [47].

Following [45], we randomly select 1000 samples from the test set of each
dataset and perform adversarial attack via each of the six methods. Note that
in this experiment, we desire a positive correlation, since higher function
values for PSF and both the baselines denote a larger spread in the hidden
representation space and a higher probability of having many points near the
non-linear decision boundaries, making the attacks easier [48].

(a) PWWS Attacker

(b) Textbugger Black-box Attacker

(c) TextFooler Attacker

(d) Genetic Attacker

(e) UAT Attacker

(f) Textbugger White-box Attacker

Figure 6: Each plots shows the corresponding attack method’s Success Rates (in %) vs the PSF values on the $10$ datasets. Full forms of dataset abbreviations are same as in section V-A .  TABLE I: Spearman correlations of PSF, ALID and AMS (row-wise) against the attack methods (column-wise). Best results in bold, second best underlined. % Improvements of PSF against each baseline are also reported. | PWWS | TB-B | TF | GA | UAT | TB-W
---|---|---|---|---|---|---
PSF (Ours) | 0.4545 | $\bm{0.5151}$ | $\bm{0.5515}$ | $\bm{0.5272}$ | 0.3212 | $\bm{0.7212}$
ALID | 0.3696 | 0.1515 | 0.0420 | 0.0780 | $\bm{0.5393}$ | -0.0900
% Improvement of PSF over ALID | _22.97%_ | _240%_ | _1213%_ | _575%_ | _-40%_ | _901%_
AMS | $\bm{0.5030}$ | 0.4060 | 0.4787 | 0.2727 | 0.4181 | 0.4545
% Improvement of PSF over AMS | _-0.09%_ | _26.8%_ | _15.2%_ | _93.3%_ | _-23.4%_ | _58.67%_

####  V-C2 Discussion

Figure 6 shows the correlation plots of PSF for each of the attack methods.
From the results demonstrating the comparison of PSF to the baselines shown in
Table I, it is evident that PSF outperforms both ALID and AMS by large margins
for most of the attack methods. The % improvements of PSF are extremely large
as compared to ALID, wherein the improvements are more than $200\%$ on four
attack methods, including both black-box and white-box. Against AMS, the
improvements are more than $25\%$ on three attack methods, while the
reductions on PWWS are negligible. UAT attacker seems to be an exception,
where ALID outperforms both PSF and AMS.

We can conclude from these results that PSF can serve as a good estimator of
the attack success rates and is able to generalize well across attack methods
from different categories measured on diverse datasets. Apart from estimating
the vulnerabilities at the dataset level, we expect that such estimators can
help us identify, from a set of multiple models trained on the same dataset,
the least vulnerable ones, along similar lines as mentioned in Section V-B1.

Note that it is non-trivial to provide theoretical claims and layout the
conditions under which a given method will show a good correlation against an
attacker due to the discrete nature of the adversarial attack task in NLP and
variation in attack strategies, for eg: UAT’s principal attack mechanism is
different from standard white-box attacks. The same holds true for the genetic
algorithm as compared to the greedy blank-out mechanism of Textbugger, and
will be our focus in future works.

##  VI Conclusion and Future Work

In this work, we proposed a novel scoring function named persistent scoring
function (PSF) using persistent homology to study the topological features
present in BERT’s hidden representation. Through exhaustive empirical studies,
we showed that PSF can capture the homology of hidden state representations of
BERT quite accurately and can be utilized as an estimator of the test set
accuracies over a diverse range of classification datasets. PSF also provides
some qualitative properties on a more fine-grained “per class” level and more
importantly, it is more stable w.r.t to perturbations in the input space, thus
realizing its practical utility. Lastly, we showed that PSF can also be used
as an estimator of adversarial success rates generalizing across different
categories of attack methods. PSF also has the advantage that it requires a
few lines of post-processing code and can be utilized with any representation
learning model.
We hope that our work will pave a new and interesting direction for the
community to realize the power of topological feature analysis to: (i) explain
and interpret large-scale transformers variants such as BERT, RoBERTa, ALBERT,
T5 etc and (ii) propose robust estimators and predictors for more diverse NLP
tasks ranging from machine translation, dialogue evaluation to summarization
in the future.

## References

  * [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” _arXiv preprint arXiv:1706.03762_ , 2017.
  * [2] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics_.
  * [3] K. Clark, U. Khandelwal, O. Levy, and C. D. Manning, “What does BERT look at? an analysis of BERT’s attention,” in _Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_.
  * [4] Y. Hao, L. Dong, F. Wei, and K. Xu, “Visualizing and understanding the effectiveness of bert,” _arXiv preprint arXiv:1908.05620_ , 2019.
  * [5] S. Abnar and W. Zuidema, “Quantifying attention flow in transformers,” in _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , Online, Jul. 2020.
  * [6] E. Reif, A. Yuan, M. Wattenberg, F. B. Viegas, A. Coenen, A. Pearce, and B. Kim, “Visualizing and measuring the geometry of bert,” in _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [7] F. Schmidt and T. Hofmann, “Bert as a teacher: Contextual embeddings for sequence-level reward,” 2020.
  * [8] H. Elsahar and M. Gallé, “To annotate or not? predicting performance drop under domain shift,” in _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , Nov.
  * [9] M. Xia, A. Anastasopoulos, R. Xu, Y. Yang, and G. Neubig, “Predicting performance for natural language processing tasks,” in _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_.
  * [10] A. Wang, Y. Pruksachatkun, N. Nangia, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Superglue: A stickier benchmark for general-purpose language understanding systems,” 2020.
  * [11] P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “Squad: 100,000+ questions for machine comprehension of text,” 2016.
  * [12] C. Sun, A. Shrivastava, S. Singh, and A. Gupta, “Revisiting unreasonable effectiveness of data in deep learning era,” in _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_ , Oct 2017.
  * [13] A. Halevy, P. Norvig, and F. Pereira, “The unreasonable effectiveness of data,” _IEEE Intelligent Systems_ , vol. 24, no. 2, pp. 8–12, 2009.
  * [14] H. Edelsbrunner, D. Letscher, and A. Zomorodian, “Topological persistence and simplification,” in _Proceedings 41st annual symposium on foundations of computer science_. IEEE, 2000, pp. 454–463.
  * [15] H. Edelsbrunner and J. Harer, _Computational topology: an introduction_. American Mathematical Soc., 2010.
  * [16] A. Gracia, S. González, V. Robles, and E. Menasalvas, “A methodology to compare dimensionality reduction algorithms in terms of loss of quality,” _Information Sciences_ , vol. 270, pp. 1–27, 2014.
  * [17] Y. Jiang, D. Krishnan, H. Mobahi, and S. Bengio, “Predicting the generalization gap in deep networks with margin distributions,” 2019.
  * [18] F. Chazal and S. Oudot, “Towards persistence-based reconstruction in euclidean spaces,” _CoRR_ , vol. abs/0712.2638, 2007.
  * [19] D. Alvarez-Melis and N. Fusi, “Geometric dataset distances via optimal transport,” in _Advances in Neural Information Processing Systems_ , vol. 33, 2020.
  * [20] A. May, J. Zhang, T. Dao, and C. Ré, “On the downstream performance of compressed word embeddings,” in _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [21] B. Rieck, M. Togninalli, C. Bock, M. Moor, M. Horn, T. Gumbsch, and K. Borgwardt, “Neural persistence: A complexity measure for deep neural networks using algebraic topology,” in _International Conference on Learning Representations_ , 2019.
  * [22] C. A. Corneanu, S. Escalera, and A. M. Martinez, “Computing the testing error without a testing set,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020.
  * [23] C. A. Corneanu, M. Madadi, S. Escalera, and A. M. Martinez, “What does it mean to learn in deep networks? and, how does one detect adversarial attacks?” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019.
  * [24] X. Zhu, “Persistent homology: an introduction and a new text representation for natural language processing,” in _Proceedings of the Twenty-Third international joint conference on Artificial Intelligence_ , 2013.
  * [25] P. Doshi, “Using topological data analysis for text classification,” Ph.D. dissertation, The University of North Carolina at Charlotte, 2018.
  * [26] L. Kushnareva, D. Cherniavskii, V. Mikhailov, E. Artemova, S. Barannikov, A. Bernstein, I. Piontkovskaya, D. Piontkovski, and E. Burnaev, “Artificial text detection via examining the topology of attention maps,” in _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , 2021.
  * [27] R. B. Gabrielsson, B. J. Nelson, A. Dwaraknath, and P. Skraba, “A topology layer for machine learning,” in _Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics_ , ser. Proceedings of Machine Learning Research. PMLR, 26–28 Aug 2020.
  * [28] P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” in _International Conference on Learning Representations_ , 2018.
  * [29] T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “Huggingface’s transformers: State-of-the-art natural language processing,” 2020.
  * [30] C. Tralie, N. Saul, and R. Bar-On, “Ripser.py: A lean persistent homology library for python,” _The Journal of Open Source Software_ , vol. 3, no. 29, p. 925, Sep 2018.
  * [31] Y.-C. Chen, “A tutorial on kernel density estimation and recent advances,” 2017\.
  * [32] K. Kamnitsas, D. C. de Castro, L. L. Folgoc, I. Walker, R. Tanno, D. Rueckert, B. Glocker, A. Criminisi, and A. V. Nori, “Semi-supervised learning via compact latent space clustering,” _CoRR_ , vol. abs/1806.02679, 2018.
  * [33] P. J. Rousseeuw, “Silhouettes: A graphical aid to the interpretation and validation of cluster analysis,” _Journal of Computational and Applied Mathematics_ , vol. 20, pp. 53–65, 1987.
  * [34] D. L. Davies and D. W. Bouldin, “A cluster separation measure,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. PAMI-1, no. 2, pp. 224–227, 1979.
  * [35] J. Chauhan, D. Nathani, and M. Kaul, “Few-shot learning on graphs via super-classes based on graph spectral measures,” in _International Conference on Learning Representations_ , 2020.
  * [36] H. Jahangir, H. Tayarani, S. Sadeghi Gougheri, M. Aliakbar Golkar, A. Ahmadian, and A. Elkamel, “Deep learning-based forecasting approach in smart grids with micro-clustering and bi-directional lstm network,” _IEEE Transactions on Industrial Electronics_ , pp. 1–1, 2020.
  * [37] M. E. Peters, S. Ruder, and N. A. Smith, “To tune or not to tune? adapting pretrained representations to diverse tasks,” in _Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)_.
  * [38] T. Zhang*, V. Kishore*, F. Wu*, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” in _International Conference on Learning Representations_ , 2020.
  * [39] D. Zhang and Z. Yang, “Word embedding perturbation for sentence classification,” _arXiv preprint arXiv:1804.08166_ , 2018.
  * [40] W. E. Zhang, Q. Z. Sheng, and A. A. F. Alhazmi, “Generating textual adversarial examples for deep learning models: A survey,” _ArXiv_ , vol. abs/1901.06796, 2019.
  * [41] X. Ma, B. Li, Y. Wang, S. M. Erfani, S. Wijewickrema, G. Schoenebeck, M. E. Houle, D. Song, and J. Bailey, “Characterizing adversarial subspaces using local intrinsic dimensionality,” in _International Conference on Learning Representations_ , 2018.
  * [42] K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples and adversarial attacks,” in _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [43] S. Ren, Y. Deng, K. He, and W. Che, “Generating natural language adversarial examples through probability weighted word saliency,” in _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_.
  * [44] M. Alzantot, Y. Sharma, A. Elgohary, B.-J. Ho, M. Srivastava, and K.-W. Chang, “Generating natural language adversarial examples,” in _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_.
  * [45] D. Jin, Z. Jin, J. T. Zhou, and P. Szolovits, “Is bert really robust? a strong baseline for natural language attack on text classification and entailment,” 2020\.
  * [46] J. Li, S. Ji, T. Du, B. Li, and T. Wang, “Textbugger: Generating adversarial text against real-world applications,” _Proceedings 2019 Network and Distributed System Security Symposium_ , 2019. [Online]. Available: http://dx.doi.org/10.14722/ndss.2019.23138
  * [47] E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, “Universal adversarial triggers for attacking and analyzing NLP,” in _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , 2019.
  * [48] H. Karimi, T. Derr, and J. Tang, “Characterizing the decision boundary of deep neural networks,” 2020.