11institutetext: LIA, Université d’Avignon et des Pays de Vaucluse,
74 Rue Louis Pasteur, 84029 Avignon, France22institutetext: École
Polytechnique de Montréal, 2900 Edouard Montpetit Blvd,
Montreal, QC H3T 1J4, Canada
{carlos-emiliano.gonzalez-gallardo,eric.sanjuan,juan-manuel.torres}@univ-
avignon.fr

# Extending Text Informativeness Measures to Passage Interestingness
Evaluation

Language Model vs. Word Embedding

Carlos-Emiliano González-Gallardo 11 Eric SanJuan 11 Juan-Manuel Torres-Moreno
1122

###### Abstract

Standard informativeness measures used to evaluate Automatic Text
Summarization mostly rely on n-gram overlapping between the automatic summary
and the reference summaries. These measures differ from the metric they use
(cosine, ROUGE, Kullback-Leibler, Logarithm Similarity, etc.) and the bag of
terms they consider (single words, word n-grams, entities, nuggets, etc.).
Recent word embedding approaches offer a continuous alternative to discrete
approaches based on the presence/absence of a text unit. Informativeness
measures have been extended to Focus Information Retrieval evaluation
involving a user’s information need represented by short queries. In
particular for the task of CLEF-INEX Tweet Contextualization, tweet contents
have been considered as queries. In this paper we define the concept of
Interestingness as a generalization of Informativeness, whereby the
information need is diverse and formalized as an unknown set of implicit
queries. We then study the ability of state of the art Informativeness
measures to cope with this generalization. Lately we show that with this new
framework, standard word embeddings outperforms discrete measures only on uni-
grams, however bi-grams seems to be a key point of interestingness evaluation.
Lastly we prove that the CLEF-INEX Tweet Contextualization 2012 Logarithm
Similarity measure provides best results.

###### Keywords:

Information Retrieval, Automatic Text Summarization, Evaluation,
Informativeness, Interestingness

##  1 Introduction

Following Bellot et al. in [2], we consider Informativeness in the context of
Focused Retrieval (FR) [9]. Given a query representing a user information
need, a FR system returns a ranked list of short passages extracted from a
document collection. Then, a user reads a passage top to bottom and tag it as:
1) informative, 2) partially informative or 3) uninformative depending if all,
only parts or no part of it contains useful information relevant to the query.
This information can be factual and explicitly linked to the query as in
Question Answering (QA) or more abstract and provide some general background
about the query. FR systems are evaluated according to the cumulative length
of extracted informative passages (Precision) and their diversity (Recall).

Interestingness, by contrast, is a much broader concept used in Data Mining as
it is defined as “the power of attracting or holding attention” and relates
the ideas of lift and information gain used to mine very large sets of
association rules between numerous features. Mining interesting associations
between features is a complex interactive process. Unlike Information
Retrieval (IR), in Interestingness there is no precise query to initiate the
search.

As expected, it has been shown that none of the numerous Interestingness
measures proposed in Business Intelligence softwares allow to catch all
associations that experts will consider as useful [10]. Each measure allows to
grasp only a facet on interestingness. However, Hilderman et. al defined in
[8] five principles to be satisfied by these type of measures: minimal and
maximal values, skewness, permutation invariance and transfer.

By reusing this concept of Interestingness, in the context of FR we define it
as: a text passage that is clearly informative for some implicit user’s
information need. More precisely; given a set of users and a set of passages
that were considered interesting by at least one of these users, the task
consists in finding new interesting passages not related to previous topics
and implicit queries. We test the ability of state of the art FR
Informativeness metrics to deal with this specific new task. Therefore, in
this paper we consider short factual passages, most of them are single
sentences instead of complete summaries. From a formal point of view we
consider text units like words, word n-grams, terms or entities as attributes
in [8]. Passages $S$ to be ranked by decreasing interestingness are then
represented as sets of unique tuples $(\omega,\omega(S))$ where $\omega$ is an
attribute and $\omega(S)$ its frequency.

The data we used consists of a large pool of $34,507$ passages extracted from
the Wikipedia by state of the art FR systems [2] over $67$ topics from
Twitter. Each passage real informativeness has been individually assessed by
CLEF-INEX Tweet Contextualization111http://inex.mmci.uni-
saarland.de/tracks/qa/ (TC@INEX) task organizers. These assessments resulted
in a reference score scaled between 0 and 2 depending on the relative length
of the informative part and assessor’s inter-agreement.

Based on this extended dataset we analyze correlations between the reference
Informativeness scores and a set of state of the art Informativeness automatic
measures including F-score, Kullback-Leibler (KL) divergence, ROUGE, Logarithm
Similarity (LogSim) and cosine measures. Because passages are extracted from
the Wikipedia, we also consider the restriction of these measures to anchor
texts inside passages. Wikipedia anchors are references to related entities;
anchors inside informative passages can be considered as informative nuggets
annotated by Wikipedia contributors.

The rest of the paper is organized as follows. In §2 we recall the state of
the art about Informativeness measures, nugget based evaluation and
Interestingness principles. In §3 we detail the method we used to evaluate the
relative effectiveness of each metric when applied to passage informativeness
and Interestingness evaluation. §4 presents and discuss the results. Finally,
§5 concludes this paper.

##  2 State of the Art

Informativeness measures have been introduced in various contexts. Depending
of the type of information unit that is used and the way this unit is
represented, it is possible to separate them in language model approaches and
continuous space approaches (word embeddings).

###  2.1 Language Model Approaches

Language model (LM) approaches of Informativenesses are based on smoothed
probabilities over all text units. The most popular is ROUGE, which compares
an automatically generated summary with a (human-produced) reference summary.
ROUGE-N is defined as the average number of common term n-grams in the summary
and a set of $k\geq 5$ reference summaries [11]. In particular, ROUGE-1
computes the distribution of uni-grams; it counts the number of uni-grams
(words or stems) that occur both in the reference summary and in the produced
summary [27].

Apart from ROUGE-1, other variants of ROUGE are also available. While ROUGE-1
(uni-grams) considers each term independently, ROUGE-2 (bi-grams) considers
sequences of two terms. Skip-grams, used in ROUGE-S, are pairs of consecutive
terms with possible insertions between them; the number of skipped terms is a
parameter of ROUGE-SU.

ROUGE-2 and ROUGE-SU4 were used to evaluate the generated summaries in the
Document Understanding Conference (DUC) in 2005 [3]. ROUGE proved itself
better correlating human judgments under readability assumption than classical
cosine measures [22]. Indeed, the various ROUGE variants were evaluated on the
three years of DUC data in [13], showing that some ROUGE versions are more
appropriate for specific contexts.

ROUGE implies that reference summaries have been built by humans. In the case
of a very large number of documents, it is easier to apply a measure that can
be used automatically to compare the content of the produced summary with the
one of the full set of documents. In this framework, measures such as KL
divergence and Jensen-Shannon (JS) probability distribution divergence used in
the Text Analysis Conference (TAC) [4] compare the distributions of words or
word sequences between the produced summary and the original documents. In
this approach, Informativeness relies on complex hided word distributions and
not on specific units.

In addition, it is shown that JS probability distribution divergence is
correlated in the same way than ROUGE when used to evaluate summaries built by
passage extraction from the documents to be summarized [15, 24]. JS is the
symmetric variant of the KL divergence used in the LM for IR and Latent
Dirichlet Allocation to reveal latent concepts. Both measures aim at
calculating the similarity between two smoothed conditional probabilistic
distributions of terms $P(w|R)$ and $P(w|S)$ where $R$ is a textual reference
and $S$ a short summary.

We can also cite the LogSim measure used in TC@INEX 2012 task [26]. This
measure was introduced because 1) there were no reference summaries produced
by humans, 2) the automatically produced summaries were of various sizes and
3) some automatically produced summaries were too short. Although these
various measures have been introduced to evaluate similar tasks as Automatic
Text Summarization (ATS), they are applied in different contexts. Moreover,
all of them can be sensitive to the text unit that is being used. The
influence of the type of text unit (uni-gram, bi-gram, skip-gram, etc.) has
been evaluated for ROUGE in [13] as they correspond to the ROUGE variants.

Informativeness measures can be classified into three main families: The most
common approach of Informativeness likelihood in IR relies on probabilistic KL
divergence. In automatic summarization evaluation the most common approaches
are $F_{\beta}$ and ROUGE similarity scores based on n-grams. This two
families rely on exact term overlapping. A third family created by word
embedding representations has recently introduced a vectorial approach less
dependent on explicit term overlapping.

More formally, let $\Omega$ be a type of text unit and $S$ a sentence which
Informativeness is to be evaluated against a textual reference $R$, we use the
following three discrete metrics and one continuous approach.

### Kullback-Leibler (KL) Divergence

We implement it as the expectation based on the reference $R$ of the $log$
difference between normalized frequencies in $R$ and smoothed probabilities
over $S$. We fix the smoothing parameter $\mu$ at its minimal value ($\mu=1$).
This divergence is not normalized.

$D_{KL}(R||S)=\sum_{\omega\in\Omega(R)}\ln\left(\frac{P(\omega|R).(|S|+1)}{P(\omega|S).|S|+P(\omega|\Omega)}\right).P(\omega|R)$
(1)

### Logarithm Similarity (LogSim)

Like the KL divergence, this is also an expectation based on the reference $R$
but of a normalized similarity that is only defined over
$\Omega(S)\cap\Omega(R)$ [26].

$LS(S||R)=\sum_{\omega\in\Omega(S)\cap\Omega(R)}e^{-\left|\ln\left(\frac{L_{R}(\omega,S)}{L_{R}(\omega,R)}\right)\right|}.P(\omega|R)\\\
$ (2)

where,

$L_{R}(\omega,X)=\ln(1+P(\omega|X).|R|)$ (3)

###  $F_{\beta}$ and ROUGE Scores

$F_{\beta}$ measures the harmonic mean between Precision ($p$) and Recall
($r$). It is normally expressed as

$F_{\beta}=\frac{(\beta^{2}+1)\times p\times r}{\beta^{2}\times p+r}$ (4)

where $\beta$ is the factor that controls the relative emphasis between $p$
and $r$. If $\beta=1$, then it is possible to rewrite Equation 4 as:

$F_{1}=2\times\frac{p\times r}{p+r}$ (5)

F1-score ($F_{1}$) is the most common normalized set theoretic similarity
giving equal emphasis to $p$ and $r$. To represent $F_{1}$ in terms of
$\Omega$, $R$ and $S$, lets first rewrite $p$ and $r$ as:

$p=\frac{|\Omega(S)\cap\Omega(R)|}{|\Omega(S)|}$ (6)
$r=\frac{|\Omega(S)\cap\Omega(R)|}{|\Omega(R)|}$ (7)

finally Equation 5 is redefined as:

$F_{1}(S|R)=2\times\frac{|\Omega(S)\cap\Omega(R)|}{|\Omega(S)|+|\Omega(R)|}$
(8)

As explained in [11], the idea behind all ROUGE metrics is to automatically
determine the quality of a candidate summary comparing it with reference
summaries written by humans. The quality is obtained by comparing the number
of overlapping n-grams, such as word sequences or word pairs, between the
candidate and a set of reference summaries.

ROUGE-N is defined as the average number of common n-grams between the
candidate summary and a set of reference summaries:

$\textrm{ROUGE-N}=\frac{\sum_{\omega\in\Omega(R)}Count_{match}(\omega)}{\sum_{\omega\in\Omega(R)}Count(\omega)}$
(9)

where $Count(\omega)$ is the frequency of the $\omega^{\prime}s$ n-gram and
$Count_{match}$ is the co-occurring frequency of the $\omega^{\prime}s$ n-gram
in $R$ and in the candidate summary $S$. All ROUGE variants base their
functionality in the lexical similarities between the candidate and the
reference summaries, this is a problem when the candidate summary do not share
the same words of the references; that is the case of abstractive summaries
and summaries with a significant amount of paraphrasing.

### Continuous Space Approach

Classic approaches of Natural Language Processing transform the words of a
dataset into a bag of words representation that leads to sparse vectors and
semantic information loose. In a dataset containing $n$ different words, each
word will be represented in a one-hot vector that is absolutely independent of
the rest of the words in the dataset.

Table 1 shows the one-hot vectors of a fictitious dataset with 10,000
different words. It is true that some kind of relation exists between
$tiger\leftrightarrow cat$ and $wolf\leftrightarrow dog$ but with the one-hot
vectors this relationship is impossible to maintain. The restriction can be
overcome with word embeddings.

Table 1: One-hot encoding vectors example ID | word | One-hot
---|---|---
1 | tiger | $[1\ 0\ 0\ 0\ 0\ ...0\ 0]$
2 | cat | $[0\ 1\ 0\ 0\ 0\ ...0\ 0]$
… | … | …
9999 | wolf | $[0\ 0\ 0\ 0\ 0\ ...1\ 0]$
10000 | dog | $[0\ 0\ 0\ 0\ 0\ ...0\ 1]$

Word embeddings are another way to represent words within a dataset. In this
representation, vectors are capable to maintain the relationship between words
in the dataset following the distributional semantics hypothesis [7]: words
that appear in the same contexts share similar meanings.

Word2vec is a popular Neural Network embedding model described in [18]. It
aims to map the vocabulary of a dataset into a multidimensional vector space
in which the distance between the projections corresponds to the semantic
similarity between them [20].

Two different model approaches are proposed in word2vec: Continuous Bag-of-
Words (CBOW) and Continuous Skip-gram. The first one predicts a target word
based on the context, while the second one predicts a context given a target
word [16]. In both cases the size of the embeddings is defined by the size of
the projection layer.

Learning the output vectors of the neural network is a very expensive task; so
in order to increase the computation efficiency of the training process, it is
necessary to limit the number of output vectors that are updated for each
training instance. In [23] two different methods are proposed: Hierarchical
Softmax and Negative Sampling.

  * •

Hierarchical Softmax [19]: It represents the output layer of the neural
network in a binary tree shape where each leaf of the tree corresponds to each
one of the $V$ words of the vocabulary within the dataset and each node
represents the relative probabilities of its child nodes. Given the nature of
the binary tree shape, there exists just one path from the root of the tree to
each one of the leafs; making it possible to use this path to estimate the
probability of each one of the words (leafs) [17, 23].

  * •

Negative Sampling [17]: The goal is to reduce the amount of output vectors
that have to be updated during each iteration of the training process. Instead
of using all the samples during the loss function evaluation, just a small
sample of them is taken into account.

As proposed in [17], it is possible to combine words by an element-wise
addition of their embeddings. Given a document D of length $n$ represented by
the set of word embeddings
$D=\\{\overline{v}_{1},\overline{v}_{2},...,\overline{v}_{n}\\}$ where
$|\overline{v}_{i}|=m$; a simple way to represent D with a unique embedding in
terms of $D$ is to add each component $j$ of each embedding $\overline{v}_{i}$
in $D$ to obtain a unique vector $\overline{d}$ of length $m$.

To measure the similarity between the vector of a reference document
($\overline{d}_{R}$) and the one of a proposed sentence ($\overline{d}_{S}$)
we calculate the cosine similarity between the two vectors as:

$cos_{RS}(\theta)=\frac{\overline{d}_{R}\cdot\overline{d}_{S}}{||\overline{d}_{R}||\cdot||\overline{d}_{S}||}$
(10)

###  2.2 Nugget Based Evaluation

LM approaches of Informativeness are based on smoothed probabilities over all
text units, but it is possible to use only a subset of them (nuggets).

In the context of QA, Dang et al. defined a nugget as an informative text unit
(ITU) about the target that is interesting; atomicity being linked to the fact
a binary decision can be made on the relevance of the nugget to answer a
question [4]. This method makes it possible to consider documents that have
not been evaluated to be labeled as relevant or not relevant (simply because
they contain relevant nuggets or not).

It has been shown that real ITUs can be automatically extracted to convert
textual references into a set of nuggets [5]. This simplifies the complex
problem of Informativeness evaluation providing a method to measure the
proximity between two sets of ITUs. For that, standard Precision-Recall or
Pyramid measures can be used if nuggets are unambiguous entities [14] and more
sophisticated nugget score metrics based on shingles if not [21].

We refer the reader to these publications [4, 5, 14, 21] for advanced and non
trivial technical details. Here we shall point out that the original Pyramid
method relies on human evaluation to:

  1. 1.

Identify in text summaries all short sequences of words that are relevant to
some query or question.

  2. 2.

Supervise the clustering of these units into coherent features that allow to
compute some informativeness score.

The global idea is that Informativeness relies on the presence or absence of
some specific text units and can be then evaluated based on their counting.
Pavlu et al. share this same idea even though they try to automatize the
process of selecting and identifying these ITUs [21].

###  2.3 Interestingness Principles

For the purpose of this work, we consider sentences as sets of items (nuggets,
words or word n-grams). There is a wide range of Interestingness measures but
they all combine the three following properties [8]:

  1. 1.

Diversity that can rely on concentration (the sentence reveals an important
information) or on dispersion (the sentence links several different entities).

  2. 2.

Permutation invariance: the order of the items has no impact on the overall
score.

  3. 3.

Transfer: a sentence with few highly informative items should be considered
more interesting than a long sentence with less informative items.

Given a text reference, only the LogSim measure covers all three properties.
KL measure, due to smoothing, does not fulfill (1); while $F_{\beta}$ measure
does not fulfill (3) since it favors long sentences with large overlaps.

##  3 Method

###  3.1 From Informativeness to Interestingness

We formally define short passage Informativeness evaluation as a ternary
relation between a set of topics $T$, a subset of short text extracts $P$ from
a large document resource and a set $S$ of graded scores such that top ranked
passages contain certainly relevant information about the related topic or its
background.

By contrast, we define short passage Interestingness evaluation as a
projection of interestingness over content and scores. Therefore a binary
relation exist between a set of short text passages and a graded score such
that top ranked passages are informative for some unknown topic.

###  3.2 Dataset

We consider the data collection from the TC@INEX 2012 task [26] and state of
the art measures to automatically evaluate summary Informativeness and
Interestingness. The participants to this task had to provide a summary
composed of extracted passages from Wikipedia that should contextualize a
tweet by revealing its implicit background and providing some factual
explanations about related concepts. During 2012 and 2013, tweets were
collected from non-personal Twitter accounts. In 2012 a total of 33 valid runs
were submitted by 13 teams from 10 countries; while in 2013, 13 teams from 9
countries submitted 24 runs to the task [1].

### Textual Relevance Judgments

Reusable assessments to evaluate text informativeness have been one of the
main goals of TC@INEX tracks; all the collections that have been built are
indeed freely available222http://tc.talne.eu. This has been performed by
organizers on a pool of 63 topics (tweets) in 2012, and 70 topics in 2013.

Assessments consist in textual relevance judgments (t-rels) to be used for
content comparison with automatically built summaries. Since summaries
returned by participant systems are composed of extracted passages from
Wikipedia, the reference is built on a pool of these passages.

An important fact in the pre-processing of this pool is that passages starting
and ending by the same 25 characters have been considered as duplicated;
therefore passages in the reference are unique, but short sub-passages could
appear twice in longer ones. Moreover, since for each topic all passages from
all participants have been merged and displayed to the assessor in
alphabetical order, each passage informativeness has been evaluated
independently from others, even in the same summary. This results in a
reference highly redundant at the level of noun phrases and consequently all
types of ITUs that can be extracted.

The soundness of the pooling procedure was verified during the TC@INEX 2011
campaign over a corpus extracted from the New York
Times333https://www.nytimes.com [6, 25]. Indeed, topics in 2011 were only
tweets coming from the New York Times, and a full article was associated to
each tweet. To check that the resulting pool of relevant answers was sound, a
second automatic evaluation for informativeness of summaries had then been
carried out, taking as the reference the textual content of the article. None
of the participants had reported having used this information available on the
New York Times website. Both rankings, one based on submitted run pooling and
the second one based on New York Times articles, appeared to be highly
correlated. Pearson’s product-moment correlation $=88\%$ and
$p$-value$<10^{-6}$.

The TC@INEX 2012 task collection provides:

  * •

A set of $63$ different topics: A topic is a short sentence (a tweet) that is
used by participants to build a query-driven summary. Each summary should be
500 words-long or less and be supposed to be built by sentence extraction from
Wikipedia;

  * •

A set of $36$ runs: a run consists of several summaries, one per topic which
has to be built by one system from a participant;

  * •

For each summary produced by a participant, the passages that have been marked
as informative (and thus the ones that are considered as non informative) by
human assessors. We have a set of $34,507$ assessed passages, either
informative or non-informative.

From this test collection we extracted the set of passages from participants’
automatic summaries that human evaluators have marked as informative regarding
a topic. Each topic was evaluated by two people including the one that chose
the original tweet as the topic. Therefore each passage obtained a graded
score among $[0,1]\cup\\{2\\}$ as the relative total length of the passage
that has been highlighted as informative by at least one evaluator. In the
case that the two evaluators agreed that the whole passage was informative, an
score of $2$ was assigned.

Another resource was built using this data. From each passage we extracted
text units. We considered in first place stems that were in turn used to build
uni-grams, bi-grams and skip-grams; secondly, Wikipedia entities in anchor
texts. Stems were simply obtained using Porter’s stemmer after a process of
stop words removal; they correspond to uni-grams. Bi-grams were composed of
two adjacent text units after stop words removal.

If all passages that have been selected as informative are considered and
separated by topic, it is possible to build a textual reference for each topic
to apply state of the art Informativeness metrics. By contrast, by merging all
the references we obtain a textual reference to evaluate Interestingness.

We also focus on other type of token characteristics. We hypothesize that the
measures also can be sensitive to term distribution. Indeed, it is likely that
automatic summaries that contain highly frequent terms are less interesting
for a user than a summary that contains less frequent and thus probably more
informative terms. In the same way, it is likely that summaries that contain
name entities are more informative than summaries that do not contain any.
These phenomenons have not been studied in the literature justifying the
purpose of this paper. While in previous studies tokens are words or stems, in
this paper we also study other types of tokens such as DBpedia entities.

To perform experiments with word embeddings we used four different word2vec
models. All of them with an embedding size of 300 dimensions and a negative
sampling of 15 units. Table 2 shows the number of embeddings and the text unit
used in each model.

Table 2: Word2vec models Model |  # of embeddings |  Text Unit
---|---|---
$\textrm{google}\\_\textrm{news}_{1gram}$ | 3,000,000 |  uni-grams
$\textrm{clef}\\_\textrm{inex}_{1gram}$ | 30,000 |  uni-grams
$\textrm{clef}\\_\textrm{inex}_{2gram}$ | 600,000 |  bi-grams
$\textrm{wiki}_{2gram}$ | 10,000,000 |  bi-grams

The $\textrm{google}\\_\textrm{news}_{1gram}$ model was the same used by [20]
to calculate ROUGE-WE. Both $\textrm{clef}\\_\textrm{inex}_{1gram}$ and
$\textrm{clef}\\_\textrm{inex}_{2gram}$ models where trained with the $34,507$
passages of the TC@INEX 2012 task. Finally, the $\textrm{wiki}_{2gram}$ model
was trained with a $3.4Gb$ partition of the 2012’s Wikipedia.

###  3.3 Meta Evaluation

Rather than calculating the correlation between systems’ average measure
scores and their human assigned average coverage scores as in [12] with a
limited number of systems and abstracts, we considered each passage
individually for which we have a human assessment. We then calculated all
scores for each metric.

Following the same approach as in TREC 2015 Microblog
Track444https://github.com/lintool/twitter-tools/wiki/TREC-2015-Track-
Guidelines [28], to evaluate one specific metric we ranked passages in
decreasing order following this metric. Considering different cut-off values,
we computed the normalized Cumulative Gain (nCG) over top ranked passages.
This score is the sum of the graded human judgments top ranked passages
obtained in the TC@INEX 2012 divided by the maximum score that could have been
expected at this precise cut-off value.

More precisely, given a set of topics $\cal T$ and a set of passages $\Omega$
for which there is a graded evaluation $ref_{\tau}(\omega)$ of their
Informativeness for at least one topic $\tau\in{\cal T}$, $nCG_{k}(m)$ is
computed as follows for any metric $m$ and any cut-off value $k$:

$nCG_{k}(m)=\frac{\max\\{\sum_{\omega\in S}m(\omega):S\subset\Omega,|S|\leq
k\\}}{\max\\{\sum_{\omega\in S}ref_{\tau}(\omega):S\subset\Omega,|S|\leq
k,\tau\in{\cal T}\\}}$ (11)

where $|S|$ is the cardinal of $S$.

For values of $k$ lower than the number of passages $\omega$ such that
$ref_{\tau}(\omega)\geq 1$, $nCG_{k}(m)$ shows the measure Precision. On the
contrary, for cut-off values higher than the number of passages $\omega$ such
that $ref_{\tau}(\omega)>0$, $nCG_{k}(m)$ indicates the maximum Recall that
can be expected using this measure.

We considered and evaluated the following set ${\cal M}$ of 13 textual overlap
measures:

F1_1:

F1-score among uni-grams

F1_2:

F1-score among bi-grams

F1_sk:

F1-score among skip-grams with a gap of one word

KL_1:

KL divergence among uni-grams

KL_2:

KL divergence among bi-grams

KL_sk:

KL divergence among skip-grams with a gap of one word

LS_1:

LogSim score among uni-grams

LS_2:

LogSim score among bi-grams

LS_sk:

LogSim score among skip-grams with a gap of one word

w2v_g:

Word2vec cosine similarity over the Google News model
($\textrm{google}\\_\textrm{news}_{1gram}$)

w2v_c:

Word2vec cosine similarity over the set of all passages in $\Omega$
($\textrm{clef}\\_\textrm{inex}_{1gram}$)

w2v_c_bi:

Word2vec cosine similarity the same set $\Omega$ but considering word bi-grams
instead of single words ($\textrm{clef}\\_\textrm{inex}_{2gram}$)

w2v_wp_bi:

Word2vec cosine similarity over the Wikipedia version used as copora in
TC@INEX 2012 considering word bi-grams ($\textrm{wiki}_{2gram}$)

The first nine measures are discrete measures that we also applied to sibling
operators over Wikipedia anchor texts. We consider the anchors associated with
Wikipedia entities as potential nuggets as defined in Pyramid evaluations [3].

To evaluate Informativeness, each measure was applied to estimate the overlap
between the reference informative passages of the topic and the passage
itself. Statistical signification was tested over topics. Meanwhile, to
evaluate Interestingness each measure is applied to estimate the overlap
between the set of all informative passage, disregarding its specific topic
and the passage itself. Statistical signification is tested based on a 12-fold
split of the corpus.

##  4 Results

###  4.1 Which Measure Ranks First Most Informative Passages per Topic?

We used nCG over TC@INEX 2012 data to test the ability of state of the art
Informativeness measures to evaluate ATS systems. The goal is to distinguish
informative from non-informative short text passages rather than entire
abstracts. This is done by ranking the Informativeness score of all passages
manually assessed in TC@INEX 2012 for any topic and sorting them in decreasing
order. Then, the ability of a specific measure to find the $k$ most
informative passages is evaluated.

For each measure $\mu\in{\cal M}$ and passage $\omega\in\Omega$ associated
with a topic $\tau\in{\cal T}$, we compute $\mu(\varphi_{\tau},\omega)$ to
estimate the semantic overlap between $\omega$ with the textual reference
$\varphi_{\tau}$ defined as:

$\displaystyle\varphi_{\tau}$ $\displaystyle=$
$\displaystyle\bigcup\\{\omega\in\Omega:ref_{\tau}(\omega)>0\\}$ (12)

### Discrete vs. Continuous Metrics

Figure 1 presents the results of the F1-scores over uni-grams, bi-grams and
skip-grams and compares them against the word2vec uni-gram models. It shows
that word2vec approaches are less efficient to evaluate short passage
informativeness even over standard uni-grams. Meanwhile, F1-scores over bi-
grams and skip-grams reach similar performance.

Figure 1: Informativeness nCG scores about F1-scores vs. word2vec approaches

The figure also shows that for low cut-off values, Precision is lower that
$40$% and that bi-grams do not improve the measure performance. For higher
cut-off values it shows that maximal Recall is only reached by the F1-score
over bi-grams and skip-grams.

A slightly improvement can be seen with the local word2vec model
($\textrm{clef}\\_\textrm{inex}_{1gram}$) over the model trained with the
Google News dataset ($\textrm{google}\\_\textrm{news}_{1gram}$). This behavior
can be explained by the fact that the amount of unknown words in the
$\textrm{clef}\\_\textrm{inex}_{1gram}$ model is smaller that the one in the
$\textrm{google}\\_\textrm{news}_{1gram}$ model. Despite that this improvement
in the cumulative score is very small, it shows that is better to create the
word embeddings with a smaller but more specialized dataset.

### Best Discrete informativeness Metric

In Figure 2 it can be seen that all KL metrics show similar performance for
medium cut-off values. Among all metrics over uni-grams, F1-score achieves the
better Precision and Recall over all cut-off values, obtaining a pick in the
recall of almost $60$% in high cut-off values. F1-scores over bi-grams and
skip-grams outperforms LogSim over the same units, however this improvement is
not significant. Again only bi-gram based measures reach maximal recall for
high cut-off values.

Figure 2: Informativeness nCG scores about F1-scores vs. KL and LogSim
approaches

### Discrete Metrics Restricted to Entities

As shown in Figure 3, it appears that for low cut-off values, restricting
references and passages to DBpedia entities provides equivalent scores to
complete passages for F1-scores. By contrast, for high cut-off values and
maximal recall there is a clear gap between restricted entities and complete
passages.

Figure 3: Informativeness nCG scores about DBpedia entities vs. complete
passages

An interesting remark is the behavior of the LogSim measures with DBpedia
entities. For low cut-off values all three LogSim scores are extremely low and
fail to highlight informative passages; for high cut-off values, LogSim scores
over bi-grams and skip-grams reach a recall near the $70\%$.

In regard to the F1-scores with uni-grams, a slightly outperform with
restricted entities over complete passages can be seen for high cut-off values
but this difference is not significant.

###  4.2 Which Measure Ranks First Most Interesting Passages?

We now used nCG over TC@INEX 2012 data to test the ability of the same
Informativeness measures in distinguishing interesting from non-interesting
text passages without considering an explicit topic.

For each measure $\mu\in{\cal M}$ and passage $\omega\in\Omega$ associated
with a topic $\tau\in{\cal T}$, we compute $\mu(\Delta_{\tau},\omega)$ to
estimate the semantic overlap between $\omega$ and the textual reference
$\Delta_{\tau}$ defined as the concatenation of passages that are informative
for at least one different topic $t^{\prime}\neq t$:

$\displaystyle\Delta_{\tau}$ $\displaystyle=$
$\displaystyle\bigcup\\{\omega\in\Omega:(\exists t^{\prime}\in{\cal
T}-\\{t\\})ref_{t^{\prime}}(\omega)>0\\}$ (13)

However in oder to avoid any overfitting effect, we’ve split the dataset of
$34,507$ passages ranked per topic into $12$ folds and restricted
$\Delta_{\tau}$ to passages in a different fold than the one including $\tau$.

### Discrete vs. Continuous Metrics

Figure 4 presents the results of F1 scores and word2vec models over uni-grams
and bi-grams. For small cut-off values, word2vec models and the F1-score over
uni-grams show a similar low performance. The improvement of F1-scores over
bi-grams and skip-grams against uni-grams for all cut-off values was striking
for this task.

Figure 4: Interestingness nCG scores about F1-scores vs. word2vec approaches

F1-scores do not reach this time maximal recall for high cut-off values.
However best compromise between recall and precision is reached for a cut-off
value around $2,500$ passages over $3,000$ informative passages in the
reference. Word2vec models trained over Wikipedia using bi-grams significantly
improves precision over small cut-off values but then become less efficient
after $2,500$ passages because of the proportion of missing bi-grams in the
model. Other word2vec models do not outperform the baseline taking inverse
passage length (len_inv) as measure with the idea that short passages could be
more specific.

### Best Discrete Interestingness Metric

As shown in Figure 5, among discrete measures there is another contrast with
previous results over Informativeness. It appears that LogSim measures over
bi-grams significantly outperform F1-scores and reach total recall after
$5,000$ retrieved passages. Again, only bi-gram and skip-gram based measures
reach maximal recall for high cut-off values. LogSim bi-gram and skip-gram
measures significantly outperform all other measures for any cut-off value. It
also appears that KL metrics provide the best results over uni-grams among all
metrics with the same units.

Figure 5: Interestingness nCG scores about F1-scores vs. KL and LogSim
approaches

### Discrete Metrics Restricted to Entities

Finally, in Figure 6 we look at the impact of restricting references and
passages to DBpedia entities in the case of interestingness. For low and
medium cut-off values, F1-scores over bi-grams and skip-grams with complete
passages show a better performance than the ones with DBpedia entities. This
difference in performance is reduced for high cut-off values where the recall
for bi-grams with DBpedia entities rises to a $70\%$.

From Figures 5 and 6 we see that LogSim measures over bi-grams and skip-grams
with both complete and restricted passages outperform all the other measures.
In general, LogSim scores with complete passages show a better performance
than those with restricted entities. Surprisingly, it appears that LogSim
measures over bi-grams remain almost stable and outperform F1-scores both over
complete and restricted passages.

Figure 6: Interestingness nCG scores about DBpedia entities vs. complete
passages

##  5 Conclusion

In this paper we have defined the concept of Interestingness in FR as a
generalization of the concept of Informativeness where the information need is
diverse and formalized as an unknown set of implicit queries.

We then studied the ability of state of the art Informativeness measures to
cope with this generalization, showing that on this new framework, cosine
similarity between word embedding vector representations outperform discrete
measures only on uni-grams. Moreover discrete bi-gram and skip-gram LMs
appeared to be a key point of Interestingness evaluation, significantly
outperforming all experiments wit word2vec models. However we did show that an
alternative word2vec bi-gram model learned on Wikipedia outperforms the uni-
gram word2vec models on nCG scores for small cut-off value, but its
performance decreases for higher values. Finally we showed that TC@INEX 2012
LogSim measure provides indeed best results for efficient Interestingness
detection over this corpus, both on complete passages and on passages
restricted to their anchor texts referring to entities.

## References

  * [1] Bellot, P., Moriceau, V., Mothe, J., Sanjuan, E., Tannier, X.: Overview of the inex 2013 tweet contextualization track. In: CLEF 2013 Evaluation Labs and Workshop, Online Working Notes (2013)
  * [2] Bellot, P., Moriceau, V., Mothe, J., SanJuan, E., Tannier, X.: INEX Tweet Contextualization task: Evaluation, results and lesson learned. Information Processing and Management 52(5), 801–819 (2016)
  * [3] Dang, H.T.: Overview of DUC 2005. In: Proceedings of the Document Understanding Conference. pp. 1–12 (2005)
  * [4] Dang, H.T., Kelly, D., Lin, J.J.: Overview of the trec 2007 question answering track. In: TREC. vol. 7, p. 63 (2007)
  * [5] Ekstrand-Abueg, M., Pavlu, V., Aslam, J.A.: Live nuggets extractor: a semi-automated system for text extraction and test collection creation. In: Jones, G.J.F., Sheridan, P., Kelly, D., de Rijke, M., Sakai, T. (eds.) SIGIR. pp. 1087–1088. ACM (2013)
  * [6] Geva, S., Kamps, J., Schenkel, R. (eds.): Focused Retrieval of Content and Structure, 10th International Workshop of the Initiative for the Evaluation of XML Retrieval, INEX 2011, Saarbrücken, Germany, December 12-14, 2011, Revised Selected Papers, Lecture Notes in Computer Science, vol. 7424. Springer (2012)
  * [7] Harris, Z.S.: Distributional structure. Word 10(2-3), 146–162 (1954)
  * [8] Hilderman, R.J., Hamilton, H.J.: Measuring the interestingness of discovered knowledge: A principled approach. Intell. Data Anal. 7(4), 347–382 (2003)
  * [9] Kamps, J., Geva, S., Trotman, A.: Report on the SIGIR 2008 workshop on focused retrieval. ACM SIGIR Forum 42(2), 59–65 (2008)
  * [10] Koh, Y.S., O’Keefe, R., Rountree, N.: Interestingness Measures for Association Rules: What Do They Really Measure? In: Data Mining and Knowledge Discovery Technologies, chap. 2, pp. 36–58. IGI PUBLISHING (2008)
  * [11] Lin, C.Y.: Rouge: a package for automatic evaluation of summaries. pp. 74–81 (July 25-26 2004)
  * [12] Lin, C.Y., Hovy, E.H.: Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics. In: HLT-NAACL. vol. 1, pp. 71–78 (2003)
  * [13] Lin, C.Y., Och, F.: Looking for a few good metrics: Rouge and its evaluation. In: NTCIR Workshop (2004)
  * [14] Lin, J.J., Zhang, P.: Deconstructing nuggets: the stability and reliability of complex question answering evaluation. In: Kraaij, W., de Vries, A.P., Clarke, C.L.A., Fuhr, N., Kando, N. (eds.) SIGIR. pp. 327–334. ACM (2007)
  * [15] Louis, A., Nenkova, A.: Performance Confidence Estimation for Automatic Summarization. In: EACL. pp. 541–548. ACL (2009)
  * [16] Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013)
  * [17] Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems. pp. 3111–3119 (2013)
  * [18] Mikolov, T., Yih, W.t., Zweig, G.: Linguistic regularities in continuous space word representations. In: HLT-NAACL. vol. 13, pp. 746–751 (2013)
  * [19] Morin, F., Bengio, Y.: Hierarchical probabilistic neural network language model. In: Proceedings of Aistats. vol. 5, pp. 246–252 (2005)
  * [20] Ng, J.P., Abrecht, V.: Better summarization evaluation with word embeddings for rouge. arXiv:1508.06034 [cs.CL] (2015)
  * [21] Pavlu, V., Rajput, S., Golbus, P.B., Aslam, J.A.: Ir system evaluation using nugget-based test collections. In: Adar, E., Teevan, J., Agichtein, E., Maarek, Y. (eds.) WSDM. pp. 393–402. ACM (2012)
  * [22] Radev, D.R., Teufel, S., Saggion, H., Lam, W., Blitzer, J., Qi, H., Çelebi, A., Liu, D., Drábek, E.: Evaluation Challenges in Large-Scale Document Summarization. In: ACL’03. pp. 375–382 (2003)
  * [23] Rong, X.: word2vec Parameter Learning Explained. arXiv:1411.2738 [cs.CL] (2014)
  * [24] Saggion, H., Torres-Moreno, J.M., da Cunha, I., SanJuan, E., Velázquez-Morales, P.: Multilingual Summarization Evaluation without Human Models. In: Huang, C.R., Jurafsky, D. (eds.) COLING. pp. 1059–1067 (2010)
  * [25] SanJuan, E., Moriceau, V., Tannier, X., Bellot, P., Mothe, J.: Overview of the INEX 2011 question answering track (qa@inex). In: Geva et al. [6], pp. 188–206
  * [26] SanJuan, E., Moriceau, V., Tannier, X., Bellot, P., Mothe, J.: Overview of the INEX 2012 Tweet Contextualization Track. In: Forner, P., Karlgren, J., Womser-Hacker, C. (eds.) CLEF (Working Notes/Labs/Workshop) (2012)
  * [27] Torres-Moreno, J.M.: Automatic Text Summarization. John Wiley and Sons, London (2014)
  * [28] Yilmaz, E., Verma, M., Mehrotra, R., Kanoulas, E., Carterette, B., Craswell, N.: Overview of the trec 2015 tasks track. In: Text Retrieval Conference (TREC) (2015)


11institutetext: Dept. de Astrofísica Molecular, Instituto de Física
Fundamental (IFF-CSIC), C/ Serrano 121, 28006 Madrid, Spain
11email<EMAIL_ADDRESS><EMAIL_ADDRESS>22institutetext:
Department of Applied Chemistry, Science Building II, National Yang Ming Chiao
Tung University, 1001 Ta-Hsueh Rd., Hsinchu 300098, Taiwan 33institutetext:
Observatorio Astronómico Nacional (OAN, IGN), C/ Alfonso XII, 3, 28014,
Madrid, Spain 44institutetext: Observatorio de Yebes (IGN), Cerro de la Palera
s/n, 19141 Yebes, Guadalajara, Spain

# Discovery of the interstellar cyanoacetylene radical cation HC3N+††thanks:
Based on observations carried out with the Yebes 40m telescope (projects
19A003, 20A014, 20D023, 21A011, 21D005 and 23A024). The 40m radio telescope at
Yebes Observatory is operated by the Spanish Geographic Institute (IGN,
Ministerio de Transportes y Movilidad Sostenible).

C. Cabezas 11 M. Agúndez 11 Y. Endo 22 B. Tercero 3344 N. Marcelino 3344 P. de
Vicente 44 and J. Cernicharo 11

(Received; accepted)

We report the first identification in space of HC3N+, the simplest member of
the family of cyanopolyyne cations. Three rotational transitions with half-
integer quantum numbers from $J$=7/2 to 11/2 have been observed with the Yebes
40m radio telescope and assigned to HC3N+, which has an inverted ${}^{2}\Pi$
ground electronic state. The three rotational transitions exhibit several
hyperfine components due to the magnetic and nuclear quadrupole coupling
effects of the H and N nuclei. We confidently assign the characteristic
rotational spectrum pattern to HC3N+ based on the good agreement between the
astronomical and theoretical spectroscopic parameters. We derived a column
density of (6.0$\pm$0.6)$\times$1010 cm-2 and a rotational temperature of
4.5$\pm$1 K. The abundance ratio between HC3N and HC3N+ is 3200$\pm$320\. As
found for the larger members of the family of cyanopolyyne cations (HC5N+ and
HC7N+), HC3N+ is mainly formed through the reactions of H2 and the cation C3N+
and by the reactions of H+ with HC3N. In the same manner than other
cyanopolyyne cations, HC3N+ is mostly destroyed through a reaction with H2 and
a dissociative recombination with electrons.

###### Key Words.:

molecular data — line: identification — ISM: molecules — ISM: individual
(TMC-1) — astrochemistry

##  1 Introduction

The cold dark cloud TMC-1 shows a rich and complex chemistry in which
molecular ions play roles as important intermediates. These charged species
can rapidly react with neutral partners or recombine with electrons to form
other ionic and neutral molecules under astrophysical conditions (Agúndez &
Wakelam 2013). TMC-1 is known to contain a variety of protonated species of
abundant neutral molecules. In the past few years, the list has increased
considerably with the discovery of the cations HC5NH+ (Marcelino et al.
2020), HC3O+ (Cernicharo et al. 2020a), HC3S+ (Cernicharo et al. 2021a),
CH3CO+ (Cernicharo et al. 2021b), HCCNCH+ (Agúndez et al. 2022), HCCS+
(Cabezas et al. 2022a), HC7NH+ (Cabezas et al. 2022b), NC4NH+ (Agúndez et al.
2023), C5H+ (Cernicharo et al. 2022a), H2C3H+ (Silva et al.  2023), and HC5N+
and HC7N+ (Cernicharo et al. 2024). The interstellar identification of only a
few of these protonated molecules has been performed through the direct
comparison of the astronomical spectra with laboratory spectroscopic
measurements. HC3O+ (Cernicharo et al. 2020a), HC3S+ (Cernicharo et al.
2021a), and CH3CO+ (Cernicharo et al. 2021b) were observed in the laboratory
thanks to their astronomical rotational frequencies, and only the cation
H2C3H+ (Silva et al.  2023) was observed in the laboratory prior to its
discovery in TMC-1.

These protonated species are considered ’short-lifetime’ or transient
molecules under the terrestrial physical conditions; hence, their
spectroscopic characterisations require experiments with in situ generation,
such as those employing electric discharges (Cernicharo et al. 2020a, 2021a);
glow discharges (Cernicharo et al. 2021b); or electron impact ionisation
techniques (Silva et al.  2023). However, most of these positive ions have
eluded characterisation in the laboratory, and therefore their identification
in space has been carried out using quantum chemical calculations and direct
spectroscopic analysis of high spectral resolution line surveys of molecular
clouds, such as QUIJOTE111Q-band Ultrasensitive Inspection Journey to the
Obscure TMC-1 Environment (Cernicharo et al. 2021c). Examples of these kinds
of identifications are the cations HC5NH+ (Marcelino et al.  2020), HCCNCH+
(Agúndez et al. 2022), HCCS+ (Cabezas et al. 2022a), HC7NH+ (Cabezas et al.
2022b), NC4NH+ (Agúndez et al.  2023), C5H+ (Cernicharo et al. 2022a), and the
recently detected cations HC5N+ and HC7N+ (Cernicharo et al. 2024). Hence,
high-level quantum chemical calculations are the main driver for the
identification of most of these species.

In this work, we present a new case of molecular spectroscopy done in space.
Thanks to sensitive high spectral resolution observations of the cold dark
cloud TMC-1 achieved with QUIJOTE, we have detected various groups of
unidentified lines, revealing a complex hyperfine structure, which we assign
to HC3N+. This species is the simplest member of the family of cyanopolyyne
cations, for which two other species have also been observed in TMC-1, HC5N+
and HC7N+ (Cernicharo et al. 2024). As our observations of TMC-1 show narrow
lines and high spectral resolution, it has been possible to resolve the
hyperfine structure of the rotational lines of HC3N+ and hence to fully
characterise the carrier and derive very precisely its spectroscopic
parameters. Based on these constants and their agreement with those obtained
by high-level quantum chemical calculations, we confidently assign the
observed lines to HC3N+, and we report its first detection in space.

##  2 Quantum chemical calculations

Several theoretical calculations for HC3N+ have been published (Zhang et al.
2012; Leach et al.  2014; Dai et al.  2015; Desrier et al.  2016; Gans et al.
2016; Steenbakkers et al. 2023). However, we carried out quantum chemical
calculations in order to estimate the molecular parameters for HC3N+ not
reported in the literature before. The equilibrium molecular geometry,
rotational constant, and the electric dipole moment were calculated using the
spin-restricted coupled cluster method with single, double, and perturbative
triple excitations and with an explicitly correlated approximation
(RCCSD(T)-F12; Knizia et al. 2009) and all electrons (valence and core)
correlated together with the Dunning’s correlation consistent basis sets with
a polarised core-valence correlation triple-$\zeta$ for explicitly correlated
calculations (cc-pCVTZ; Hill et al. 2010). These calculations were carried out
using the Molpro 2024.1 program (Werner et al. 2024). In addition, we
calculated other parameters necessary to interpret the rotational spectrum. At
the optimised geometry, we calculated the magnetic hyperfine constants for the
non-zero nuclear spin nuclei, hydrogen and nitrogen in this case, which are
the Fermi ($b_{F}$) and the dipole-dipole constants ($T_{aa}$ and $T_{bb}$).
Additionally, the nuclear electric quadrupole constants for the nitrogen
nucleus ($\chi_{aa}$ and $\chi_{bb}$) have also been calculated using the
B3LYP hybrid density functional (Becke  1993) with the polarised valence
triple-$\zeta$ basis set (cc-pVTZ; Woon & Dunning 1993). Harmonic frequencies
were computed at the same level of theory to estimate the centrifugal
distortion constant. These calculations were done using the Gaussian16 program
(Frisch et al. 2016). Table 1 summarises the results of these calculations.

The Frosch and Foley (Frosch & Foley 1952) hyperfine constants were derived
from the dipole-dipole interaction tensor T and the Fermi contact constant bF
– both were obtained from quantum chemical calculation – using the relations

$\displaystyle b={b_{F}-{c\over 3}}={b_{F}-{1\over 2}T_{aa}},$ (1)

$\displaystyle c={{3\over 2}T_{aa}},$ (2)

$\displaystyle d={T_{bb}-T_{cc}}.$ (3)

We utilised the approximate relation (Townes & Schawlow  1975)

$\displaystyle c=3(a-d)$ (4)

between $a$, $c$, and $d$ to estimate the $a$ constant as

$\displaystyle a={{c\over 3}+d}={{3\over 2}T_{aa}+T_{bb}}.$ (5)

The main contribution to the hyperfine coupling in the ${}^{2}\Pi_{3/2}$ state
comes from the constant h1, defined as

$\displaystyle h_{1}={a+{(b+c)\over 2}}.$ (6)

Table 1: Molecular constants (all in megahertz) of HC3N+ in
$\tilde{X}$${}^{2}\Pi_{3/2}$.

Constant | TMC-1 |  Theoreticala
---|---|---
$B$ |  4533.31711(148)b | 4535
$D$ | 0.000456(36) | 0.000469
$b_{F}$(H) | -39.9(36) | -36.37
$T_{aa}$(H) | - | 22.20
$T_{bb}$(H) | - | -3.87
$a$(H) | 18.53(160) | 25.55
$c$(H) |  [33.3]c | 33.3
$b_{F}$(N) | 9.52(276) | 0.13
$T_{aa}$(N) | - | -20.55
$T_{bb}$(N) | - | 38.01
$a$(N) | 33.67(129) | 45.18
$c$(N) |  [-30.82]c | -30.82
$eQq$(N) | -5.442(62) | -6.21
—$\mu$— (D) | - | 5.5
222
$a$$a$footnotetext: These values have been obtained from quantum chemical
calculations at different levels of theory. (See text for details.)
$b$$b$footnotetext: The uncertainties (in parentheses) are in units of the
last significant digits. $c$$c$footnotetext: Values in brackets have been kept
fixed to those obtained theoretically.

##  3 Astronomical observations

The observational data presented in this work are part of the QUIJOTE spectral
line survey (Cernicharo et al. 2021c) in the Q-band towards TMC-1(CP)
($\alpha_{J2000}=4^{\rm h}41^{\rm m}41.9^{\rm s}$ and
$\delta_{J2000}=+25^{\circ}41^{\prime}27.0^{\prime\prime}$) that was performed
at the Yebes 40m radio telescope. This survey was done using a receiver built
within the Nanocosmos project333https://nanocosmos.iff.csic.es/ consisting of
two cooled high electron mobility transistor (HEMT) amplifiers covering the
31.0-50.3 GHz band with horizontal and vertical polarisations. Fast Fourier
transform spectrometers (FFTSs) with $8\times 2.5$ GHz and a spectral
resolution of 38.15 kHz provided the whole coverage of the Q-band in both
polarisations. Receiver temperatures were 16 K at 32 GHz and 30 K at 50 GHz.
The experimental setup has been described in detail by Tercero et al. (2021).

All observations were performed using the frequency-switching observing mode
with a frequency throw of 10 and 8 MHz. The total observing time on the source
for data taken with frequency throws of 10 MHz and 8 MHz was 465 and 737
hours, respectively. Hence, the total observing time on source was 1202 hours.
The QUIJOTE sensitivity varies between 0.08 mK at 32 GHz and 0.2 mK at 49.5
GHz, and it is around 50 times better than that of previous line surveys in
the Q-band of TMC-1 (Kaifu et al. 2004). A detailed description of the line
survey and the data analysis procedure followed are provided in Cernicharo et
al. (2021c, 2022b). The main beam efficiency can be given across the Q-band by
$B_{\rm eff}$=0.797 exp[$-$($\nu$(GHz)/71.1)2]. The forward telescope
efficiency is 0.97. The telescope beam size at half-power intensity is 54.4′′
at 32.4 GHz and 36.4′′ at 48.4 GHz. The absolute calibration uncertainty is 10
$\%$. The data were analysed with the GILDAS
package.444http://www.iram.fr/IRAMFR/GILDAS

Figure 1: Observed transitions of HC3N+ in TMC-1 with $F_{1}$ and $F$ quantum
numbers indicated. The abscissa corresponds to the rest frequency, and the
ordinate is the antenna temperature corrected for atmospheric and telescope
losses in milliKelvin. Blank channels correspond to negative features produced
when folding the frequency-switched data. The red lines correspond to the
synthetic spectra calculated for a column density of 6.0$\times$1010 cm-2 and
a rotational temperature of 4.5 K.

##  4 Results and discussion

Table 2: Observed line parameters for HC3N+.

$J^{\prime}$-$J$ |  $F_{1}^{\prime}$-$F_{1}$ |  $F^{\prime}$-$F$ |  $\nu_{rest}$ a | (o-c)b |  $\int T_{A}^{*}dv$ c |  $\Delta v$ d |  $T_{A}^{*}$ e | Notes
---|---|---|---|---|---|---|---|---
|  |  | (MHz) | (MHz) | (mK km s-1) | (km s-1) | (mK) |
7/2-5/2 | 4-5 | 3-4 | 31621.877$\pm$0.010 | 0.002 | 1.53$\pm$0.08 | 1.07$\pm$0.07 | 1.33$\pm$0.10 |
7/2-5/2 | 3-4 | 2-3 | 31623.173$\pm$0.010 | -0.000 | 1.05$\pm$0.07 | 0.90$\pm$0.01 | 1.16$\pm$0.10 |
7/2-5/2 | 4-4 | 3-3 | 31626.161$\pm$0.010 | 0.001 | 1.07$\pm$0.13 | 0.75$\pm$0.08 | 1.34$\pm$0.13 | A
7/2-5/2 | 3-3 | 2-2 | 31627.170$\pm$0.010 | 0.005 | 0.91$\pm$0.13 | 0.99$\pm$0.18 | 0.87$\pm$0.10 |
7/2-5/2 | 4-3 | 3-2 | 31629.665$\pm$0.010 | 0.000 | 0.92$\pm$0.13 | 0.92$\pm$0.18 | 0.94$\pm$0.10 | B
7/2-5/2 | 3-2 | 2-1 | 31630.840$\pm$0.020 | -0.006 | 0.88$\pm$0.13 | 0.95$\pm$0.23 | 0.61$\pm$0.10 |
9/2-7/2 | 5-6 | 4-5 |  |  |  |  |  | C
9/2-7/2 | 4-5 | 3-4 | 40659.446$\pm$0.010 | 0.002 | 1.37$\pm$0.08 | 0.97$\pm$0.07 | 1.33$\pm$0.09 |
9/2-7/2 | 5-5 | 4-4 | 40660.934$\pm$0.010 | 0.001 | 1.44$\pm$0.08 | 0.81$\pm$0.05 | 1.67$\pm$0.09 |
9/2-7/2 | 4-4 | 3-3 | 40661.701$\pm$0.010 | 0.009 | 1.45$\pm$0.09 | 0.98$\pm$0.07 | 1.39$\pm$0.09 |
9/2-7/2 | 5-4 | 4-3 | 40662.912$\pm$0.010 | -0.004 | 0.92$\pm$0.07 | 0.72$\pm$0.06 | 1.20$\pm$0.09 |
9/2-7/2 | 4-3 | 3-2 | 40663.761$\pm$0.010 | -0.008 | 0.90$\pm$0.09 | 1.10$\pm$0.09 | 0.76$\pm$0.09 |
11/2-9/2 | 6-7 | 5-6 | 49694.811$\pm$0.050 | -0.049 | 2.41$\pm$0.24 | 1.51$\pm$0.18 | 1.51$\pm$0.25 |
11/2-9/2 | 5-6 | 4-5 | 49695.446$\pm$0.050 | 0.037 | 1.00$\pm$0.19 | 0.69$\pm$0.17 | 1.38$\pm$0.25 |
11/2-9/2 | 6-6 | 5-5 | 49696.225$\pm$0.050 | -0.020 | 1.21$\pm$0.19 | 0.83$\pm$0.14 | 1.43$\pm$0.25 |
11/2-9/2 | 5-5 | 4-4 | 49696.847$\pm$0.050 | 0.008 | 1.65$\pm$0.21 | 0.85$\pm$0.13 | 1.82$\pm$0.25 | B
11/2-9/2 | 6-5 | 5-4 | 49697.548$\pm$0.050 | 0.026 | 0.98$\pm$0.22 | 1.01$\pm$0.30 | 0.91$\pm$0.25 |
11/2-9/2 | 5-4 | 4-3 |  |  |  |  |  | D

555 $a$$a$footnotetext: Rest frequency assuming a vLSR of 5.83 km s-1
(Cernicharo et al. 2020c). $b$$b$footnotetext: Observed frequency minus
calculated frequency from the fit. $c$$c$footnotetext: Integrated intensity
(in mK km s-1). $d$$d$footnotetext: Line width (in km s-1).
$e$$e$footnotetext: Antenna temperature (in mK). $A$$A$footnotetext: Data only
from the set of data with frequency switching throw of 8 MHz.
$B$$B$footnotetext: Partially blended with a U feature. $C$$C$footnotetext:
The line is fully blended with a strong line of CH2CN. $D$$D$footnotetext:
Affected by a negative feature produced in the folding of the frequency
switching data.

Since HC5N+ and HC7N+ have been detected in TMC-1, the smaller member of this
family of cyanopolyyne cations is expected to be present as well in this
molecular cloud. A first attempt was made to find the rotational signatures of
HC3N+ in our data, based on the estimation of the $B$ rotational constant
reported in several theoretical works (Cernicharo et al. 2024). However, we
could not find unidentified lines with frequencies in harmonic relation
attributable to HC3N+ in the explored frequency ranges. Hence, we concluded
that the abundance of HC3N+ in TMC-1 should be $\leq$1011 cm-2.

Similarly to HC5N+ and HC7N+, HC3N+ has an inverted ${}^{2}\Pi$ electronic
ground state with the $\Omega$=3/2 ladder lower in energy than the
$\Omega$=1/2 ladder (Gans et al. 2016). Consequently, the three rotational
transitions $J$=7/2-5/2, $J$=9/2-7/2, and $J$=11/2-9/2 that are within the
frequency coverage of QUIJOTE will exhibit significant hyperfine structure due
to electric and magnetic effects caused by the interaction between H and N
nuclear spin and electron orbital and electron spin angular momenta. As it
occurs for HC5N+, the $\Lambda$-doubling components are not expected to be
resolved at the spectral resolution of our observations due to the values of
$\Delta$(${}^{2}\Sigma-^{2}\Pi$) – the energy difference between the
${}^{2}\Sigma$ and the ${}^{2}\Pi$ electronic states – and $A_{SO}$ (the spin-
orbit constant), which are 1.914 eV (Leach et al.  2014) and $-$44$\pm$2 cm-1
(Gans et al. 2016), respectively. We predicted the rotational spectrum of
HC3N+ in the 30-50 GHz region using the theoretical values for the molecular
constants shown in Table 1. The predicted pattern for the most intense lines
consists of six lines with a separation of 1.2-1.6 MHz. This can be seen as a
triplet of doublets produced by the nitrogen and hydrogen nuclei,
respectively. We found a group of unidentified lines at 31.6 GHz, which is
only 15 MHz from the predicted frequencies of the $J$=7/2-5/2 transition and
shows a comparable hyperfine structure. The lines, with intensities ranging
from 0.5 mK to $\sim$1.2 mK, are mixed with stronger transitions of other
species such as HC13CCCCN or DC7N, as shown in Fig.1. The frequency separation
between the six unidentified lines is larger than predicted, which indicates
that the values of the $b_{F}$ constants of H and N nuclei must be larger too.
Another group of unidentified lines was found in the 40.6 GHz frequency
region, 30 megahertz from the frequency predicted for the $J$=9/2-7/2
transition, showing a similar pattern to the lines at 31.6 GHz. In this case,
the group of lines consists of five lines with intensities between 0.5 mK and
$\sim$1.6 mK that appear close to the transitions of the CH2CN radical. In
fact, it seems that one of the HC3N+ hyperfine components could be fully
blended with a strong line from CH2CN. Assuming that these two groups of lines
correspond to HC3N+, we accurately predicted the frequency for the
$J$=11/2-9/2 transition, and we also found several weak lines at 49.7 GHz.
This frequency region is the least sensitive of our QUIJOTE data, and those
lines are strongly affected by the noise. Nonetheless, the expected hyperfine
pattern can be easily distinguished, as seen in Fig.1. The derived
frequencies, intensities, and line widths of the observed lines are given in
Table 2.

The observed frequencies were analysed with the SPFIT program (Pickett et al.
1991) using an effective Hamiltonian for a linear molecule in a ${}^{2}\Pi$
electronic state. The employed Hamiltonian has the following form:

$\textbf{H}_{eff}=\textbf{H}_{rot}+\textbf{H}_{mhf}+\textbf{H}_{Q},$ (7)

where Hrot contains rotational and centrifugal distortion parameters,
$\textbf{H}_{mhf}$ represents the magnetic hyperfine coupling interaction term
due to the N and H nuclei, and $\textbf{H}_{Q}$ represents the nuclear
electric quadrupole interaction due to the N nucleus. The coupling scheme used
was F1 = J \+ I1 and F = F1 \+ I2, where I1 = I(H) and I2 = I(N).

We considered uncertainties of 10 kHz for the $J$=7/2-5/2 and $J$=9/2-7/2
rotational transitions, while the uncertainty assigned to the components of
the $J$=11/2-9/2 transition was 50 kHz, due to the lower signal-to-noise ratio
of the lines in the 49.6 GHz region. The results obtained from the fit are
shown in Table 1. We experimentally determined the values of the rotational
constant, $B$; the centrifugal distortion constant, $D$; the magnetic
hyperfine constants due to the nitrogen and hydrogen nuclei, $a$ and $b_{F}$;
and the nuclear electric quadrupole constant of the nitrogen nucleus, $eQq$.
The $c$ magnetic hyperfine constants of nitrogen and hydrogen nuclei were kept
fixed to the theoretical values calculated in this work. The standard
deviation of the fit is 18.2 kHz, about a half of the frequency resolution in
our observations.

The identification of HC3N+ as the carrier of the observed lines is
unequivocal. The experimental rotational constant $B$ deviates from the
theoretical value by just 0.03 %. The level of theory employed in this work,
RCCSD(T)-F12/cc-pCVTZ-F12, also reports very accurate values for the
rotational constants of the neutral counterpart HC3N molecule. Although the
prediction of the centrifugal distortion constant is usually less accurate,
the derived value from our fit also agrees very well with that predicted by
quantum chemical calculations. These two constants provide information on the
mass distribution of the molecular species, and thus, the good accordance
between the observed and calculated values could serve by itself to identify
the spectral carrier. However, the high resolution of our survey allowed us to
scrutinise the fine and/or hyperfine structure of the rotational transitions.
This structure depends strongly on the electronic environment of the molecular
species, which is related to the number of nuclei with a non-zero nuclear spin
and their relative positions. Hence, the fine and hyperfine coupling constants
determined are ideal fingerprints of the spectral carrier, providing key
information for its identification. As can be seen in Fig.1, the spectral
pattern for the three transitions is very well described by the employed
Hamiltonian, not only in terms of frequencies but also in relation to the
relative intensities of each hyperfine component. Regarding this point, it
should be noted that the hyperfine components may be affected by both negative
components produced when folding the frequency-switched data and emissions of
some other still unidentified chemical species. The limited number of
rotational transitions observed and the fact that only one spin component,
$\Omega$=3/2, is observed makes it so that only the values for two hyperfine
constants for each nuclei can be determined, and not very accurately.
Nevertheless, the values of $a$ and $b_{F}$ for nitrogen and hydrogen nuclei
are in agreement with those predicted theoretically. In addition, the $eQq$
constant for the nitrogen nucleus is also in accordance with that obtained by
quantum chemical calculations. The $eQq$ constant is a sensitive measure of
the electric field gradient in a molecule. The experimental value, $-$5.394
MHz, is larger than that found for the neutral counterpart HC3N ($eQq$=$-$4.31
MHz), and this is compatible with a cationic species with a terminal nitrogen.
This fact has been observed before for NNH+ (Caselli et al. 1995) and NC2NH+
(Gottlieb et al.  2000). In light of all of these arguments, we definitively
conclude that the carrier of the unidentified lines observed in TMC-1 is
HC3N+.

Using the derived molecular constants from Table 1 and adopting for HC3N+ the
dipole moment of 5.5 D derived in this work (see Table 1), we derived a
rotational temperature of 4.5$\pm$0.5 K and a column density of
(6.0$\pm$0.6)$\times$1010 cm-2. We assumed a source of uniform brightness with
a diameter of 80′′. The derived rotational temperature is not well
constrained, as the energies of the upper levels of the three observed
transitions range between 4 and 9 K. The derived column density is almost two
times smaller than the upper limit estimated by Cernicharo et al. (2024). The
derived rotational constant of HC3N+, $B\sim$4533 MHz, is 15 MHz below the
lower range explored for $B$ by Cernicharo et al. (2024).

Adopting the column density derived for HC3N of 1.9$\times$1014 cm-2 (Tercero
et al. 2024), the HC3N/HC3N+ abundance ratio is 3200$\pm$320, which is a
factor of five larger than the ratio between HC5N and HC5N+ and a factor of
three larger than that of HC7N and HC7N+ (Cernicharo et al. 2024). Adopting
the column density derived for C3N of 1.8$\times$1013 cm-2 (Cernicharo et al.
2020b), the C3N/HC3N+ abundance ratio is 300$\pm$30, while the C5N/HC5N+
abundance ratio was found to be 4.8$\pm$0.8 (Cernicharo et al. 2024). These
results indicate that there are significant differences in the formation and
destruction paths of these cations. The cation HC3N+ can be seen as the
protonated form of C3N, whose proton affinity has been calculated to be 772 kJ
mol-1. Even though this value is a bit smaller than those calculated for C5N
and C7N, the reactions of the proton transfer from abundant cations such as
HCO+ are exothermic and can provide a formation pathway to HC3N+ in TMC-1.
Nevertheless, according to our previous chemical modelling calculations
(Cernicharo et al. 2024), HC3N+ is mostly formed by the reactions H2 \+ C3N
and H+ \+ HC3N. The abundance calculated in that work for HC3N+ is low, about
two orders of magnitude lower than the value derived here, due to the non-
negligible reactivity with H2 (Knight et al. 1985). Therefore, it would be
interesting to investigate the low-temperature chemical kinetics of HC3N+,
HC5N+, and HC7N+ with H2 in order to better understand the formation and
survival of these highly unstable radical cations in TMC-1. The cation HCN+
(${}^{2}\Pi_{i}$) is worth being searched for, but its first transition,
$J$=5/2-3/2, is outside the frequency ranges of our surveys.

##  5 Conclusions

We report the discovery of the radical cation HC3N+ in TMC-1. Thanks to the
sensitive high spectral resolution observations of the QUIJOTE survey, we have
observed three rotational transitions for this molecule. The transitions
reveal a complex hyperfine structure, which was fully analysed and assigned to
HC3N+, allowing us to derive its rotational spectroscopic parameters for the
first time. This represents an invaluable alternative to overcome the lack of
laboratory experimental data for very unstable molecules. The column density
derived for HC3N+ is (6.0$\pm$0.6)$\times$1010 cm-2, and the rotational
temperature is 4.5$\pm$1 K. The abundance ratio between HC3N and HC3N+ is
3200$\pm$320.

###### Acknowledgements.

We thank Ministerio de Ciencia e Innovación of Spain (MICIU) for funding
support through projects PID2019-106110GB-I00, and PID2019-106235GB-I00. We
also thank ERC for funding through grant ERC-2013-Syg-610256-NANOCOSMOS and
the Consejo Superior de Investigaciones Científicas (CSIC; Spain) for funding
through project PIE 202250I097. C.C., M.A, and J.C. thank Ministerio de
Ciencia e Innovación of Spain (MICIU) for funding support through projects
PID2019-106110GB-I00, PID2019-107115GB-C21 / AEI / 10.13039/501100011033, and
PID2019-106235GB-I00. The present study was also supported by Ministry of
Science and Technology of Taiwan and Consejo Superior de Investigaciones
Científicas under the MoST-CSIC Mobility Action 2021 (Grants 11-2927-I-A49-502
and OSTW200006).

## References

  * Agúndez & Wakelam (2013) Agúndez, M. & Wakelam, V. 2013, Chem. Rev., 113, 8710
  * Agúndez et al. (2022) Agúndez, M., Cabezas, C., Marcelino, N. et al. 2022, A&A, 659, L9
  * Agúndez et al. (2023) Agúndez, M., Cabezas, C., Marcelino, N. et al. 2023, A&A, 669, L1
  * Becke (1993) Becke, A. D. 1993, J. Chem. Phys., 98, 1372
  * Cabezas et al. (2022a) Cabezas, C., Agúndez, M., Marcelino, N., et al. 2022a, A&A, 657, L4
  * Cabezas et al. (2022b) Cabezas, C., Agúndez, M., Marcelino, N. et al. 2022b, A&A, 659, L8
  * Caselli et al. (1995) Caselli, P., Myers, P. C., & Thaddeus, P., 1995, ApJ, 455, L77
  * Cernicharo et al. (2020a) Cernicharo, J., Marcelino, N., Agúndez, M. et al. 2020a, A&A, 642, L17
  * Cernicharo et al. (2020b) Cernicharo, J., Marcelino, N., Agúndez, M. et al. 2020b, A&A, 641, L9
  * Cernicharo et al. (2020c) Cernicharo, J., Marcelino, N., Agúndez, M. et al. 2020c, A&A, 642, L8
  * Cernicharo et al. (2021a) Cernicharo, J., Cabezas, C., Endo, Y. et al. 2021a, A&A, 646, L3
  * Cernicharo et al. (2021b) Cernicharo, J., Cabezas, C., Bailleux, S. et al. 2021b, A&A, 646, L7
  * Cernicharo et al. (2021c) Cernicharo, J., Agúndez, M., Kaiser, R., et al. 2021c, A&A, 652, L9
  * Cernicharo et al. (2022a) Cernicharo, J., Agúndez, M., Cabezas, C. et al. 2022a, A&A, 657, L16
  * Cernicharo et al. (2022b) Cernicharo, J., Fuentetaja, R., Agúndez, M. et al. 2022b, A&A, 663, L9
  * Cernicharo et al. (2024) Cernicharo, J., Cabezas, C., Agúndez, M., et al. 2024, A&A, 686, L15
  * Dai et al. (2015) Dai, Z., Sun, W., Wang, J. & Mo, Y. 2015, J. Chem. Phys., 143, 054301
  * Desrier et al. (2016) Desrier, A., Romanzin, C., Lamarre, N., et al. 2016, J. Chem. Phys., 145, 234310
  * Frisch et al. (2016) Frisch M. J., et al. 2016, Gaussian 16 Revision A.03
  * Frosch & Foley (1952) Frosch, R. A. & Foley, H. M., Phys. Rev., 1952, 88, 1337
  * Gans et al. (2016) Gans, B., Lamarre, N., Broquier, M. et al. 2016, J. Chem. Phys., 145, 234309
  * Gottlieb et al. (2000) Gottlieb, C. A., Apponi, A. J., McCarthy, M. C., et al. 2000, J. Chem. Phys., 113, 1910
  * Hill et al. (2010) Hill, J. G., Mazumder, S., Peterson, K. A., 2010, J. Chem. Phys., 132, 054108
  * Kaifu et al. (2004) Kaifu, N., Ohishi, M., Kawaguchi, K., et al., 2004, Publ. Astron. Soc. Japan, 56, 69
  * Knight et al. (1985) Knight, J. S., Freeman, C. G., McEwan, M. J. et al 1985, Int. J. Mass Spectrom. Ion Proc., 67, 317
  * Knizia et al. (2009) Knizia, G., Adler, T. B., Werner. H.-J., 2009 J. Chem. Phys. 130, 054104
  * Leach et al. (2014) Leach, S., A. García, G., Mahjoub, A. 2014, J. Chem. Phys., 140, 174305
  * Marcelino et al. (2020) Marcelino, N., Agúndez, M., Tercero, B. et al. 2020, A&A, 643, L6
  * Pickett et al. (1991) Pickett, H.M. 1991, J. Mol. Spectrosc., 148, 371
  * Silva et al. (2023) Silva, W. G. D. P., Cernicharo, J., Schlemmer, S. et al. 2023, A&A, 676, L1
  * Steenbakkers et al. (2023) Steenbakkers, K., N. Marimuthu, A., Redlich, B., et al. 2023, J. Chem. Phys., 158, 084305
  * Tercero et al. (2021) Tercero, F., López-Pérez, J. A., Gallego, J. D., et al. 2021, A&A, 645, A37
  * Tercero et al. (2024) Tercero, B., Marcelino, N., Roueff, E. et al. 2024, A&A, 682, L12
  * Townes & Schawlow (1975) Townes, C. H. & Schawlow, A. L., 1975, Microwave Spectroscopy (Dover, New York)
  * Werner et al. (2024) Werner H.-J., Knowles, P. J., Knizia G., et al., MOLPRO, version 2024.1, a package of ab initio programs, 2024, see https://www.molpro.net
  * Woon & Dunning (1993) Woon, D. E., & Dunning, T. H., Jr., 1993, J. Chem. Phys., 98, 1358
  * Zhang et al. (2012) Zhang, Y., Guo, J., Zhang, J. 2012, Int. J. Mass Spectom., 309, 56