# Multi-facet Universal Schema

Rohan Paul Haw-Shiuan Chang11footnotemark: 1 Andrew McCallum
CICS, University of Massachusetts Amherst
<EMAIL_ADDRESS>
* indicates equal contribution

###### Abstract

Universal schema (USchema) assumes that two sentence patterns that share the
same entity pairs are similar to each other. This assumption is widely adopted
for solving various types of relation extraction (RE) tasks. Nevertheless,
each sentence pattern could contain multiple facets, and not every facet is
similar to all the facets of another sentence pattern co-occurring with the
same entity pair. To address the violation of the USchema assumption, we
propose multi-facet universal schema that uses a neural model to represent
each sentence pattern as multiple facet embeddings and encourage one of these
facet embeddings to be close to that of another sentence pattern if they co-
occur with the same entity pair. In our experiments, we demonstrate that
multi-facet embeddings significantly outperform their single-facet embedding
counterpart, compositional universal schema (CUSchema) (Verga et al., 2016),
in distantly supervised relation extraction tasks. Moreover, we can also use
multiple embeddings to detect the entailment relation between two sentence
patterns when no manual label is available.

##  1 Introduction

Relation extraction (RE) is a crucial step in automatic knowledge base
construction (AKBC). A major challenge of RE is that the frequency of
relations in the real world is a long-tail distribution but collecting
sufficient human annotations for every relation is infeasible (Han et al.,
2020).

Distant supervision is proposed to alleviate the issue (Mintz et al., 2009).
Distant supervision assumes that a sentence pattern expresses a relation if
the sentence pattern co-occurs with an entity pair and the entity pair has the
relation. For example, we assume the sentence pattern “$ARG1, the partner of
fellow $ARG2” is likely to express the spouse relation if we observe a text
clip “… Angelina Jolie, the partner of fellow Brad Pitt …” in our training
corpus and a knowledge base tells us that Angelina Jolie and Brad Pitt has the
spouse relation. Accordingly, we can infer that another entity pair is likely
to have the spouse relation if we observe the text “, the partner of fellow”
between them in a new corpus.

Figure 1: Comparison between the multi-facet and compositional universal
schema. In our training loss, we encourage one of the facet embeddings from a
sentence pattern to be similar to its co-occurred entity pair.

Universal schema (Riedel et al., 2013) extends this assumption by treating
every sentence pattern as a relation, which means we assume that sentence
patterns or relations in a knowledge base are similar if they co-occur with
the same entity pair. For example, we assume “$ARG1, the partner of fellow
$ARG2” and “$ARG1, the wife of fellow $ARG2” are similar if they both co-occur
with (Kristen Bell, Dax Shepard). Consequently, we can infer that “$ARG1, the
wife of fellow $ARG2” also implies spouse relation as “$ARG1, the partner of
fellow $ARG2” even if the knowledge base does not record the spouse relation
between Kristen Bell and Dax Shepard.

Compositional universal schema (Verga et al., 2016) realizes the idea by using
a LSTM (Hochreiter and Schmidhuber, 1997) to encode each sentence pattern into
an embedding and encouraging the embedding to be similar to the embedding of
the co-occurred entity pair. As in the lower part of Figure 1, the model makes
the embeddings of two sentence patterns similar if they co-occur with the same
entity pair. Baldini Soares et al. (2019) rely on a similar assumption and
achieve state-of-the-art results on supervised RE tasks by replacing the LSTM
with a large pre-trained language model.

Figure 2: An illustration of the proposed method. The training signal comes
from the co-occurrence matrices of the KB and training text corpus on the
right. On the lower left, we visualize our neural encoder, which captures the
compositional meaning of tokens in the sentence pattern, and our neural
decoder, which models the dependency among multiple facet embeddings. When a
sentence pattern co-occurs with an entity pair, the training loss minimizes
the distance between the entity pair embedding and the closest facet embedding
of the sentence pattern (e.g., 0.2 between $s_{i,2}$ and $e_{1}$). Trainable
parameters in our model are highlighted using red borders. On the upper left,
we visualize the embedding space to establish the connection between our
method and clustering.

The variants of universal schema have many different applications, including
multilingual RE Verga et al. (2016), knowledge base construction Toutanova et
al. (2015); Verga et al. (2017), question answering Das et al. (2017),
document-level RE Verga et al. (2018), N-ary RE Akimoto et al. (2019), open
information extraction Zhang et al. (2019), and unsupervised relation
discovery Percha and Altman (2018).

Nevertheless, one sentence pattern could contain multiple facets, and each
facet could imply a different relation. In Figure 1, “$ARG1, the partner of
fellow $ARG2” could imply the entity pair has the spouse relation, the co-
worker relation, or both. “$ARG1 moved in with $ARG2” could imply the spouse
relation, the parent relation, …, etc. If we squeeze the facets of a sentence
pattern into a single embedding, the embedding is more likely to be affected
by the irrelevant facets from other patterns co-occurred with the same entity
pair (e.g., “$ARG1 moved in with $ARG2” might incorrectly imply the co-worker
relation).

Another limitation is that single embedding representation can only provide
symmetric similarity measurement between two sentence patterns. Thus, an open
research challenge is to predict the entailment direction of two sentence
patterns only based on their co-occurring entity pair information.

To overcome the challenges, we propose multi-facet universal schema, where we
assume that two sentence patterns share a similar facet if they co-occur with
the same entity pair. As in Figure 1, we use a neural encoder and decoder to
predict multiple facet embeddings of each sentence pattern and encourage one
of the facet embeddings to be similar to the entity pair embedding. As a
result, the facets that are irrelevant to the relation between the entity
pairs are less likely to affect the embeddings of entity pairs and other
related sentence patterns. For example, the parent facet of “$ARG1 moved in
with $ARG2” could be excluded when updating the embeddings of (Angelina Jolie,
Brad Pitt).

In our experiments, we first compare the multi-facet embeddings with the
single-facet embedding in distantly supervised RE tasks. The results
demonstrate that multiple facet embeddings significantly improve the
similarity measurement between the sentence patterns and knowledge base
relations. Besides RE, we also apply multi-facet embeddings to unsupervised
entailment detection tasks. In a newly collected dataset, we show that multi-
facet universal schema significantly outperforms the other unsupervised
baselines.

##  2 Methods

Our method is illustrated in Figure 2. In Section 2.1, we first provide our
problem setup: We are given a knowledge base (KB) and a text corpus during
training. Our goal is to extract relations by measuring the similarity between
KB relations and an (unseen) sentence pattern or to detect entailment between
two sentence patterns. In Section 2.2, we introduce our neural model, which
predicts multi-facet embeddings of each sentence pattern. Next, in Section
2.3, we describe our objective function, which encourages the embeddings of
co-occurred entity pairs to be close to the embeddings of their closest
pattern facets. Finally, in Section 2.4, we explain that multi-facet
embeddings could be viewed as the cluster centers of possibly co-occurred
entity pairs, and in Section 2.5, we provide our scoring functions for
distantly supervised RE and unsupervised entailment tasks.

###  2.1 Background and Problem Setup

Our RE problem setup is the same as compositional universal schema (Verga et
al., 2016). First, we run named entity recognition (NER) and entity linking on
a raw corpus. After identifying the entity pairs in each sentence, we prepare
a co-occurrence matrix as in Figure 2. Similarly, we represent the KB
relations between entity pairs as a co-occurrence matrix and merge the
matrices from the KB and the training corpus. The merged matrix has
$y_{i,j}=1$ if the $i$th sentence pair or KB relation co-occurs with the $j$th
entity pair and $y_{i,j}=0$ otherwise.

During testing, we use NER to extract an entity pair and the sentence pattern,
which might not have been seen in the training corpus. Next, we extract
relations by computing the similarity between the sentence pattern embeddings
and the embeddings of the applicable KB relations. Besides RE, we also detect
the entailment between two sentence patterns by comparing their embeddings.

###  2.2 Neural Encoder and Decoder

We use a neural model to predict $K$ facet embeddings of each sentence
pattern. The goal is similar to Chang et al. (2021), which predict a fixed
number of embeddings of a sentence, so we adopt their neural model as shown in
Figure 2.

For the $i$th sentence pattern $S_{i}$, we append an <eos> to its end and use
a 3-layer Transformer Vaswani et al. (2017) encoder $TE$ to model the
compositional meaning of the input word sequence:
$\underline{{\bm{u}}}_{i,1}...\underline{{\bm{u}}}_{i,|S_{i}|}\underline{{\bm{u}}}_{i,\text{\textless{eos}\textgreater{}}}=TE(S_{i}\text{\textless{eos}\textgreater{}}),$
where $\underline{{\bm{u}}}_{i,l}$ is an embedding contextualized by the
encoder. In the experiment, we also replace the Transformer with a
bidirectional LSTM (bi-LSTM) to show that the improvement of multi-facet
embeddings is independent of the encoder choice.

The embedding $\underline{{\bm{u}}}_{i,\text{\textless{eos}\textgreater{}}}$
represents the whole sentence pattern; we use $K$ different linear layers
$L^{d}_{k}$ to transform the embedding into the inputs of our decoder:
$\underline{{\bm{b}}}_{i,k}=L^{d}_{k}(\underline{{\bm{u}}}_{i,\text{\textless{eos}\textgreater{}}})$.

The facets in a sentence pattern often have some dependency. For example, the
patterns that express the partnership between two people might also express
the collaboration relation between two companies. To leverage the dependency,
we use another 3-layer Transformer as our decoder $TD$. Besides the self-
attention, we allow the hidden states in the decoder to query the
contextualized word embeddings $\underline{{\bm{u}}}_{i,l}$ from the encoder
Vaswani et al. (2017) and output the embeddings corresponding to the different
facets $\underline{{\bm{d}}}_{i,k}$:
$\underline{{\bm{d}}}_{i,1}...\underline{{\bm{d}}}_{i,K}=TD(\underline{{\bm{b}}}_{i,1}...,\underline{{\bm{b}}}_{i,K},\underline{{\bm{u}}}_{i,1}...\underline{{\bm{u}}}_{i,\text{\textless{eos}\textgreater{}}}).$
Notice that we do not use autoregressive decoding as in Vaswani et al. (2017),
so our decoder could also be viewed as another encoder with attention to the
output of the encoder $TE$. Finally, to convert the hidden state size to the
entity embedding size, we let the outputs of decoder go through another linear
layer $L^{o}$ to get the facet embedding (i.e., sentence pattern embedding):
$\underline{{\bm{s}}}_{i,k}=L^{o}(\underline{{\bm{d}}}_{i,k})$.

###  2.3 Objective Function

When measuring the distance between the $j$th entity pair and the $i$th
sentence pattern, we compute the Euclidean distance between the entity pair
embedding $\tilde{\underline{{\bm{e}}}}_{j}$ and its closest facet embedding
of the $i$th sentence pattern. The distance is defined as

$\displaystyle
D(\\{\underline{{\bm{s}}}_{i,k}\\}_{k=1}^{K},\tilde{\underline{{\bm{e}}}}_{j})=\min_{k=1}^{K}\min_{0\leq\eta_{k}\leq
1}||\tilde{\underline{{\bm{e}}}}_{j}-\eta_{k}\underline{{\bm{s}}}_{i,k}||^{2},$
(1)

where the entity pair embedding is normalized (i.e.,
$||\tilde{\underline{{\bm{e}}}}_{j}||=1$). During testing, we ignore the
magnitude of facet embeddings, so we use $\eta_{k}$ to eliminate the magnitude
of facet embeddings $\underline{{\bm{s}}}_{i,k}$ during training. We do not
allow negative $\eta_{k}$ to prevent the gradient flow from pushing
$\underline{{\bm{s}}}_{i,k}$ toward the inverse direction of
$\tilde{\underline{{\bm{e}}}}_{j}$ and we ensure $\eta_{k}\leq 1$ to avoid the
neural model from outputting $\underline{{\bm{s}}}_{i,k}$ with a very small
magnitude.

As in Figure 2, we minimize the distance
$D(\\{\underline{{\bm{s}}}_{i,k}\\}_{k=1}^{K},\tilde{\underline{{\bm{e}}}}_{j})$
in our loss function when the $i$th sentence pair co-occurs with the $j$th
entity pair (i.e., $y_{i,j}=1$). For negative samples (i.e., $y_{i,j}=0$), we
maximize the distance instead. That is, the major term of our loss function is
defined as

$\displaystyle\sum_{(i,j)\in R}(2\cdot
y_{i,j}-1)r_{i,j}D(\\{\underline{{\bm{s}}}_{i,k}\\}_{k=1}^{K},\tilde{\underline{{\bm{e}}}}_{j}),$
(2)

and the other regularization term $\Omega$ in the loss function will be
described in the appendix. $R$ is a set that includes all positive and
negative samples. Positive samples are $(i,j)$ such that $y_{i,j}=1$ and the
negative samples are constructed by pairing a randomly selected sentence
pattern with the $j$th entity pair. To balance the influence of popular entity
pairs (i.e., entity pairs that co-occur with many sentence patterns) and rare
entity pairs on our model, we set the weight of each pair,
$r_{i,j}\propto\frac{1}{\sum_{i}{y_{i,j}}}$ and $\frac{\sum_{(i,j)\in
R}r_{i,j}}{|R|}=1$.

We generate the embeddings for KB relations in a similar way. We use a single
token to represent the relation and append an <eos> (e.g., _per:spouse_ <eos>)
to form the input of our neural model. The KB relations usually co-occur with
more entity pairs, so we set the number of facet embeddings for KB relations
$K_{rel}$ to be larger than the number of facet embeddings for sentence
patterns $K$.

###  2.4 Connection to Clustering

If a sentence pattern contains multiple facets that describe different
relations between the entity pairs, the pattern often co-occurs with different
kinds of entity pairs. For example, “$ARG1 ’s partner $ARG2” in Figure 2 could
express the collaboration relationship between two companies or the
partnership between two people, so the sentence patterns could co-occur with
two companies such as (Google, Facebook) and two people such as (Bob Bryan,
Mike Bryan).

Different kinds of entity pairs often have very different embeddings, so we
could discover the facets of sentence patterns by clustering the embeddings of
entity pairs. Here, a facet refers to a mode of the embedding distribution of
the entity pairs that could possibly co-occur with the sentence pattern. A
facet could be represented by multiple facet embeddings and each facet
embedding corresponds to a cluster center of the entity pair embeddings.
Hence, although the number of facet embeddings $K$ is fixed for all the
sentence patterns, our model can capture the facets of the sentence patterns
well when the number of facets is less than $K$.

In equation 1, we choose the closest facet embedding of the sentence pattern
for each co-occurring entity pair embedding and minimize their distance. For
example, $\underline{{\bm{s}}}_{i,2}$ and the embedding of (Bob Bryan, Mike
Bryan) are pulled closer in Figure 2. Minimizing equation 1 by passing the
gradient through the scaled facet embedding
$\eta_{k}\underline{{\bm{s}}}_{i,k}$ is the same as minimizing a Kmeans loss,
so the loss term induced by positive sample pairs encourage each
$\underline{{\bm{s}}}_{i,k}$ to become the cluster center of its nearby co-
occurring entity pair embeddings. The details of our training algorithm could
be found in the appendix.

The co-occurrence matrices in RE tasks are usually extremely sparse, and most
of the sentence patterns only co-occur with a few entity pairs, which makes it
difficult to derive multiple high-quality embeddings by clustering the co-
occurring entity pair embeddings as in multi-sense word embedding methods such
as Neelakantan et al. (2014). The proposed method solves this sparsity
challenge by predicting the cluster centers using a neural model. For
instance, even if “$ARG1 ’s partner $ARG2” does not co-occur with many entity
pairs, its embeddings are encouraged to be close to the embeddings of entity
pairs that co-occur with other similar patterns (e.g., “$ARG1 and her partner
$ARG2”).

Figure 3: Comparison of the asymmetric similarities.
$Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})>Asym(\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\},\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\})$
because the average cosine distance on the left is smaller than that on the
right.

###  2.5 Scoring Functions

In compositional universal schema, the similarity between the $i$th and $j$th
sentence patterns are measured by the symmetric cosine similarity
$\tilde{\underline{{\bm{s}}}}^{T}_{i,1}\tilde{\underline{{\bm{s}}}}_{j,1}$,
where
$\tilde{\underline{{\bm{s}}}}_{i,1}=\frac{\underline{{\bm{s}}}_{i,1}}{||\underline{{\bm{s}}}_{i,1}||}$.
When using multiple embeddings to represent a sentence pattern, we can compute
the asymmetric similarity as

$\displaystyle
Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})=\frac{\sum_{m=1}^{K}\max\limits_{k=1}^{K}(\tilde{\underline{{\bm{s}}}}^{T}_{i,k}\tilde{\underline{{\bm{s}}}}_{j,m})}{K}.$
(3)

In an example of Figure 3, a red square $\tilde{\underline{{\bm{s}}}}_{i,k}$
is close to all the blue points, which leads to a high
$Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})$.

Between two sentence patterns with entailment relation, we empirically find
that the embeddings of a premise (the more specific pattern) often have some
facet embeddings that are far away from all the embeddings of its hypothesis
(the more general pattern). Relying on the tendency, we could detect the
direction of the entailment relation. For example, the $i$th sentence pattern
(red squares) in Figure 3 is more likely to be premise if the $i$th and $j$th
(blue circles) sentence patterns have an entailment relation.

We suspect the reason is that more specific patterns could contain more words
that are similar to the words of other patterns expressing different
relations. For example, “$ARG1 , the wife of fellow $ARG2” have a facet
embedding for spouse relation and another facet embedding for the co-worker
relation because the pattern has high word overlapping with “$ARG1 , the wife
of $ARG2” and “$ARG1 and her fellow $ARG2”. Another possible reason is that
the articles in our corpus tend to use more specific patterns to express the
relation between a pair of entities Shwartz et al. (2017).

When performing RE, we compute the symmetric similarity between $i$th sentence
pattern and $j$th KB relation
$Sim(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})$
by

$\displaystyle\frac{Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})+Asym(\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\},\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\})}{2}.$
(4)

##  3 Experiments

We primarily compare our method with compositional universal schema (CUSchema)
(Verga et al., 2016) because CUSchema is one of the state-of-the-art RE
methods in the small model regime (without using large pre-trained language
models) Chang et al. (2016); Chaganty et al. (2017).111We have not yet applied
the multi-facet embeddings approach to the models that rely on a large
pretrained language model (LM) Baldini Soares et al. (2019) due to
computational and evaluation considerations. Computationally speaking,
training state-of-the-art models requires intensive GPU resources. Besides, a
smaller model size might be desired when we need to construct a knowledge base
from a large corpus in real time. Moreover, there is no existing pretrained LM
in some domains Zhang et al. (2019), and training the LM in a new domain from
scratch requires even more GPU resources. In terms of the evaluation
consideration, our method is an improvement over CUSchema, so we want to
compare it with CUSchema fairly. Furthermore, evaluating entailment between
two full sentences is more difficult than between the sentence patterns, and
we are not aware of a LM-based model that only considers the text between the
entity pairs.

In Section 3.1, we visualize and analyze the facet embeddings. Next, we use
distant-supervised RE tasks to evaluate our symmetric similarity measurement
in Section 3.2, and detect entailment between sentence patterns to evaluate
our asymmetric similarity measurement in Section 3.3.

Figure 4: Facet embedding visualization of Ours (Single-Trans) on the left and
Ours (Trans) on the right. Dots are the facet embeddings outputted by our
models and crosses are their nearby entity pair embeddings

###  3.1 Embedding Visualization

We visualize the embeddings of sentence patterns and a KB relation from the
single embedding model and multi-facet embedding model that perform the best
in the RE tasks (i.e., Ours (Single-Trans) and Ours (Trans) in Table 1). We
project the facet embeddings to a 2-dimensional space using multidimensional
scaling (MDS) Borg and Groenen (2005) and visualize the embeddings of one KB
relation and three related sentence patterns in Figure 4. The three sentence
patterns are selected from our validation set, so the model is not aware of
the entity pairs that actually co-occur with the patterns during training. For
each facet embedding, we show two among five of its closest entity pairs to
visualize the meaning of the embedding space.222Notice that our training
signal is sparse and noisy and the projection does not necessarily preserve
the original distances, so the entity pairs with similar relations might be
relatively far away from each other.

In the single embedding model, the embedding of org:city_of_headquarter is
close to the embedding of (school, location) while “$ARG1 headoffice in $ARG2”
is close to (company, location) and “$ARG1 headquarter in $ARG2”.

In the multi-facet embedding model, some embeddings of org:city_of_headquarter
are closer to (school, location) and others are closer to (company, location).
In addition to these entity pairs, “$ARG1 headoffice in $ARG2” and “$ARG1
headquarter in $ARG2” also co-occur with (people, location) and
(people/organization, year). Using the visualization of multi-facet embedding,
we can understand which facets of org:city_of_headquarter are similar or
dissimilar to “$ARG1 headoffice in $ARG2”, which cannot be done if all facets
are averaged into a single embedding as in the traditional models.

The facet embeddings of “$ARG1 is now at $ARG2” are close to (people,
organization) where the organization could be school, sports team, and
company. Using multiple embeddings could avoid enforcing the closeness of
these entity pairs with different relations. The results also indicate that
our model can output reasonable cluster centers despite learning from the
sparse and noisy training data. Finally, we can see that if a sentence pattern
has fewer facets than $K$, our model learns to output some very similar facet
embeddings, which makes the performance less sensitive to the setting of $K$.

###  3.2 Relation Extraction

We follow the same training data and testing protocol in compositional
universal schema (CUSchema) (Verga et al.,
2016)333https://github.com/patverga/torch-relation-extraction to highlight the
benefit of predicting multiple facet embeddings, and the relation extraction
step in TAC KBP slot-filling tasks is used to compare the different models.

Method | TAC 2012 (Validation) | TAC 2013 | TAC 2014
---|---|---|---
Prec | Recall | F1 | Prec | Recall | F1 | Prec | Recall | F1
USchema* | 34.8 | 23.7 | 28.2 | 42.6 | 29.4 | 34.8 | 35.5 | 24.3 | 28.8
CUSchema (LSTM)* | 27.0 | 32.7 | 29.6 | 39.6 | 32.2 | 35.5 | 32.9 | 27.3 | 29.8
Ours (Single-LSTM) | 25.7 | 21.7 | 23.5 | 30.4 | 26.3 | 28.2 | 22.1 | 20.5 | 21.3
Ours (Single-Trans) | 26.1 | 21.6 | 23.7 | 29.5 | 25.2 | 27.2 | 19.0 | 21.2 | 20.0
Ours (LSTM) | 32.0 | 28.9 | 30.3 | 41.3 | 33.9 | 37.2 | 34.1 | 29.5 | 31.6
Ours (Trans) | 33.6 | 27.7 | 30.4 | 42.5 | 33.2 | 37.3 | 34.6 | 28.5 | 31.3
USchema + CUSchema (LSTM)* | 29.3 | 32.8 | 30.9 | 41.9 | 34.4 | 37.7 | 29.3 | 34.1 | 31.5
USchema + Ours (LSTM) | 29.2 | 33.7 | 31.3 | 38.1 | 38.9 | 38.5 | 31.5 | 34.4 | 32.9
USchema + Ours (Trans) | 30.4 | 33.9 | 32.1 | 39.0 | 38.8 | 38.9 | 32.0 | 34.0 | 33.0

Table 1: Distantly supervised relation extraction using different versions of
the universal schema. All numbers are %. CUSchema refers to compositional
universal schema. Trans is an abbreviation of Transformer. The best scores of
the single models and ensemble models are highlighted. *The performance of TAC
2013 and 2014 are copied from Verga et al. (2016).

Setup: The training data for our RE models are prepared by distant supervision
without requiring any manually labeled data. The relations in Freebase
(Bollacker et al., 2008) are mapped to TAC relations (e.g.,
org:city_of_headquarter) and the NER tagger and entity linker are run in a raw
text corpus. Then, the training data is cleaned using the methods in Roth et
al. (2013).

During testing, we are given a query containing the head entity and a query
TAC relation in the slot-filling tasks, and the goal is to extract the tail
entity from the candidate sentences. The NER tagger and query expansion are
used to gather the candidate sentence patterns, and we compute the similarity
scores from different models between the candidate sentence patterns and query
relation. Finally, we compare the extracted second entity with the ground
truth using exact string matching and report the precision, recall, and F1
scores.

Following Verga et al. (2016), we use TAC 2012 as our validation set to
determine the threshold score for each TAC relation. Each model’s
hyperparameters are tuned separately using the validation set (TAC 2012) to
ensure a fair comparison.

We compare the following methods:
Ours (Trans): Our method that measures the similarity between the sentence
pattern $\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\}$ and TAC relation
$\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\}$ using
$Sim(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})$
in equation 4. Trans is an abbreviation of the Transformer encoder. We set
$K=5$ and $K_{rel}=11$ based on the validation set.
Ours (LSTM): The same as Ours (Trans) except that we use bi-LSTM as our
encoder instead.
Ours (Single-*): Our methods that use single facet embedding to represent each
sentence pattern or KB relation. When setting $K=K_{rel}=1$, our decoder
becomes the interleaving feedforward layers and cross-attention layers
attending to the output embeddings of the encoder.
CUSchema (LSTM): Compositional universal schema (Verga et al., 2016). The
method is similar to Ours (Single-LSTM) but uses a different loss function,
neural architecture (no decoder), and hyperparameter search procedure.
USchema: Universal schema (Riedel et al., 2013) estimates every sentence
pattern embedding by factorizing the co-occurrence matrices (i.e., replacing
the bi-LSTM in CUSchema with a look-up table).
USchema + *: Verga et al. (2016) show that taking the maximal similarity
between USchema and CUSchema model improves the F1. We also apply the same
merging procedure to our model.

Results: In Table 1, the proposed method Ours (Trans) significantly outperform
CUSchema (LSTM) before and after combining with universal schema. As far as we
know, our proposed multi-facet embedding is the first method that outperforms
compositional universal schema using the same training signal in the distant-
supervised RE benchmark they proposed.

Although the recall of USchema is low because it does not exploit the
similarity between the patterns (e.g., “$ARG1 happily married $ARG2” is
similar to “$ARG1 married $ARG2”), USchema has a high precision because it
also won’t be misled by the similarity (e.g., “$ARG1, and his wife $ARG2”
expresses the spouse relation but “$ARG1, his wife, and $ARG2” does not)
(Verga et al., 2016). Thus, combining USchema and Ours (Trans) leads to the
best performance.

Ours (Trans) and Ours (LSTM) perform similarly. Furthermore, Ours (LSTM)
performs much better than Ours (Single-LSTM), which demonstrates the
effectiveness of using multiple embeddings. Notice that multiple facet
embeddings could improve the performance even after the training data have
been cleaned. This indicates that our method is complementary to the noise
removal methods in Roth et al. (2013).

Premise (Specific Pattern) | Hypothesis (General Pattern) | Label | Ours | CUSchema | Ours Diff | Freq Diff
---|---|---|---|---|---|---
$ARG1 , the president of the $ARG2 | $ARG1 , the leader of the $ARG2 | Entailment | 0.98 | 0.94 | + | +
$ARG1 ’s chairman , $ARG2 | $ARG1 ’s leader , $ARG2 | Entailment | 0.95 | 0.87 | + | -
$ARG1 ’s father , $ARG2 | $ARG1 ’s leader , $ARG2 | Other | 0.08 | 0.52 | NA | NA
$ARG1 worked with $ARG2 | $ARG1 deal with $ARG2 | Entailment | 0.92 | 0.83 | + | -
$ARG1 had with $ARG2 | $ARG1 deal with $ARG2 | Other | 0.96 | 0.88 | NA | NA
$ARG1 said the $ARG2 | $ARG1 say the $ARG2 | Paraphrase | 0.93 | 0.92 | NA | NA

Table 2: Example of sentence pattern pairs, its label, and our predictions in
our entailment experiment. Ours and Ours Diff are the predictions from Ours
(Trans). Freq Diff is the frequency difference baseline.

###  3.3 Entailment Detection

Entailment is a common and fundamental relation between two sentence patterns.
Some examples could be seen in Table 2. Unsupervised hypernym detection (i.e.,
entailment at the word level) is extensively studied Shwartz et al. (2017),
but we are not aware of any previous work on unsupervised entailment detection
at the sentence level, nor any existing entailment dataset between sentence
patterns. Thus, we create one.

Dataset Creation: We use WordNet (Miller, 1998) to discover the entailment
candidates of sentence pattern pairs and manually label the candidates. For
each sentence pattern in the training data of Verga et al. (2016), we replace
one word at a time with its hypernym based on the WordNet hierarchy. The two
sentence patterns before and after replacement form an entailment candidate.

We label 1,500 pairs of the most popular sentence pattern, which co-occurs
with the highest number of unique entity pairs. Each candidate could be
labeled as entailment, paraphrase, or other. Finally, around 20% of the
candidates are randomly chosen to form the validation set, and the rest are in
the test set. More details of the dataset creation process could be seen in
the appendix

In this dataset, only 22% and 10% of candidates are labeled as entailment and
paraphrase, respectively. This suggests that entailment relation between two
sentence patterns is hard to be inferred by only the hypernym relation (i.e.,
entailment relation at the word level) in WordNet.

Setup: We evaluate entailment detection using the typical setup and metrics in
hypernym detection (Shwartz et al., 2017). Negative examples include the
candidates labeled as paraphrases and others. We compare the average precision
of different methods (i.e., AUC in the precision-recall curve) (Hastie et al.,
2009). In addition, we predict the direction of entailment relation in each
pair (i.e., which pattern is the premise) and report the accuracy. Many
hypotheses have the same hypernyms such as the leader in Table 2, so we also
report the macro accuracy of direction detection averaged across every
hypernym in the hypotheses.

The task is challenging because all the candidates have a word-level
entailment relation if their compositional meaning is ignored. Furthermore, we
cannot infer the entailment direction based on the tendency that longer
sentence patterns tend to be more specific because most of the candidate pairs
in this dataset have the same length.

As described in Section 2.5, our models detect the direction by computing Ours
Diff as
$Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})-Asym(\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\},\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\})$
and predict the $i$th sentence pattern to be premise if Ours Diff $>0$. When
performing entailment classification, we use as the asymmetric similarity
scores
$Asym(\\{\tilde{\underline{{\bm{s}}}}_{i,k}\\},\\{\tilde{\underline{{\bm{s}}}}_{j,m}\\})$.
We report the performance of Ours (Trans), which is the same best model in the
RE experiment.

In entailment classification, we compare the results with cosine similarity
from Ours (Single-Trans) and CUschema. We also test the frequency difference,
which is a strong baseline in hypernym direction detection Chang et al.
(2018). Freq Diff = Freq($S_{j}$) - Freq($S_{i}$) where Freq($S_{i}$) is the
number of unique entity pairs co-occurred with the $i$th sentence pattern. The
baseline predicts $S_{i}$ to be premise if Freq Diff $>0$ because more general
sentence patterns should co-occur with more entity pairs. As a reference, we
also report the performance of random scores.

Method | Classification | Direction Detection
---|---|---
AP@all | Micro Acc | Macro Acc
Random | 21.9 | 50.0 | 50.0
Freq Diff | 21.4 | 54.5 | 47.3
CUSchema | 31.2 | 50.0 | 50.0
Ours (Single) | 23.6 | 50.0 | 50.0
Ours | 37.8 | 63.1 | 55.4

Table 3: Comparison of entailment detection methods. AP and Acc are average
precision and accuracy, respectively. All numbers are %. Our methods use a
Transformer as their encoder.

Results: The quantitative and qualitative comparison are presented in Table 3
and Table 2, respectively. Our model that uses multi-facet embeddings
significantly outperforms the other baselines. We hypothesize that a major
reason is that the sentence patterns with an entailment relation are often
similar in some but not all of the facets, and our asymmetric similarity
measurement is better at capturing the facet overlapping.

##  4 Related Work

Relation extraction (RE) is widely studied. Han et al. (2020) summarize the
trend of recent studies and point out one of the major challenges is the cost
of collecting the labels. Distant supervision (Mintz et al., 2009) and its
follow-up work enable us to collect a large amount of training data at a low
cost, but the violation of its assumptions often introduces substantial noise
into the supervision signal. Our goal is to alleviate the noise issue by
representing every sentence pattern using multiple embeddings.

Other noise reduction methods have also been proposed Roth et al. (2013). For
instance, we can adopt multi-instance learning techniques Yao et al. (2010);
Surdeanu et al. (2012); Amin et al. (2020), global topic model Alfonseca et
al. (2012), or both Roth and Klakow (2013). We can also reduce the noise by
counting the number of shared entity pairs between a sentence pattern and a KB
relation Takamatsu et al. (2012); Su et al. (2018). Nevertheless, the studies
focus on mitigating the noise caused by assuming similarity between the
sentence patterns and KB relations that co-occur with the same entity pairs,
while our method can also reduce the noise from two sentence patterns sharing
the same entity pair. Besides, our method is complementary to popular noise
reduction methods because our improvement is shown in the training data that
have been cleaned (Verga et al., 2016).

Our method is conceptually related to some studies for lexical semantics. For
example, word sense induction or unsupervised hypernymy detection can be
addressed by clustering the co-occurring words (Neelakantan et al., 2014;
Athiwaratkun and Wilson, 2017; Chang et al., 2018). However, the clustering-
based methods do not apply to RE because the co-occurring matrix for RE is
much sparser (see Section 2.4 for more details).

Finally, our work is inspired by Chang et al. (2021), but they focus on
improving the sentence representation rather than RE. We encourage the facet
embeddings to become the centers in Kmeans clustering instead of NNSC (non-
negative sparse coding) clustering used in Chang et al. (2021), due to its
simplicity, efficiency, and better RE performance. Moreover, we discover that
an additional regularization described in the appendixis crucial to overcome
the sparsity challenge in RE.

##  5 Conclusion

In this work, we address the limitation of representing each sentence pattern
using only a single embedding, and our approach improves the distantly-
supervised RE performances of compositional universal schema.

Relying on only a very sparse co-occurrence matrix between the sentence
patterns and entity pairs, we show that it is possible to predict reasonable
cluster centers of entity pair embeddings and to predict the entailment
relation between two sentence patterns without any labels.

## Acknowledgements

We thank Ge Gao for the preliminary exploration of this project. We also thank
the anonymous reviewers for their constructive feedback.

This work was supported in part by the Center for Data Science and the Center
for Intelligent Information Retrieval, in part by the Chan Zuckerberg
Initiative under the project Scientific Knowledge Base Construction, in part
using high performance computing equipment obtained under a grant from the
Collaborative R&D Fund managed by the Massachusetts Technology Collaborative,
in part by the National Science Foundation (NSF) grant numbers DMR-1534431 and
IIS-1514053.

Any opinions, findings, conclusions, or recommendations expressed in this
material are those of the authors and do not necessarily reflect those of the
sponsor.

## References

  * Akimoto et al. (2019) Kosuke Akimoto, Takuya Hiraoka, Kunihiko Sadamasa, and Mathias Niepert. 2019.  Cross-sentence n-ary relation extraction using lower-arity universal schemas.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6225–6231, Hong Kong, China. Association for Computational Linguistics.
  * Alfonseca et al. (2012) Enrique Alfonseca, Katja Filippova, Jean-Yves Delort, and Guillermo Garrido. 2012\.  Pattern learning for relation extraction with a hierarchical topic model.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 54–59, Jeju Island, Korea. Association for Computational Linguistics.
  * Amin et al. (2020) Saadullah Amin, Katherine Ann Dunfield, Anna Vechkaeva, and Guenter Neumann. 2020\.  A data-driven approach for noise reduction in distantly supervised biomedical relation extraction.  In _Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing_ , pages 187–194, Online. Association for Computational Linguistics.
  * Arora et al. (2017) Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017.  A simple but tough-to-beat baseline for sentence embeddings.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Athiwaratkun and Wilson (2017) Ben Athiwaratkun and Andrew Wilson. 2017.  Multimodal word distributions.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1645–1656, Vancouver, Canada. Association for Computational Linguistics.
  * Baldini Soares et al. (2019) Livio Baldini Soares, Nicholas FitzGerald, Jeffrey Ling, and Tom Kwiatkowski. 2019\.  Matching the blanks: Distributional similarity for relation learning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 2895–2905, Florence, Italy. Association for Computational Linguistics.
  * Bollacker et al. (2008) Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008\.  Freebase: a collaboratively created graph database for structuring human knowledge.  In _Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008_ , pages 1247–1250. ACM.
  * Borg and Groenen (2005) Ingwer Borg and Patrick JF Groenen. 2005.  _Modern multidimensional scaling: Theory and applications_.  Springer Science & Business Media.
  * Chaganty et al. (2017) Arun Chaganty, Ashwin Paranjape, Percy Liang, and Christopher D. Manning. 2017.  Importance sampling for unbiased on-demand evaluation of knowledge base population.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 1038–1048, Copenhagen, Denmark. Association for Computational Linguistics.
  * Chang et al. (2021) Haw-Shiuan Chang, Amol Agrawal, and Andrew McCallum. 2021.  Extending multi-sense word embedding to phrases and sentences for unsupervised semantic applications.  In _Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence_.
  * Chang et al. (2016) Haw-Shiuan Chang, Abdurrahman Munir, Ao Liu, Johnny Tian-Zheng Wei, Aaron Traylor, Ajay Nagesh, Nicholas Monath, Patrick Verga, Emma Strubell, and Andrew McCallum. 2016.  Extracting multilingual relations under limited resources: TAC 2016 cold-start KB construction and slot-filling using compositional universal schema.  In _Proceedings of the 2016 Text Analysis Conference, TAC 2016, Gaithersburg, Maryland, USA, November 14-15, 2016_. NIST.
  * Chang et al. (2018) Haw-Shiuan Chang, Ziyun Wang, Luke Vilnis, and Andrew McCallum. 2018.  Distributional inclusion vector embedding for unsupervised hypernymy detection.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 485–495, New Orleans, Louisiana. Association for Computational Linguistics.
  * Das et al. (2017) Rajarshi Das, Manzil Zaheer, Siva Reddy, and Andrew McCallum. 2017.  Question answering on knowledge bases and text using universal schema and memory networks.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 358–365, Vancouver, Canada. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Han et al. (2020) Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Jie Zhou, and Maosong Sun. 2020.  More data, more relations, more context and more openness: A review and outlook for relation extraction.  In _Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing_ , pages 745–758, Suzhou, China. Association for Computational Linguistics.
  * Hastie et al. (2009) Trevor Hastie, Robert Tibshirani, and Jerome H. Friedman. 2009.  _The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition_.  Springer Series in Statistics. Springer.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural Comput._ , 9(8):1735–1780.
  * Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Miller (1998) George A Miller. 1998.  _WordNet: An electronic lexical database_.  MIT press.
  * Mintz et al. (2009) Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009.  Distant supervision for relation extraction without labeled data.  In _Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP_ , pages 1003–1011, Suntec, Singapore. Association for Computational Linguistics.
  * Neelakantan et al. (2014) Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. 2014\.  Efficient non-parametric estimation of multiple embeddings per word in vector space.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1059–1069, Doha, Qatar. Association for Computational Linguistics.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Percha and Altman (2018) Bethany Percha and Russ B. Altman. 2018.  A global network of biomedical relationships derived from text.  _Bioinform._ , 34(15):2614–2624.
  * Riedel et al. (2013) Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. 2013.  Relation extraction with matrix factorization and universal schemas.  In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 74–84, Atlanta, Georgia. Association for Computational Linguistics.
  * Roth et al. (2013) Benjamin Roth, Tassilo Barth, Michael Wiegand, and Dietrich Klakow. 2013.  A survey of noise reduction methods for distant supervision.  In _Proceedings of the 2013 workshop on Automated knowledge base construction_.
  * Roth and Klakow (2013) Benjamin Roth and Dietrich Klakow. 2013.  Feature-based models for improving the quality of noisy training data for relation extraction.  In _22nd ACM International Conference on Information and Knowledge Management, CIKM’13, San Francisco, CA, USA, October 27 - November 1, 2013_ , pages 1181–1184. ACM.
  * Shwartz et al. (2017) Vered Shwartz, Enrico Santus, and Dominik Schlechtweg. 2017.  Hypernyms under siege: Linguistically-motivated artillery for hypernymy detection.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_ , pages 65–75, Valencia, Spain. Association for Computational Linguistics.
  * Su et al. (2018) Yu Su, Honglei Liu, Semih Yavuz, Izzeddin Gür, Huan Sun, and Xifeng Yan. 2018\.  Global relation embedding for relation extraction.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 820–830, New Orleans, Louisiana. Association for Computational Linguistics.
  * Surdeanu et al. (2012) Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012\.  Multi-instance multi-label learning for relation extraction.  In _Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning_ , pages 455–465, Jeju Island, Korea. Association for Computational Linguistics.
  * Takamatsu et al. (2012) Shingo Takamatsu, Issei Sato, and Hiroshi Nakagawa. 2012.  Reducing wrong labels in distant supervision for relation extraction.  In _Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 721–729, Jeju Island, Korea. Association for Computational Linguistics.
  * Toutanova et al. (2015) Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. 2015.  Representing text for joint embedding of text and knowledge bases.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 1499–1509, Lisbon, Portugal. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Verga et al. (2016) Patrick Verga, David Belanger, Emma Strubell, Benjamin Roth, and Andrew McCallum. 2016.  Multilingual relation extraction using compositional universal schema.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 886–896, San Diego, California. Association for Computational Linguistics.
  * Verga et al. (2017) Patrick Verga, Arvind Neelakantan, and Andrew McCallum. 2017.  Generalizing to unseen entities and entity pairs with row-less universal schema.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers_ , pages 613–622, Valencia, Spain. Association for Computational Linguistics.
  * Verga et al. (2018) Patrick Verga, Emma Strubell, and Andrew McCallum. 2018.  Simultaneously self-attending to all mentions for full-abstract biological relation extraction.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 872–884, New Orleans, Louisiana. Association for Computational Linguistics.
  * Yao et al. (2010) Limin Yao, Sebastian Riedel, and Andrew McCallum. 2010.  Collective cross-document relation extraction without labelled data.  In _Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing_ , pages 1013–1023, Cambridge, MA. Association for Computational Linguistics.
  * Zhang et al. (2019) Dongxu Zhang, Subhabrata Mukherjee, Colin Lockard, Luna Dong, and Andrew McCallum. 2019.  OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 762–772, Minneapolis, Minnesota. Association for Computational Linguistics.

##  Appendix A Method Details

Our objective function includes the loss defined in equation 2 and a
regularization term $\Omega$. We describe $\Omega$ in Section A.1 and some
implementation details in Section A.2. Finally, our training procedure is
summarized in Algorithm 1.

###  A.1 Regularization by Autoencoder

The co-occurrence matrix between the sentence patterns and entity pairs is
very sparse because most of the sentence patterns only co-occur with a few
entity pairs. The sparsity might make the training process of multi-facet
embeddings sensitive to the hyperparameters.

We discover that adding a simple autoencoder regularization is an effective
way to stabilize the training. This regularization term aims to make the
average of our facet embeddings of a sentence pattern similar to the weighted
average of our word embeddings in that sentence pattern. The regularization is
a kind of autoencoder because it reconstructs the weighted average embeddings
of words in the input sentence pattern using the output facet embeddings. The
regularization term $\Omega$ is defined as

$\displaystyle\gamma\sum_{(i,j)\in
R^{auto}}(2\cdot\mathbbm{1}_{i=j}-1)||\underline{{\bm{s}}}^{aw^{\prime}}_{j}-\mu(S_{i})||^{2},$
(5)

where $\gamma$ is a weight for the regularization term,
$R^{auto}=\cup_{i=1}^{I}\\{(i,i),(i,q)\\}$ is the set of all positive and
negative training pairs, $I$ is the number of sentence patterns, and $q$ is a
randomly selected index of sentence patterns which serves as our negative
example, $\mathbbm{1}_{i=j}=1$ if $i=j$ and $0$ otherwise,
$\mu(S_{i})=\frac{\sum_{k=1}^{K}\underline{{\bm{s}}}_{i,k}}{K}$ is the average
of facet embeddings of the sentence pattern $S_{i}$.
$\underline{{\bm{s}}}^{aw^{\prime}}_{j}$ is a weighted average embedding of
words in the $j$th sentence pattern that passes through a linear
transformation $\bm{H}$. Weighting each word embedding by a smoothed inverse
frequency provides a better text similarity measurement (Arora et al., 2017)
because the frequently occurring words often do not contribute much to the
semantic meaning (e.g., stop words). Similarly , we compute

$\displaystyle\underline{{\bm{s}}}^{aw^{\prime}}_{j}=\bm{H}\underline{{\bm{s}}}^{aw}_{j}=\bm{H}\sum_{w\in
S_{j}}\frac{\nu}{\nu+p(w)}\underline{{\bm{w}}},$ (6)

where $\bm{H}$ linearly transforms the word embedding into the entity pair
embedding space. $\nu=10^{-4}$ is a constant set suggested in Arora et al.
(2017), $p(w)$ is the frequency of the word $w$ divided by the number of words
in the corpus, and $\underline{{\bm{w}}}$ is the pretrained embedding of word
$w$. We use 840B GloVe Pennington et al. (2014) as our word embedding in this
work.

Ours (Single-Trans) | Ours (Trans) | BERT Base
---|---|---
1.0M | 1.1M | 86.0M
Ours (Single-LSTM) | Ours (LSTM) |
2.6M | 3.0M |

Table 4: Comparison of neural model sizes (i.e., the number of trainable
parameters excluding the word embedding layer).

Input : Sentence patterns and KB relations $S$, co-occurrence matrix
$\\{y_{i,j}\\}$, entity pair embeddings from USchema, and pre-trained word
embeddings.

Output : Our neural encoder and decoder

Initialize the entity pair embeddings using the embeddings learned by USchema.

Initialize the word embeddings of our neural encoder using pre-trained word
embeddings and randomly initialize other parameters

foreach  _$S_{i}$ in training corpus_ do

Run forward pass on the neural encoder and decoder to compute facet embeddings
$\\{\underline{{\bm{s}}}_{i,k}\\}_{k=1}^{K}$

Collect positive examples (i.e., $\\{j|y_{i,j}=1\\}$) and negative examples
for the $i$th sentence pattern or KB relation

foreach  _Positive and negative examples $(i,j)$_ do

Compute
$\tilde{\underline{{\bm{e}}}}_{j}=\frac{\underline{{\bm{e}}}_{j}}{||\underline{{\bm{e}}}_{j}||}$

Compute
$\eta_{k}=\min(1,\max(0,\frac{\tilde{\underline{{\bm{e}}}}^{T}_{j}\underline{{\bm{s}}}_{i,k}}{||\underline{{\bm{s}}}_{i,k}||^{2}})),\forall
1\leq k\leq K$

Select $k_{best}$th facet embedding by
$k_{best}=\operatorname*{arg\,min}_{k}||\tilde{\underline{{\bm{e}}}}_{j}-\eta_{k}\underline{{\bm{s}}}_{i,k}||^{2}$

Add $(2\cdot
y_{i,j}-1)r_{i,j}||\tilde{\underline{{\bm{e}}}}_{j}-\eta_{k_{best}}\underline{{\bm{s}}}_{i,k_{best}}||^{2}$
to the loss

end foreach

Add the autoencoder loss $\Omega$ in equation 5 using pre-trained word
embeddings

Update neural encoder and decoder, entity pair embeddings, and $\bm{H}$ by
backpropagation

end foreach

Algorithm 1 Training procedure (using batch size = 1)

###  A.2 Implementation Details

In our Transformer encoder and decoder, we set the number of layers to be 3
and the size of the hidden state to be 300. In our bi-LSTM encoder, the number
of layers is set to 2. After the bi-LSTM encoder, we use the last embedding in
the hidden state to encode a sequence into a single embedding. In Table 4, we
can see that the size of the neural model outputting multi-facet embeddings is
similar to the size of the single embedding model, and both are much smaller
than the BERT base model (Devlin et al., 2019). The size of CUSchema is
smaller than ours because it uses a smaller hidden state size, but we find
that increasing the model size of CUSchema does not lead to better
performance.

Before training, we initialize the embeddings of entity pairs using USchema as
in Verga et al. (2016). Besides, we initialize the word embedding layer in our
neural model using GloVe. CUSchema initializes its word embedding layer
randomly, but we also find that initializing it using pre-trained word
embeddings does not increase the performance.

We use Adam Kingma and Ba (2015) to optimize the parameters of our neural
model and entity pair embeddings. For the linear layer $\bm{H}$ in equation 6,
we adopt SGD to make the training more stable. Due to its small model size,
one 1080Ti GPU is sufficient to train our model in 3 days.

##  Appendix B Experiment Details

The details of preparing the entailment dataset are included in Section B.1,
and the details of our experiment setup are included in Section B.2. We
present the results of our ablation studies in Section B.3.

###  B.1 Entailment Dataset Creation

When finding the entailment candidates using WordNet, we iterate over all the
words in every sentence pattern. For each word, we retrieve all of its
senses/synsets and the possible hypernym synsets. Finally, we replace the word
with every lemma of each hypernym synset. After the replacement, if the
sentence pattern appears in our training data, we pair the sentence patterns
before and after replacement as a candidate.

Our goal is to find entailment rather than paraphrase relation, so we exclude
the candidates where the replaced word is both the hypernym and hyponym of the
replacing word. To measure each sentence pattern’s popularity, we compute the
number of unique entity pairs co-occurring with the sentence pattern as the
pattern frequency and take the minimum of the frequency between the two
sentence patterns in a candidate as the candidate frequency. For each
hypothesis, we only consider the top 6 premise candidates with the highest
frequencies to diversify the hypotheses in our dataset. The hypothesis
popularity score is the average candidate frequencies across its top 6
premises.

Before labeling, we sort the hypotheses based on their popularity scores and
label the most popular 1,500 candidates (with the highest minimal
frequencies). The labeling is done by a PhD student who has RE research
experiences because we hypothesize that it is hard to clearly explain the task
to crowdsourcing workers. After the dataset is built, we separate the
validation set and test set such that all hypernyms in the test hypotheses are
unseen in the validation set.

###  B.2 Experiment Setup Details

In our visualization experiment, we filter out a few entity pairs that co-
occur with less than 5 sentence patterns or become far away from its closest
facet embedding after the projection. To prevent the facet embeddings from
overlapping, we add a small random vector to each facet embedding.

In the co-occurrence matrix, we use 5% of the unique sentence patterns as our
validation set. All the sentence patterns in the validation set are unseen in
the training set. We use the validation set to tune the number of epochs
during training.

We use coordinate descent to search the hyperparameters that result in the
best F1 score in TAC 2012 during training (i.e., change one hyperparameter at
a time). Compared with the grid search used in CUSchema, this tuning method is
less computationally expensive and less likely to overfit the validation data.

We first optimize the hyperparameters in Ours (Trans). The search ranges are
$\gamma=[0.1,0.2,0.3]$, $K=[1,2,3,4,5,6,11]$,
$K_{rel}=[1,8,9,10,11,12,13,14,15]$, encoder dropout rate $=[0.25,0.3,0.35]$,
learning rate for updating $\bm{H}=[1,0.1,0.01]$ and maximal epoch number
$=[15,20,25,30,50]$. Our best Transformer model Ours (Trans) used
$\gamma=0.2$, $K=5$, $K_{rel}=11$, encoder dropout rate $=0.3$, learning rate
for updating $\bm{H}=0.1$ and maximal number of epochs = 50. Then, we start
from these best hyperparameters for Ours (Trans) and tune only encoder dropout
rate $=[0.25,0.3,0.35]$, and maximal epoch number $=[15,20,25,30,50]$ for Ours
(LSTM). The best performing LSTM model used maximal 30 epochs while all other
hyperparameters are found to be the same as the best Transformer model.
Finally, we fix $K=K_{rel}=1$ and tune the hyperparameters using the same
range as above, for Ours (Single-Trans) and Ours (Single-LSTM).

###  B.3 Ablation Study

In Table 5, we justify using the autoencoder loss and using different facet
numbers for sentence patterns ($K$) and for KB relations ($K_{rel}$). We can
see that performance drops if we remove these techniques from our models using
a Transformer.

Method | 2012 | 2013 | 2014
---|---|---|---
Ours | 30.4 | 37.3 | 31.3
Ours ($K=K_{rel}=11$) | 29.9 | 36.1 | 29.8
Ours (No autoencoder) | 27.5 | 33.5 | 30.2

Table 5: Ablation study on TAC datasets. All numbers are F1 (%). All models
use the Transformer encoder.


myfont

# Gain-compensated metal cavity modes and a million-fold improvement of
Purcell factors

Becca VanDrunen  Department of Physics, Engineering Physics, and Astronomy,
Queen’s University, Kingston, Ontario K7L 3N6, Canada  Juanjuan Ren
Department of Physics, Engineering Physics, and Astronomy, Queen’s University,
Kingston, Ontario K7L 3N6, Canada  Sebastian Franke  Technische Universität
Berlin, Institut für Theoretische Physik, Nichtlineare Optik und
Quantenelektronik, Hardenbergstraße 36, 10623 Berlin, Germany  Department of
Physics, Engineering Physics, and Astronomy, Queen’s University, Kingston,
Ontario K7L 3N6, Canada  Stephen Hughes  Department of Physics, Engineering
Physics, and Astronomy, Queen’s University, Kingston, Ontario K7L 3N6, Canada

###### Abstract

Using a rigorous mode theory for gain-compensated plasmonic dimers, we
demonstrate how quality factors and Purcell factors can be dramatically
increased, improving the quality factors from 10 to over 26,000 and the peak
Purcell factors from around 3000 to over 10 billion. Full three-dimensional
calculations are presented for gold dimers in a finite-size gain medium, which
allows one to easily surpass fundamental Purcell factor limits of lossy media.
Within a regime of linear system response, we show how the Purcell factors are
modified from the contributions from the projected local density of states as
well as a non-local gain. Further, we show that the effective mode volume and
radiative beta factors remain relatively constant, despite the significant
enhancement of the Purcell factors.

Introduction.—Plasmonic resonators, formed by metal nanoparticles (MNPs), have
become a prominent topic in nanophotonics, due in part to their unique
abilities for enhancing light-matter interactions in extremely small spatial
scales [1, 2]. Plasmonic resonators exploit surface plasmons, which occur at
the interface of a metal and a dielectric material, and are the result of
mixed electronic-optical excitations on the surface of the metal [3].
Plasmonic resonators yield electromagnetic modes that are well below the
diffraction limit [4, 1], leading to improved sensing and fast generation of
single photons [5, 6].

Metal based cavity modes have been explored theoretically [7, 8] and
experimentally [9, 10], for a wide range of photonics applications. However,
plasmonic resonators have significant decay rates, since metal is inherently a
lossy material. Thus, an important goal in optical plasmonics is to find
methods for alleviating this loss. Historically, optical mode theories of
plasmonics were thought to be problematic [11], but this is largely caused by
the use of ill-defined mode models, treating such systems like regular “normal
modes”, without much consideration for losses.

Recently, accurate cavity mode theories used to describe the optical response
of MNPs have been formulated, which are based on “quasinormal modes”
(QNMs)—the formal solution for open cavity modes [12, 13, 14, 15]. Similar to
normal modes, QNMs are solutions to the source-free Helmholtz equation, but
with open boundary conditions, where the solutions have complex
eigenfrequencies and spatially diverging fields (due to temporal losses) [12,
13]. By exploiting QNM theory, it has become clear that cavity physics applies
to plasmonic resonator structures [16, 15]. Theories based on QNMs offer a
significant advantage over common all-numerical approaches, which can be
tedious, limited in scope, and often do not even explain the basic mechanisms
of field enhancement; direct numerical approaches also have to be repeated
many times, e.g., for studying the emission decay of oscillating dipoles as a
function of dipole position. In contrast, a mode theory provides many physical
insights, is efficient, has wide applications, and lends itself to mode
quantization [17]. Moreover, the modes form a basis for computing the photonic
Green function (GF), which can be used to describe a wide range of optical
phenomena in both classical and quantum optics [18, 19, 20, 14, 21].

Accounting for both material loss and radiative loss, the mode quality factor
is defined from $Q=\omega_{c}/(2\gamma_{c})$ ($2\gamma_{c}$ is the energy loss
rate of the mode c), which for plasmonic resonators is much smaller than
typical dielectric cavities [22]. Thus, the MNP resonances typically generate
quality factors of around $Q\approx 10-20$ [23, 24, 25, 26], which manifest in
a very short cavity mode lifetime, $\tau_{c}={1}/{\gamma_{c}}$, e.g., a
resonance of $\hbar\omega_{c}=1.780-i0.068$ eV [27], corresponds to
$\tau_{c}\approx 0.01$ ps. Such losses prohibit many applications in coherent
optics, including surface plasmon lasing and spasing [28, 29].

One potential method for mitigating this significant loss is through gain
compensation, which uses material gain to suppress some of the dissipation,
using a linear amplification regime [30, 31, 32]. It is important to note that
the total cavity structure must be overall lossy for the properties of cavity
physics and QNM theory to apply, and also to maintain a linear medium
response. More specifically, the entire GF is only allowed to have complex
poles in the lower complex half plane. To model such a structure, both the
metal and the gain are defined by a complex dielectric constant, where the
imaginary part describes loss or gain.

Gain media has been utilized in several applications to suppress metallic
losses [33, 34, 35]. For example, loss suppression in MNPs has been studied by
doping them in gain [36, 37] or by adding gain media to plasmonic resonators
[38, 39] to observe the impact on the response of an emitter. While there
exist some studies on how the quality factor changes as a result of gain-
compensation [37, 38, 40, 41], little has been done to quantify how enhanced
spontaneous emission (SE) and Purcell factors change from coupled dipole
emitters. Furthermore, the concept of spasing has been an enticing area of
study for decades [42, 43, 44], yet many approaches lack the robustness of a
rigorous mode theory, which is a requirement in quantum optics.

A particularly important metric in nanophotonics is the Purcell factor [45],
which describes the enhanced SE rate of a dipole emitter,
$\Gamma(\mathbf{r}_{0},\omega)$, normalized to the rate from a background
homogeneous medium, $\Gamma_{\rm B}(\mathbf{r}_{0},\omega)$:

$F_{\rm P}\equiv\frac{3}{4\pi^{2}}\left(\frac{\lambda_{0}}{{n}_{\rm
B}}\right)^{3}\frac{Q}{V_{\rm eff}},$ (1)

where $Q$ and $V_{\rm eff}$ are the mode quality factor and effective mode
volume, respectively, $\lambda_{0}$ is the free space wavelength, and $n_{\rm
B}$ is the background refractive index. This well known formula assumes the
emitter is on resonance with the cavity mode and at a field maximum position,
with the same polarization as the cavity mode. Traditionally, the effective
mode volume describes the volume associated with mode localization, but that
is no longer correct using QNMs (as the modes diverge in space), and the
generalized mode volume can be both complex and position dependent [46].
Moreover, when gain is added to the cavity, this formula no longer applies
(even when using QNMs), and one must include a non-local correction from gain
[47, 48].

In this work, we show how material gain can significantly improve the MNP
cavity mode properties, using a rigorous QNM theory with linear amplification.
The key findings are: (i) the effective mode volume of the mode profiles stays
nearly constant as gain is introduced to the system; (ii) the Purcell factor
yields a million-fold improvement when material gain is added; and (iii) the
radiative beta factor also remains relatively constant. Our gain theory has a
wide range of applications, including quantum sensing, as well as lasing and
fundamental topics in quantum topics.

Theory.—The QNMs, ${\tilde{\bf f}}_{\mu}$, are the mode solutions to the
Helmholtz equation, with open boundary conditions [14]:

${\bm{\nabla}}\crossproduct{\bm{\nabla}}\crossproduct\tilde{{\bf
f}}_{\mu}({\bf
r})-\left(\frac{\tilde{\omega}_{\mu}}{c}\right)^{2}\epsilon({\bf
r},\tilde{\omega}_{\mu})\tilde{{\bf f}}_{\mu}({\bf r})=0,$ (2)

where $c$ is the speed of light in a vacuum, $\tilde{\omega}_{\mu}$ is the QNM
complex eigenfrequency $\tilde{\omega}_{\mu}=\omega_{\mu}-i\gamma_{\mu}$, and
$\epsilon({\bf r},\tilde{\omega}_{\mu})$ is the complex dielectric function.
The inhomogeneous Helmholtz equation for an arbitrary polarization source can
be used to define the GF, from
$c^{2}\mathbf{\nabla}\times\mathbf{\nabla}\times{\mathbf{G}}(\mathbf{r},\mathbf{r}^{\prime},\omega)-\omega^{2}\epsilon(\mathbf{r},\omega){\mathbf{G}}(\mathbf{r},\mathbf{r}^{\prime},\omega)=\omega^{2}{\mathbf{1}}\delta(\mathbf{r}-\mathbf{r}^{\prime}),$
where the electric field solution is at $\mathbf{r}$, when a source field
dipole is at $\mathbf{r}^{\prime}$. Within or near the cavity region, the GF
can be expressed as a sum of normalized QNMs [27, 49, 19]:

${\bf G}({\bf r},{\bf r}_{0},\omega)=\sum_{\mu}A_{\mu}(\omega)\tilde{{\bf
f}}_{\mu}({\bf r})\tilde{{\bf f}}_{\mu}({\bf r}_{0}),$ (3)

where $A_{\mu}(\omega)={\omega}/{[2(\tilde{\omega}_{\mu}-\omega)]}$ [7], and
we note the vector product is unconjugated (i.e., not $\tilde{\bf
f}_{\mu}\tilde{\bf f}_{\mu}^{*}$), which is a consequence of using a non-
Hermitian theory. When a single QNM dominates, $\mu=c$, then the GF is simply

${\bf G}_{c}({\bf r},{\bf r}_{0},\omega)\approx A_{c}(\omega)\tilde{{\bf
f}}_{c}({\bf r})\tilde{{\bf f}}_{c}({\bf r}_{0}).$ (4)

In a lossy material system with no gain, the SE rate for a dipole emitter at a
location ${\bf r}_{0}$ is determined from the (projected) local density of
states (LDOS) [12, 27]:

$\Gamma_{\rm LDOS}^{\rm SE}({\bf
r}_{0},\omega)=\frac{2}{\hbar\epsilon_{0}}{\bf d}\cdot{\rm Im}[{\bf G}({\bf
r}_{0},{\bf r}_{0},\omega)]\cdot{\bf d},$ (5)

and using a single QNM expansion approximation, ${\bf G}$ becomes ${\bf
G}_{c}$, as shown in Eq. (4). The SE rate for a dipole in a homogeneous
medium, $\Gamma_{\rm B}({\bf r}_{0},\omega)$, is similarly obtained by
replacing ${\bf G}$ by $\mathbf{G}_{\rm B}$ (the GF of a homogeneous medium),
which is known analytically. The LDOS Purcell factor is then given by [50, 12,
27]

$\displaystyle F_{\rm P}^{\rm LDOS}({\bf r}_{0},\omega)$
$\displaystyle=1+\frac{6\pi c^{3}}{\omega^{3}n_{\rm B}}{\bf n}_{\rm
d}\cdot{\rm Im}[{\bf G}({\bf r}_{0},{\bf r}_{0},\omega)]\cdot{\bf n}_{\rm d},$
(6)

and $F_{\rm P}^{\rm QNM,LDOS}$ is obtained by using ${\rm G}\rightarrow{\rm
G}_{c}$.

However, this classical LDOS formalism for the Purcell factor is not correct
in the presence of a linear gain medium. Instead, the total SE rate can be
written as [47, 48]

$\displaystyle\Gamma^{\rm SE}_{\rm tot}({\bf r}_{0},\omega)=\Gamma^{\rm
SE}_{\rm LDOS}({\bf r}_{0},\omega)+\Gamma^{\rm SE}_{\rm gain}({\bf
r}_{0},\omega),$ (7)

which notably contains an extra net-positive term related to gain region added
to the traditional LDOS term; this correction can be derived quantum
mechanically [47, 14] or classically [48]. The total Purcell factor is then

$F_{\rm P}(\mathbf{r}_{0},\omega)=1+\frac{\Gamma^{\rm SE}_{\rm
tot}(\mathbf{r}_{0},\omega)}{\Gamma_{\rm B}(\mathbf{r}_{0},\omega)}.$ (8)

In origin, the well known LDOS-SE formula is linked to the GF identity,
$\int_{\mathbb{R}^{3}}{\rm
d}\mathbf{s}\epsilon_{I}(\mathbf{s})\mathbf{G}(\mathbf{r},\mathbf{s})\cdot\mathbf{G}^{*}(\mathbf{s},\mathbf{r}^{\prime})={\rm
Im}[\mathbf{G}(\mathbf{r},\mathbf{r}^{\prime})]$, which involves an
integration over all space. However, from this identity, one must subtract the
contribution from the gain, and this results in adding the separate gain
contribution term, which is given by [47]

$\Gamma_{\rm gain}^{\rm
SE}(\mathbf{r}_{0},\omega)=\frac{2}{\hbar\epsilon_{0}}{\bf d}\cdot{\bf K}({\bf
r}_{0},{\bf r}_{0},\omega)\cdot{\bf d},$ (9)

where

$\mathbf{K}(\mathbf{r}_{0},\mathbf{r}_{0},\omega)=\\!\int_{V_{\rm
gain}}\\!d\mathbf{s}\left|{\rm Im}[\epsilon^{\rm
gain}(\mathbf{s})]\right|\mathbf{G}(\mathbf{r}_{0},\mathbf{s},\omega)\cdot\mathbf{G}^{*}(\mathbf{s},\mathbf{r}_{0},\omega).$
(10)

In the case of a single dominant QNM, $\mu=c$, then

$\displaystyle\begin{split}\Gamma_{\rm gain}^{\rm SE}({\bf
r}_{0},\omega)&=\frac{2|\mathbf{d}|^{2}}{\hbar\epsilon_{0}}\Big{|}A_{c}(\omega)\Big{|}^{2}\Big{|}\mathbf{n}_{\rm
d}\cdot\tilde{{\bf f}_{c}}(\mathbf{r}_{0})\Big{|}^{2}\times\\\ &\int_{V_{\rm
gain}}d\mathbf{s}\Big{|}{\rm Im}[\epsilon^{\rm
gain}(\mathbf{s})]\Big{|}\Big{|}\tilde{{\bf f}_{c}}({\bf
s})\Big{|}^{2}.\end{split}$ (11)

From the perspective of classical power flow arguments [48], the total Purcell
factor can be written as

$\displaystyle F_{\rm P}(\mathbf{r}_{0},\omega)=\frac{P_{\rm
LDOS}(\mathbf{r}_{0},\omega)+P_{\rm
gain}(\mathbf{r}_{0},\omega)}{P_{0}({\omega})},$ (12)

where $P_{\rm LDOS}$ and $P_{0}$ are the power flow from the point dipole with
and without the cavity structure (background medium), respectively, and
$P_{\rm gain}$ is the power flowing out from the gain region (net-positive).
By removing the $P_{\rm gain}$ term, this gives the LDOS Purcell factor.
Equation (12) can be solved using QNMs or numerically, which is useful to
justify the QNM solutions. The total Purcell factors can also be written as

$\displaystyle F_{\rm P}(\mathbf{r}_{0},\omega)=\frac{P_{\rm
far}(\mathbf{r}_{0},\omega)+P_{\rm
loss}(\mathbf{r}_{0},\omega)}{P_{0}({\omega})},$ (13)

where $P_{\rm far}$ and $P_{\rm loss}$ are power radiated to far field region
and dissipated within lossy region, respectively.

From this general SE decay theory, one can also determine the radiative beta
factor ($\beta$-factor), which represents the probability that an emitted
photon will decay radiatively to the far field [27]. Generally, there are both
radiative and non-radiative $\beta$-factors, and the sum of these must equal
one [8]. The radiative beta factor is [48]

$\beta^{\rm rad}(\mathbf{r}_{0},\omega)=\frac{P_{\rm
far}(\mathbf{r}_{0},\omega)}{P_{\rm far}(\mathbf{r}_{0},\omega)+P_{\rm
loss}(\mathbf{r}_{0},\omega)},$ (14)

which can also be written equivalently in terms of $P_{\rm LDOS}$ and $P_{\rm
gain}$ based on the power conservation law [48].

Results.—The main MNP cavity structure we model consists of a gold dimer
enclosed in a finite-sized cylindrical region of gain, as shown in Fig. 1. The
dielectric constant for the gold is described by the Drude model,
$\epsilon^{\rm Drude}(\omega)=1-\frac{\omega_{\rm
p}^{2}}{\omega^{2}+i\omega\gamma_{\rm p}}$, with $\hbar\omega_{\rm p}=8.2934$
eV and $\hbar\gamma_{\rm p}=0.0928$ eV. The gap region between the nanorods
(where the dipole sits) is considered to have a real permittivity. The gain
region has a permittivity of $\epsilon^{\rm gain}=2.25-i\alpha_{g}$, so the
gap region simply takes the real component of this. We assume the gain is
dispersionless, but gain dispersion results are discussed in the Supplementary
Material (SM) [51], and do not affect our general findings. For all
calculations below, we obtain the QNMs numerically using a complex frequency
approach, implemented in COMSOL [7, 27].

(a)(b) Figure 1: (a) A 2D view of the resonator system. The dielectric
functions for each material are labelled, where the background medium has
$\epsilon_{\rm B}=2.25$ ($\epsilon_{\rm B}=n_{\rm B}^{2}$), the gain region
has $\epsilon^{\rm gain}=2.25-i\alpha_{g}$ (where $\alpha_{g}$ is the gain
parameter), the gold nanorods have $\epsilon^{\rm Drude}$ which is governed by
the Drude model (see text), and the gap region has $\epsilon^{\rm gap}=2.25$.
The gold nanorods have a length of $80\leavevmode\nobreak\ $nm, a radius of
$10\leavevmode\nobreak\ $nm, and the gap distance is $20\leavevmode\nobreak\
$nm; $h$ represents the height of the gain region, and is
$400\leavevmode\nobreak\ $nm, and $r$ represents the radius of the gain
region, which is $200\leavevmode\nobreak\ $nm. (b) 3D version of the system
(as used in our model).  Figure 2:  Surface plots of the computed QNM profile
(using the dominant field component) for the plasmonic resonator system (a)
without gain, and (b) with gain, using $\alpha_{g}=2.54\cdot 10^{-1}$.

(a) small gain values

(b) larger gain values

(c) total Purcell factors, for cases in (b)

Figure 3:  (a) LDOS Purcell factors for various gain coefficients,
$\alpha_{g}$, corresponding to the model in Fig. 1. The curves show the QNM
method, calculated with Eq. (6), and the symbols represent the full dipole
numerical solution with Eq. (12). The orange dashed line/symbols are for the
case with no gain ($\alpha_{g}=0$), the green line/points represent
$\alpha_{g}=1\cdot 10^{-1}$, the magenta line/points represent
$\alpha_{g}=2\cdot 10^{-1}$, and the black line/points represent
$\alpha_{g}=2.2\cdot 10^{-1}$. The $\gamma_{c}$ values for each curve in
increasing order of $\alpha_{g}$ are: $\gamma_{0}$, $0.61\gamma_{0}$,
$0.21\gamma_{0}$, $0.13\gamma_{0}$ , where $\gamma_{0}=9.016\cdot 10^{13}$
rads/s. (b) LDOS Purcell factors for larger values of $\alpha_{g}$. The blue
line/points represent the case where $\alpha_{g}=2.5\cdot 10^{-1}$, the red
line/point represents the case where $\alpha_{g}=2.53\cdot 10^{-1}$, and
finally the gold line/point represents the case where $\alpha_{g}=2.54\cdot
10^{-1}$. The $\gamma_{c}$ values for each curve in increasing order of
$\alpha_{g}$ are: $0.06\gamma_{0}$, $0.004\gamma_{0}$, and $0.0004\gamma_{0}$.
(c) Total Purcell factor over a range of energies for three different values
of $\alpha_{g}$ for the model in Fig. 1, calculated with Eq. (8) for the QNM
method, and a full numerical dipole method to confirm the results, using Eq.
(12). The blue line/points are for $\alpha_{g}=2.5\cdot 10^{-1}$, the red
line/points are for $\alpha_{g}=2.53\cdot 10^{-1}$, and the gold line and
points are for $\alpha_{g}=2.54\cdot 10^{-1}$, all plotted on a logarithmic
$y$-axis. The gold curve peaks at $2.08\cdot 10^{10}$. The dashed blue line
represents the LDOS Purcell factor when $\alpha_{g}=2.5\cdot 10^{-1}$, which
shows a significant difference (and becomes negative at larger frequencies).

When gain is introduced to the cavity system, it is important to first analyze
the gain-modified mode profiles of these systems to determine how the dominant
QNM behaves in the presence of material gain. The vector-field QNM is
dominated by $z$-polarization, that peaks in frequency near
$\hbar\omega\approx 1.2\leavevmode\nobreak\ $eV. As can be seen in Fig. 2, the
spatial profile of the QNM is similar with and without the gain medium.
Furthermore, this can be verified quantitatively, by calculating the effective
mode volume, $V_{\rm eff}^{-1}({\bf r}_{0})=\epsilon(\mathbf{r}_{0}){\rm
Re}[\tilde{\bf f}_{c}^{2}({\bf r}_{0})]$ from QNM theory. See SM [51] for the
computed effective mode volume at gap center.

For the two cases shown in Fig. 2, the complex QNM eigenfrequencies are
$\hbar\tilde{\omega}_{c}=1.198-i5.934\cdot 10^{-2}\leavevmode\nobreak\ {\rm
eV}$ and $\hbar\tilde{\omega}_{c}=1.195-i2.238\cdot
10^{-5}\leavevmode\nobreak\ {\rm eV}$, respectively. This leads to a QNM
quality factor $Q$ of 10 for the case with no gain and 26,698 for the case
with the largest amount of gain, showing that a significant improvement in $Q$
is possible.

In the presence of gain, the LDOS Purcell factor of a dipole placed at the
center of the dimer gap can be calculated again using Eq. (6) and checked
against a full numerical dipole calculation [27, 48]. By comparing the LDOS
Purcell factor over a range of frequencies for both approaches, we can
determine if the analytical single QNM expansion method is accurate.
Additionally, the LDOS Purcell factor gives insights into how the gain medium
impacts the SE rates. We can then add in the non-local gain contribution to
obtain the (correct) total Purcell factors.

A summary of the computed Purcell factors with and without gain is shown in
Fig. 3. In Fig. 3a, we show the LDOS Purcell factors for $\alpha_{g}$ ranging
from $0\leavevmode\nobreak\ {\rm to}\leavevmode\nobreak\ 2.2\cdot 10^{-1}$;
the agreement between the QNM method (curves) and the full dipole method
(symbols) is excellent, and we stress there are no fitting parameters used in
this model.

Moreover, both Figs. 3a and 3b (which increases the gain further to
$\alpha_{g}=2.5\cdot 10^{-1}\sim 2.54\cdot 10^{-1}$, like in Fig. 3c)
demonstrate that a substantial increase in the LDOS Purcell factor is possible
as gain is added to the system. Specifically, the LDOS Purcell factor
experiences an increase by a factor of nearly 3000, when comparing the LDOS
Purcell factors with no gain and with the largest amount of gain. Since the
effective mode volume is similar with and without gain, this increase is
primarily caused by gain compensating the loss. As anticipated, there are
values of the LDOS Purcell factor that are negative for certain frequency
values (as the LDOS becomes negative), and it is necessary to look to the
total Purcell factors. We note again that there is nothing unphysical about a
negative LDOS; it means that there is more local power flow back to the dipole
location than out of the dipole, but the entire cavity system is still net
lossy, as rigorously quantified by the QNM complex poles.

Next, we show the total Purcell factor in Fig. 3c, for three different values
of gain. There are no longer any negative values for the total Purcell factor,
and the (correct) enhanced SE has been further increased, by many orders of
magnitude, and peaks at a value of $2.08\cdot 10^{10}$ when the gain value is
$2.54\cdot 10^{-1}$, yielding an increase by a factor of $7.1\cdot 10^{6}$
from the case with no gain. As anticipated by our analytical QNM theory, the
lineshapes also deviate significantly from Lorentzian and become similar to
Lorentzian-squared, as can be seen by the change in lineshape between the LDOS
and total Purcell factor curves, sharing some features and applications of
resonances near exceptional points [52, 53, 14], such as enhanced sensing.

As mentioned before, it is crucial for the region of linear amplification that
we ensure the QNM peak eigenfrequency $\tilde{\omega}_{\rm c}=\omega_{\rm
c}-i\gamma_{\rm c}$ has a positive value for $\gamma_{\rm c}$. This threshold
has been found heuristically in other works [37], however the QNM method
provides a more rigorous definition of the linear amplification regime through
the resonant eigenfrequency, and quantifies the physics in terms of the
resonant modes. The highest peak in Figs. 3b and 3c, corresponding to
$\alpha_{g}=2.54\cdot 10^{-1}$, has a peak frequency of
$\hbar\tilde{\omega}_{c}=1.195-2.238\cdot 10^{-5}i$ eV, where indeed
$\gamma_{\rm c}$ is positive.

Finally, we examine the modified beta factors. Using Eq. (14), the $\beta^{\rm
rad}$ factor can be calculated for the model with and without gain. In the
linear regime, the $\beta$-factor must have an upper limit of 1, which is the
$\beta$-factor for an ideal lossless dielectric system, and values surpassing
this limit are indicative of the lasing regime [54]. When there is no gain,
the beta factor from the metal dimer cavity (on resonance) is 0.33, and when
$\alpha_{g}=0.254$, the beta factor is 0.376, which is only slightly higher
than the case with no gain. Further insights about the beta factor over a
wider frequency range can be found in the SM [51]. Having a large radiative
beta factor is useful for many applications, such as lasing/spasing and
sensing applications. [55, 56, 57, 58].

In summary, using a rigorous and powerful QNM theory, the enhanced SE rates of
a dipole emitter in a plasmonic resonator system were studied, with and
without gain. Using material gain to compensate for the lossy nature of the
gold dimer, the LDOS and total Purcell factors were shown to be substantially
increased; the Purcell factor with no gain peaks around $2900$, but with a
maximum gain value of $\alpha_{g}=2.54\cdot 10^{-1}$ (for linear
amplification) the LDOS and total Purcell factors peak at $7.7\cdot 10^{6}$
and $2.08\cdot 10^{10}$, respectively. We also discussed how we ensure a
regime of linear amplification, and demonstrated that a single QNM theory
worked quantitatively well by comparing with numerically exact simulations
(subject to numerical limitations). Studying these systems and determining how
gain-compensation of loss impacts properties, such as the enhanced SE, are
important for developing accurate models of plasmonic lasers, quantum sensors,
and lossy cavity systems that can possibly achieve the regime of ultrastrong
light-matter coupling ($g/\omega_{c}>0.1$ [59, 60, 61], where $g$ is the
cavity-dipole coupling rate).

We acknowledge funding from Queen’s University, Canada, the Canadian
Foundation for Innovation (CFI), the Natural Sciences and Engineering Research
Council of Canada (NSERC), and CMC Microsystems for the provision of COMSOL
Multiphysics. We also acknowledge support from the Alexander von Humboldt
Foundation through a Humboldt Research Award.

## References

  * Gonçalves  _et al._ [2020] M. R. Gonçalves, H. Minassian, and A. Melikyan, Plasmonic resonators: fundamental properties and applications, Journal of Physics D: Applied Physics 53, 443002 (2020).
  * Krasnok and Alù [2020] A. Krasnok and A. Alù, Active nanophotonics, Proceedings of the IEEE 108, 628 (2020).
  * Pitarke  _et al._ [2006] J. M. Pitarke, V. M. Silkin, E. V. Chulkov, and P. M. Echenique, Theory of surface plasmons and surface-plasmon polaritons, Reports on Progress in Physics 70, 1 (2006).
  * Wang  _et al._ [2009] T.-B. Wang, X.-W. Wen, C.-P. Yin, and H.-Z. Wang, The transmission characteristics of surface plasmon polaritons in ring resonator, Opt. Express 17, 24096 (2009).
  * Xiao  _et al._ [2010] Y.-F. Xiao, B.-B. Li, X. Jiang, X. Hu, Y. Li, and Q. Gong, High quality factor, small mode volume, ring-type plasmonic microresonator on a silver chip, Journal of Physics B: Atomic, Molecular and Optical Physics 43, 035402 (2010).
  * Østerkryger [2018] A. Østerkryger,  _Engineering of nanophotonic structures for quantum information applications._ , Ph.D. thesis (2018).
  * Bai  _et al._ [2013] Q. Bai, M. Perrin, C. Sauvan, J.-P. Hugonin, and P. Lalanne, Efficient and intuitive method for the analysis of light scattering by a resonant nanostructure, Opt. Express 21, 27371 (2013).
  * Hughes  _et al._ [2019] S. Hughes, S. Franke, C. Gustin, M. Kamandar Dezfouli, A. Knorr, and M. Richter, Theory and limits of on-demand single-photon sources using plasmonic resonators: A quantized quasinormal mode approach, ACS Photonics 6, 2168 (2019).
  * Chae  _et al._ [2016] J. Chae, B. Lahiri, and A. Centrone, Engineering near-field seira enhancements in plasmonic resonators, ACS Photonics 3, 87 (2016).
  * Chikkaraddy  _et al._ [2016] R. Chikkaraddy, B. de Nijs, F. Benz, S. J. Barrow, O. A. Scherman, E. Rosta, A. Demetriadou, P. Fox, O. Hess, and J. J. Baumberg, Single-molecule strong coupling at room temperature in plasmonic nanocavities, Nature 535, 127 (2016).
  * Koenderink [2010] A. F. Koenderink, On the use of purcell factors for plasmon antennas, Optics Letters 35, 4208 (2010).
  * Kristensen and Hughes [2014] P. T. Kristensen and S. Hughes, Modes and Mode Volumes of Leaky Optical Cavities and Plasmonic Nanoresonators, ACS Photonics 1, 2 (2014).
  * Kristensen  _et al._ [2020] P. T. Kristensen, K. Herrmann, F. Intravaia, and K. Busch, Modeling electromagnetic resonators using quasinormal modes, Adv. Opt. Photonics 12, 612 (2020).
  * Ren  _et al._ [2021] J. Ren, S. Franke, and S. Hughes, Quasinormal modes, local density of states, and Classical Purcell factors for coupled loss-gain resonators, Phys. Rev. X 11, 041020 (2021).
  * Lalanne  _et al._ [2018] P. Lalanne, W. Yan, K. Vynck, C. Sauvan, and J.-P. Hugonin, Light interaction with photonic and plasmonic resonances, Laser Photonics Rev. 12, 1700113 (2018).
  * Perrin  _et al._ [2016] M. Perrin, J. Yang, and P. Lalanne, Analytical treatment of the interaction between light, plasmonic and quantum resonances: quasi-normal mode expansion, in  _Quantum Sensing and Nano Electronics and Photonics XIII_, Vol. 9755, edited by M. Razeghi, International Society for Optics and Photonics (SPIE, 2016) p. 97551J.
  * Franke  _et al._ [2019] S. Franke, S. Hughes, M. K. Dezfouli, P. T. Kristensen, K. Busch, A. Knorr, and M. Richter, Quantization of quasinormal modes for open cavities and plasmonic cavity quantum electrodynamics, Phys. Rev. Lett. 122, 213901 (2019).
  * [18] C. P. Van Vlack, Dyadic Green functions and their applications in classical and quantum nanophotonics, phD Thesis, Queen’s University (2012).
  * Ge  _et al._ [2014] R.-C. Ge, P. T. Kristensen, J. F. Young, and S. Hughes, Quasinormal mode approach to modelling light-emission and propagation in nanoplasmonics, New J. Phys. 16, 113048 (2014).
  * Dung  _et al._ [1998] H. T. Dung, L. Knöll, and D.-G. Welsch, Three-dimensional quantization of the electromagnetic field in dispersive and absorbing inhomogeneous dielectrics, Phys. Rev. A 57, 3931 (1998).
  * Righini  _et al._ [2007] M. Righini, A. Zelenina, and C. e. a. Girard, Parallel and selective trapping in a patterned plasmonic landscape, Nature Physics 3, 477–480 (2007).
  * Sharma and Shrivastava [2008] A. Sharma and S. C. Shrivastava, Analysis of resonant frequency and quality factor of dielectric resonator at different dielectric constant materials, in  _2008 International Conference on Recent Advances in Microwave Theory and Applications_ (2008) pp. 593–595.
  * Cognée  _et al._ [2019] K. G. Cognée, H. M. Doeleman, P. Lalanne, and A. F. Koenderink, Cooperative interactions between nano-antennas in a high-Q cavity for unidirectional light sources, Light: Sci. Appl. 8, 115 (2019).
  * Melli  _et al._ [2013] M. Melli, A. Polyakov, D. Gargas, C. Huynh, L. Scipioni, W. Bao, D. F. Ogletree, P. J. Schuck, S. Cabrini, and A. Weber-Bargioni, Reaching the theoretical resonance quality factor limit in coaxial plasmonic nanoresonators fabricated by helium ion lithography, Nano Letters 13, 2687 (2013).
  * Lilley  _et al._ [2015] G. Lilley, M. Messner, and K. Unterrainer, Improving the quality factor of the localized surface plasmon resonance, Opt. Mater. Express 5, 2112 (2015).
  * Kuttge  _et al._ [2010] M. Kuttge, F. J. García de Abajo, and A. Polman, Ultrasmall mode volume plasmonic nanodisk resonators, Nano Letters 10, 1537 (2010), pMID: 19813755.
  * Ren  _et al._ [2020] J. Ren, S. Franke, A. Knorr, M. Richter, and S. Hughes, Near-field to far-field transformations of optical quasinormal modes and efficient calculation of quantized quasinormal modes for open cavities and plasmonic resonators, Phys. Rev. B 101, 205402 (2020).
  * Stockman [2008] M. I. Stockman, Spasers explained, Nature Photonics 2, 327 (2008).
  * Stockman [2011a] M. I. Stockman, Spaser action, loss compensation, and stability in plasmonic systems with gain, Physical Review Letters 106, 156802 (2011a).
  * Stockman [2011b] M. I. Stockman, Loss compensation by gain and spasing, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 369, 3510 (2011b).
  * Russev  _et al._ [2012] S. C. Russev, G. G. Tsutsumanova, and A. N. Tzonev, Conditions for loss compensation of surface plasmon polaritons propagation on a metal/gain medium boundary, Plasmonics 7, 151 (2012).
  * Veltri and Aradian [2012] A. Veltri and A. Aradian, Optical response of a metallic nanoparticle immersed in a medium with optical gain, Phys. Rev. B 85, 115429 (2012).
  * De Leon and Berini [2010] I. De Leon and P. Berini, Amplification of long-range surface plasmons by a dipolar gain medium, Nature Photonics 4, 382–387 (2010).
  * Noginov [2008] M. A. Noginov, Compensation of surface plasmon loss by gain in dielectric medium, Journal of Nanophotonics 2, 021855 (2008).
  * Berini and De Leon [2011] P. Berini and I. De Leon, Surface plasmon–polariton amplifiers and lasers, Nature Photonics 6, 16–24 (2011).
  * Pustovit  _et al._ [2015] V. Pustovit, F. Capolino, and A. Aradian, Cooperative plasmon-mediated effects and loss compensation by gain dyes near a metal nanoparticle, J. Opt. Soc. Am. B 32, 188 (2015).
  * Liu  _et al._ [2011] S.-Y. Liu, J. Li, F. Zhou, L. Gan, and Z.-Y. Li, Efficient surface plasmon amplification from gain-assisted gold nanorods, Optics Letters 36, 1296 (2011).
  * Cai  _et al._ [2018] J. Cai, Y. J. Zhou, Y. Zhang, and Q. Y. Li, Gain-assisted ultra-high-q spoof plasmonic resonator for the sensing of polar liquids, Opt. Express 26, 25460 (2018).
  * Fang  _et al._ [2011] A. Fang, Z. Huang, T. Koschny, and C. M. Soukoulis, Overcoming the losses of a split ring resonator array with gain, Opt. Express 19, 12688 (2011).
  * Ding  _et al._ [2013] P. Ding, J. He, J. Wang, C. Fan, G. Cai, and E. Liang, Low-threshold surface plasmon amplification from a gain-assisted core–shell nanoparticle with broken symmetry, Journal of Optics 15, 105001 (2013).
  * Wang  _et al._ [2017] Z. Wang, X. Meng, A. V. Kildishev, A. Boltasseva, and V. M. Shalaev, Nanolasers enabled by metallic nanoparticles: From spasers to random lasers, Laser & Photonics Reviews 11, 1700212 (2017).
  * Zhong and Li [2013] X.-L. Zhong and Z.-Y. Li, All-analytical semiclassical theory of spaser performance in a plasmonic nanocavity, Phys. Rev. B 88, 085101 (2013).
  * Zhong  _et al._ [2013] X. L. Zhong, M. H. Hong, and Z. Y. Li, Spaser in plasmonic nano-antenna evaluated by an analytical theory, Applied Physics A 115, 5–11 (2013).
  * Warnakula  _et al._ [2018] T. Warnakula, M. I. Stockman, and M. Premaratne, Improved scheme for modeling a spaser made of identical gain elements, J. Opt. Soc. Am. B 35, 1397 (2018).
  * Gaponenko [2010] S. V. Gaponenko,  _Introduction to Nanophotonics_ (Cambridge University Press, 2010).
  * Kristensen  _et al._ [2014] P. T. Kristensen, J. R. de Lasson, and N. Gregersen, Calculation, normalization, and perturbation of quasinormal modes in coupled cavity-waveguide systems, Opt. Lett. 39, 6359 (2014).
  * Franke  _et al._ [2021] S. Franke, J. Ren, M. Richter, A. Knorr, and S. Hughes, Fermi’s golden rule for spontaneous emission in absorptive and amplifying media, Phys. Rev. Lett. 127, 013602 (2021).
  * Ren  _et al._ [2023] J. Ren, S. Franke, B. VanDrunen, and S. Hughes, Classical Purcell factors and spontaneous emission decay rates in a linear gain medium (2023), arXiv:2305.12049 [physics, physics:quant-ph].
  * Leung  _et al._ [1994] P. T. Leung, S. Y. Liu, and K. Young, Completeness and orthogonality of quasinormal modes in leaky optical cavities, Phys. Rev. A 49, 3057 (1994).
  * Anger  _et al._ [2006] P. Anger, P. Bharadwaj, and L. Novotny, Enhancement and quenching of single-molecule fluorescence, Phys. Rev. Lett. 96, 113002 (2006).
  * [51] See supplemental material at [url will be inserted by publisher] for [give brief description of material].
  * Pick  _et al._ [2017] A. Pick, B. Zhen, O. D. Miller, C. W. Hsu, F. Hernandez, A. W. Rodriguez, M. Soljačić, and S. G. Johnson, General theory of spontaneous emission near exceptional points, Opt. Express 25, 12325 (2017).
  * Miri and Alù [2019] M.-A. Miri and A. Alù, Exceptional points in optics and photonics, Science 363, eaar7709 (2019).
  * Carlson and Hughes [2020] C. Carlson and S. Hughes, Dissipative modes, Purcell factors, and directional beta factors in gold bowtie nanoantenna structures, Phys. Rev. B 102, 155301 (2020).
  * Kewes  _et al._ [2017] G. Kewes, K. Herrmann, R. Rodríguez-Oliveros, A. Kuhlicke, O. Benson, and K. Busch, Limitations of particle-based spasers, Phys. Rev. Lett. 118, 237402 (2017).
  * Mikhailova  _et al._ [2019] T. Mikhailova, A. Shaposhnikov, S. Tomilin, and D. Alentiev, Nanostructures with magnetooptical and plasmonic response for optical sensors and nanophotonic devices, in  _Journal of Physics: Conference Series_ , Vol. 1410 (IOP Publishing, 2019) p. 012163.
  * Gao  _et al._ [2020] Y. Gao, H. Wu, N. Zhou, Y. Yang, X. Guo, P. Wang, J. You, Y. Gu, G. Lu, Q. Gong, and L. Tong, Single-nanorod plasmon nanolaser: A route toward a three-dimensional ultraconfined lasing mode, Phys. Rev. A 102, 063520 (2020).
  * Caicedo  _et al._ [2022] K. Caicedo, A. Cathey, M. Infusino, A. Aradian, and A. Veltri, Gain-driven singular resonances in active core-shell and nano-shell plasmonic particles, J. Opt. Soc. Am. B 39, 107 (2022).
  * Kockum  _et al._ [2019] A. F. Kockum, A. Miranowicz, S. D. Liberato, S. Savasta, and F. Nori, Ultrastrong coupling between light and matter, Nature Reviews Physics 1, 19 (2019).
  * Forn-Díaz  _et al._ [2019] P. Forn-Díaz, L. Lamata, E. Rico, J. Kono, and E. Solano, Ultrastrong coupling regimes of light-matter interaction, Rev. Mod. Phys. 91, 025005 (2019).
  * Salmon  _et al._ [2022] W. Salmon, C. Gustin, A. Settineri, O. D. Stefano, D. Zueco, S. Savasta, F. Nori, and S. Hughes, Gauge-independent emission spectra and quantum correlations in the ultrastrong coupling regime of open system cavity-QED, Nanophotonics 11, 1573 (2022).