# Achieving Conversational Goals with
Unsupervised Post-hoc Knowledge Injection

Bodhisattwa Prasad Majumder Department of Computer Science and Engineering, UC
San Diego
{bmajumde, tberg<EMAIL_ADDRESS>Harsh Jhamtani School of Computer
Science, Carnegie Mellon University
<EMAIL_ADDRESS>Taylor Berg-Kirkpatrick Department of Computer Science and
Engineering, UC San Diego
{bmajumde, tberg<EMAIL_ADDRESS>Julian McAuley Department of Computer
Science and Engineering, UC San Diego
{bmajumde, tberg<EMAIL_ADDRESS>
###### Abstract

A limitation of current neural dialog models is that they tend to suffer from
a lack of specificity and informativeness in generated responses, primarily
due to dependence on training data that covers a limited variety of scenarios
and conveys limited knowledge. One way to alleviate this issue is to extract
relevant knowledge from external sources at decoding time and incorporate it
into the dialog response. In this paper, we propose a post-hoc knowledge-
injection technique where we first retrieve a diverse set of relevant
knowledge snippets conditioned on both the dialog history and an initial
response from an existing dialog model. We construct multiple candidate
responses, individually injecting each retrieved snippet into the initial
response using a gradient-based decoding method, and then select the final
response with an unsupervised ranking step. Our experiments in goal-oriented
and knowledge-grounded dialog settings demonstrate that human annotators judge
the outputs from the proposed method to be more engaging and informative
compared to responses from prior dialog systems. We further show that
knowledge-augmentation promotes success in achieving conversational goals in
both experimental settings.

##  1 Introduction

Generic responses which lack specificity have been a major issue in existing
dialog models Hosseini-Asl et al. (2020); Dinan et al. (2019a). The issue in
part stems from bottlenecks in dialog models due to a limited scope of
scenarios and access to limited knowledge available during training. On the
other hand, encoding all possible world knowledge at training time is not
feasible, and even undesirable in cases where knowledge sources are
dynamically varying Ghazvininejad et al. (2018); Majumder et al. (2020b); Zhao
et al. (2020); Bruyn et al. (2020); Kim et al. (2020); Prabhumoye et al.
(2021). One possible approach is to incorporate relevant knowledge at
decoding-time. For example, in Figure 1, the user is seeking options for a fun
activity around Cambridge. While the initial dialog response suggests watching
a movie as an option, it does not provide any information behind that choice.

Figure 1: Augmenting initial response from an existing dialog model with
relevant external knowledge leads to more engaging and informative responses
improving the success in achieving the conversational goal (here, finding a
fun activity).  Figure 2: Pipeline of POKI: It first retrieves post-hoc
knowledge from external sources based on dialog history and an initial
response from a dialog model. Then the most relevant and diverse knowledge
snippets are selected from the retrieved set. Each selected snippet is
individually combined with the initial response through constrained decoding
to generate a candidate final response. At last, the final response is
selected via an unsupervised ranking step. Note that POKI requires no
additional training.

We propose and evaluate an approach for unsupervised knowledge injection into
a dialog model’s response at decoding time111Code:
https://github.com/majumderb/poki\---not addressed in any previous work. We
first sample a response from the model (trained on dialog data) conditioned on
the dialog context. Next, we utilize the dialog context and the sampled
response to query external knowledge sources. Finally, the retrieved knowledge
is used to construct a more informative and engaging response (Figure 1). A
major advantage of such post-hoc knowledge injection is its flexibility in
adding newer knowledge sources especially where the success of achieving
conversational goals relies upon the availability of relevant knowledge. Post-
hoc injection also promotes efficiency in NLP applications Schwartz et al.
(2020); Strubell et al. (2019): it mitigates the need to retrain dialog models
to accommodate dynamically evolving knowledge.

We experiment with two types of knowledge sources: language models, which we
treat as parametric knowledge bases Petroni et al. (2019); Brown et al.
(2020); and user review datasets such as Yelp reviews Hajas et al. (2014) as
non-parametric knowledge sources (§ 2). Since it is possible to gather a large
amount of related knowledge given a query, we select a relevant and diverse
(estimated via information-theoretic measures) subset of knowledge snippets
using an unsupervised method (§ 3.1). Then, a gradient-based inference
approach is used to construct an updated response that incorporates the
selected knowledge (§ 3.2). Note that our framework does not require
retraining the existing dialog model---it only relies upon updating the
model’s output hidden states at decoding time for unsupervised knowledge
injection.

We experiment with two scenarios: goal-oriented and knowledge-grounded dialog
where the training data covers only a fraction of the needed knowledge.
Automatic evaluation reveals that our method is capable of generating highly
diverse responses in both settings. In some cases, the generated response
shows high overlap with the original target response showing that our
unsupervised method bridges the knowledge gap between available knowledge and
human-written responses present in the existing dialog corpus. An extensive
human evaluation confirms that generated responses are indeed engaging,
interesting, and human-like without any loss in fluency.

To pinpoint the usefulness of knowledge injection in the above settings, we
design a real-time study (§ 5.3) where users interact with our system to reach
a conversational goal (e.g. planning a holiday or knowing more about the solar
system). We find that external knowledge enables users to achieve their goals
more efficiently. Additionally, we observe that the our approach of sub-
selecting relevant but diverse knowledge leads to responses that promote
success in achieving conversational goals.

##  2 Post-hoc Knowledge for Dialog

Our goal is to construct a dialog response by injecting knowledge (from
external textual sources) at decoding time, without having to retrain the
models. Consider a dialog model $\mathcal{M}$ from which we can sample a
dialog response $x^{d}$ given a dialog history $\mathcal{H}$. We shall refer
to the response $x^{d}$ sampled from such a model without any decoding time
knowledge injection as the _initial_ response.

However, as motivated earlier, samples from such a dialog model often lack
detail. To improve such responses, we retrieve and incorporate relevant
external knowledge $k$ into the initial response. To achieve our goal, we
construct a query using both dialog history $\mathcal{H}$ and the initial
response $x^{d}$, and gather a relevant knowledge candidate $k$ from a
knowledge source $\mathcal{K}$. The retrieved snippet can provide useful
information to the end-user to achieve the conversational goal (see § 5.3). We
explore both parametric (e.g querying a language model) and non-parametric
(e.g. deterministic retrieval using word-overlap) ways to obtain post-hoc
knowledge.

###  2.1 Parametric knowledge sources

Pretrained language models (PTLM) are typically trained with a vast amount of
text that spans a diverse range of domains. Petroni et al. (2019); Brown et
al. (2020) showed that such PTLMs can be used as a source of knowledge when
queried with suitable textual prompts (e.g. Seattle is famous for ). To use
PTLMs in our use-case, we construct useful prompts from dialog history and the
initial response. We assemble simple prompts inspired from various knowledge-
seeking situations in dialog Shwartz et al. (2020) such as [KP] is famous for
, Here is what I know about [KP]:  , where [KP] is a key-phrase222It possible
that a lack of key-phrases results in no knowledge. Key-phrase extraction
details are in Appendix B.  extracted from dialog context. We use gpt2-large
as the PTLM. For example, a query ‘‘Here is what I know about fun things
around Cambridge:" results in ‘‘There are plenty of museums to visit around
Cambridge. If you love hiking, you can enjoy the trails alongside the river…"
as shown in Figure 1. A complete list of prompts is provided in Appendix B. We
finally rank each knowledge snippet $k$ using the likelihood obtained from the
PTLM for a concatenated input of $k$ and dialog history and choose the most
likely.

###  2.2 Non-parametric knowledge sources

External knowledge in the form of a text corpus can be used as a non-
parametric knowledge source available at decoding time. Compared to parametric
knowledge sources, such sources do not generate text as knowledge snippets,
but offer the advantage of high quality and reliability of human written text.
We consider the dialog history and the _initial_ response as a query to
retrieve relevant knowledge instances from the corpus. Next, we identify the
top relevant instances in the given corpus with respect to the constructed
query using cosine similarity on TF-IDF based representations Robertson et al.
(1995).

##  3 Unsupervised Knowledge Injection in Generated Dialog

Effectively utilizing the retrieved knowledge snippets to construct an
enriched dialog response encompasses two major challenges. Firstly, it is not
practical to use potentially hundreds of knowledge snippets obtained from the
retrieval step for a single response generation. Thus, we need to find a
relevant but diverse subset of the snippets. Secondly, the dialog model
$\mathcal{M}$ is trained to condition only on the dialog context, and not on
the external knowledge. Hence, to leverage the knowledge snippets, we need a
decoding strategy to rewrite the initial response $x^{d}$ such that the
resulting final response $x^{f}$ should closely follow the knowledge snippet
to be injected without a loss in the fluency and consistency. Thus, our method
requires no additional training and only assumes a language model trained on
dialog context (i.e. $\mathcal{M}$). We refer to our proposed framework
(Figure 2) as POKI (Post-hoc Knowledge Injection in Generated Dialog).

###  3.1 Relevance-Redundancy Tradeoff for Knowledge Selection

At each turn, we obtain $N$ knowledge snippets from both the parametric and
non-parametric sources. We wish to select a subset of $B$ (out of $N$)
relevant but diverse knowledge snippets.

We define relevance score of a snippet $k_{i}$ with respect to the dialog
history $H$ using pointwise mutual information (PMI) as follows:

$\mathbb{REL}_{i}=\operatorname{PMI}(k_{i},\mathcal{H})=\log\left(\frac{p(\mathcal{H}|k_{i})}{p(\mathcal{H})}\right),$

Thus, a high PMI score would imply a larger semantic similarity between the
snippet $k_{i}$ and $H$. To account for redundancy between the snippet pair
$k_{i}$, $k_{j}$ we again use the PMI score as follows:

$\mathbb{RED}_{ij,j>i}=\operatorname{PMI}(k_{i},k_{j})=\log\left(\frac{p(k_{j}|k_{i})}{p(k_{j})}\right).$

The redundancy score is symmetric i.e. $\mathbb{RED}_{ij}=\mathbb{RED}_{ji}$
as $\operatorname{PMI}$ is a symmetric measure.

We estimate probabilities (both conditional and marginal) $p(.)$ in the above
equations using GPT2 language model, following past work Padmakumar and He
(2021). The PMI measure is often considered better than other n-gram-based
overlap metrics to measure the degree of association between two sentences
Kedzie et al. (2018); Padmakumar and He (2021). Semantically similar phrases
occur in both sentences that can easily be ignored by overlap based metrics.

#### Selection via Determinantal Point Processes.

To select $B$ knowledge snippets out of $N$ with a relevance-redundancy trade-
off, we use a subset selection process named Determinantal Point Process (DPP)
Kulesza and Taskar (2011). DPP employs a non-uniform selection that assigns
low probability to subsets (here, of knowledge snippets) that are less diverse
by modeling the repulsive correlation between independently occurring
datapoints (see Figure 2).

We build an $N\times N$ kernel matrix $\mathcal{D}$, which is real, symmetric
and positive semi-definite. The diagonal entries $\mathcal{D}_{ii}$ are
populated by the squared relevance score of the $i$-th knowledge
$\mathbb{REL}_{i}$ and the off-diagonal entries $\mathcal{D}_{ij}$ are
$\beta\,\times$ squared redundancy scores $\mathbb{RED}_{ij}$. We adjust
$\beta$ in such a way that $\mathcal{D}$ always remains positive semi-definite
(more details in Wilhelm et al. (2018)). To select a subset of $B$, a DPP
assigns a probability of sampling such a subset proportional to the
determinant of the submatrix $\mathcal{D}_{B}$ of $\mathcal{D}$, constructed
using the indices of the subsetted items. The DPP probability is geometrically
related to the volume of the parallelepiped spanned by the selected knowledge
snippets. Diverse knowledge snippets tend to be orthogonal in their space
hence span larger volume Kulesza and Taskar (2012).

Choosing $B$-size submatrix from $N$-size $\mathcal{D}$ is a combinatorial
problem and can become prohibitively costly when $N$ is very high. Hence, we
use a greedy method Wilhelm et al. (2018) where we initialize the selection
with the most relevant $k_{i}$ and subsequently select the next $k_{j}$ that
maximizes the determinant of the resultant submatrix.

###  3.2 Gradient-based Constrained Decoding for Knowledge Injection

Upon selecting $B$ knowledge snippets, we want to individually inject each
knowledge snippet into $x^{d}$ to construct a candidate final response $x^{f}$
at inference time.

Previous works have addressed the problem of unsupervised modification of
already-generated text using gradient-based decoding Dathathri et al. (2020);
Qin et al. (2020) that employs an iterative procedure consisting of a forward
and a backward pass. The forward pass on the generative model (here,
$\mathcal{M}$) encourages fluency of the generated text while the backward
pass performs gradient ascent on certain desired constraints. Note that due to
the discrete nature of $x_{d}$, it is not possible to directly update it via
back-propagation. Therefore, we maintain the sequence of hidden
representations of each output token as $z$ from the dialog model. Each output
token $x^{d}_{(t)}$ is realized via
$p(x^{d}_{(t)})\sim\operatorname{softmax}(Wz_{(t)}/\tau)$, where $\tau$ is the
temperature hyperparameter, $W$ is the output embedding matrix (shared with
the input), and $Wz_{(t)}\in\mathcal{R}^{V}$ ($V$ is the size of the
vocabulary).

Constraints. Following Majumder et al. (2021a), we define a knowledge fidelity
objective that encourages $x^{f}$ to be minimally different from the knowledge
snippet $k$. We achieve this by minimizing the cross entropy loss
($\operatorname{CE}$) between knowledge tokens $k_{(1)},\ldots,k_{(T)}$ as
labels and $Wz_{(1)},\ldots,Wz_{(T)}$ as the logits.

We further notice that injected knowledge can influence the generation in such
a way that it contradicts with responses uttered during previous turns. Hence,
we also want $x^{f}$ to be entailed with the dialog history $\mathcal{H}$. We
build an entailment classifier $\theta(z,\mathcal{H})$ that predicts the
probability of $x^{f}$ (ideally, the hidden representation $z$ of $x^{f}$)
entailing $\mathcal{H}$. The classifier $\theta(z,\mathcal{H})$ is a bag-of-
words classification layer with hidden states $z$ from $\mathcal{M}$ and fine-
tuned using the DNLI dataset Welleck et al. (2019) to predict whether the
current response is entailed with previous responses or not.

Decoding. In the subsequent forward and backward passes, the hidden
representation $z$ is gradually perturbed via gradient ascent on the
respective objectives. During backward pass, the objective with constraints is

$\mathcal{L}(\mathcal{H},k;z)=\alpha\log\theta(z,\mathcal{H})-\lambda\operatorname{CE}(k,Wz)$

with hyperparameters $\alpha$ and $\lambda$. We use back-propagation to update
$z$ with the gradient $\nabla_{z}\mathcal{L}(\mathcal{H},k;z)$ while the
parameters of $\mathcal{M}$ remain fixed. The updated latent representations
of $z$ after the backward pass are denoted as $z^{\mathit{bw}}$.

A forward pass with $\mathcal{M}$ is required to regularize the hidden states
$z$ toward the original dialog model objective to obtain $z^{\mathit{fw}}$.
Corresponding to the $t^{\text{th}}$ token, the hidden states for the
${t+1}^{\text{th}}$ time step are computed via a weighted addition of backward
and forward hidden states, i.e., $z_{(t+1)}=\gamma\times
z^{\mathit{bw}}_{(t)}+(1-\gamma)\times z^{\mathit{fw}}_{(t)}$ where
$\gamma\in(0,1)$ is a hyperparameter.

During generation, we start by sampling the initial response $x^{d}$ with
greedy decoding from $\mathcal{M}$. The hidden states $z$ (of $x^{d}$) are
iteratively updated by alternate backward and forward passes. The final
response is sampled as $x^{f}\sim\operatorname{softmax}(Wz/\tau)$. The number
of iterations ($=5$) and the $\gamma$ ($=0.45$) were chosen by maximizing the
Z-normalized sum of dialog model perplexity and linguistic diversity (% of
distinct bigrams) in a greedy hyperparameter search. More details are in
Appendix B.

System | Acc | BLEU | BRTSc | D-2 | ENTR
---|---|---|---|---|---
KCopy | 70.1 | 4.1 | 62.3 | 3.16 | 2.41
SimpleTOD Hosseini-Asl et al. (2020) | 70.1 | 15.0 | 79.2 | 0.56 | 0.90
SimpleTOD+ Sun et al. (2021) | 69.8 | 12.1 | 68.1 | 0.81 | 1.11
Arranger Sun et al. (2021) | 70.2 | 12.3 | 68.5 | 0.93 | 1.15
Rewriter Sun et al. (2021) | 70.2 | 12.1 | 69.4 | 1.03 | 1.45
POKI | 71.1 | 13.7 | 74.5 | 3.78 | 2.67
w/o Entailment | 69.9 | 10.9 | 67.8 | 3.67 | 2.56
w/o Kw Fidelity | 70.0 | 12.3 | 71.2 | 0.95 | 1.19
Gold | 100 | 100 | 100 | 0.78 | 0.86
Table 1:  Automatic metrics on the test set of MultiWoZ. Difference between
bold and non-bold numbers is statistically significant ($p<0.001$).

System | BLEU | BRTSc | D-2 | ENTR
---|---|---|---|---
KCopy | 13.4 | 74.3 | 3.64 | 3.12
KGuide Zhao et al. (2017) | 16.7 | 71.5 | 2.54 | 2.12
KGround Wolf et al. (2019) | 18.3 | 72.5 | 2.87 | 2.35
BART Lewis et al. (2020a) | 19.8 | 73.4 | 2.97 | 2.55
RAG Lewis et al. (2020b) | 19.9 | 73.1 | 1.03 | 1.45
POKI | 19.4 | 76.8 | 3.65 | 3.44
w/o Entailment | 18.1 | 74.2 | 3.17 | 3.39
w/o Kw Fidelity | 18.8 | 73.3 | 2.75 | 2.54
Gold | 100 | 100 | 2.98 | 2.59
Table 2:  Automatic metrics on the test set of Wizard-of-Wikipedia. Difference
between bold and non-bold numbers is statistically significant ($p<0.001$).

###  3.3 Unsupervised Ranking of Candidate Final Responses

Several previous works often over-generate and use an additional ranking step
in order to select the final candidate in unsupervised text generation Qin et
al. (2020); Shwartz et al. (2020); Paranjape and Manning (2021). Similarly,
here we want to rank the generated candidate final responses according to the
diversity of the generated text as well as the conditional likelihood of
generation given the dialog history. For diversity, we measure the percentage
of distinct bigrams present in the response. For conditional likelihood, we
use the pre-trained GPT2 model to obtain the log probability when the dialog
history, followed by the generated response, passed as a concatenated input.
Since these two scores can have varied scale, we perform Z-normalization on
the individual scores and add them to obtain a single score for ranking. The
highest ranked candidate response is finally rendered to the user.

##  4 Experimental Setup

###  4.1 Scenarios and Datasets

We experiment with two dialog scenarios: goal-oriented and knowledge grounded.
Both setups are knowledge intensive but the training data in such setups often
contains only a fraction of the needed knowledge. For the goal-oriented
setting, we use the Multi-domain Wizard-of-Oz Budzianowski et al. (2018)
dataset. For knowledge grounded dialog, we use the Wizard-of-Wikipedia Dinan
et al. (2019b) dataset. More details are in Appendix A.

Multi-domain Wizard-of-Oz (MultiWOZ) is a multi-domain dialog dataset (we use
v2.0 Hosseini-Asl et al. (2020)) consisting of goal-oriented human-human
conversations. The dataset spans seven domains (restaurant, train, attraction,
hotel, taxi, hospital, police) and contains 10,438 dialogs with 13.68 average
turns. Since, we do not need any training data, we only use an evaluation set
(of 7K utterances).

Wizard-of-Wikipedia (WoW) is a knowledge grounded dialog dataset which
involves retrieving relevant knowledge from Wikipedia, reading and
conditioning on it, and finally generating dialog responses Dinan et al.
(2019b). The dataset contains 201K utterances from 22K dialogues spanning 1300
diverse topics, from which we use only the test set. The associated Wikipedia
knowledge base has 5.4M articles and 93M sentences.

###  4.2 Baselines and Ablations

Baselines for MultiWOZ. For MultiWOZ, we consider several baselines following
Sun et al. (2021) for knowledge injection. First, we use the current state-of-
the-art model, SimpleTOD, for goal-oriented dialog Hosseini-Asl et al. (2020).
Sun et al. (2021) extends SimpleTOD by adding chitchat candidates to dialog
histories during training. They also have other variants that either
concatenate output from SimpleTOD and candidate chitchats (Arranger) or
rewrite by combining both output and chitchat snippets (Rewriter). We also
have a trivial baseline (KCopy) which appends the retrieved knowledge snippet
$k$ from POKI with the initial response $x_{d}$.

Baselines for WoW. For WoW, we use two current-best knowledge-grounded models,
KGround Wolf et al. (2019) and BART Lewis et al. (2020a) that concatenate the
associated knowledge snippets (present in WoW) and the dialog history as
inputs to generate the response with supervision. KGuide Zhao et al. (2017)
and RAG Lewis et al. (2020b) have an additional knowledge selection step
modeled by a latent variable before response generation similar to knowledge
grounded models. We also use the KCopy baseline, as described for MultiWOZ.

Variants of POKI. To investigate the impact of various decoding constraints in
POKI, we consider the following two variants of POKI\---w/o Entailment and w/o
Knowledge (Kw) Fidelity (§ 3.2). In POKI, we use SimpleTOD as the base dialog
model in goal-oriented scenarios and use BART (which is a state-of-the-art
model for WoW) as the base dialog model in the knowledge-grounded scenario.
For all variants of POKI, we use gradient-based inference for decoding the
final response.

| POKI vs | SimpleTOD | Rewriter | w/o Entailment | w/o Kw Fidelity | Gold
---|---|---|---|---|---|---
| Criteria | win | loss | $\kappa$ | win | loss | $\kappa$ | win | loss | $\kappa$ | win | loss | $\kappa$ | win | loss | $\kappa$
MultiWOZ | Coherent | 93.2 | 4.4 | 0.76 | 85.6 | 10.2 | 0.75 | 98.7 | 0.8 | 0.72 | 77.8 | 17.8 | 0.78 | 26.2 | 34.4 | 0.69
Engaging | 94.3 | 4.5 | 0.78 | 89.7 | 7.9 | 0.79 | 98.7 | 0.6 | 0.80 | 71.5 | 20.5 | 0.80 | 42.4 | 37.4 | 0.78
Interesting | 92.7 | 5.4 | 0.72 | 91.2 | 8.3 | 0.73 | 88.6 | 8.9 | 0.68 | 98.7 | 0.8 | 0.75 | 49.7 | 45.6 | 0.67
Humanlike | 85.4 | 10.7 | 0.68 | 87.4 | 7.3 | 0.65 | 61.9 | 30.5 | 0.71 | 81.7 | 14.0 | 0.74 | 29.7 | 37.8 | 0.66
|  | RAG | BART | w/o Entailment | w/o Kw Fidelity | Gold
WoW | Coherent | 95.4 | 4.5 | 0.78 | 88.5 | 9.6 | 0.72 | 94.3 | 3.4 | 0.68 | 83.6 | 10.7 | 0.65 | 23.8 | 25.3 | 0.73
Engaging | 89.3 | 7.7 | 0.72 | 87.8 | 8.3 | 0.71 | 97.7 | 0.8 | 0.70 | 71.5 | 25.4 | 0.69 | 25.4 | 26.7 | 0.73
Interesting | 96.3 | 3.5 | 0.74 | 83.3 | 9.9 | 0.75 | 79.8 | 17.2 | 0.70 | 93.5 | 4.5 | 0.71 | 35.9 | 37.8 | 0.76
Humanlike | 91.4 | 7.1 | 0.68 | 92.4 | 6.5 | 0.66 | 84.5 | 10.5 | 0.67 | 81.8 | 13.5 | 0.71 | 42.3 | 41.9 | 0.68

Table 3: Pairwise comparison (% win/loss cases, tie not reported) between
responses from POKI and from other baselines as well as ground truth.
Difference between bold and non-bold numbers is statistically significant
($p<0.001$). $\kappa$ denotes Cohen’s Kappa Cohen (1960) between a pair of
annotators. Complete details of the human evaluation are in Appendix C.

##  5 Results and Discussion

###  5.1 Automatic Evaluation

Our primary goal is to generate responses enriched with relevant external
knowledge. Arguably, a system which can effectively leverage additional
knowledge at decoding time should generate more diverse responses. We measure
percentage of distinct bigrams as Distinct-(D-2) Li et al. (2016) and
geometric mean of entropy values of empirical frequency distributions of
n-grams ($n=1,2,3$) as Entropy (ENTR) Jhamtani et al. (2018) for diversity.
Additionally, we report overlap between generated responses and corresponding
ground truth as per BLEU and BERTScore (BRTSc). For MultiWOZ, we also report
the final goal accuracy (Acc) following Hosseini-Asl et al. (2020).

MultiWOZ. Table 2 shows POKI outperforms all the baselines in terms of
diversity of generated responses. More importantly, we see POKI promotes
accuracy of reaching the final dialog state i.e. the goal. For ablated
versions of POKI, we find the entailment constraint has little effect on
diversity while dropping the knowledge adherence constraint negatively
influences accuracy and diversity. All variants of SimpleTOD and all versions
of POKI show departure from the results obtained by SimpleTOD on BLEU and
BERTScore since all of these versions add external knowledge that were not
explicitly present in the data. However, we observe that the departure is not
significant and POKI achieves a much closer BERTScore to SimpleTOD compared to
baselines.

WoW. Despite all systems for WoW use knowledge explicitly in the knowledge-
grounded dialog generation task, Table 2 shows POKI generates the most diverse
responses. Similar to MultiWOZ, the knowledge adherence constraint still
remains a significant factor for increasing diversity, one of the main goals
of knowledge injection. For WoW, we instead see POKI outperform even BART
(previous SOTA) in terms of BERTScore when injected with external knowledge
indicating the need of the external knowledge for modeling WoW dialogs.

Figure 3: POKI converses with a user who is looking for some restaurant
options (left column). In each turn, a knowledge snippet (right column) is
injected into an initial response (middle column). More examples are in
Appendix D.

###  5.2 Human Evaluation

We conduct a comparative human evaluation with 300 samples to evaluate the
quality of generated dialog responses following ACUTE-Eval Li et al. (2019).
We show a generated response from POKI to an annotator with its associated
dialog history to annotate if knowledge injection makes the final response
more engaging, interesting and humanlike compared to a baseline response. As
sanity check, we also investigate if the response remain coherent after
knowledge injection. Each sample is evaluated by two annotators333More details
of the setup are in Appendix C. .

MultiWOZ. Table 3 records the pairwise comparison showing POKI consistently
outperforms baselines on all criteria. Responses from POKI are more engaging
and interesting compared to SimpleTOD and Rewriter, demonstrating that
gradient-based decoding is effective for knowledge injection. In POKI,
entailment constraint mostly influences coherence whereas knowledge fidelity
constraint is important for engagingness and interestingness.

WoW. Table 3 shows POKI outperforms baselines that use grounding knowledge
during training in all criteria showing that external knowledge can be useful
even in the knowledge-grounded setting to make the conversation engaging and
interesting. It also indicates the limitation of the training signal or lack
of access to sufficient knowledge and room for improvement in terms of how
knowledge is utilized. A large gap in win percentages in favor of POKI for
evaluating how ‘humanlike’ is a response when compared to state-of-the-art
methods suggests knowledge injection leads to more natural conversation. Here
too, both decoding constraints show similar trends to MultiWOZ.

Qualitative Analysis. Figure 3 shows a conversation by POKI with a user who
seeks to find restaurant options around Cambridge. We observe that in most of
the turns the injected knowledge appeared as an additional justification over
the initial responses making the dialog engaging and effective to reach the
user’s goal (also noted by human judges in § 5.3). For example, in turn 3, we
observe that adding the extra information about Indian cuisine helped user to
reach a conclusion when their original choice of English cuisine was absent.

Effect of Response Length. Qualitatively, as seen in Figure 3, responses
generated by POKI are longer than those from the initial response due to the
post-hoc knowledge injection. In the human evaluation sample, we found that
37% of responses from POKI are similar or smaller in length compared to
responses from the best baseline. We investigate if response length acted as a
confounding factor during human evaluation. Among all the cases where POKI was
_lost_ over a baseline, 45% ($\pm$ 2% when bootstrapped with 1000 subsets of
size 50) of responses from POKI were longer than those from the comparing
baseline. Among _win_ cases for POKI, we observe 49% ($\pm$ 3% when
bootstrapped with 1000 subsets of size 50) POKI responses were longer than
those from the comparing method. This indicates that human users did not only
choose longer responses as better.

###  5.3 User Study for Effectiveness of Knowledge Injection

Relevant knowledge injection has the benefit of adding more justification to
terse dialog outputs and hence influencing the task outcome positively.
Mirroring observations from Ghandeharioun et al. (2019), a real-time full
conversation evaluation is needed to investigate if POKI could achieve the
conversational goal any better than baselines.

We recruited 60 users for this study444More details of the participants and
the study setup are in Appendix C.. One half of the users interacted with
POKI, while the other half interacted with the best baseline model that does
not augment dialog responses with external knowledge. We construct a
speculative goal for each user to accomplish via the conversation. We allow
users to end the conversation any time they would like and ask them whether
the system helped them to reach their conversation goal along with additional
comments to justify their annotation. Users who interacted with a knowledge-
augmented system also asked if the system provided any knowledge that user has
not explicitly asked for but indeed the extra information helped them to reach
the conversational goal Majumder et al. (2021b). Finally, we also ask if they
would like to engage with the system they interacted with in future.

For goal-oriented dialog, we construct speculative goals (e.g. looking for
entertainment options) manually from the ground truth for 300 dialog samples.
Since we are not using the underlying databases, we made sure speculative
goals do not require specific information (e.g. booking availability, flight
information, etc.). For knowledge-grounded dialog, we provide the intended
topic of discussion (e.g. science fiction) present in the data; the
speculative goal here is to know more about, or to have an engaging
conversation about the topic.

MultiWOZ | # turns $\downarrow$ | Goal | Know | Would use
---|---|---|---|---
Rewriter |  8 $\pm$ 2 | 69% | 35% | 56%
POKI | 4 $\pm$ 3 | 86% | 84% | 76%
WoW | # turns $\uparrow$ | Goal | Know | Would use
BART |  10 $\pm$ 2 | 56% | 70% | 48%
POKI | 16 $\pm$ 3 | 76% | 89% | 71%

Table 4: Real-time user study with average # of turns for successful goal
completion, % of time the goal was achieved, % of success cases users were
helped by an additional knowledge (Know) that was not explicitly asked to
reach their goal, and if users would like to use the system in future.

Results. First of all, we find that POKI is unanimously preferred by users
compared to the baseline during the user study. More importantly, we see that
when the user successfully accomplished their goal, 84% of those times they
found the additional knowledge helpful in the goal-oriented setting (MultiWOZ)
as compared to a baseline (Rewriter) that did not use any external knowledge.
Most importantly, POKI takes significantly fewer turns for users to accomplish
the goal as compared to Rewriter implicitly indicating injected knowledge (we
observe high correlation, 0.67) contributes toward more efficient
conversations.

For the knowledge-grounded setting (WoW), both BART and POKI have access to
external knowledge sources. However, 89% (compared to 70%) of success
scenarios were directly influenced by the additional post-hoc knowledge. For
knowledge-grounded dialog, a longer conversation is indicative of engagingness
on a particular topic Gopalakrishnan et al. (2019), hence users preferred to
converse with POKI for more turns as compared to a BART baseline. We quote a
comment from a user who found a conversation about the Korean culture with
POKI was particularly engaging---‘‘Before this conversation, I had less
knowledge about Korean movies and art-forms. This gave me a new perspective
and a handful of popular opinions to look at it.’’.

###  5.4 Discussion

| Relevant | Factual | BRTSc for WoW
---|---|---|---
Source | Random | DPP | Random | DPP | Random | DPP
Parametric | 82% | 89% | 65% | 83% | 74.2 | 81.3
Non-parametric | 81% | 83% | 97% | 98% | 65.2 | 76.8

Table 5: Evaluation for the quality of the knowledge snippets for random and
DPP-based selection.

System | MultiWOZ | WoW
---|---|---
Supervised |  17.6 $\pm$ 5.2 ms |  23.6 $\pm$ 4.6 ms
PPCM Madotto et al. (2020) |  30.9 $\pm$ 7.5 ms |  32.6 $\pm$ 4.2 ms
POKI |  34.2 $\pm$ 8.4 ms |  35.7 $\pm$ 5.7 ms
POKI, only decoding |  31.6 $\pm$ 2.7 ms |  32.3 $\pm$ 3.4 ms

Table 6: Mean and std. error of clock-time taken per token

Performance of Knowledge Selection. The knowledge selection step in POKI acts
an information bottleneck where the quality of the generated response directly
depends on the quality of the selected knowledge555A statistical analysis on
number of knowledge snippets retrieved/generated and selected is provided in
Appendix B.. We perform a human evaluation on 200 snippets to measure the
relevance and the factual correctness in two scenarios: when we randomly
select a retrieved snippet or select via DPP. In Table 5, we see that the
parametric knowledge source (gpt2-large) generates more relevant knowledge
snippets than a non-parametric one. We attribute this to 1) a large and
diverse dataset (webtext) used during pretraining of gpt2 as compared to yelp
reviews (restricted domains) we used for retrieval, and 2) the limited recall
of relevant knowledge when using word-overlap based retrieval. However, large
language models are still prone to generate non-factual knowledge. We observe
that DPP-based selection in POKI is able to sub-select more factual knowledge
which then positively influences the final response quality. For WoW, we also
compare the selected snippets with the gold knowledge available in the dataset
that in turn show high fidelity in terms of BERTScore.

Time Complexity.  Madotto et al. (2020) shows that iterative gradient-based
decoding could be slower than generating response using single forward pass
from an existing model. When we benchmark POKI in an Nvidia 2080Ti GPU, in
Table 6, we see that knowledge generation (or retrieval) could be a
computational bottleneck for POKI. However the greedy selection and the
constrained decoding step do not add significant computational load.
Furthermore, POKI’s performance is comparable with PPCM Madotto et al.
(2020)\---a more efficient version of gradient-based decoding. The efficiency
of the knowledge retrieval step can be improved with better indexing Johnson
et al. (2021) which we leave as a future work.

##  6 Related Work

Knowledge grounded dialog datasets such as Wizard-of-Wikipedia Dinan et al.
(2019a) and Topical chat Gopalakrishnan et al. (2019) typically consist of
dialog responses paired with relevant knowledge available as collected
annotations. Hence, models trained on such datasets are restricted to the
knowledge sources they were exposed to at training time. Past work Sun et al.
(2021); Majumder et al. (2020a); Su et al. (2020); Komeili et al. (2021);
Adolphs et al. (2021); Ghazvininejad et al. (2018); Tuan et al. (2020); Lewis
et al. (2020c); Guu et al. (2020) has looked into injecting extra knowledge
sources at training time in a bid to add knowledge not available originally as
paired to dialog responses. However, such approaches require re-training the
model if some new knowledge source were to be used. Moreover, while previous
work focuses on just improving specificity of dialog response using external
knowledge, we also study the effect of additional knowledge in achieving
conversational goals.

Improving the diversity of dialog responses by using diversity-promoting
sampling has been explored in past work Fan et al. (2018); Holtzman et al.
(2020). We use a gradient-based decoding method, building on past work in this
direction Dathathri et al. (2020); Qin et al. (2020); Madotto et al. (2020);
Majumder et al. (2021a). However, we propose new objectives to inject post-hoc
knowledge obtained based on already generated dialog---an unsupervised
knowledge injection method that has not been explored so far.

##  7 Conclusion

We propose a framework for unsupervised knowledge injection into dialog
responses. We show that knowledge can be obtained post-hoc from any knowledge
sources that can improve users’ ability to reach their conversational goal
more effectively. In future, our idea can be generalized to setups where
external knowledge can justify model’s predictions such as conversational
recommendation.

## Acknowledgements

We thank anonymous reviewers for providing valuable feedback. BPM is partly
supported by a Qualcomm Innovation Fellowship, a Friends of the International
Center Fellowship--UC San Diego, NSF Award #1750063, and MeetElise.

## References

  * Adolphs et al. (2021) Leonard Adolphs, Kurt Shuster, Jack Urbanek, Arthur Szlam, and Jason Weston. 2021\.  Reason first, then respond: Modular generation for knowledge-infused dialogue.  _CoRR_ , abs/2111.05204.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, et al. 2020.  Language models are few-shot learners.  In _NeurIPS_.
  * Bruyn et al. (2020) Maxime De Bruyn, Ehsan Lotfi, Jeska Buhmann, and Walter Daelemans. 2020.  BART for knowledge grounded conversations.  In _Converse@KDD_ , volume 2666. CEUR-WS.org.
  * Budzianowski et al. (2018) Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018.  Multiwoz - A large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling.  In _EMNLP_.
  * Campos et al. (2020) Ricardo Campos, Vítor Mangaravite, Arian Pasquali, Alípio Jorge, Célia Nunes, and Adam Jatowt. 2020.  Yake! keyword extraction from single documents using multiple local features.  _Information Sciences_ , 509.
  * Cohen (1960) Jacob Cohen. 1960.  A coefficient of agreement for nominal scales.  _Educational and psychological measurement_ , 20(1):37--46.
  * Dathathri et al. (2020) Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020.  Plug and play language models: A simple approach to controlled text generation.  In _ICLR_.
  * Dinan et al. (2019a) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019a.  Wizard of wikipedia: Knowledge-powered conversational agents.  In _ICLR_.
  * Dinan et al. (2019b) Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019b.  Wizard of wikipedia: Knowledge-powered conversational agents.  In _ICLR_.
  * Fan et al. (2018) Angela Fan, Mike Lewis, and Yann N. Dauphin. 2018.  Hierarchical neural story generation.  In _ACL_.
  * Gautier et al. (2019) Guillaume Gautier, Guillermo Polito, Rémi Bardenet, and Michal Valko. 2019\.  DPPy: DPP Sampling with Python.  _Journal of Machine Learning Research - Machine Learning Open Source Software (JMLR-MLOSS)_.
  * Ghandeharioun et al. (2019) Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Àgata Lapedriza, and Rosalind W. Picard. 2019.  Approximating interactive human evaluation with self-play for open-domain dialog systems.  In _NeurIPS_.
  * Ghazvininejad et al. (2018) Marjan Ghazvininejad, Chris Brockett, Ming-Wei Chang, Bill Dolan, Jianfeng Gao, Wen-tau Yih, and Michel Galley. 2018.  A knowledge-grounded neural conversation model.  In _AAAI_.
  * Gopalakrishnan et al. (2019) Karthik Gopalakrishnan, Behnam Hedayatnia, Qinglang Chen, Anna Gottardi, Sanjeev Kwatra, Anu Venkatesh, Raefer Gabriel, and Dilek Hakkani-Tür. 2019\.  Topical-chat: Towards knowledge-grounded open-domain conversations.  In _Interspeech_.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020\.  REALM: retrieval-augmented language model pre-training.  _CoRR_ , abs/2002.08909.
  * Hajas et al. (2014) Peter Hajas, Louis Gutierrez, and Mukkai S. Krishnamoorthy. 2014.  Analysis of yelp reviews.  _CoRR_ , abs/1407.1443.
  * Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020.  The curious case of neural text degeneration.  In _ICLR_.
  * Hosseini-Asl et al. (2020) Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020.  A simple language model for task-oriented dialogue.  In _NeurIPS_.
  * Jhamtani et al. (2018) Harsh Jhamtani, Varun Gangal, Eduard Hovy, Graham Neubig, and Taylor Berg-Kirkpatrick. 2018.  Learning to generate move-by-move commentary for chess games from large-scale social forum data.  In _ACL 2018_.
  * Johnson et al. (2021) Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021.  Billion-scale similarity search with gpus.  _IEEE Trans. Big Data_.
  * Kedzie et al. (2018) Chris Kedzie, Kathleen R. McKeown, and Hal Daumé III. 2018.  Content selection in deep learning models of summarization.  In _EMNLP_.
  * Kim et al. (2020) Byeongchang Kim, Jaewoo Ahn, and Gunhee Kim. 2020.  Sequential latent knowledge selection for knowledge-grounded dialogue.  In _ICLR_. OpenReview.net.
  * Komeili et al. (2021) Mojtaba Komeili, Kurt Shuster, and Jason Weston. 2021.  Internet-augmented dialogue generation.  _CoRR_ , abs/2107.07566.
  * Kulesza and Taskar (2011) Alex Kulesza and Ben Taskar. 2011.  k-dpps: Fixed-size determinantal point processes.  In _ICML_. Omnipress.
  * Kulesza and Taskar (2012) Alex Kulesza and Ben Taskar. 2012.  Determinantal point processes for machine learning.  _Found. Trends Mach. Learn._ , 5(2-3):123--286.
  * Lewis et al. (2020a) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a.  BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _ACL_.
  * Lewis et al. (2020b) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _NeurIPS_.
  * Lewis et al. (2020c) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020c.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _NeurIPS_.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016.  A diversity-promoting objective function for neural conversation models.  In _NAACL HLT_.
  * Li et al. (2019) Margaret Li, Jason Weston, and Stephen Roller. 2019.  ACUTE-EVAL: improved dialogue evaluation with optimized questions and multi-turn comparisons.  _CoRR_ , abs/1909.03087.
  * Madotto et al. (2020) Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri, and Pascale Fung. 2020.  Plug-and-play conversational models.  In _Findings of EMNLP_.
  * Majumder et al. (2021a) Bodhisattwa Prasad Majumder, Taylor Berg-Kirkpatrick, Julian J. McAuley, and Harsh Jhamtani. 2021a.  Unsupervised enrichment of persona-grounded dialog with background stories.  In _ACL_.
  * Majumder et al. (2020a) Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Berg-Kirkpatrick, and Julian J. McAuley. 2020a.  Like hiking? you probably enjoy nature: Persona-grounded dialog with commonsense expansions.  In _EMNLP_.
  * Majumder et al. (2020b) Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian J. McAuley. 2020b.  Interview: Large-scale modeling of media dialog with discourse patterns and knowledge grounding.  In _EMNLP_.
  * Majumder et al. (2021b) Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, and Julian J. McAuley. 2021b.  Ask what’s missing and what’s useful: Improving clarification question generation using global knowledge.  _NAACL_.
  * Padmakumar and He (2021) Vishakh Padmakumar and He He. 2021.  Unsupervised extractive summarization using pointwise mutual information.  In _EACL_.
  * Paranjape and Manning (2021) Ashwin Paranjape and Christopher D. Manning. 2021.  Human-like informative conversations: Better acknowledgements using conditional mutual information.  In _NAACL-HLT_.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019.  Language models as knowledge bases? In _EMNLP-IJCNLP_.
  * Prabhumoye et al. (2021) Shrimai Prabhumoye, Kazuma Hashimoto, Yingbo Zhou, Alan W. Black, and Ruslan Salakhutdinov. 2021.  Focused attention improves document-grounded generation.  In _NAACL-HLT_.
  * Qin et al. (2020) Lianhui Qin, Vered Shwartz, Peter West, Chandra Bhagavatula, Jena D. Hwang, Ronan Le Bras, Antoine Bosselut, and Yejin Choi. 2020.  Back to the future: Unsupervised backprop-based decoding for counterfactual and abductive commonsense reasoning.  In _EMNLP_.
  * Robertson et al. (1995) Stephen E. Robertson, Steve Walker, and Micheline Hancock-Beaulieu. 1995.  Large test collection experiments on an operational, interactive system: Okapi at TREC.  _Inf. Process. Manag._ , 31(3):345--360.
  * Schwartz et al. (2020) Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. 2020.  Green AI.  _Commun. ACM_ , 63(12):54--63.
  * Shwartz et al. (2020) Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020\.  Unsupervised commonsense question answering with self-talk.  In _EMNLP_.
  * Strubell et al. (2019) Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019.  Energy and policy considerations for deep learning in NLP.  In _ACL_.
  * Su et al. (2020) Hui Su, Xiaoyu Shen, Sanqiang Zhao, Xiao Zhou, Pengwei Hu, Randy Zhong, Cheng Niu, and Jie Zhou. 2020.  Diversifying dialogue generation with non-conversational text.  In _ACL_.
  * Sun et al. (2021) Kai Sun, Seungwhan Moon, Paul A. Crook, Stephen Roller, Becka Silvert, Bing Liu, Zhiguang Wang, Honglei Liu, Eunjoon Cho, and Claire Cardie. 2021.  Adding chit-chats to enhance task-oriented dialogues.  _NAACL_.
  * Tuan et al. (2020) Yi-Lin Tuan, Wei Wei, and William Yang Wang. 2020.  Unsupervised injection of knowledge into dialogue generation via language models.  _CoRR_ , abs/2004.14614.
  * Welleck et al. (2019) Sean Welleck, Jason Weston, Arthur Szlam, and Kyunghyun Cho. 2019.  Dialogue natural language inference.  In _ACL_.
  * Wilhelm et al. (2018) Mark Wilhelm, Ajith Ramanathan, Alexander Bonomo, Sagar Jain, Ed H. Chi, and Jennifer Gillenwater. 2018.  Practical diversified recommendations on youtube with determinantal point processes.  In _CIKM_. ACM.
  * Wolf et al. (2019) Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019.  Transfertransfo: A transfer learning approach for neural network based conversational agents.  _CoRR_ , abs/1901.08149.
  * Zhao et al. (2017) Tiancheng Zhao, Ran Zhao, and Maxine Eskénazi. 2017.  Learning discourse-level diversity for neural dialog models using conditional variational autoencoders.  In _ACL_.
  * Zhao et al. (2020) Xueliang Zhao, Wei Wu, Can Xu, Chongyang Tao, Dongyan Zhao, and Rui Yan. 2020.  Knowledge-grounded dialogue generation with pre-trained language models.  In _EMNLP_.

##  Appendix A Datasets

#### MultiWOZ.

To compare with previous works, we use MultiWoz 2.0 following Hosseini-Asl et
al. (2020). Note that we do not need any training data for our models since we
perform post-hoc knowledge injection.

#### WoW

For Wizard-of-Wikipedia, all baselines and the original dialog model for POKI
use available paired knowledge present in the training data (not a part of our
pipeline). However, POKI additionally uses the external knowledge snippets
selected via DPP.

##  Appendix B Implementation Details

We open-source our code at: https://github.com/majumderb/poki. We use the
publicly available implementation666https://github.com/guilgautier/DPPy for
DPP Gautier et al. (2019).

We obtain the MultiWOZ 2.0 from the official release
777https://github.com/budzianowski/multiwoz. Similarly, we obtain the Wizard-
of-Wikipedia from ParlAI repository
888https://parl.ai/projects/wizard_of_wikipedia/. We adapted codes from
original PPLM Dathathri et al. (2020) repository999https://github.com/uber-
research/PPLM and modified them for our own objective function. We obtained
the Yelp review dataset from the official
website101010https://www.yelp.com/dataset. Yelp dataset contains 8,635,403
reviews. For diversity calculation (in automatic evaluation), we use
NLTK111111https://www.nltk.org/_modules/nltk/util.html to extract n-grams.

#### Network architecture

For MultiWOZ, we use the
SimpleTOD121212https://github.com/salesforce/simpletod as the base model.
Whereas for WoW, we use
BART131313https://huggingface.co/transformers/model_doc/bart.html as the base
model. For the parametric knowledge source, we use
gpt2-large141414https://huggingface.co/transformers/model_doc/gpt2.html.

#### Hyperparameters

POKI does not require any training since we perform gradient-based decoding at
the inference time. For hyperparameters involved in the decoding stage, we
maximize the Z-normalized sum of dialog model perplexity and linguistic
diversity (% of distinct bigrams) of the generated response in a greedy
fashion to select the best values. For our best method, in objective function
$\mathcal{L}$, we use $\alpha$ as 1 and $\lambda$ as 1. We keep generation
length to be 100 to encourage longer generations. We train the entailment
classifier using code from PPLM repository151515https://github.com/uber-
research/PPLM/blob/master/run_pplm_discrim_train.py. The weight $\gamma$ for
mixing forward and backward passes was set to 0.45. We run 5 backward-forward
passes to obtain a candidate final response.

#### Filtering knowledge candidates from PTLMs

Our initial experiments suggests that that knowledge generated from PTLMs can
be inappropriate (contains bias or toxic content) and misleading/nonfactual.
Sun et al. (2021) collected annotations of dialog responses with labels
positive (useful, social), negative (inappropriate and misleading). We learn a
binary classifier to classify a knowledge snippet as positive or negative and
use it as a filtering criteria.

#### Key-phrase extraction

Given a sentence from the context, we first extract n-gram (n $\in$ 1,2,3,4)
key-phrases using YAKE (Yet-Another-Keyword-Extractor) Campos et al. (2020)
and retain only those that contain at least a noun.

#### Prompts

We curated prompts inspired by various knowledge-seeking situations (such as
for: more information, opinion, review) Shwartz et al. (2020) and are listed
in Table 7.

[KP] is famous for
---
The popular opinion about [KP] is
Here is what I know about [KP]:
My friend says that [KP] is:
Here is some information about [KP]:
Here are some reviews about [KP]:
I think [KP] is:
I read on the internet about [KP] and found that
Today I learned about [KP] that
Table 7: Manually curated prompts to query the PTLM

#### Statistics on generated and selected knowledge snippets

For both datasets, we retrieve 100 most relevant knowledge snippets from non-
parametric source (here, yelp reviews), and generate 5 candidate knowledge
snippets (using nucleus sampling Holtzman et al. (2020), $p=0.95$) for each
key-phrase extracted from an input instance (dialog history + initial
response). After knowledge selection by DPP, on an average (over validation
set), 5 snippets were selected for MultiWoz and 8 snippets were selected for
WoW.

##  Appendix C Human Evaluation and User Study Setup

#### Human Evaluation

We hired two Anglophone (Lifetime HIT acceptance % $>$ 85) annotators for
every test sample. Figure 4 shows a sample question for the pairwise
comparison between response generated by POKI and a baseline for
informativeness. The exact formulations for all criteria are provided as
below:

  * •

Coherent: Which version is more consistent with the dialog history?

  * •

Engaging: Which version is more likely to hold your attention and make you
want to hear more?

  * •

Interesting: Which version arouses your curiosity or tells you something new
or useful?

  * •

Humanlike: Which version is more natural and personable?

All differences in values from human evaluations are significant with $p<0.05$
from bootstrap tests on 1000 subsets of size 50. A snapshot of our human
evaluation interface is shown in Figure 4. The order of two candidate
responses (R1 and R2) is made random for each question.

#### User Study

For user study, we similarly recruited 60 Anglophone users who have at least
high-school level of education and are comfortable with handling internet-
based technologies. Each session (depending on the systems they interacted)
lasted on an average 30 minutes (for MultiWOZ) and 60 minutes (for WoW)
including on-boarding, performing actual task and answering post-task
questions.

Figure 4: Human evaluation setup for pairwise comparison between POKI and
another baseline Figure 5:  POKI converses with a user who is discussing about
science fiction, in a knowledge-grounded dialog scenario (left column). In
each turn, a initial response (middle column) is augmented with a knowledge
snippet (right column) using constrained gradient-based decoding. Human judges
unanimously noted this conversation as more engaging as compared to the
initial responses. Figure 6: Utterance level examples (left column) in (a) and
(b) goal oriented scenario; and (c) knowledge-grounded scenario. POKI updates
the initial response (middle column) with a knowledge snippet (right column)
using constrained gradient-based decoding.

##  Appendix D Qualitative Examples

Figure 5 shows a complete dialog in the knowledge-grounded scenario where the
user discusses about ‘science-fiction’. Figure 6 shows more utterance level
examples for both goal-oriented and knowledge-grounded scenarios.

## Ethical considerations

We do not foresee any immediate ethical concerns for our method as we use
several constraints (less divergence from the extracted knowledge, consistency
with the dialog context) that allow the generation to be restricted to the
context. In general, we expect our dialog system to be more engaging and
accessible to the user. Since we use PTLMs as knowledge source, we inherit the
general risk of generating biased or toxic language, which should be carefully
filtered. In our work, we perform explicit filtering steps to make sure that
the knowledge is appropriate. Furthermore, our selection step promotes more
factually correct knowledge to be selected. However, the generations may
incorporate biases that are already present in the dialog datasets due to
crowd-sourced data collection. Finally, our generations are limited only to
the English language. Hence we suggest that a system like ours should likely
not be used as a ‘black box,’ but would best be used in a setting where its
outputs can be ‘audited’. Carbon footprint: Our system uses post-hoc knowledge
injection which refrains from retraining newer dialog models to accommodate
dynamically evolving external knowledge. This promotes green NLP applications
Schwartz et al. (2020); Strubell et al. (2019) reducing carbon footprints that
stem from training (or even finetuning) large language models.


# Few-cycle vacuum squeezing in nanophotonics

Rajveer Nehra,1∗, Ryoto Sekine1,∗, Luis Ledezma1,2,
Qiushi Guo1, Robert M. Gray1, Arkadev Roy1, and Alireza Marandi1,† 1Department
of Electrical Engineering,
California Institute of Technology, Pasadena, California 91125, USA.,
2Jet Propulsion Laboratory,
California Institute of Technology, Pasadena, California 91109, USA.
∗These authors contributed equally to this work.
†E-mail<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

One of the most fundamental quantum states of light is squeezed vacuum, in
which noise in one of the quadratures is less than the standard quantum noise
limit. Significant progress has been made in the generation of optical
squeezed vacuum and its utilization for numerous applications. However, it
remains challenging to generate, manipulate, and measure such quantum states
in nanophotonics with performances required for a wide range of scalable
quantum information systems. Here, we overcome this challenge in lithium
niobate nanophotonics by utilizing ultrashort-pulse phase-sensitive amplifiers
for both generation and all-optical measurement of squeezed states on the same
chip. We generate a squeezed state spanning over more than 25 THz of bandwidth
supporting only a few optical cycles, and measure a maximum of 4.9 dB of
squeezing ($\sim$11 dB inferred). This level of squeezing surpasses the
requirements for a wide range of quantum information systems. Our results on
generation and measurement of few-optical-cycle squeezed states in
nanophotonics enable a practical path towards scalable quantum information
systems with THz clock rates and open opportunities for studying non-classical
nature of light in the sub-cycle regime.

Quantum information processing offers great promises for computation, secure
communication, metrology and sensing. Many physical platforms such as nuclear
spins, superconducting circuits, photonics, trapped ions, quantum dots, and
neutral atoms have widely been explored in the pursuit to build quantum
information processors Clarke and Wilhelm (2008); O’brien  _et al._ (2009);
Cirac and Zoller (1995); Imamog¯lu  _et al._ (1999); Saffman  _et al._ (2010);
Vandersypen  _et al._ (2001). Among these, photonics stands out because of its
potential for scalability, room-temperature logical operations, and ease of
encoding Chen  _et al._ (2014); Roslund  _et al._ (2014); Asavanant  _et al._
(2019); Larsen  _et al._ (2019); Zhong  _et al._ (2020) quantum information in
both discrete Knill  _et al._ (2001) and continuous variables Lloyd and
Braunstein (1999); Gu  _et al._ (2009).

In continuous-variable (CV) quantum photonics, information is encoded in
continuous amplitude and phase values of the quantized electromagnetic field.
The single-mode and multimode squeezed states are widely used for various
applications including quantum-enhanced interferometry such as in LIGO Yu  _et
al._ (2020), microscopy Casacio  _et al._ (2021), and quantum teleportation
Furusawa  _et al._ (1998). Moreover, highly entangled CV quantum states, i.e.,
cluster states Asavanant  _et al._ (2019); Larsen  _et al._ (2019); Chen  _et
al._ (2014), serve as a universal resource for one-way quantum computation
Raussendorf and Briegel (2001).

Typically, such high-quality CV states are generated from single or two-mode
squeezed vacuum generated using quadratic ($\chi^{(2)}$) parametric processes
either in bulk crystals or waveguides with large ($\sim$100 $\mu\text{m}^{2}$)
mode areas Chen  _et al._ (2014); Kanter  _et al._ (2002); Roslund  _et al._
(2014); Asavanant  _et al._ (2019); Larsen  _et al._ (2020). While such
experiments using bulky discrete components have been successful to
demonstrate small and medium-scale quantum circuits, it is highly desired to
achieve CV quantum states with comparable qualities in nanophotonics to enable
large-scale integrated quantum circuits.

In nanophotonics, silicon nitride (SiN) and silica platforms have been used
for many quantum photonic experiments such as entangled photon-pair
generation, squeezing, error correction, and small scale Gaussian boson
sampling Joshi  _et al._ (2020); Vaidya  _et al._ (2020); Yang  _et al._
(2021); Vigliar  _et al._ (2021); Arrazola  _et al._ (2021). However, their
inherently weak cubic ($\chi^{(3)}$) nonlinearity typically necessitates using
high-Q resonators, which imposes limitations on accessible squeezing levels
and bandwidths. Despite significant advances, the measured squeezing levels
have so far remained below 2 dB in nanophotonics Dutt  _et al._ (2015); Vaidya
_et al._ (2020); Yang  _et al._ (2021); Cernansky and Politi (2020).

Figure 1: Illustration of generating squeezed vacuum and measuring it all
optically in nanophotonics. Phase-space distributions of (a) phase-quadrature
squeezed vacuum, and its amplification in the (b) anti-squeezed and (c)
squeezed quadratures. The small (large) dashed circles correspond to shot-
noise (amplified shot-noise level, see text) and the small (large) filled
ellipses are for squeezed vacuum (amplified squeezed vacuum). d) Sketch of our
nanophotonic device consisting of the Squeezer OPA, tapered adiabatic coupler,
and Measurement OPA. When pumped, the Squeezer OPA, generates a squeezed
vacuum state, which is selectively coupled into an adjacent waveguide. There,
it is subsequently amplified by the Measurement OPA to macroscopic power
levels.

On the other hand, the measurements in CV quantum systems have typically
relied on balanced homodyne detection (BHD) using highly-efficient and low-
noise photodetectors which are limited in bandwidths to MHz and GHz ranges
Tasker  _et al._ (2021). Moreover, in nanophotonics, the loss associated with
transferring the microscopic quantum states from a tightly confined mode to a
photodetector has imposed barriers in the measurement capabilities of such
states Zhao  _et al._ (2020); Javid  _et al._ (2021); Chen  _et al._ (2021);
Vaidya  _et al._ (2020). A potential solution for these measurement challenges
lies in all-optical measurement schemes based on a noiseless phase-sensitive
amplifier with sufficiently large gain Caves (1982); Shaked  _et al._ (2018);
Takanashi  _et al._ (2020); Frascella  _et al._ (2021); Li  _et al._ (2020)
that can eliminate the limitations of homodyne detection and the sensitivity
to detection losses. However, achieving such large gains ($>$30 dB) over broad
optical bandwidths is challenging in nanophotonics with cubic nonlinearity Ye
_et al._ (2021).

Recently, lithium niobate (LN) nanophotonics has opened promising avenues in
optical communication, sensing, and computation due to its extraordinary
optical, electrical, and acoustic properties Zhu  _et al._ (2021); Jankowski
_et al._ (2021). A combination of sub-wavelength confinement of the optical
mode, strong $\chi^{(2)}$ nonlinearity, high-fidelity quasi-phasematching
(QPM) by periodic poling, and dispersion engineering for longer interaction
lengths has enabled devices outperforming the traditional LN devices Ledezma
_et al._ (2021); Jankowski  _et al._ (2020); Guo  _et al._ (2021); Hu  _et
al._ (2021).

In this work, we use a nanophotonic circuit in LN and experimentally
demonstrate record-level generation and all-optical measurement of ultra-
short-pulse squeezed vacuum as the building block of a scalable CV quantum
nanophotonics. Our circuit combines two dispersion-engineered phase-sensitive
optical parametric amplifiers (OPAs) Ledezma  _et al._ (2021), as shown in
Fig. 1. The first OPA generates a microscopic squeezed vacuum which is then
amplified with a high-gain OPA to macroscopic levels within the same
nanophotonic chip. The resulting macroscopic field carries information about
the microscopic squeezed state, which can be measured with a high tolerance to
loss.

In Figure 1a, the phase-space distributions for vacuum (dashed circle) and
phase-quadrature squeezed vacuum (filled ellipse) are displayed. For the
figure, we consider a particular case of 10-dB squeezing with mean photon-
number $\langle N_{\text{sq-v}}\rangle\approx 2$. The squeezed light is then
amplified by an OPA with 50 dB of phase-sensitive gain, thereby amplifying the
few-photon squeezed signal to a macroscopic power level. In Fig. 1b and 1c,
the phase-space distributions corresponding to amplified anti-squeezed and
amplified squeezed quadratures are shown for two particular pump phases of the
Measurement OPA, $\phi_{\text{ Pump 2}}$. The dashed circles in Fig. 1b and 1c
represent the amplified shot-noise level corresponding to the phase-space
distribution of amplified vacuum. For brevity, we represent the distribution
of the amplified vacuum as a circle because the vacuum state is phase-
insensitive and therefore, the amplification of any phase space quadrature by
the Measurement OPA will lead to the same noise level. In the case of Fig. 1b
with $\phi_{\text{ Pump 2}}=0$, the anti-squeezed quadrature (Q) is amplified
while the orthogonal phase quadrature (P) is de-amplified such that the output
field is dominated entirely by the Q quadrature and the P quadrature can be
considered negligible. In such a high-gain amplification regime, the total
average photon-number (power) of the output field is:
$\langle\hat{N}_{+}\rangle\propto\langle\hat{Q}_{\text{amp}}^{2}\rangle\approx\mathcal{O}(10^{5})$.
Likewise, by changing the pump phase of the Measurement OPA to $\phi_{\text{
Pump 2}}=\pi$, the original squeezed quadrature is amplified to achieve
$\langle\hat{N}_{-}\rangle\propto\langle\hat{P}_{\text{amp}}^{2}\rangle\approx\mathcal{O}(10^{3})$,
as shown in Figure 1c. As a result, the macroscopic output of the Measurement
OPA provides a direct all-optical measurement of the microscopic squeezed
state. The squeezing ($S_{-}$) and anti-squeezing ($S_{+}$) can then be
determined as
$S_{\pm}[\text{dB}]=10\text{log}_{10}[\langle\hat{N}_{\pm}\rangle/\langle\hat{N}_{\text{v}}\rangle]$,
where $\langle\hat{N}_{\text{v}}\rangle$ denotes the amplified vacuum. In the
ideal case, the squeezing (anti-squeezing) can be determined as Sup

$S_{\pm}[\text{dB}]=10\text{log}\bigg{(}\frac{\text{sinh}^{2}(r_{2}\pm
r_{1})}{\text{sinh}^{2}r_{2}}\bigg{)},$ (1)

where $r_{1}$ and $r_{2}$ are the gain parameters for the Squeezer and
Measurement OPAs, respectively. Sufficient gain ($>$ 33 dB for $\sim$ 11 dB of
Squeezer OPA gain, see Ref. Sup , Part 6 for details) in the Measurement OPA
allows a direct measurement of the phase-squeezed vacuum generated in the
Squeezer OPA Sup . Importantly, the high-gain Measurement OPA makes our
measurement tolerant to coupling losses and photodetection inefficiencies as
high as $\sim$ 7 dB Sup .

In experiments, the Squeezer (low-gain) and Measurement (high-gain) OPAs of
our circuit are periodically-poled with the lengths of 2.5 mm and 5.0 mm,
respectively. The output of the Squeezer OPA (microscopic squeezed vacuum) is
coupled to the Measurement OPA through a directional coupler. To make our
directional coupler broadband and less susceptible to fabrication
imperfections, we employ an adiabatic design where both of the waveguides are
tapered while keeping the gap constant throughout the coupler length. The
coupler directs the squeezed vacuum to the adjacent waveguide towards the
Measurement OPA, and keeps the residual pump of the Squeezer OPA in the
original waveguide as shown in Fig. 1d. In our current device, the coupler
causes $\sim$30% loss for the squeezed vacuum, and leaks $\sim$20% of the
squeezer pump to the Measurement OPA Sup . Our numerical simulations suggest
that the coupling performance can be significantly improved to $>95\%$ for
squeezed signal and $<5\%$ for squeezer pump, which will lead to better
measurement quality Sup .

Figure 2: Generation and measurement of squeezed light in the LN nanophotonic
chip. a) Experimental setup. A MLL is split to pump the squeezer (Pump 1) and
measurement (Pump 2) OPAs. The phase difference between the pumps is modulated
by a piezoelectric transducer (PZT) in the Pump 2 arm. At the output of the
nanophotonic chip, light from the measurement amplifier waveguide is
collected. The amplified squeezed signal (red) and residual Pump 2 (green) are
separated using a DM, and measured by two different OSAs. (i) a false-colored
SEM image of our nanophotonic circuit, (ii) a zoomed-in SEM image of the
coupler region, and (iii) a false-colored second harmonic microscope image of
the periodically-poled region prior to etching the waveguides. b) Squeezing
measurement by an OSA in the zero-span mode at 2090 nm. The green trace shows
the amplified squeezed vacuum when the PZT is modulated by a ramp signal. The
other traces represent amplified vacuum (shot-noise) when Pump 1 is blocked at
different Pump 2 powers (see main text). c) The squeezing measured at 2090 nm
for several values of Pump 1 while keeping Pump 2 constant. d) Loss analysis
of the squeezing measurements. The solid (dashed) curves show the degradation
of anti-squeezing (squeezing) as the photon loss increases and the solid data
points correspond to measured values of minimum and maximum squeezing. e) The
squeezer gain dependence on the energy of Pump 1. Blue points are measured
from anti-squeezing and red points are directly obtained from optical
parametric generation (OPG) measurements. Error bars are obtained from the
statistics of the measurements. MLL: mode-locked laser, PBS: polarized
beamsplitter, BS: beam splitter, PZT: piezoelelectric transducer, HWP: half-
wave plate, DM: dichroic mirror, Obj.: reflective objective, VND: variable
neutral-density filter, FC: fiber coupler, OSA: optical spectrum analyzer,
TEC: thermoelectric cooler.

Our experimental setup is shown in Fig. 2a. The Squeezer and Measurement OPAs
are pumped by a mode-locked laser (Menlo Systems Orange A) generating
$\sim$75-fs-long nearly transform-limited pulses at a 250-MHz repetition rate.
The relative phase between Pump 1 (Squeezer OPA) and Pump 2 (Measurement OPA)
pulses is modulated by a piezoelectric transducer (PZT) on the Pump 2 arm. At
the output of the nanophotonic chip, the amplified squeezed signal and
Measurement OPA pump are first separated using a dichroic mirror and then are
detected by two different optical spectrum analysers (OSAs). In Fig. 2a, we
show, (i) a false-colored scanning electron microscope (SEM) image of our
nanophotonic circuit, (ii) a zoomed-in SEM image of the coupler region, and
(iii) a false-colored second harmonic microscope image of the periodically-
poled region prior to etching the waveguides.

Figure 2b shows an example measurement of our squeezed state. The green trace
shows the output signal of the Measurement OPA using an OSA in a zero-span
mode at 2090 nm while keeping both Pump 1 and Pump 2 on and modulating the PZT
by a 1-Hz ramp signal. To accurately measure the squeezing, we need to
eliminate the effect of residual interference of the two pumps at the output
of the measurement. We achieve this by determining the maximum and minimum of
this residual interference and then calibrating our amplified shot-noise
levels by subsequently varying the power of Pump 2 to these maximum and
minimum pump powers while blocking Pump 1. These two levels of Pump 2 result
in “shot-noise maximum” and “shot-noise minimum” as shown in Fig. 2b, while
“original shot-noise” corresponds to the Pump 2 level during the squeezing
measurement. Hence, in the squeezing measurement, the shaded area below
(above) the “shot-noise minimum” (“shot-noise maximum”) corresponds to
squeezing (anti-squeezing) at the input of the high-gain OPA. A detailed
discussion on our shot-noise calibration measurements can be found in Ref. Sup
, Part 2.

We further characterize the dependence of squeezing at 2090 nm on the pump
power while keeping the Pump 2 constant and performing the shot-noise
calibration for each power level as shown in Fig. 2c. As we increase the pump
power in the Squeezer OPA, the level of measured squeezing increases at first.
However, above 0.8 pJ of pump pulse energy, we observe that further increasing
the squeezer pump decreases the level of measured squeezing. The degradation
of measured squeezing at high pump powers may be due to the existence of a
small phase noise and relative chirp between Pump 1 and Pump 2 which can mix
the loss-degraded squeezed quadrature with the relatively large anti-squeezed
quadrature Werner  _et al._ (1995); Oelker  _et al._ (2016). Additionally,
parasitic nonlinear effects such as the photorefractive effect and nonlinear
absorption mechanisms in the waveguide can also account for the degradation of
squeezing at higher pump powers.

Figure 2d shows how squeezing levels degrade in the presence of photon loss
$(1-\eta)$. The solid and dashed curves represent anti-squeezing and
squeezing, respectively. Analytically,
$S_{\pm}[\text{dB}]=10\text{log}[(1-\eta)+\eta e^{\pm 2r}]$, where $(1-\eta)$
determines the loss present in the detection protocol and $r$ is the squeezing
parameter characterizing nonlinear interaction strength Sup . The solid dots
are the experimental data points for the minimum and maximum amount of
measured squeezing at 2090 nm in Fig. 2c. From these measurements, we estimate
the total loss $L=1-\eta\approx 0.3$ experienced by the microscopic squeezed
signal before being fully amplified by the measurement OPA. This agrees well
with our measured coupling efficiency of the adiabatic coupler using an
auxiliary signal centered at 2090 nm (See Ref. Sup , Part 4). From the fit, we
infer that we successfully generated 10.6 dB of squeezing with the pump energy
of $<$1 pJ. This paves the way for fault-tolerant CV quantum processors in LN
nanophotonics, as 10.0 dB of squeezing is sufficient for many architectures
including recent proposals with Gottesman-Kitaev-Preskill (GKP) qubit
encodings Bourassa  _et al._ (2021); Fukui  _et al._ (2018).

Figure 2e depicts the gain in the Squeezer OPA as a function of Pump 1 pulse
energy . The gain for lower pump energies ($<$2.4 pJ) is determined from the
anti-squeezing measurements, while for higher pump energies ($>$2.4 pJ) we
obtain the gain from a direct measurement of average photon-number Sup ;
Ledezma  _et al._ (2021). For a vacuum-seeded phase-sensitive OPA, the average
number of photons in the high parametric gain regime
($\langle\hat{N}\rangle\sim G/4$) allows us to estimate the gain Sup . The
solid curve is the fit that includes the overall detection efficiency
(including off-chip coupling losses and imperfect detection after the
Measurement OPA) and the nonlinear strength as fitting parameters. From the
fit, we extract the overall detection efficiency of $\eta^{\text{off-
chip}}_{\text{overall}}\sim 0.20$ Sup . This level of linear loss puts an
upper limit of $<1$ dB to the measurable squeezing for a standard BHD.
Remarkably, this is not a limiting factor for our all-optical squeezing
measurements because of the noiseless amplification by the Measurement OPA.
Note that such lossy measurements are even more detrimental for highly
squeezed states, as they are extremely sensitive to losses. This can be seen
in Fig. 2d, where $\sim$11 dB of initial squeezing degrades by $\sim$10 dB in
the presence of the detection losses of $L^{\text{off-
chip}}_{\text{overall}}=1-\eta^{\text{off-chip}}_{\text{overall}}=0.80$.
However, our all-optical measurement is not affected by $L^{\text{off-
chip}}_{\text{overall}}$ losses due to the amplification by the Measurement
OPA and allows us to measure the squeezing levels as high as 4.9 dB Sup .
Thus, our measured squeezing is mostly limited by the coupling losses at the
adiabatic coupler, which can be fabricated with losses of $<0.05$, as
suggested by our numerical simulations Sup .

Figure 3: Broadband squeezing measurements. a, b, c) Three measurements over
different selected bandwidths of the OSA when the PZT is modulated with a 300
mHz ramp signal. The shot noise traces (blue, black and red) were acquired
with Pump 1 blocked. d) Optical parametric generation from the squeezer OPA
(green) and measurement amplifier OPA (blue). Both traces are acquired at
$\sim$ 6 pJ of pulse energy.

Figures 3a, 3b, and 3c show the measured squeezing over a broad bandwidth. The
amplified shot-noise is calibrated using the same method as discussed earlier
over the entire spectrum. Green traces correspond to measurements by the OSA
over three different spectral windows when the PZT is modulated by a slow ramp
signal at 300 mHz. Squeezing is present over the entire spectrum with a slight
spectral dependence. The measured squeezing is 3.8 dB around 1950 nm, 4.2 dB
around 2090 nm, and 4.9 dB around 2200 nm. This is attributed to the
wavelength dependence of the coupling efficiency of our adiabatic coupler Sup
. We measured the squeezing bandwidth to be 25.1 THz. The bandwidth is
expected to increase to 36.4 THz, as confirmed by the optical parametric
generation (OPG) from the Squeezer OPA in Fig. 3d. The measured squeezing
bandwidth is mostly limited by the slight mismatch of Measurement OPA gain in
the wings of the spectrum, as evident from its OPG signal. Due to this
difference in the gain spectrum, the Measurement OPA does not amplify the
squeezed vacuum over its entire generation bandwidth to macroscopic levels,
leading to a reduced measured squeezing bandwidth. These measurements indicate
that our generated squeezed state can occupy a record-level time window of
$\sim$ 4 optical cycles (See Ref. Sup , Part 3), thereby opening many
promising opportunities in ultra-high-speed optical quantum information
processing.

Our results on generating and measuring squeezed states all-optically in
nanophotonics mark an important milestone for achieving scalable CV quantum
photonic systems. Figure 4 compares our measured squeezing and bandwidth with
state-of-the-art demonstrations in nanophotonic platforms including SiN and
Silica Sup . These experimental demonstrations utilize relatively weaker
$\chi^{(3)}$ nonlinearities and require microresonators for enhancement.
Additionally, the squeezed light is typically detected using off-chip BHDs
which impose limitations on the measured squeezing (due to off-chip coupling
losses) and the accessible squeezing bandwidths. Our measured squeezing
substantially surpasses these other works both in the magnitude and bandwidth.

Figure 4: Performance comparison of measured squeezing levels and squeezing
bandwidths on various nanophotonic platforms.

In summary, we demonstrated few-cycle vacuum squeezing in the LN nanophotonic
platform. Our results represent a paradigm shift for on-chip squeezed light
sources for quantum information processing. The extraordinary performance of
quadratic parametric amplifiers is achieved by spatio-temporal confinement of
the pump and signal pulses. Immediate improvements would include reducing the
coupler loss, improving the input coupling losses (which are currently
limiting the gain of the Measurement OPA), and improving the phase noise of
the pump.

All-optical measurements through high-gain amplification brings the
microscopic signal to macroscopic levels, making it resilient to coupling
losses and imperfect detection, and allows one to measure squeezing over the
optical bandwidths that are not accessible through conventional homodyne
detection Chen  _et al._ (2014); Yang  _et al._ (2021). The broadband gain
allows for the generation and measurement of squeezing over a bandwidth
corresponding to a few-cycle temporal pulse. This can be beneficial for time-
multiplexed quantum systems Asavanant  _et al._ (2019); Larsen  _et al._
(2020); Marandi  _et al._ (2014) by enabling THz clock rates. Moreover, our
results can open paths toward studying quantum fields in the extremely short-
pulse regime Kizmann  _et al._ (2019); Riek  _et al._ (2017).

Our all-optical measurement in the same nanophotonic circuit eliminates the
need for the spatio-temporal shaping of the local oscillator which is
typically done in squeezing experiments Kim and Kumar (1994); Kanter  _et al._
(2002). Such an all-optical measurement can greatly simplify the quadrature
measurements for CV quantum information processing using quantum frequency
combs Chen  _et al._ (2014); Roslund  _et al._ (2014). Our all-optical loss-
tolerant measurement scheme can also be used to purify noisy quantum
measurements for various applications in optical quantum information
processing, including all-optical quantum communication, and for protection
and characterization of quantum resources Dall’Arno  _et al._ (2010); Fukui
_et al._ (2021); Ralph (1999).

## Data Availability

The data supporting the plots within this paper and other findings of this
study are available from the corresponding author upon reasonable request.

## Code Availability

The computer codes used to perform the numerical simulations in this paper are
available from the corresponding author upon reasonable request.

## Acknowledgements

The device nanofabrication was performed at the Kavli Nanoscience Institute
(KNI) at Caltech. The authors gratefully acknowledge support from ARO grant
no. W911NF-18-1-0285, NSF grant no. 1846273 and 1918549, AFOSR award
FA9550-20-1-0040, and NASA/JPL. This project was funded in part by the
President’s and Director’s Research and Development Fund of Caltech and JPL.
The authors wish to thank NTT Research for their financial and technical
support. The authors thank Carlos Gonzàlez-Arciniegas and Olivier Pfister for
fruitful discussions.

## Authors Contributions

R.N. and A.M. conceived the idea and designed the experiments; R.N. designed
the devices with assistance from L.L. and Q.G.; R.S. fabricated the devices
and L.L. performed the periodic poling; R.N. carried out the experiments with
assistance from R.S., R.M.G., Q.G. and A.R.; R.N. performed the theoretical
and numerical analysis with contributions from L.L.; R.N. and A.M. wrote the
manuscript with input from all other authors. A.M. supervised the project.

## References

  * Clarke and Wilhelm (2008) J. Clarke and F. K. Wilhelm, Nature 453, 1031 (2008).
  * O’brien  _et al._ (2009) J. L. O’brien, A. Furusawa, and J. Vučković, Nature Photonics 3, 687 (2009).
  * Cirac and Zoller (1995) J. I. Cirac and P. Zoller, Physical review letters 74, 4091 (1995).
  * Imamog¯lu  _et al._ (1999) A. Imamog¯lu, D. D. Awschalom, G. Burkard, D. P. DiVincenzo, D. Loss, M. Sherwin, and A. Small, Phys. Rev. Lett. 83, 4204 (1999).
  * Saffman  _et al._ (2010) M. Saffman, T. G. Walker, and K. Mølmer, Reviews of modern physics 82, 2313 (2010).
  * Vandersypen  _et al._ (2001) L. M. Vandersypen, M. Steffen, G. Breyta, C. S. Yannoni, M. H. Sherwood, and I. L. Chuang, Nature 414, 883 (2001).
  * Chen  _et al._ (2014) M. Chen, N. C. Menicucci, and O. Pfister, Phys. Rev. Lett. 112, 120505 (2014).
  * Roslund  _et al._ (2014) J. Roslund, R. M. De Araujo, S. Jiang, C. Fabre, and N. Treps, Nature Photonics 8, 109 (2014).
  * Asavanant  _et al._ (2019) W. Asavanant, Y. Shiozawa, S. Yokoyama, B. Charoensombutamon, H. Emura, R. N. Alexander, S. Takeda, J.-i. Yoshikawa, N. C. Menicucci, H. Yonezawa, _et al._ , Science 366, 373 (2019).
  * Larsen  _et al._ (2019) M. V. Larsen, X. Guo, C. R. Breum, J. S. Neergaard-Nielsen, and U. L. Andersen, Science 366, 369 (2019).
  * Zhong  _et al._ (2020) H.-S. Zhong, H. Wang, Y.-H. Deng, M.-C. Chen, L.-C. Peng, Y.-H. Luo, J. Qin, D. Wu, X. Ding, Y. Hu, _et al._ , Science 370, 1460 (2020).
  * Knill  _et al._ (2001) E. Knill, R. Laflamme, and G. J. Milburn, Nature 409, 46 (2001).
  * Lloyd and Braunstein (1999) S. Lloyd and S. L. Braunstein, Physical Review Letters 82, 1784 (1999).
  * Gu  _et al._ (2009) M. Gu, C. Weedbrook, N. C. Menicucci, T. C. Ralph, and P. van Loock, Phys. Rev. A 79, 062318 (2009).
  * Yu  _et al._ (2020) H. Yu, L. McCuller, M. Tse, N. Kijbunchoo, L. Barsotti, and N. Mavalvala, Nature 583, 43 (2020).
  * Casacio  _et al._ (2021) C. A. Casacio, L. S. Madsen, A. Terrasson, M. Waleed, K. Barnscheidt, B. Hage, M. A. Taylor, and W. P. Bowen, Nature 594, 201 (2021).
  * Furusawa  _et al._ (1998) A. Furusawa, J. L. Sørensen, S. L. Braunstein, C. A. Fuchs, H. J. Kimble, and E. S. Polzik, science 282, 706 (1998).
  * Raussendorf and Briegel (2001) R. Raussendorf and H. J. Briegel, Phys. Rev. Lett. 86, 5188 (2001).
  * Kanter  _et al._ (2002) G. S. Kanter, P. Kumar, R. V. Roussev, J. Kurz, K. R. Parameswaran, and M. M. Fejer, Optics express 10, 177 (2002).
  * Larsen  _et al._ (2020) M. V. Larsen, X. Guo, C. R. Breum, J. S. Neergaard-Nielsen, and U. L. Andersen, arXiv preprint arXiv:2010.14422 (2020).
  * Joshi  _et al._ (2020) C. Joshi, A. Farsi, A. Dutt, B. Y. Kim, X. Ji, Y. Zhao, A. M. Bishop, M. Lipson, and A. L. Gaeta, Physical review letters 124, 143601 (2020).
  * Vaidya  _et al._ (2020) V. Vaidya, B. Morrison, L. Helt, R. Shahrokshahi, D. Mahler, M. Collins, K. Tan, J. Lavoie, A. Repingon, M. Menotti, _et al._ , Science advances 6, eaba9186 (2020).
  * Yang  _et al._ (2021) Z. Yang, M. Jahanbozorgi, D. Jeong, S. Sun, O. Pfister, H. Lee, and X. Yi, Nature Communications 12, 1 (2021).
  * Vigliar  _et al._ (2021) C. Vigliar, S. Paesani, Y. Ding, J. C. Adcock, J. Wang, S. Morley-Short, D. Bacco, L. K. Oxenløwe, M. G. Thompson, J. G. Rarity, _et al._ , Nature Physics 17, 1137 (2021).
  * Arrazola  _et al._ (2021) J. Arrazola, V. Bergholm, K. Brádler, T. Bromley, M. Collins, I. Dhand, A. Fumagalli, T. Gerrits, A. Goussev, L. Helt, _et al._ , Nature 591, 54 (2021).
  * Dutt  _et al._ (2015) A. Dutt, K. Luke, S. Manipatruni, A. L. Gaeta, P. Nussenzveig, and M. Lipson, Physical Review Applied 3, 044005 (2015).
  * Cernansky and Politi (2020) R. Cernansky and A. Politi, APL Photonics 5, 101303 (2020).
  * Tasker  _et al._ (2021) J. F. Tasker, J. Frazer, G. Ferranti, E. J. Allen, L. F. Brunel, S. Tanzilli, V. D’Auria, and J. C. Matthews, Nature Photonics 15, 11 (2021).
  * Zhao  _et al._ (2020) J. Zhao, C. Ma, M. Rüsing, and S. Mookherjea, Phys. Rev. Lett. 124, 163603 (2020).
  * Javid  _et al._ (2021) U. A. Javid, J. Ling, J. Staffa, M. Li, Y. He, and Q. Lin, Phys. Rev. Lett. 127, 183601 (2021).
  * Chen  _et al._ (2021) P.-K. Chen, I. Briggs, S. Hou, and L. Fan, arXiv preprint arXiv:2107.02250 (2021).
  * Caves (1982) C. M. Caves, Physical Review D 26, 1817 (1982).
  * Shaked  _et al._ (2018) Y. Shaked, Y. Michael, R. Z. Vered, L. Bello, M. Rosenbluh, and A. Pe’er, Nature communications 9, 1 (2018).
  * Takanashi  _et al._ (2020) N. Takanashi, A. Inoue, T. Kashiwazaki, T. Kazama, K. Enbutsu, R. Kasahara, T. Umeki, and A. Furusawa, Optics Express 28, 34916 (2020).
  * Frascella  _et al._ (2021) G. Frascella, S. Agne, F. Y. Khalili, and M. V. Chekhova, npj Quantum Information 7, 1 (2021).
  * Li  _et al._ (2020) J. Li, Y. Liu, N. Huo, L. Cui, S. Feng, X. Li, and Z. Y. Ou, Phys. Rev. A 101, 053801 (2020).
  * Ye  _et al._ (2021) Z. Ye, P. Zhao, K. Twayana, M. Karlsson, V. Torres-Company, and P. A. Andrekson, Science advances 7, eabi8150 (2021).
  * Zhu  _et al._ (2021) D. Zhu, L. Shao, M. Yu, R. Cheng, B. Desiatov, C. Xin, Y. Hu, J. Holzgrafe, S. Ghosh, A. Shams-Ansari, _et al._ , Advances in Optics and Photonics 13, 242 (2021).
  * Jankowski  _et al._ (2021) M. Jankowski, J. Mishra, and M. M. Fejer, Journal of Physics: Photonics (2021).
  * Ledezma  _et al._ (2021) L. Ledezma, R. Sekine, Q. Guo, R. Nehra, S. Jahani, and A. Marandi, arXiv preprint arXiv:2104.08262 (2021).
  * Jankowski  _et al._ (2020) M. Jankowski, C. Langrock, B. Desiatov, A. Marandi, C. Wang, M. Zhang, C. R. Phillips, M. Lončar, and M. M. Fejer, Optica 7, 40 (2020).
  * Guo  _et al._ (2021) Q. Guo, R. Sekine, L. Ledezma, R. Nehra, D. J. Dean, A. Roy, R. M. Gray, S. Jahani, and A. Marandi, arXiv preprint arXiv:2107.09906 (2021).
  * Hu  _et al._ (2021) Y. Hu, M. Yu, D. Zhu, N. Sinclair, A. Shams-Ansari, L. Shao, J. Holzgrafe, E. Puma, M. Zhang, and M. Lončar, Nature 599, 587 (2021).
  * (44) See supplementary information .
  * Werner  _et al._ (1995) M. J. Werner, M. G. Raymer, M. Beck, and P. D. Drummond, Phys. Rev. A 52, 4202 (1995).
  * Oelker  _et al._ (2016) E. Oelker, G. Mansell, M. Tse, J. Miller, F. Matichard, L. Barsotti, P. Fritschel, D. McClelland, M. Evans, and N. Mavalvala, Optica 3, 682 (2016).
  * Bourassa  _et al._ (2021) J. E. Bourassa, R. N. Alexander, M. Vasmer, A. Patil, I. Tzitrin, T. Matsuura, D. Su, B. Q. Baragiola, S. Guha, G. Dauphinais, _et al._ , Quantum 5, 392 (2021).
  * Fukui  _et al._ (2018) K. Fukui, A. Tomita, A. Okamoto, and K. Fujii, Phys. Rev. X 8, 021054 (2018).
  * Marandi  _et al._ (2014) A. Marandi, Z. Wang, K. Takata, R. L. Byer, and Y. Yamamoto, Nature Photonics 8, 937 (2014).
  * Kizmann  _et al._ (2019) M. Kizmann, T. L. d. M. Guedes, D. V. Seletskiy, A. S. Moskalenko, A. Leitenstorfer, and G. Burkard, Nature Physics 15, 960 (2019).
  * Riek  _et al._ (2017) C. Riek, P. Sulzer, M. Seeger, A. S. Moskalenko, G. Burkard, D. V. Seletskiy, and A. Leitenstorfer, Nature 541, 376 (2017).
  * Kim and Kumar (1994) C. Kim and P. Kumar, Physical review letters 73, 1605 (1994).
  * Dall’Arno  _et al._ (2010) M. Dall’Arno, G. M. D’Ariano, and M. F. Sacchi, Phys. Rev. A 82, 042315 (2010).
  * Fukui  _et al._ (2021) K. Fukui, R. N. Alexander, and P. van Loock, Phys. Rev. Research 3, 033118 (2021).
  * Ralph (1999) T. C. Ralph, Optics letters 24, 348 (1999).