# Generating Benchmarks for Factuality Evaluation of Language Models

Dor Muhlgay Ori Ram Inbal Magar Yoav Levine Nir Ratner
Yonatan Belinkov Omri Abend Kevin Leyton-Brown Amnon Shashua Yoav Shoham

AI21 Labs  Corresponding author<EMAIL_ADDRESS>
###### Abstract

Before deploying a language model (LM) within a given domain, it is important
to measure its tendency to generate factually incorrect information in that
domain. Existing factual generation evaluation methods focus on facts sampled
from the LM itself, and thus do not control the set of evaluated facts and
might under-represent rare and unlikely facts. We propose FACTOR: Factual
Assessment via Corpus TransfORmation, a scalable approach for evaluating LM
factuality. FACTOR automatically transforms a factual corpus of interest into
a benchmark evaluating an LM’s propensity to generate true facts from the
corpus vs. similar but incorrect statements. We use our framework to create
two benchmarks: Wiki-FACTOR and News-FACTOR. We show that: (i) our benchmark
scores increase with model size and improve when the LM is augmented with
retrieval; (ii) benchmark score correlates with perplexity, but the two
metrics do not always agree on model ranking; and (iii) when perplexity and
benchmark score disagree, the latter better reflects factuality in open-ended
generation, as measured by human annotators. We make our data and code
publicly available.111https://github.com/AI21Labs/factor

##  1 Introduction

Figure 1: Each example in our evaluation task (dubbed FACTOR) consists of a
prefix and four completions, of which only one is factually correct
(completion (a) in this example). The non-factual completions (b), (c) and
(d), marked in red, are generated according to different factual error types,
detailed in Table 2. The evaluated model assigns likelihood scores to each
completion separately. It is considered “correct” if it assigns the highest
likelihood to the factually correct completion over all non-factual
alternatives.

Despite rapid improvements in their capabilities, large Language Models (LMs)
still tend to generate factually inaccurate or erroneous text Lin et al.
(2022); Maynez et al. (2020); Huang et al. (2020). Such phenomena can pose a
significant hurdle to deploying LMs in important or sensitive settings,
motivating the development of methods for evaluating LM factuality in
generation. Previous work evaluates the knowledge stored in LMs by using
static sets of facts Petroni et al. (2019, 2021); Roberts et al. (2020), e.g.,
via measuring their ability to answer a predefined set of knowledge-intensive
questions. Since these facts are tested in isolation and without context, they
serve only as a proxy measure of an LM’s behavior during open-ended
generation.

Alternative methods for directly evaluating an LM’s propensity towards factual
generations were recently proposed by Lee et al. (2022) and Min et al. (2023).
These methods take the straightforward approach of sampling long outputs from
a model and applying an automatic pipeline for fact verification, assigning a
score corresponding to the percentage of generated facts which are correct.
However, the sampling approach may introduce bias: by scoring the accuracy of
facts that an LM tends to generate in an open-ended setting, high-likelihood
common facts are over-represented, while the “long-tail" of rarer facts is
under-represented. This limits our control over which facts are evaluated.

This paper introduces a new and complementary method for uniformly and
inexpensively testing a model’s tendency to generate factual information from
a given factual corpus: Factual Assessment via Corpus TransfORmation (FACTOR).
The key idea is automatically perturbing factual statements taken from the
corpus to create a constant number of false variations (hereafter, $3$) for
each true statement (Figure 1). The LM’s FACTOR accuracy on our benchmark is
defined as the percentage of examples for which it assigns higher likelihood
to the factual completion than to any of the false variations. We employed
InstructGPT Ouyang et al. (2022) to generate the false variations for each
true statement, according to the FRANK factual error typology (Pagnoni et al.,
2021).

We applied FACTOR to Wikipedia and to the news domain, constructing two new,
challenging benchmarks that we dub Wiki-FACTOR and News-FACTOR. We used these
datasets to evaluate a large suite of LMs from the OPT Zhang et al. (2022),
GPT-2 Radford et al. (2019), and GPT-Neo Black et al. (2021) families, ranging
from 110M to 66B parameters. We show in §5.1 that, as expected, FACTOR scores
increase with model size. However, even the largest models we evaluated
achieved scores of only $58$% for Wiki-FACTOR and $68$% for News-FACTOR,
indicating that these benchmarks are challenging even for large LMs.

We discuss three additional experimental findings. First, in §5.2 we report
significant improvements in FACTOR when the LMs are augmented with the simple
retrieval component used by Ram et al. (2023). While Ram et al. (2023)
demonstrate the gains of their method via the LM perplexity measure, our
results directly demonstrate that retrieval augmentation improves factuality
in the LM setting. Second, we show that FACTOR accuracy and LM perplexity tend
to be highly correlated but sometimes induce different orderings between LMs
(see Figure 2). Of course, while perplexity is indeed affected by the
likelihood of facts mentioned in the corpus, it is also affected by many
additional linguistic phenomena. Third, in §5.3 we report on pairs of models
that share similar perplexity but differ significantly in terms of FACTOR
accuracy. In §6, we report findings of a manual annotation effort over $1,200$
genrated completions, which reinforces FACTOR accuracy as predictive of
factuality in open-ended generation.

Figure 2: Wiki-FACTOR scores versus LM perplexity on Wikipedia for LMs from
the GPT-Neo model family (blue circle, sizes (1.3B-20B)) and the OPT model
family (red triangle, 1.3B-66B). Labels indicate sizes (in billions). The two
measures correlate but can disagree on ranking, e.g., the OPT-66B LM has
higher perplexity but better Wiki-FACTOR accuracy than the GPT-J-6B LM. In §6
we annotate text generated out of both models and show that better Wiki-FACTOR
is predictive of more factual text generation.

##  2 Related Work

##### Factuality Evaluation for LMs

Many well-established benchmarks for evaluating factual knowledge are focused
on knowledge-intensive downstream tasks like question answering, fact
verification, and slot filling (Petroni et al., 2019, 2021; Roberts et al.,
2020; Jiang et al., 2020; Elazar et al., 2021; Mallen et al., 2023). This body
of work tests models in a simplified uncontextualized setting, testing
knowledge of each fact in isolation. Tam et al. (2023) proposed evaluating
LMs’ factual consistency against a grounding context through summarization.
FACTOR extends those benchmarks to a text generation setting, in which the
evaluated facts are contextualized.

Recent work by Lee et al. (2022) and Min et al. (2023) propose automatically
scoring the factuality of a sample of free-form LM generations on a certain
topic. These approaches make it hard to control the facts contributing to the
evaluation score, and moreover they are biased towards common facts, since
they are generated by the evaluated LM itself.

##### Contrastive Datasets

Contrastive evaluation, in which the model’s ability to discern between
similar positive and negative examples is tested, is a long-standing approach
used in a wide range of tasks Sennrich (2017); Burlot and Yvon (2017);
Glockner et al. (2018); Kaushik et al. (2020); Glockner et al. (2018). For
factuality evaluation, the contrastive approach is applied by perturbing
factual claims to obtain negative examples. Schuster et al. (2021) performed
synthetic edits to Wikipedia sentences with crowd workers, while Liu et al.
(2022) used BERT Devlin et al. (2019) to make local replacements, verified by
human annotators. Gupta et al. (2022) used model and rule-based heuristics to
perturb factual claims in dialogue.

Following recent work, which showed that contemporary large LMs can be
instructed to generate high quality evaluation sets Perez et al. (2023), our
FACTOR approach leverages those capabilities to generate contextualized,
fluent, diverse and challenging non-factual claims, as described in the
following section.

Original text (completion in bold) | …In 1982, Donne was appointed as the first Queen’s Representative
---|---
to the Cook Islands. After completing his term, he became Chief
Justice of Nauru and Tuvalu in 1985.
Error Type | Example
Entity | After completing his term, he became the Queen’s Representative to
the Cook Islands in 1985.
Predicate | After completing his term, he declined the position of Chief Justice
of Nauru and Tuvalu in 1985.
Circumstance | After completing his term, he became Chief Justice of Nauru and
Tuvalu in 1987.
Coreference | After completing her term, she became Chief Justice of Nauru and
Tuvalu in 1985.
Link | Before completing his term, he became Chief Justice of Nauru and
Tuvalu in 1985.

Table 1: Error types examples. The original text (top) consists of a prefix
and a completion sentence (marked in bold). Each example introduce different
perturbation over the original completion of different type (edit marked in
red).

##  3 The FACTOR Evaluation Approach

This section describes our proposed approach of Factual Assessment via Corpus
TransfORmation, or FACTOR. Given an (unlabeled) evaluation corpus, we define a
multi-choice task where each example is comprised of a multi-sentence prefix,
a single factual next sentence completion, and three non-factual alternative
completions (Figure 1). In §3.1 we present several properties required of a
FACTOR benchmark, and describe the error verticals along which we generate
non-factual alternatives. We then describe our FACTOR dataset creation
pipeline, which given a corpus generates a FACTOR benchmark automatically
(§3.2). Lastly, we describe our application of this pipeline to two corpora:
Wikipedia and news, creating Wiki-FACTOR and News-FACTOR, and verify the
quality of these datasets through manual annotations against the required
properties (§3.3).

###  3.1 The Evaluation Task: FACTOR

We now formally describe the FACTOR multi-choice factual evaluation task. Each
instance (or example) of our task contains a prefix text $t$ comprised of
several sentences, along with four possible full sentence completions, of
which only one is factually correct. We choose the original completion (i.e.,
the continuation of $t$ in the corpus) as the factually correct one. We denote
this correct completion as $c^{+}$, and the non-factual completions as
$\mathcal{C}^{-}=\\{c^{-}_{1},c^{-}_{2},c^{-}_{3}\\}$. We evaluate models by
measuring the percentage of examples for which they assign the highest mean
log-probability to $c^{+}$. To put formally, the model is correct on a given
example if

$c^{+}=\operatorname*{argmax}_{c\in\\{c^{+}\\}\cup\mathcal{C}^{-}}\frac{\log
p(c|t)}{|c|},$ (1)

where $|c|$ is the length of completion $c$ in tokens. We refer to the
percentage of correct examples as the FACTOR accuracy.

Generally, we require each of the “incorrect” completions
$c^{-}\in\mathcal{C}^{-}$ to satisfy the following properties:

  1. 1.

Non-factual: $c^{-}$ contains a false claim;

  2. 2.

Fluent: $c^{-}$ is grammatical;

  3. 3.

Similar to the factual completion: $c^{-}$ has a small edit-distance from
$c^{+}$.

The second and third properties are aimed at making it harder to distinguished
between the factual and non-factual completions for reasons other than their
factual correctness, such as fluency or style. Furthermore, it is desirable
that the non-factual completions be logical and self-consistent, to make them
more difficult to eliminate. For example, for $c^{+}=$“They got married in
2010 and divorced in 2017”, then $c^{-}=$“They got married in 2010 and
divorced in 2019” is a good non-factual completion since it checks whether the
model knows the divorce year, while $c^{-}=$“They got married in 2010 and
divorced in 2009” only evaluates the model’s knowledge of the temporal
relations between marriage and divorce.

##### Error Types

Ideally, non-factual completions in a FACTOR dataset should span a diverse
variety of factuality error types. To achieve this, we adopt the error
typology introduced in FRANK Pagnoni et al. (2021) for factual evaluation in
summarization. While they introduced their error typology to categorize
factual inconsistencies of generated summaries w.r.t. the source document, we
instead leverage this typology to vary the type of factual inconsistencies
that hold between between our non-factual completions and the prefix and
completion ($t$ and $c^{+}$). We focus on the five error types from two
categories: semantic frame errors and discourse errors (examples shown in
Table 2):

  * •

Predicate error: a predicate that is inconsistent with $c^{+}$ or $t$.

  * •

Entity error: The subject or object of a predicate are inconsistent with
$c^{+}$ or $t$.

  * •

Circumstance error: The completion contains information describing the
circumstance of a predicate (e.g., location, time, manner) that is
inconsistent with $c^{+}$ or $t$.

  * •

Coreference error: The contradiction is inconsistent with a pronoun/reference
in $c^{+}$ or $t$, referring to a wrong or non-existing entity.

  * •

Link error: $c^{-}$ is inconsistent with $c^{+}$ or $t$ in the way that
different statements are linked together (causal/temporal links).

###  3.2 Generating FACTOR Benchmarks

Given an evaluation corpus, we generate a FACTOR benchmark automatically, in a
process designed to meet the requirements presented in §3.1. The dataset
creation process follows a four-stage pipeline: (1) prefix and completion
selection, (2) non-factual completion generation, (3) non-factual completion
filtering, and (4) non-factual completion selection.

####  3.2.1 Prefix and Factual Completion Selection

We select a single sentence from each document as a factually correct
completion $c^{+}$. We exclude headlines and sentences with fewer than 10
words. The prefix $t$ is set to be the entire text preceding $c^{+}$ in the
document.

####  3.2.2 Non-factual Completion Generation

Given a prefix $t$ and its original completion $c^{+}$, we use InstructGPT
(davinci-003; Ouyang et al. 2022) to generate a set of contradictory
completions. We designed a specific prompt instructing the model to generate
contradictions corresponding to each type of error. We only apply each prompt
to sentences that are relevant to its error type (determined through simple
heuristics, see Appendix A.1 for details).

The prompts are designed as follows:

  * •

Multiple contradiction generation: the model is prompted to generate multiple
subsequent contradictions in each sampling operation. Therefore, the model
creates new completions conditioned on its previous ones. In preliminary
experiments, we found this sampling practice to improve diversity when
compared to multiple independent sampling of completions.

  * •

Edit planning: for each contradiction, the model first explicitly generates
the planned edits over the original completion, and then applies those edits
by writing the entire modified completion (similar to chain-of-thought
prompting; Wei et al. 2022). For example, generating the coreference error in
Table 2 is done by first explicitly writing the edits (“Changes: ‘his’ to
‘her’, ‘he’ to ‘she’ ”), before generating the contradiction. This encourages
the model to make minimal edits. See full prompts in the appendix.

Appendix D lists the full prompts that we used for all error types.

Property | Wiki | News
---|---|---
Non-factual | 97.6 | 98.3
Fluent | 94.0 | 97.0
Self-Consistent | 87.4 | 87.3
Edit-Distance | 2.3$\pm$(1.4) | 2.1$\pm$(1.4)
Table 2: Validation results: (1) percentage of generation that meet each
desired property in Wiki-FACTOR (Wiki) and News-FACTOR (News), estimated by
manual annotation over sub-samples (top), (2) mean edit-distance between the
generations and their original completion (bottom).

####  3.2.3 Non-factual Completion Filtering

We considered the set of generated completions as candidates for non-factual
completions. We applied automatic tools to filter out (i) non-contradictory
and (ii) non-fluent completions.

##### Non-Contradictory Completions

Given a candidate completion $c$, we asserted that it was indeed contradictory
to the original completion $c^{+}$ by applying a NLI model.222We used DeBERTa-
large model He et al. (2021) fine-tuned on the MNLI dataset Williams et al.
(2018) from Hugging Face: microsoft/deberta-large-mnli. The premise was set to
be $c^{+}$ along with its near context (i.e., the last tokens of the prefix
$t$; denoted by $t_{\text{near}}$). The hypothesis was set to be $c$, also
preceded by $t_{\text{near}}$. We selected generations that the NLI model
classified as contradictory with a probability higher than a fixed threshold
$\tau_{\textrm{NLI}}$, i.e.:

$p_{\textrm{NLI}}(\text{contradiction}\ |\
[t_{\textrm{near}};c^{+}],[t_{\textrm{near}};c]))>\tau_{\textrm{NLI}}$

We chose $\tau_{\textrm{NLI}}=0.6$ (except for contradictions generated by the
coreference error prompt, where we set $\tau_{\textrm{NLI}}=0.3$) after using
a manual validation process detailed Appendix A.2.

##### Non-Fluent Completions

To verify that $c$ is a fluent completion we used GPT2-Small Radford et al.
(2019) scores, similar to Gupta et al. (2022): We filtered out generations
with mean log-likelihood lower than the original completion’s by a fixed
margin $\tau_{\textrm{LM}}$. Formally, we selected a completion $c$ if it
satisfied:

$\frac{\log p(c|t)}{|c|}>\frac{\log p(c^{+}|t)}{|c^{+}|}-\tau_{\textrm{LM}}$

Using a manual validation detailed in Appendix A.2, we set
$\tau_{\textrm{LM}}=0.2$.

####  3.2.4 Non-factual Completion Selection

Finally, we selected the non-factual completions
$c^{-}_{1},c^{-}_{2},c^{-}_{3}$ from the candidates that passed the filtering
phase. In order to increase diversity in error types, we first selected a
single completion from each type. In cases where fewer than three generations
overall met the criteria from §3.2.3, we added completions from already used
error types.

(a) Wiki-FACTOR

(b) News-FACTOR

Figure 3: Accuracy per model size for Wiki-FACTOR (left) and News-FACTOR (right), for models from GPT-2 (blue circle), GPT-Neo (red triangle), and OPT (yellow square) families. Type | Wiki | News
---|---|---
Predicate | 25.4 | 31.3
Entity | 42.8 | 48.0
Circumstance | 24.2 | 16.0
Coreference |  4.4 |  2.3
Link |  3.2 |  2.3
Table 3: Annotated error type distribution for Wiki-FACTOR (Wiki), and News-
FACTOR (News).

###  3.3 Applying FACTOR to Wikipedia and News Domains

We focused on two knowledge intensive domains: Wikipedia (encyclopedic
knowledge) and news (current events). We constructed two evaluation datasets
following the above described procedure:

  * •

Wiki-FACTOR: a dataset based on the Wikipedia section of The Pile’s validation
split Gao et al. (2021), containing $2994$ examples.

  * •

News-FACTOR: a dataset based on Reuters articles published after $1/10/2021$,
extracted from The RefinedWeb Dataset Penedo et al. (2023). The dataset
consists of $1036$ examples.

####  3.3.1 Dataset Validation

To validate that Wiki-FACTOR and News-FACTOR meet the required FACTOR
properties detailed in §3.1, we manually evaluated a sub-sample from each
dataset. We sampled $138$ examples from Wiki-FACTOR and $100$ examples from
News-FACTOR, containing $414$ and $300$ generations overall. Each generation
was annotated w.r.t. the properties manifested in §3.1, namely whether they
were (1) non-factual, (2) fluent, and (3) self-consistent. To assess the
dataset diversity, we annotated the contradictions in accordance with the
error typology of Pagnoni et al. (2021), described in §3.1. In addition, we
verified that the non-factual completions were similar to the factual
completion by measuring their mean edit distances over all the examples.

Table 2 shows the validation results. For both datasets, almost every
generated completion indeed contradicted the corresponding original
completion, was fluent, and was self consistent. Table 3 shows the error type
distribution. For both datasets we observed that FACTOR yielded diverse
contradiction types. Semantic frame errors (Entity, Predicate, and
Circumstance) were more prevalent than discourse errors (Link and
Coreference), as more sentences are suited for these type of errors.

##  4 Experimental Setup

We use Wiki-FACTOR and News-FACTOR (§3.2) to evaluate factual knowledge of
various LLMs from different model families. In addition, we explore how
grounding on an external corpus can affect factual knowledge. Specifically, we
leveraged an existing Wikipedia retrieval-corpus to evaluate retrieval
augmented models over Wiki-FACTOR. We further investigate our FACTOR approach,
compare it to perplexity, and examine how its links to factuality in
generation (§6).

###  4.1 Datasets

The Wiki-FACTOR and News-FACTOR datasets are described in §3.3. For perplexity
evaluation (§5.3) , we selected a subset of $300$ Wikipedia articles from the
documents Wiki-FACTOR is based on, containing approximately $367$K tokens.

(a)

(b)

Figure 4: Factual accuracy over Wiki-FACTOR for GPT-Neo and OPT models,
compared to their IC-RALM variants. IC-RALM leads to consistent improvement
for all models.

###  4.2 Models

We performed our experiments over a set of open source models. We evaluated
four models of GPT-2 family (110M–1.5B parameters; Radford et al. 2019), five
models from the GPT-Neo family (125M–20B; Black et al. 2021, 2022; Wang and
Komatsuzaki 2021), and eight models of OPT (125M–66B; Zhang et al. 2022). We
capped the sequence length at $1024$ tokens, so all models could be directly
compared to each other.

The corpora that Wiki-FACTOR and News-FACTOR were constructed from were not
used for training any of the examined models. News-FACTOR is based on articles
published after 1/10/2021, beyond the models’ data cutoff date. Wiki-FACTOR is
based on Wikipedia documents from The Pile’s validation split, which is not
part in any of the models’ training sets (OPT and GPT-Neo models were trained
on The Pile’s training split, GPT-2 models were not trained on Wikipedia).

###  4.3 Retrieval-Augmented Models

In §5.2, we present evaluations of retrieval-augmented variants of the models.
To that end, we adopted the In-Context RALM (IC-RALM) framework of Ram et al.
(2023), where the retrieved document is prepended to the LLM’s input, without
any further training or specialized LLM architecture. In IC-RALM, a retriever
is called every $s$ tokens (i.e., the retrieval stride), with a query
comprised of the last $\ell$ tokens. The top-scored retrieved document is then
prepended to the LLM’s input. The LLM is run with the concatenated input to
assign log-probabilities to the next $s$ tokens.

Similar to Ram et al. (2023), we used the lexical BM25 Robertson and Zaragoza
(2009) over the Wikipedia corpus,333We used the Wikipedia corpus of Karpukhin
et al. (2020), based on the dump from Dec. $20,2018$. excluding the evaluated
docs. We set $s=8$ and $\ell=32$.

##  5 Factual Knowledge Evaluation Results

This section describes the experimental evaluation of LLM factuality with
Wiki-FACTOR and News-FACTOR. In §5.1 we show that FACTOR accuracy increases
with LM size but also depends on the training data (different model families
differ in scores). In §5.2 we show that retrieval augmentation of the LM
improves FACTOR accuracy, positioning FACTOR accuracy as a first automatic
measure of factuality improvement for retrieval augmented LM methods. Finally,
in §5.3 we show that while corpus perplexity and FACTOR accuracy correlate,
their pairwise model ranking can significantly differ. This outcome, along
with manual validation of the correlation between FACTOR accuracy and factual
generation in §6, solidifies FACTOR accuracy as a novel automatic measure for
evaluating the proneness of an LM to generate factual information in a certain
domain.

###  5.1 Factual Knowledge Improves with Model Size

We evaluate models from the GPT-2, GPT-Neo, and OPT families on LLMs with
Wiki-FACTOR and News-FACTOR. Results are presented in Figure 3. As expected,
within a specific model family, larger models generally outperform smaller
ones.

However, even the largest models are capped at $58.0$% (GPT-NeoX-20B) and
$68.1$% (OPT-66B) on Wiki-FACTOR and News-FACTOR respectively, indicating that
these benchmarks are challanging.

We observe that all models achieve higher FACTOR accuracy on news than
Wikipedia when comparing the two domains. We hypothesize that since news
articles tend to cover more specific events, the prefix may be more useful for
detecting the factual completions. In addition, when comparing between
different model-families, we find that the OPT models leads on News-FACTOR,
while the GPT-Neo family leads on Wiki-FACTOR. This implies that the different
data sources used for training these two model families are suited to
different domains.

###  5.2 The Effect of Retrieval Augmentation on Factual Knowledge

Next, we ask: Can FACTOR accuracy be improved by augmenting models with a
retrieval component? Importantly, while a clear motivation for retrieval
augmentation is factual grounding of the LM, no existing metrics allow direct
measurement of the factuality enhancement over a corpus in a retrieval
augmented LMs. The imperfect measure of LM perplexity in often used to assess
these methods Khandelwal et al. (2020); Borgeaud et al. (2022); Ram et al.
(2023); Shi et al. (2023), and we propose FACTOR accuracy as a more direct
measure of factuality.

To explore this question, we compared the accuracy of LLMs to that of their
retrieval-augmented counterparts, implemented following the IC-RALM framework
(§4.3; Ram et al. 2023). Figure 4 compares the FACTOR accuracy of models in
GPT-Neo and OPT families against their IC-RALM variants over Wiki-FACTOR. We
observed consistent gains from augmenting the models with retrieval. These
results highlight that grounding the model in an external corpus can improve
its factual predictions. Since the retriever used in our experiments is used
in an “off-the-shelf” manner, we speculate that further performance boosts may
be gained by a retriever or reranker specialized for this task (Izacard et
al., 2022; Ram et al., 2023).

Another interesting finding is that the relative gains in FACTOR accuracy
obtained by IC-RALM, are more moderate compared to the relative gains in
perplexity over WikiText-103 (Merity et al., 2016), reported by Ram et al.
(2023). In the next section we further explore the connection between
perplexity and FACTOR accuracy.

###  5.3 Perplexity Correlates but is not Always Aligned with FACTOR Accuracy

We investigate whether FACTOR evaluation adds additional information beyond
perplexity, when used as a comparative metric for selecting which LM to use
within a certain corpus. Figure 2 shows the FACTOR accuracy of models on Wiki-
FACTOR, compared to their token-level perplexity on the Wikipedia section of
The Pile’s validation set (§4.1) (Appendix C includes all evaluated models).
Overall, we observe a high correlation between the two metrics. However, there
are cases where they disagree (i.e., a pair of models where one is better when
measured by perplexity but worse in terms of FACTOR accuracy). For example,
GPT-Neo-2.7B is significantly better than OPT-2.7B in terms of perplexity
($9.0$ vs. $10.1$), but slightly worse in terms of FACTOR accuracy ($46.3$%
vs. $46.6$%). In addition, GPT-J-6B has lower perplexity compared to OPT-66B
($7.4$ vs. $7.6$), while OPT-66B is significantly better in terms of FACTOR
accuracy ($57.7$% vs. $53.5$%). This finding suggests that (i) FACTOR accuracy
offers a complementary view of models’ performance, not necessarily captured
by the standard perplexity metric, and (ii) improvements in perplexity do not
necessarily imply better factuality.

##  6 Factuality in Open-Ended Generation

This section explores the connection between FACTOR accuracy and factuality in
open-ended generation, via an experiment based on human annotations.

###  6.1 Experimental Setup

We considered pairs of prefix, original completion and non-factual completion
$(t,c^{+},c^{-})$ from Wiki-FACTOR. We then manually identified the minimal
factual claim modified by $c^{-}$. For the example given in Table 2, replacing
“became” with “declined the position of” relates to the minimal fact “Donne
became Chief Justice of Nauru and Tuvalu”. We denote the minimal fact by $f$.

Also, let $c$ be the common prefix of $c^{+}$ and $c^{-}$. For the example
given in Table 2, along with the above replacement of $c^{+}$ with $c^{-}$,
the common prefix $c$ is: “After completing his term, he". We let LLMs
generate free text, conditioned on the concatenation of $t$ and $c$. In other
words, the model was conditioned on the prefix and the text of the completion
until the edit induced by $c^{-}$. Note that in this open-ended generative
setting the LM might generate the correct fact, text violating it, or any
other fluent continuation that does not refer to the specific fact. For each
example we manually annotated whether the generated text is true, false, or
neutral w.r.t. $f$.

We analyzed two models with a similar token-level perplexity but a significant
gap in FACTOR accuracy: GPT-J 6B and OPT-66B (marked in a green circle in
Figure 2).

For each model, we considered two groups of examples: examples with
$c^{+},c^{-}$ pairs for which the model was right, i.e., $c^{+},c^{-}$ pairs
for which:

$\frac{\log p(c^{+}|t)}{|c^{+}|}>\frac{\log p(c^{-}|t)}{|c^{-}|},$ (2)

and $c^{+},c^{-}$ pairs for which the model was wrong (i.e the complement of
Eq. 2).

We sampled three generations per example for $100$ examples from each group
and for each model. Overall, we created $2\times 2\times 100\times 3=1200$
generations.

We filtered some of the samples due to either ill-formatted generations, non-
factual original completions or non-contradictory completions (overall we
removed $14.5$% of the samples).

Model | Subset | Fact. Accuracy
---|---|---
GPT-J 6B | Right | 30.0%
Wrong | 10.5%
All (Weighted) | 24.8%
OPT-66B | Right | 46.6%
Wrong |  4.6%
All (Weighted) |  38.8%
Table 4: Manual factuality annotation results for OPT-66B and GPT-J 6B. For
each model, we present the results per right and wrong subsets. Bottom row
shows the weighted average between the right and wrong variants w.r.t to the
right/wrong pairs of Wiki-FACTOR.

###  6.2 Results

We assess each model’s knowledge of the minimal facts for each example through
manual annotation. To this end, we only considered generations that are
relevant to their minimal fact $f$, excluding “neutral” generations (which
constituted 59.5% and 54.3% for GPT-J 6B and OPT-66B, respectively). For each
model, we measure the percentage of generated texts that are true w.r.t. $f$
per the right and wrong subsets separately. We obtained the overall FACTOR
accuracy by weighting between the subsets results according to their
distribution in Wiki-FACTOR. Table 4 shows the results of our manual
annotation process (full results are given in App. B)

##### Accuracy over Wiki-FACTOR is linked with factuality in open-ended
generation.

We found that for examples where models were wrong, they also tended to
generate significantly more false claims w.r.t their minimal fact. For
example, OPT-66B generated a true claim only $4.6$% of the times for examples
it got wrong, compered to $46.6$% for examples for which it was right. This
suggests that the overall performance over FACTOR can shed light on the
model’s ability to generate factual claims accurately in open-ended
generation.

##### As a comparative metric, accuracy over Wiki-FACTOR aligns with
factuality in open-ended generation.

We observed significant gaps in manual factuality annotation between OPT-66B
and GPT-J 6B: OPT-66B generated true claims $38.8$% of the time, compared to
$24.8$% generated by GPT-J 6B. This aligns with the models’ performance over
Wiki-FACTOR, despite the fact that the two models share roughly the same
perplexity on Wiki.

This finding suggests that FACTOR can serve as a better proxy than perplexity
when measuring the factuality of a model on a given domain.

##  7 Discussion

This paper introduced FACTOR, a new approach for quantitatively evaluating LM
factuality. FACTOR creates an evaluation benchmark from a given corpus,
consisting of contextualized, factual statements from the corpus along with
non-factual variations. The LM’s FACTOR score describes its tendency to assign
greater likelihood to the original fact than to any of the non-factual
alternatives. By directly comparing the LM likelihood of factual claims with
their non-factual variants, FACTOR correlates with the LM’s propensity to
generate factual information.

While the approach of generating text and checking for its factuality Lee et
al. (2022); Min et al. (2023) evaluates a model’s likeliest behavior, our
approach is oriented towards assessing a model’s factual grasp of a broader
set of facts regardless of their respective likelihoods. We see these two
approaches as complementary; together, they can be used to yield a more
holistic assessment of LM factuality.

These two methods also differ in terms of costs. The expensive step in FACTOR
evaluation is the automatic benchmark creation pipeline for a given corpus,
described in §3.2. Once a FACTOR benchmark is created, it is very cheap to
evaluate different LMs (or different retrieval augmentation methods, etc) on
it. On the other hand, methods that directly evaluate LM generated text must
reapply their automatic fact verification pipeline for every generation of
every evaluated LM, so they incur a constant high cost for every evaluation.
FACTOR allows for a controlled uniform examination of facts within a corpus.
Overall, this paper establishes FACTOR as an scalable and effective approach
for assessing an LM’s propensity for generating factual text.

## References

  * Black et al. (2021) Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. 2021.  GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow.
  * Black et al. (2022) Sidney Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, Usvsn Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022.  GPT-NeoX-20B: An open-source autoregressive language model.  In _Proceedings of BigScience Episode #5 – Workshop on Challenges & Perspectives in Creating Large Language Models_, pages 95–136, virtual+Dublin. Association for Computational Linguistics.
  * Borgeaud et al. (2022) Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. 2022\.  Improving language models by retrieving from trillions of tokens.  In _ICML_.
  * Burlot and Yvon (2017) Franck Burlot and François Yvon. 2017.  Evaluating the morphological competence of machine translation systems.  In _Proceedings of the Second Conference on Machine Translation_ , pages 43–55, Copenhagen, Denmark. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Elazar et al. (2021) Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, and Yoav Goldberg. 2021.  Measuring and improving consistency in pretrained language models.  _Transactions of the Association for Computational Linguistics_ , 9:1012–1031.
  * Gao et al. (2021) Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2021.  The Pile: An 800gb dataset of diverse text for language modeling.
  * Glockner et al. (2018) Max Glockner, Vered Shwartz, and Yoav Goldberg. 2018.  Breaking NLI systems with sentences that require simple lexical inferences.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 650–655, Melbourne, Australia. Association for Computational Linguistics.
  * Gupta et al. (2022) Prakhar Gupta, Chien-Sheng Wu, Wenhao Liu, and Caiming Xiong. 2022.  DialFact: A benchmark for fact-checking in dialogue.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3785–3801, Dublin, Ireland. Association for Computational Linguistics.
  * He et al. (2021) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021.  DeBERTa: Decoding-enhanced BERT with disentangled attention.  In _International Conference on Learning Representations_.
  * Huang et al. (2020) Minlie Huang, Xiaoyan Zhu, and Jianfeng Gao. 2020.  Challenges in building intelligent open-domain dialog systems.  _ACM Trans. Inf. Syst._ , 38(3).
  * Izacard et al. (2022) Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022.  Atlas: Few-shot learning with retrieval augmented language models.
  * Jiang et al. (2020) Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020.  How can we know what language models know? _Transactions of the Association for Computational Linguistics_ , 8:423–438.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6769–6781, Online. Association for Computational Linguistics.
  * Kaushik et al. (2020) Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020.  Learning the difference that makes a difference with counterfactually-augmented data.  In _International Conference on Learning Representations_.
  * Khandelwal et al. (2020) Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020\.  Generalization through memorization: Nearest neighbor language models.  In _International Conference on Learning Representations_.
  * Lee et al. (2022) Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022.  Factuality enhanced language models for open-ended text generation.  In _Advances in Neural Information Processing Systems_.
  * Lin et al. (2022) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022.  TruthfulQA: Measuring how models mimic human falsehoods.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics.
  * Liu et al. (2022) Tianyu Liu, Yizhe Zhang, Chris Brockett, Yi Mao, Zhifang Sui, Weizhu Chen, and Bill Dolan. 2022.  A token-level reference-free hallucination detection benchmark for free-form text generation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 6723–6737, Dublin, Ireland. Association for Computational Linguistics.
  * Mallen et al. (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 9802–9822, Toronto, Canada. Association for Computational Linguistics.
  * Maynez et al. (2020) Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020.  On faithfulness and factuality in abstractive summarization.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 1906–1919, Online. Association for Computational Linguistics.
  * Merity et al. (2016) Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016.  Pointer sentinel mixture models.
  * Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023.  FActscore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation.  _arXiv preprint arXiv:2305.14251_.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022.  Training language models to follow instructions with human feedback.  In _Advances in Neural Information Processing Systems_ , volume 35, pages 27730–27744. Curran Associates, Inc.
  * Pagnoni et al. (2021) Artidoro Pagnoni, Vidhisha Balachandran, and Yulia Tsvetkov. 2021.  Understanding factuality in abstractive summarization with FRANK: A benchmark for factuality metrics.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4812–4829, Online. Association for Computational Linguistics.
  * Penedo et al. (2023) Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023.  The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only.
  * Perez et al. (2023) Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. 2023.  Discovering language model behaviors with model-written evaluations.  In _Findings of ACL_.
  * Petroni et al. (2021) Fabio Petroni, Aleksandra Piktus, Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021\.  KILT: a benchmark for knowledge intensive language tasks.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2523–2544, Online. Association for Computational Linguistics.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019.  Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2463–2473, Hong Kong, China. Association for Computational Linguistics.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Ram et al. (2023) Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023.  In-context retrieval-augmented language models.
  * Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020.  How much knowledge can you pack into the parameters of a language model? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5418–5426, Online. Association for Computational Linguistics.
  * Robertson and Zaragoza (2009) Stephen Robertson and Hugo Zaragoza. 2009.  The probabilistic relevance framework: BM25 and beyond.  _Found. Trends Inf. Retr._ , 3(4):333–389.
  * Schuster et al. (2021) Tal Schuster, Adam Fisch, and Regina Barzilay. 2021.  Get your vitamin C! robust fact verification with contrastive evidence.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 624–643, Online. Association for Computational Linguistics.
  * Sennrich (2017) Rico Sennrich. 2017.  How grammatical is character-level neural machine translation? assessing MT quality with contrastive translation pairs.  In _Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers_ , pages 376–382, Valencia, Spain. Association for Computational Linguistics.
  * Shi et al. (2023) Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023.  Replug: Retrieval-augmented black-box language models.  _arXiv preprint arXiv:2301.12652_.
  * Tam et al. (2023) Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, and Colin Raffel. 2023.  Evaluating the factual consistency of large language models through summarization.  In _Findings of ACL_.
  * Wang and Komatsuzaki (2021) Ben Wang and Aran Komatsuzaki. 2021.  GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  In _Advances in Neural Information Processing Systems_.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022.  OPT: Open pre-trained transformer language models.

##  Appendix A Technical Details of FACTOR Data Pipeline

###  A.1 Identifying Sentences’ Relevant Error Types

For each sentence, we identify the types of edits we can apply to it. First,
we use a part-of-speech tagger to detect relevance for entity error (detecting
nouns), predicate error (detecting verbs) and coreference error (detecting
pronouns). For circumstances errors, we use Named-Entity Recognition taggers
to identify sentences containing locations, dates, and time entities. Finally,
we search for temporal/causal link words from a predefined set of words, which
implies relevance for link errors.

###  A.2 Setting Filters Thresholds

As discussed in §3.2.3, we applied two filters to ensure the quality of the
potential completions–an NLI filter (to filter out non-contradictory
completions) and an LM filter (to filter out non-fluent completions). To
choose the thresholds $\tau_{\textrm{NLI}}$ and $\tau_{\textrm{LM}}$, we
manually annotated 40 samples w.r.t to the properties specified in §3.1 (i.e.,
(1) contradictory and (2) fluent and self-consistent). We have tested
thresholds 0.1-0.9, and chose the threshold which achieved highest precision
without filtering out too many samples (max 35% of the samples). For the NLI
filter we used DeBERTa-largs model fine-tuned on the MNLI dataset. Best
threshold was $\tau_{\textrm{NLI}}=0.6$, with precision of 0.96. Manually
evaluating the different contradiction types we have noticed this threshold
was too harsh for corefrence contradiction (87.5% of the completions were
filtered out. Therefore we reduced its threshold to 0.3 which filtered out 75%
of the samples). For the LM filter we used GPT2-Small. Best threshold was
$\tau_{\textrm{LM}}=0.2$, with precision of 0.78.

##  Appendix B Factuality in Open-ended Generation: Extended Results

Table 5 shows the extended results for the manual factuality annotation for
open-ended generation experiment §6. In addition to the overall results, we
include the distribution of Neutral/True/False annotations. Notably, most
generations are neutral for both models. This highlights the limitation of
sampled-based approach for assessing model’s factual knowledge.

##  Appendix C Comparison between Perplexity and FACTOR Accuracy over
Wikipedia

Figure 5: Accuracy per token perplexity over Wiki-FACTOR.

Figure 5 presents Wiki-FACTOR scores versus LM perplexity on Wikipedia. The
figure extends Figure 2, presenting all evaluated LMs: models from the GPT-Neo
family (blue circle), OPT family (red triangle) and GPT2 family (yellow
square).

Model | Variant | Neutral | True (T) | False (F) | Fact. Accuracy $\left(=\frac{\text{T}}{\text{T+F}}\right)$
---|---|---|---|---|---
GPT-J 6B | Right | 62.4% | 11.3% | 26.3% | 30.0%
Wrong | 48.8% |  5.4% | 45.8% | 10.5%
All (Weighted) | 59.5% | 10.0% | 30.5% | 24.8%
OPT-66B | Right | 54.1% | 21.4% | 24.5% | 46.6%
Wrong | 55.1% |  2.1% | 42.8% |  4.6%
All (Weighted) | 54.3% | 17.7% | 28.4% | 38.8%
Table 5: Manual factuality annotation results for OPT-66B and GPT-J 6B. For
each model, we present the results per right and wrong subsets. Bottom row
shows the weighted average between the right and wrong variants w.r.t to the
right/wrong pairs of Wiki-FACTOR.

##  Appendix D Prompts for Contradictions Generation

We prompted the model to generate multiple candidate completions, For each of
the five error types: entity (Table 6), circumstance (Table 7), coreference
(Table 8), predicate (Table 9 and 10) and link (Table 11). The prompts are
concatenated to a given a completion and its near context, with the exception
of link-prompt where only the completion is given (we found that the instruct
model tends to repeat the context when it’s appended to this particular
prompt). The prompts instruct the model to first plan its local edits, and
then generate the contradiction.

Type | Prompt
---|---
Entity | Given a context and a completion, write diverse alternative completions that contradict
| the original completion meaning.
| First, identify if the completion contains an entity. Then, write the
contradiction by
| modifying an entity or it’s property, add additional modifications if
necessary.
| Make sure the changes you make are minimal (so only change necessary details
to make
| the sentence plausible). Do not modify dates or quantities.
| ##
| Context: "Sorry" is a song by American singer Madonna from her tenth studio
album
| Confessions on a Dance Floor (2005). It was written and produced by Madonna
and
| Stuart Price, and released as the second single from the album on February
7, 2006.
| It later appeared on Celebration, her 2009 greatest hits album. An uptempo
dance song,
| " Sorry " was one of the first tracks developed for the album and had
numerous remix
| treatments before the ultimate version of the track was finalized.
| Completion: One of the remixes was done by the known band the Pet Shop Boys,
| featuring added lyrics by the band.
| 1\. Change: "Pet Shop Boys" to "Maddona".
| Contradiction: One of the remixes was done by the known singer Maddona,
| featuring added lyrics by the singer. 2\. Change: "Pet Shop Boys" to
"Depeche Mode".
| Contradiction: One of the remixes was done by the known band Depeche Mode,
| featuring added lyrics by the band.
| 3\. Change: "known" to "unfamiliar".
| Contradiction: One of the remixes was done by the unfamiliar band Pet Shop
Boys,
| featuring added lyrics by the band.
| 4\. Change: "Pet Shop Boys" to "the Killers".
| Contradiction: One of the remixes was done by the known band the Killers,
| featuring added lyrics by the band.
| ##
| Context: {context}
| Completion: {completion}
Table 6: Prompt for entity-errors generation Type | Prompt
---|---
Circumstance | Given a context and a completion, write diverse alternative completions that contradict the
| original completion meaning.
| First, identify if the completion describes the circumstances of an event
(location or time). If
| circumstances are mentioned, modify it to contradict the completion. Do not
add time or location if
| they didn’t appear in the original completion. Make sure the changes you
make are minimal.
| ##
| Context: The kingdom had been in long gradual decline since the early 13th
century. Had Pagan
| possessed a stronger central government, the collapse could have been
temporary, and the country
| "could have risen again". But the dynasty could not recover, and because the
Mongols refused to fill
| the power vacuum, no viable center emerged in the immediate aftermath. As a
result, several minor
| states fought it out for supremacy for the better part of the 14th century.
| Completion: It was only in the late 14th century that two relatively strong
powers emerged in the
| Irrawaddy basin, restoring some semblance of normalcy.
| 1\. Change: "14th" to "15th".
| Contradiction: It was only in the late 15th century that two relatively
strong powers emerged in the
| Irrawaddy basin, restoring some semblance of normalcy. 2\. Change:
"Irrawaddy" to "Chindwin".
| Contradiction: It was only in the late 14th century that two relatively
strong powers emerged in the
| Chindwin basin, restoring some semblance of normalcy.
| 3\. Change: "late" to "mid".
| Contradiction: It was only in the mid 14th century that two relatively
strong powers emerged in the
| Irrawaddy basin, restoring some semblance of normalcy.
| ##
| Context: {context}
| Completion: {completion}
Table 7: Prompt for circumstance-errors generation Type | Prompt
---|---
Coreference | Given a context and a completion, write diverse alternative completions that contradict
| the original completion meaning. First, decide if the completion contains a
pronoun
| (such as: he, she, it, they, his, her, its, theirs…) and write the entity it
refers to.
| Write the contradiction by modifying the pronoun to contradict the original
coreference.
| ##
| Context: His stance in favor of prohibition cost him the votes of four
legislators in his
| own party and the seat went to Republican William O. Bradley. Six years
later
| Beckham secured the seat by popular election, but he lost his re-election
bid largely
| because of his pro-temperance views and his opposition to women’s suffrage.
| Completion: Though he continued to play an active role in state politics for
| another two decades, he never returned to elected office, failing in his
gubernatorial
| bid in 1927 and his senatorial campaign in 1936.
| 1\. Pronoun: he
| Change: "he" to "Bradley".
| Contradiction: Though Bradley continued to play an active role in state
politics for
| another two decades, he never returned to elected office, failing in his
gubernatorial
| bid in 1927 and his senatorial campaign in 1936.
| 2\. Pronoun: he
| Change: "he" to "Bradley".
| Contradiction: Though he continued to play an active role in state politics
for
| another two decades, Bradley never returned to elected office, failing in
his
| gubernatorial bid in 1927 and his senatorial campaign in 1936.
| 3\. Pronoun: his
| Change: "his" to "Bradley’s".
| Contradiction: Though he continued to play an active role in state politics
for
| another two decades, he never returned to elected office, failing in
Bradley’s
| gubernatorial bid in 1927 and his senatorial campaign in 1936.
| ##
| Context: The early 6th century saw another queen ruling the city, known only
as the
| "Lady of Tikal", who was very likely a daughter of Chak Tok Ich ’aak II.
| Completion: She seems never to have ruled in her own right, rather being
partnered
| with other rulers.
| 1\. Pronoun: She
| Change: "She" to "He" and "her" to "his".
| Contradiction: He seems never to have ruled in his own right, rather being
partnered
| with other rulers.
| 2\. Pronoun: She
| Change: "She" to "The king" and "her" to "his".
| Contradiction: The king seems never to have ruled in his own right, rather
| being partnered with other rulers.
| 3\. Pronoun: She
| Change: "She" to "Chak Tok Ich".
| Contradiction: Chak Tok Ich seems never to have ruled in her own right,
rather
| being partnered with other rulers.
| ##
| Context: {context}
| Completion: {completion}
Table 8: Prompt for coreference-errors generation Type | Prompt
---|---
Predicate | Given a context and a completion, write diverse alternative completions, that contradict the original
| completion meaning by modifying verbs.
| First, Identify a verb in the original completion, and then write the
contradiction by modifying it. Make sure
| the contradictions are grammatically correct, fluent and consistent. Make
any necessary additional
| modifications to ensure that.
| ##
| Context: Homarus gammarus is a large crustacean, with a body length up to 60
centimetres (24 in) and
| weighing up to 5 – 6 kilograms (11 – 13 lb), although the lobsters caught in
lobster pots are usually
| 23 – 38 cm (9 – 15 in) long and weigh 0.7 – 2.2 kg (1.5 – 4.9 lb).
| Completion: Like other crustaceans, lobsters have a hard exoskeleton which
they must shed in order to grow,
| in a process called ecdysis (moulting).
| 1\. Change: "shed" to "retain". Additional changes: "in order to grow" to
"in order to survive".
| Contradiction: Like other crustaceans, lobsters have a hard exoskeleton
which they must retain in order to
| survive, in a process called ecdysis (moulting).
| 2\. Change: "grow" to "maintain their size".
| Contradiction: Like other crustaceans, lobsters have a hard exoskeleton
which they must shed in order to
| maintain their size, in a process called ecdysis (moulting).
| 3\. Change: "shed" to "keep". Additional changes: "in order to grow" to "in
order to strengthen".
| Contradiction: Like other crustaceans, lobsters have a hard exoskeleton
which they must keep in order to
| strengthen, in a process called ecdysis (moulting).
| ##
| Context: The ridge offered a natural avenue of approach to the airfield,
commanded the surrounding area
| and was almost undefended. Edson and Thomas tried to persuade Vandegrift to
move forces to defend
| the ridge, but Vandegrift refused, believing that the Japanese were more
likely to attack along the coast.
| Completion: Finally, Thomas convinced Vandegrift that the ridge was a good
location for Edson’s Raiders
| to rest from their actions of the preceding month.
| 1\. Change: "rest" to "keep up".
| Contradiction: Finally, Thomas convinced Vandegrift that the ridge was a
good location for Edson’s
| Raiders to keep up with their actions of the preceding month.
| 2\. Change: "convinced Vandegrift" to "made Vandegrift doubt".
| Contradiction: Finally, Thomas made Vandegrift doubt that the ridge was a
good location for Edson’s
| Raiders to rest from their actions of the preceding month. 3\. Change:
"rest" to "continue".
| Contradiction: Finally, Thomas convinced Vandegrift that the ridge was a
good location for Edson’s
| Raiders to continue their actions of the preceding month.
| ##
| Context: According to a report titled Wolves in Sheep’s Clothing, which
documents the increase in
| potentially violent, profane, and sexual content in children’s programming,
the Parents Television Council,
| a watchdog media group, and fans believed the SpongeBob SquarePants episode"
Sailor Mouth "was
| an implicit attempt to promote and satirize use of profanity among children.
| Completion: The episode originally aired during the 2001 – 02 television
season, ironically the season
| in which the PTC named SpongeBob SquarePants among the best programs on
cable television,
| but the report cited a repeat broadcast of the episode from 2005 to prove
its point that it promoted use of
| profanity among children.
| 1\. Change: "prove" to "refute". Additional changes: "best" to "most
profane".
| Contradiction: The episode originally aired during the 2001 – 02 television
season, ironically the season
| in which the PTC named SpongeBob SquarePants among the most profane programs
on cable television,
| but the report cited a repeat broadcast of the episode from 2005 to refute
its point that it promoted use of
| profanity among children.
| 2\. Change: "originally aired" to "pulled off".
| Contradiction: The episode was pulled off from the 2001 – 02 television
season, ironically the season
| in which the PTC named SpongeBob SquarePants among the best programs on
cable television,
| but the report cited a repeat broadcast of the episode from 2005 to prove
its point that it promoted use of
| profanity among children.
| ##
| Context: {context}
| Completion: {completion}
Table 9: Prompt for predicate-errors generation (the rest of the prompt is in table 10) Type | Prompt
---|---
Predicate | Context: By Part II of the series, Shikamaru is capable of utilizing multiple shadow-based techniques at
| once and can lift his shadow from the ground in order to interact with
physical objects; for instance, he can
| pierce enemies with the shadow tendrils or use them to throw weapons.
Shikamaru approaches the exams
| with a sense of apathy; when he battles the Sunagakure ninja Temari, he
defeats her
| but forfeits his match to her, due to his chakra being low.
| Completion: Despite this loss, he is the only ninja among his peers to be
promoted to the rank of Chunin,
| as the overseers of the exams were impressed by the insight and intelligence
he demonstrated against Temari.
| 1\. Change: "promoted" to "demoted". Additional changes: "Despite" to "Due",
"as" to "although".
| Contradiction: Due to this loss, he is the only ninja among his peers to be
demoted to the rank of Chunin,
| although the overseers of the exams were impressed by the insight and
intelligence he demonstrated against
| Temari.
| 2\. Change: "were impressed" to "underappreciated". Additional changes: "as"
to "although".
| Contradiction: Despite this loss, he is the only ninja among his peers to be
promoted to the rank of Chunin,
| although the overseers of the exams underappreciated the insight and
intelligence he demonstrated against
| Temari.
| 3\. Change: "demonstrated" to "failed to demonstrate". Additional changes:
"as" to "although",
| "impressed" to "disappointed".
| Contradiction: Despite this loss, he is the only ninja among his peers to be
promoted to the rank of Chunin,
| although the overseers of the exams were disappointed by the insight and
intelligence he failed to
| demonstrate against Temari.
| ##
| Context: {context}
| Completion: {completion}
Table 10: Prompt for predicate-errors generation (continue of the prompt in table 9) Type | Prompt
---|---
Link | Given a sentence, write contradictory sentences by modifying a temporal link.
| First, identify a link between events, and then modify it. Make sure the
contradictions are grammatically
| correct and fluent. If no such link exists, answer "NA".
| ##
| Sentence: Prior to filming, a week was spent reinforcing the roof of the
liquor store to ensure it would not
| collapse if it were to be intruded by a group of fans.
| 1\. Change: "prior to" to "after".
| Contradiction: After filming, a week was spent reinforcing the roof of the
liquor store to ensure it would not
| collapse if it were to be intruded by a group of fans.
| ##
| Sentence: Lewis McAllister, a businessman in Tuscaloosa, Alabama, was the
first Republican to serve in the
| Mississippi House of Representatives since Reconstruction, 1962-1968; he
resided in Meridian prior to 1971.
| 1\. Change: "prior to" to "after".
| Contradiction: Lewis McAllister, a businessman in Tuscaloosa, Alabama, was
the first Republican to serve
| in the Mississippi House of Representatives since Reconstruction, 1962-1968;
he resided in Meridian
| after 1971.
| 2\. Change: "since" to "before"
| Contradiction: Lewis McAllister, a businessman in Tuscaloosa, Alabama, was
the first Republican to serve
| in the Mississippi House of Representatives before Reconstruction,
1962-1968; he resided in Meridian prior
| to 1971.
| ##
| Sentence: The decline of the railroad industry caused significant job
losses, resulting in a population decline
| as workers left for other areas.
| 1\. Change: "caused" to "caused by".
| Contradiction: The decline of the railroad industry, caused by significant
job losses, resulting a
| population decline as workers left for other areas.
| 2\. Change: "resulting" to "was the result of".
| Contradiction: The decline of the railroad industry caused significant job
losses, was the result of a population
| decline, as workers left for other areas.
| ##
| Sentence: {completion}
Table 11: Prompt for link-errors generation


# Diffusion models as probabilistic neural operators for recovering unobserved
states of dynamical systems

###### Abstract

This paper explores the efficacy of diffusion-based generative models as
neural operators for partial differential equations (PDEs). Neural operators
are neural networks that learn a mapping from the parameter space to the
solution space of PDEs from data, and they can also solve the inverse problem
of estimating the parameter from the solution. Diffusion models excel in many
domains, but their potential as neural operators has not been thoroughly
explored. In this work, we show that diffusion-based generative models exhibit
many properties favourable for neural operators, and they can effectively
generate the solution of a PDE conditionally on the parameter or recover the
unobserved parts of the system. We propose to train a single model adaptable
to multiple tasks, by alternating between the tasks during training. In our
experiments with multiple realistic dynamical systems, diffusion models
outperform other neural operators. Furthermore, we demonstrate how the
probabilistic diffusion model can elegantly deal with systems which are only
partially identifiable, by producing samples corresponding to the different
possible solutions.

Index Terms—  Diffusion Models, Neural Operator, Physical Systems Modelling,
Partial Differential Equations

##  1 Introduction

Physical dynamical systems can be expressed with partial differential
equations (PDEs) and traditionally solved using methods like the finite
difference method. This conventional approach cannot easily incorporate
measurements of the process, which motivates the development of neural network
methods that can combine the prior knowledge with data-driven learning.
Differential equations can be combined with neural networks by integrating the
PDEs directly into the architecture [1, 2] or by including the PDE error in
the loss to represent prior knowledge [3]. The first option complies with the
underlying physics by design, but is system-specific and non-trivial to
extend. In the second approach the architecture does not depend on the
equations but the training requires the functional form of the equations and
knowledge of the parameters, which may be unavailable. Alternatively, it is
possible to incorporate domain knowledge through a physics simulator, which
can be integrated into the network architecture [4] or utilized to augment the
training data [5]. The simulator allows to collect a rich data set from the
underlying process by randomizing its parameters, which alleviates the need
for system identification at inference time, enables a wider class of models,
and increases robustness against noise and distortions in the data. If the PDE
is known at test time, it can be used to fine-tune the model’s prediction [6].

Neural operators are data-driven and learn a mapping from the input parameters
or observations to the output. Most of them are deterministic, which suffices
for approximating the mapping from the PDE parameters to the solution.
However, for the inverse problems of recovering parameters or reconstructing
an unobserved state, deterministic predictions may not capture all possible
outcomes. To resolve this, we opt for probabilistic generative models, the
diffusion models [7, 8], which have excelled in many domains. We view the
system state evolution, discretized across spacial and temporal dimensions, as
an image with state variables stacked as channels. Consequently, the
reconstruction of an unobserved state is analogous to a color restoration
task. The diffusion models can be pre-trained on complete image data, and
conditioning applied only at inference time [9]. This ‘unconditional’
approach, called RePaint in [9], is adaptable to many use cases, but with
increased inference complexity. Alternatively, the model’s input can be
augmented with the conditioning information already during training [10, 11],
yielding a specialized model suitable for a single task. A similar approach
has been used for multivariate time series prediction and imputation [12].

Our novel contributions are: 1) We define and compare multiple diffusion
models for unobserved state reconstruction and prediction. Unlike existing
works in this domain [13, 14], we train our model with a ‘mixed conditional’
objective, allowing it to solve multiple tasks with a single training (see
Fig. 1). 2) We empirically demonstrate the strong performance of diffusion
models in general, and the model with mixed conditional training in
particular, against other neural operator baselines. 3) We study systems which
are not fully identifiable, and demonstrate that diffusion models can
represent the variability of possible solutions.

| Full PDE solution
---
|  |  | Conditional
---
| Mixed Conditional (M-CEDM)
---
| Desired output
---
|  |  |  Model 1 … Model $5$
---
| Single model
---
$t$$x$ Velocity $u$ Height $h$ |  |  Training input |  |  $\thicksim$ each batch i.i.d.
|  |  |  | Task 1 … Task 5
---
| Task 1 Task 2 Task 3 Task 4 Task 5
---
|  |  Inference input |  |
Fig. 1: Conditional and mixed conditional training of diffusion models
demonstrated with the Shallow-Water Equation (SWE-orig). The system has two
variables (channels) represented by the colored rectangles with time ($t$) and
spatial coordinate ($x$) on the x- and y-axes. The ‘desired output’ is the
full state which we train the model to reconstruct from partial information.
The clean parts of the input represent the conditioning information (at
training and inference time) and the noisy parts are reconstructed by
denoising. Each conditional model is trained with conditioning information for
a single task. The mixed conditional training yields a single model for all
defined tasks by sampling one task for each mini-batch during training.

##  2 Related work

Neural operators [15] have strong performance in modelling of physical
systems. State-of-the-art results on rectangular domains are produced by FNO
[13]. Recently proposed neural operators use the transformer architecture,
e.g., OFormer [14]. Existing methods solve either the forward (parameters to
solution) or the inverse (solution to parameters) problem. Unlike the neural
operator approaches, a diffusion model can solve both of these problems at
once by conditioning with known information. Concurrent to our work, [16]
combines forward and inverse operators but in a non-diffusion model.

Diffusion models for physical systems have been successfully applied for
predicting the dynamics of the system in the future, either by autoregressive
unrolls [17] or by taking the desired future time as a model input [18]. These
models focus on prediction only while we consider training jointly for
prediction and reconstruction. A work [19] utilized the diffusion model for
data generation for several physical systems. It focused on unconditional
generation that adheres to physical constraints of the system while we recover
the unobserved states conditioned on partial information.

Incorporation of PDE prior information into diffusion models. A physics-
informed diffusion model [20] includes a scaled PDE residual into the training
objective, encouraging solutions that comply with the PDE. Alternatively, the
gradient of the PDE residual guide the sampling [21] or be directly fed into
the model as an extra input channel during training [22]. All of the
approaches can be applied together with the methods in this paper.

##  3 Method

| States to reconstruct
---
Conditioning
---
Binary masks
---
Fig. 2: Model input for mixed conditional training.

###  3.1 Background on diffusion models

Denoising diffusion probabilistic models (DDPMs) [8] are a class of generative
models which learn underlying data distribution by progressively removing
noise from the distorted input. In particular, the training procedure requires
defining a schedule for the magnitude of noise added to model input at every
training iteration. The highest noise magnitude in the schedule makes the
model input close to pure Gaussian noise. A neural network, predicting a
denoised version of the input, is trained by optimizing a variational lower
bound or a simplified objective [8] between the added noise and the network
prediction, or some other refined combinations (see, e.g., [23, 24]).
Denoising diffusion models use the U-net architecture [25] with conditioning
on the step $t$ of the noise schedule [8]. The original architecture has been
made more expressive by improvements in residual blocks [26, 27] while
inefficiencies in training dynamics have been remedied by input and output
preconditioning [24].

One downside of diffusion models is the slow inference, which, starting from
random noise, iteratively denoises the input into the final prediction. The
original DDPM [8] sampling uses the same number of steps as the training noise
schedule (typically around 1000 steps). However, the number of inference steps
can be significantly decreased by defining a separate inference schedule [23,
28, 24]. The approaches [23, 28] are uniformly skipping some steps in the
training noise schedule. However, a better strategy for allocating the
inference resources is skipping more steps in the beginning of the sampling
process and preserving more steps closer to the end where the noise magnitude
is smaller. This idea combined with a second order Heun method has been
proposed by [24] and together with input and output preconditioning is
referred to as EDM.

###  3.2 Mixed conditional training

We aim at training a model capable of solving several tasks, such as the
unobserved state reconstruction, the system dynamics prediction from the
parameter value, the underlying parameter recovery from the dynamics, and the
prediction of future system states. The parameter values can be concatenated
with the system states as an additional input channel making parameter
recovery and dynamics prediction analogous to the state reconstruction task.
Similar problem formulation arises in image-to-image translation considered in
Palette [10] and in time series imputation and forecasting addressed in SSSDS4
[12]. We propose to model the dynamics by a conditional diffusion model
trained on a mixture of the tasks of interest as shown in Fig. 1.

To simultaneously train for several tasks, we generate a mask corresponding to
the unknown part of the system (e.g. upper or lower channel in Fig. 1) for
each training batch. For the observed channel(s), we sample the number of time
steps observed (in the direction of the x-axis in Fig. 1). For example, Task 3
in Fig. 1 corresponds to observing both channels partially for the first half
of the time span, while in Tasks 4 and 5 only one channel is partially
observed.

Since a part of the full state is known, the diffusion process is applied only
to the unobserved part. As shown in Fig. 2, the input for one denoising step
consists of system states, such that noise is added to the unobserved part,
and the observed part is kept noise free and given as conditioning
information. The input is the same both at training and inference times. The
conditioning information can be optionally accompanied by the corresponding
binary masks as in SSSDS4 which we also found beneficial. To take into account
that the denoising should be applied only to the unknown part, the observed
part is masked out from the loss.

##  4 Experiments

(a) Darcy flow |  |  (b) Reactor |
---|---|---|---
Fig. 3: Example simulations from two of the studied systems.  Table 1:  Model comparison on state reconstruction for the forward (Task 1, columns 2 – 4) and inverse (Task 2, columns 5 – 7) problems in case of identifiable systems. For each problem we present the mean absolute error (MAE) and the PDE residual error. All MAE values should be multiplied by $10^{-3}$.  Model | Darcy, $a\rightarrow u$ | SWE, $h\rightarrow u$ | Reactor, | Darcy, $u\rightarrow a$ | SWE, $u\rightarrow h$ | Reactor,
---|---|---|---|---|---|---
$x_{p},T\rightarrow\theta,x_{a}$ | $\theta,x_{a}\rightarrow x_{p},T$
MAE | PDE | MAE | PDE | MAE | PDE | MAE | PDE | MAE | PDE | MAE | PDE
FNO [13] | 7.55 | 38.08 | 0.44 | 0.9963 | 0.07 | 0.0146 | 3.38 | 3.236 | 2.21 | 1.1370 | 5.64 | 0.1132
OFormer [14] | 6.82 | 21.00 | 0.31 | 0.9966 | 0.13 | 0.0083 | 4.06 | 6.683 | 0.81 | 0.9929 | 7.76 | 0.0282
RePaint [9] | 9.80 | 6.909 | 0.69 | 1.0462 | 0.73 | 0.1521 | 191.8 | 84.11 | 9.68 | 1.0429 | 188.3 | 0.2017
CDDPM [8] | 6.23 | 1.478 | 0.25 | 0.9925 | 0.09 | 0.0039 | 0.25 | 1.180 | 0.84 | 0.9906 | 4.62 | 0.0039
CEDM [24] | 5.52 | 1.344 | 0.10 | 0.9923 | 0.05 | 0.0015 | 0.18 | 1.192 | 0.44 | 0.9924 | 3.42 | 0.0017
M-CEDM | 5.46 | 1.272 | 0.10 | 0.9915 | 0.07 | 0.0020 | 0.17 | 1.180 | 0.43 | 0.9906 | 2.69 | 0.0017

###  4.1 Dynamical systems considered

We demonstrate the performance of the proposed diffusion models on four
systems. The first three: Darcy flow, shallow water equation with periodic
initial conditions, and the fixed-bed tubular reactor are fully identifiable
given the conditioning information. The fourth system is the shallow water
equation with an initial perturbation of the water level, and it is partially
non-identifiable, meaning that multiple solutions are compatible with the
given conditioning information. Below we specify these systems and their data
generation.

Darcy Flow equation is defined over a unit square with a 2D steady-state
solution governed by

$\displaystyle\nabla\left(a(x)\nabla u(x)\right)$ $\displaystyle=f(x)\text{,
}x\in(0,1)^{2},$ (1) $\displaystyle u(x)$ $\displaystyle=0\text{,
}x\in\partial(0,1)^{2},$ (2)

with viscosity $a(x)$ being piece-wise constant and the force term $f(x)=1$.
We use a subset of 1000 simulations from PDEBench [29], of which one instance
is shown in Fig. 3(a).

Shallow water equation (SWE-orig) is derived from the compressible Navier-
Stokes equation and describes free-surface fluid flow. The system is
characterized by height $h$ and velocity $u$ of the fluid evolving as:

$\displaystyle\frac{\partial h}{\partial t}$ $\displaystyle=-\frac{\partial
hu}{\partial x}\text{, }\quad\frac{\partial hu}{\partial
t}=-\frac{\partial(u^{2}h+0.5gh^{2})}{\partial x}$ (3)

where $g$ is the gravitational constant. Here, we set $g=1$ and randomize the
initial conditions for $h$. For $t\in[0,0.128]$, $x\in[-0.5,0.5]$, and
$u_{0}=0$, $h_{0}$ is calculated as:

$\displaystyle\tilde{h}(0,x)$
$\displaystyle=\sum_{k=-N}^{N}\lambda_{k}\cos(2\pi kx)+\gamma_{k}\sin(2\pi
kx),$ (4) $\displaystyle h(0,x)$
$\displaystyle=1+\frac{\tilde{h}(0,x)-\min(\tilde{h}(0,x))}{\max(\tilde{h}(0,x))-\min(\tilde{h}(0,x))}\text{,}$
(5)

where $N=3$ and $\lambda_{k},\gamma_{k}\sim\mathcal{N}(0,1)$. An example of
this system is used in the illustrations in Fig. 1.

Shallow water equation (SWE-init) is generated for $x\in[-2.5,2.5]$ and
$t\in[0,1.28]$ with initial conditions:

$\displaystyle h(0,x)$
$\displaystyle=h_{\text{in}}+\epsilon\exp\left(\frac{(x-x_{0})^{2}}{2\sigma^{2}}\right)\text{,
}\quad u(0,x)=u_{0}\text{,}$ (6)

where $h_{\text{in}}\sim\mathcal{U}\left(1.2,5.2\right)$,
$\epsilon\sim\mathcal{U}\left(0.05,1\right)$,
$x_{0}\sim\mathcal{U}\left(-1,1\right)$,
$\sigma\sim\mathcal{U}\left(0.2,2\right)$,
$(hu)_{0}\sim\mathcal{U}\left(-2.2,2.2\right)$. This equation models an abrupt
change in the water level, e.g. caused by a dam break. Depending on the
initial water velocity $u_{0}$, several possible outcomes may correspond to
the same observation of the height $h$ (Fig. 4).

For both SWE datasets the data are simulated using PyClaw [30] Python package
implementing a finite volume method for this equation.

Fixed-bed tubular reactor is a system with hydrogenation of aromatics with
concentration $x_{a}$ on the catalyst surface with activity $\theta$. The
reactor’s input contains a poisoning agent with concentration $x_{p}$ which
deteriorates activity $\theta$ [31]. The system dynamics in space $z$ and time
$t$ is modelled as

$\displaystyle\frac{\partial x_{a,p}}{\partial t}$
$\displaystyle=-U\frac{\partial x_{a,p}}{\partial z}-\alpha(T)r_{a,p},$ (7)
$\displaystyle\frac{\partial T}{\partial t}$
$\displaystyle=-\beta(x_{a},T)U\frac{\partial T}{\partial z}+\gamma
r_{a},\quad\frac{d\theta}{dt}=-r_{d},$ (8)

where $U$ is the fluid velocity, $\alpha(T)$ and $\beta(x_{a},T)$ are
functions of temperature $T$ and the concentration $x_{a}$, $\gamma$ is a
constant and the functions $r_{a}$, $r_{p}$ and $r_{d}$ are reaction,
poisoning adsorption, and catalyst deactivation rates (see [2] for more
details). A simulation of the system is shown in Fig. 3(b).

###  4.2 Baselines

We compared the proposed approach against the (non-diffusion) supervised
neural operators FNO [13] and OFormer [14], which have been trained using the
implementations provided by the authors. As diffusion model baselines, we
compared the mixed conditional training (M-CEDM) with the ‘unconditional’
RePaint approach and the conditional diffusion models based on the DDPM
(CDDPM) and EDM (CEDM). RePaint and CDDPM are trained on the simplified
objective [8] by predicting added noise, while CEDM and M-CEDM use the EDM
objective [24] and predict a mixture of noise and denoised image. All
diffusion models rely on the 2nd order Heun sampler [24]. In all experiments
we take 100 samples from each diffusion model and calculate the mean absolute
error (MAE) and the PDE residual error by averaging across those samples. The
number of parameters in the different models is approximately equal. The code
and the datasets will be released upon acceptance.

###  4.3 Reconstruction of the unobserved variables

In this experiment (results in Table 1), we consider two tasks: forward and
inverse. In the forward task we predict the solution $u$ conditioned on the
parameter $a$ (Darcy), the velocity $u$ from the height $h$ (SWE-orig) and
both the concentration $x_{a}$ and the catalyst $\theta$ from the temperature
$T$ and poisoning concentration $x_{p}$ (Reactor). For the inverse task the
inputs and the targets are flipped (Tasks 1, 2 in Fig. 1). For these tasks the
supervised neural operators, FNO and OFormer, provide strong results. However,
the PDE residuals are consistently smaller for the conditional and mixed
conditional diffusion models (CDDPM, CEDM and M-CEDM), which means that their
solutions are more compatible with the prior knowledge of the system dynamics.
Since CEDM results were better than CDDPM, we selected it as the basis for
training the mixed conditional approach. Comparing RePaint trained
‘unconditionally’ to generate both states jointly against the specialized
conditional models, we conclude that the conditioning during training is
beneficial. However RePaint uses a singe model for both forward and inverse
problems, while the conditional models (including FNO and OFormer) use a
dedicated model for each task. Similar to RePaint, M-CEDM uses a single model
for both tasks but with performance similar to or better than the other
conditionally trained models.

###  4.4 Future state prediction

Next, we compared the methods on Task 3 (future prediction) and Tasks 4-5
(reconstruction and prediction) in Fig. 1, results shown in Table 2. For
RePaint and M-CEDM, we used the same trained model as in Table 1. We selected
M-CEDM as a representative of the conditional diffusion models as it was the
best in the previous experiment and did not require retraining. For FNO and
OFormer we had two models, one to reconstruct the state, the other for
prediction. Both FNO and OFormer can handle inputs with different amounts of
observed time steps at test time; however, the performance drops significantly
when the models trained on the full state are applied to partial observations
(marked by * in Table 2). By retraining FNO and OFormer for the correct input
size, the performance improved but stayed below the diffusion model.

Table 2:  Results for SWE-orig dataset in reconstruction and prediction (Tasks 3-5). MAE values should be multiplied by $10^{-3}$. Models with * are the same as in Table 1 and trained for full state reconstruction.  Model | Task 3 | Task 4 | Task 5
---|---|---|---
MAE | PDE |  MAE | PDE |  MAE | PDE
FNO* [13] |  0.70 | 0.98 |  51.2 | 47.5 |  90.7 | 150
OFormer* [14] |  0.40 | 1.00 |  30.3 | 105 |  92.8 | 47.5
FNO [13] |  0.70 | 0.98 |  0.56 | 0.98 |  1.34 | 1.04
OFormer [14] |  0.40 | 1.00 |  0.35 | 1.00 |  1.04 | 1.01
RePaint [9] |  0.65 | 1.08 |  0.79 | 1.04 |  54.2 | 4.72
M-CEDM |  0.08 | 0.99 |  0.14 | 0.99 |  0.48 | 0.99

###  4.5 Recovering several possible outcomes

Table 3:  Results of state reconstruction (Task 1) in the non-identifiable case (SWE-init). All MAE values should be multiplied by $10^{-3}$. We see that samples from the diffusion model include the correct prediction (‘closest’), and additional prior information (extra observed points or PDE error) can be useful to identify the most plausible samples.  Model | MAE | PDE
---|---|---
Kalman Filter [32] | 52.2 | 0.2858
FNO [13] | 2.43 | 0.1300
OFormer [14] | 2.74 | 0.0400
Diffusion model, mean prediction | 55.1 | 0.0193
\- prediction selected by PDE error | 13.7 | 0.0159
\- prediction selected by 2 corner points | 1.92 | 0.0161
\- prediction closest to the target | 1.53 | 0.0162

Here we compare the models for velocity reconstruction (Task 1) in the SWE-
init system, and the results are summarized in Table 3. The system is
partially non-identifiability, which means that without additional information
multiple reconstructions can be compatible with the conditioning information.
The purpose of this experiment is to demonstrate the benefits of the
probabilistic diffusion models in this scenario, namely their ability to
generate multiple different samples reflecting the variability of possible
solutions, and including the correct target value. Here, we include also the
Kalman Filter [32], obtained by linearizing the PDEs [33], as another
probabilistic baseline.

As shown in Table 3, the diffusion model mean prediction, obtained by
averaging over the samples, has a higher MAE than the other methods, as
expected. To illustrate this further, Fig. 4 presents results for one test
example with a large mean MAE. We see that the diffusion model produced
samples which all have the same overall shape, but which differ in the initial
velocity, which reflects true non-determinacy of the system. Further, the
correct result is among the samples (‘closest’), and just by including two
additional points in the conditioning set it can already be identified. The
deterministic FNO and OFormer models have a relatively small MAE but they also
introduce artifacts, making the prediction not compatible with the physical
dynamics, as seen also in the large PDE-errors. The Kalman filter yields a
plausible prediction but is less precise due to the linearization.

Finally, Fig. 5 highlights the potential of the PDE-error as a prior to
identify the most plausible samples. For all test examples the PDE-error is
clearly correlated with the MAE (left and middle panels in Fig. 5). However,
even among the samples with the smallest PDE-errors, there can still be
variability, as demonstrated on the middle row in Fig. 4. The rightmost panel
in Fig. 5 confirms the earlier finding that the diffusion model produces
samples with a smaller PDE-error, meaning that they better comply with the
underlying physics.

Fig. 4: State reconstruction results for non-identifiable system SWE-init for
a test sample with multiple possible outcomes.  Fig. 5: Detailed results for
the non-identifiable system SWE-init. Left: PDE error vs. MAE for 100 samples
for a single test case. Middle: Histogram of correlations between MAE and PDE
error across the test set (red line shows the case on the left). Right: PDE
errors of different models in the test set.

##  5 Discussion and Future work

We proposed a diffusion model that can be used as a neural operator to solve
several tasks without retraining. Additionally, when the problem is not fully
defined, the probabilistic model, unlike the deterministic alternatives, can
produce several possible results compatible with the conditioning information,
and we studied using prior knowledge, either additional measured points or the
PDE error, as a way of selecting the most plausible samples. As a limitation,
the inference with the diffusion model requires sampling, which, despite
recent improvements, is still slower than with the deterministic baselines. As
future work, the model architecture could be tailored for the specifics of the
dynamical systems.

## References

  * [1] S. Greydanus et al.,  “Hamiltonian neural networks,”  in NeurIPS., 2019, vol. 32.
  * [2] K. Haitsiukevich et al.,  “A grid-structured model of tubular reactors,”  in 2021 IEEE 19th Int. Conf. on Ind. Informatics (INDIN), 2021, pp. 1–6.
  * [3] M. Raissi et al.,  “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,”  J. Comput. Physics, vol. 378, pp. 686–707, 2019.
  * [4] T. Yang et al.,  “Learning physics constrained dynamics using autoencoders,”  NeurIPS, vol. 35, pp. 17157–17172, 2022.
  * [5] R. Lam et al.,  “Learning skillful medium-range global weather forecasting,”  Science, p. eadi2336, 2023.
  * [6] Z. Li et al.,  “Physics-informed neural operator for learning partial differential equations,”  ACM/JMS J. Data Science, 2021.
  * [7] J. Sohl-Dickstein et al.,  “Deep unsupervised learning using nonequilibrium thermodynamics,”  in ICML, 2015, pp. 2256–2265.
  * [8] J. Ho et al.,  “Denoising diffusion probabilistic models,”  NeurIPS, vol. 33, pp. 6840–6851, 2020.
  * [9] A. Lugmayr et al.,  “Repaint: Inpainting using denoising diffusion probabilistic models,”  in Proc. IEEE/CVF Conf. on Comp. Vision and Pattern Recognition, 2022, pp. 11461–11471.
  * [10] C. Saharia et al.,  “Palette: Image-to-image diffusion models,”  in ACM SIGGRAPH 2022 Conf. Proc., 2022, pp. 1–10.
  * [11] S. Yang et al.,  “Uni-paint: A unified framework for multimodal image inpainting with pretrained diffusion model,”  in Proc. 31st ACM Int. Conf. on Multimedia, 2023, pp. 3190–3199.
  * [12] J. Lopez Alcaraz et al.,  “Diffusion-based time series imputation and forecasting with structured state space models,”  Trans. Mach. Learn. Research, 2023.
  * [13] Z. Li et al.,  “Fourier neural operator for parametric partial differential equations,”  in ICLR, 2021.
  * [14] Z. Li et al.,  “Transformer for partial differential equations’ operator learning,”  Trans. Mach. Learn. Research, 2023.
  * [15] N. Kovachki et al.,  “Neural operator: Learning maps between function spaces with applications to pdes,”  J. Mach. Learn. Research, vol. 24, no. 89, pp. 1–97, 2023.
  * [16] D. Long et al.,  “Invertible fourier neural operators for tackling both forward and inverse problems,”  arXiv preprint arXiv:2402.11722, 2024.
  * [17] G. Kohl et al.,  “Turbulent flow simulation using autoregressive conditional diffusion models,”  arXiv preprint arXiv:2309.01745, 2023.
  * [18] G. Yang et al.,  “A denoising diffusion model for fluid field prediction,”  arXiv preprint arXiv:2301.11661, 2023.
  * [19] R. Apte et al.,  “Diffusion model based data generation for partial differential equations,”  arXiv preprint arXiv:2306.11075, 2023.
  * [20] J. Bastek et al.,  “Physics-informed diffusion models,”  arXiv preprint arXiv:2403.14404, 2024.
  * [21] C. Jacobsen et al.,  “Cocogen: Physically-consistent and conditioned score-based generative models for forward and inverse problems,”  arXiv preprint arXiv:2312.10527, 2023.
  * [22] D. Shu et al.,  “A physics-informed diffusion model for high-fidelity flow field reconstruction,”  J. Comput. Physics, vol. 478, pp. 111972, 2023.
  * [23] A.Q. Nichol et al.,  “Improved denoising diffusion probabilistic models,”  in ICML, 2021, pp. 8162–8171.
  * [24] T. Karras et al.,  “Elucidating the design space of diffusion-based generative models,”  NeurIPS, vol. 35, pp. 26565–26577, 2022.
  * [25] O. Ronneberger et al.,  “U-net: Convolutional networks for biomedical image segmentation,”  in Med. Image Comput. and Computer-Assisted Intervention. Springer, 2015, pp. 234–241.
  * [26] Y. Song et al.,  “Score-based generative modeling through stochastic differential equations,”  ICLR, 2021.
  * [27] P. Dhariwal et al.,  “Diffusion models beat gans on image synthesis,”  NeurIPS, vol. 34, pp. 8780–8794, 2021.
  * [28] J. Song et al.,  “Denoising diffusion implicit models,”  ICLR, 2021.
  * [29] M. Takamoto et al.,  “Pdebench: An extensive benchmark for scientific machine learning,”  NeurIPS, vol. 35, pp. 1596–1611, 2022.
  * [30] D.I. Ketcheson et al.,  “Pyclaw: Accessible, extensible, scalable tools for wave propagation problems,”  SIAM J. Scientific Comput., vol. 34, no. 4, pp. C210–C231, 2012.
  * [31] T.H. Price et al.,  “Catalyst poisoning and fixed bed reactor dynamics—II: Adiabatic reactors,”  Chem. Eng. Sci., vol. 32, no. 4, pp. 393–412, 1977.
  * [32] R.E. Kalman,  “A new approach to linear filtering and prediction problems,”  J. Basic Eng., vol. 82, no. 1, pp. 35–45, 1960.
  * [33] M. Rafiee et al.,  “Kalman filter based estimation of flow states in open channels using lagrangian sensing,”  in Proc. 48h IEEE Conf. on Decision and Control (CDC), 2009, pp. 8266–8271.