# Counterfactual Fairness Is Basically Demographic Parity

Lucas Rosenblatt,†1 R. Teal Witter†1

###### Abstract

Making fair decisions is crucial to ethically implementing machine learning
algorithms in social settings. In this work, we consider the celebrated
definition of counterfactual fairness. We begin by showing that an algorithm
which satisfies counterfactual fairness also satisfies demographic parity, a
far simpler fairness constraint. Similarly, we show that all algorithms
satisfying demographic parity can be trivially modified to satisfy
counterfactual fairness. Together, our results indicate that counterfactual
fairness is basically equivalent to demographic parity, which has important
implications for the growing body of work on counterfactual fairness. We then
validate our theoretical findings empirically, analyzing three existing
algorithms for counterfactual fairness against three simple benchmarks. We
find that two simple benchmark algorithms outperform all three existing
algorithms—in terms of fairness, accuracy, and efficiency—on several data
sets. Our analysis leads us to formalize a concrete fairness goal: to preserve
the order of individuals within protected groups. We believe transparency
around the ordering of individuals within protected groups makes fair
algorithms more trustworthy. By design, the two simple benchmark algorithms
satisfy this goal while the existing algorithms for counterfactual fairness do
not.

$\dagger$$\dagger$footnotetext: Authors contributed equally.

## Introduction

A pressing challenge in the deployment of algorithms in social settings is the
risk of “unfair” decisions with respect to protected attributes like race,
gender, religion, sexual-orientation, or age. In this work, we consider the
supervised learning problem of predicting outcomes from labelled observations
where the existing outcomes are already biased. The observations consist of
protected attributes and other remaining attributes like test scores, grade
point averages, credit scores, health risks, and more context dependent
variables. The goal is to “fairly” predict outcomes like student success,
credit worthiness, and health care needs.

One approach is to ignore the protected attributes and train algorithms only
on the remaining attributes i.e. “fairness through blindness” (Dwork et al.
2012). However, there could be variables like height or zip code which are
proxies for protected attributes (Pedreshi, Ruggieri, and Turini 2008;
Corbett-Davies and Goel 2018). Even in the absence of these proxy variables,
seemingly reasonable remaining predictive attributes like study time or prior
health care can still depend on protected attributes. A natural
consequentialist approach in the face of concerns over group equity is
demographic parity, which ensures an identical distribution over the outcomes
for each protected group. However, demographic parity has been critiqued
because it can allow for blatantly unfair choices to individuals (Dwork et al.
2012; Corbett-Davies and Goel 2018).

Figure 1: A causal model in a law school context. Directed edges between a
source and a target denote a causal relationship in the sense that changing
the value of the source (probabilistically) changes the value of the target.
In this context, counterfactual fairness tries to make decisions based on
knowledge (which cannot directly be measured) without biases from race or
gender.

Counterfactual fairness offers a solution by advocating for a causal
perspective, which provides statistical tools to help us minimize the direct
effects of protected attributes on decisions (Kusner et al. 2017; Nilforoshan
et al. 2022), and comports with a Rawlsian perspective of moral philosophy
(Rawls 2004). It follows a surge of papers over the past decade that borrow
intuition from moral philosophy to formally define what makes an algorithm
fair (Dwork et al. 2012; Feldman et al. 2015; Hardt, Price, and Srebro 2016;
Mitchell et al. 2021). Drawing from (Pearl et al. 2000)’s work on causal
models, (Kusner et al. 2017) imagine a counterfactual world where only the
protected attributes of observations are changed. Then, an algorithm is
counterfactually fair if it makes the same predictions for a hypothetical
counterfactual version of an observation as it would for the real version of
the observation. This individual notion of fairness promises to avoid the
pitfalls of the group based fairness measures we described earlier, though the
relationship between causality, counterfactuals and moral philosophy is
complicated (Kasirzadeh and Smart 2021).

### Preliminaries and Notation

We follow notation laid out by (Kusner et al. 2017) for our formal analysis.
The variables are partitioned into $A$ (the set of protected attributes), $U$
(the set of latent attributes), $X$ (the set of remaining variables), and $Y$
(the outcomes).

Figure 2 shows a general causal model. Each variable is generated
(probabilistically) from a governing distribution. The parameters of a
variable’s distribution are fixed or derived from the incoming edges in the
causal model (or both). For example, we may have $a\sim\textrm{Binom}(.3,1)$,
$u\sim\mathcal{N}(0,1)$, and $x\sim\textrm{Pois}(au)$.

Figure 2: A general causal model consists of a directed acyclic graph and
governing equations for the (probabilistic) generation of each variable.

A counterfactually fair algorithm uses $U$ to predict $Y$ since every other
attribute is influenced by the protected attributes $A$. However, $U$ cannot
be measured directly. The solution of counterfactual fairness is to model the
joint distribution of each attribute in $X$ induced by the values of $A$ and
$U$. Then $U$ is estimated by the posterior distribution on instances of $A$
and $X$.

Building a causal model requires domain expert knowledge to establish both the
direction of relationships between attributes and the family of distributions
(e.g., normal, Poisson, binomial, etc.) that generates the distribution of
each variable. Once a causal model is determined, it is computationally
intensive to learn the parameters of each distribution and then to estimate
the posterior distribution of each attribute in $U$.

### Fairness Definitions

In this section, we formalize the definitions of demographic parity and
counterfactual fairness. We use capital letters $A$, $X$, $U$, and $Y$ to
refer to sets of random variables, and we use lowercase letters $a$, $x$, $u$,
and $y$ to refer to realizations of these random variables. Consider a
predictor $\hat{Y}:A\times U\times X\rightarrow Y$. If a predictor $\hat{Y}$
does not use a particular set of random variables then we omit that set. We
use $\hat{Y}_{A\leftarrow a}(u)$ for some protected group $a\in A$ to denote
the prediction on $u$ where attributes in $A$ are set to $a$.

######  Definition 1 (Demographic Parity (Calders, Kamiran, and Pechenizkiy
2009; Zliobaite 2015; Zafar et al. 2015)).

Given a predictor $\hat{Y}:X\times A\rightarrow Y$, we say $\hat{Y}$ satisfies
demographic parity if, for all instances of protected attributes $a$ and
$a^{\prime}$,

$\displaystyle\Pr(\hat{Y}(x,a)=y|A=a)=\Pr(\hat{Y}(x,a^{\prime})=y|A=a^{\prime})$

where the probability is taken over the conditional distribution of $X$ and
the possible randomness of $\hat{Y}$.

######  Definition 2 (Counterfactual Fairness (Kusner et al. 2017)).

Let $A$, $U$, $X$, and $Y$ be sets of random variables in a given causal model
with specified distributions for each attribute. Consider any protected group
$a^{\prime}\in A$ and any outcome $y\in Y$. Then a predictor $\hat{Y}:U\times
A\rightarrow Y$ is counterfactually fair if for all observations $x\in X$ and
$a\in A$,

$\displaystyle\Pr(\hat{Y}_{A\leftarrow a}(u)=y|X=x,A=a)$
$\displaystyle=\Pr(\hat{Y}_{A\leftarrow a^{\prime}}(u)=y|X=x,A=a)$

where the probability is with respect to the posterior distribution of $u$
induced by fixed observations $x$ and $a$.

Definition 2 states that it must be the case that the final prediction of a
counterfactually fair predictor $\hat{Y}_{A\leftarrow a}(u)$ is distributed
like $\hat{Y}_{A\leftarrow a^{\prime}}(u)$ when $a\neq a^{\prime}$ and where
the distribution of $u$ is induced by observations of $X$ and $A$.

Now we have established the two primary definitions of fairness that we
consider in our work. However, it is worth noting that difficulties in
defining algorithmic fairness have led to many competing approaches.
Furthermore, formal results have demonstrated the incompatibility of applying
some fairness definitions in conjunction with others, which hints at
irreconcilable philosophical groundings and a need for formalizations for
different policy settings (Kleinberg, Mullainathan, and Raghavan 2016;
Chouldechova 2016; Corbett-Davies and Goel 2018).

### Our Contributions

Our work starts with a simple observation on the distribution of latent
variables: by their definition, latent variables must be independent of
protected attributes for the model to be counterfactually fair. A careful look
at this statement suggests a connection to demographic parity, which requires
that predictions must be independent of protected attributes. A natural
question is then:

Do algorithms that satisfy demographic parity also satisfy counterfactual
fairness?

But for an algorithm to even satisfy the definition of counterfactual
fairness, we need a causal model, latent variables, and a way of estimating
latent variables. Our first result answers the question in the affirmative if
we trivially construct the necessary counterfactual fairness machinery.

The next natural question is whether the converse holds:

Do algorithms that satisfy counterfactual fairness also satisfy demographic
parity?

We answer this question in the affirmative, as well.111 After posting an early
draft of this work, we were made aware of a paper released several months
before which independently shows counterfactual fairness implies demographic
parity (see section 5.3 of (Fawkes, Evans, and Sejdinovic 2022)). We believe
this contemporary interest points to the importance of our work. Together, the
two results suggest that counterfactual fairness is basically demographic
parity. This relationship is important for the fairness community since
counterfactual fairness is often celebrated as a novel tool for fair decisions
while demographic parity is considered simple and flawed. For example, in
(Caton and Haas 2020), they note that demographic parity is flawed because it
does not account for potential differences between groups, but do not mention
limitations of counterfactual fairness. Other influential work takes a similar
position (Gajane and Pechenizkiy 2017; Corbett-Davies and Goel 2018; Coston et
al. 2020).

We also analyze three existing algorithms for counterfactual fairness in
comparison to three benchmark algorithms. The data set is based on the running
law school example and we corroborate our findings on two additional data sets
and causal models. Our analysis uses a relaxation of counterfactual fairness
and Kruskal-Wallis H tests (as one-sided tests for independence). We find that
two simple benchmark algorithms outperform the existing algorithms in terms of
satisfying the requirements of counterfactual fairness, computational
efficiency, and accuracy.

We also introduce a notion of preserving group ordering. Put simply, it
requires that an individual who performs better than another individual in the
same protected group under unfair labels must also perform better than that
individual under fair labels. We believe that this definition is important for
transparency and consistency, laying the foundations for trust in fair
algorithms. Why? Because inexplicable decisions in purportedly “fair” decision
making processes can mask unforeseen technical bias (Abdollahi and Nasraoui
2018; Bhatt et al. 2020). Interestingly, we show that preserving group
ordering can be mutually exclusive with counterfactual fairness for certain
causal models. Empirically, we find the existing algorithms for counterfactual
fairness have remarkably unstable orderings while the simple benchmark
algorithms are consistent by design.

#### Outline

  * •

We first show that all counterfactually fair predictors satisfy demographic
parity and all predictors satisfying demographic parity can be trivially
modified to satisfy counterfactual fairness.

  * •

We subsequently analyze six algorithms (including those presented in (Kusner
et al. 2017)) in the context of the following two questions:

    * –

How do these algorithms satisfy three special fairness constraints
(counterfactual fairness, demographic parity, and the independence required
for latent variables) while still maintaining reasonably accurate predictions?

    * –

How do these algorithms compare on the predictions they make at the individual
level?

We corroborate our empirical results on additional data sets in the context of
healthcare and loans. All our code and results are available online222
github.com/lurosenb/simplifying_counterfactual_fairness.

### Related Work

Since its introduction, counterfactual fairness has been highly influential.
Many recent works on equity in statistics and machine learning directly use or
modify only slightly Definition 2 to enable training classifiers and other
decision making algorithms (Zhang and Bareinboim 2018; Chiappa 2019; Wu,
Zhang, and Wu 2019; Coston et al. 2020; Black, Yeom, and Fredrikson 2020;
Mhasawade and Chunara 2021; Chikahara et al. 2021; von Kügelgen et al. 2022).
(Zhang and Bareinboim 2018) create a procedure for identifying discrimination
and applying causal explanations, and then use counterfactuals to design
repairs while offering evaluations of this system on a host of examples. (Wu,
Zhang, and Wu 2019) and (Chiappa 2019) expand theoretical groundings for
counterfactual fairness, with the former improving on the latent attribute
identifiability by offering bounds and techniques. Prior work also expands
counterfactual fairness to deal with between-group rankings (Yang, Loftus, and
Stoyanovich 2020). Furthermore, task specific variations and implementations
of Definition 2 also exist for the medical domain (Pfohl et al. 2019),
variational autoencoders in computer vision (Kim et al. 2021), generative-
adversarial networks (Xu et al. 2019), and even natural language processing
(Sarı, Hasegawa-Johnson, and Yoo 2021).

There are several existing concerns about counterfactual fairness. Recent work
by (Nilforoshan et al. 2022) present a result that unites many causal notions
of fairness, and further cautions that a gap exists between the effects of
these popular approaches on fairness and their consequences, highlighting both
practical and mathematical limitations. With particular relevance to our work,
they show that when a predictor satisfying demographic parity and a causal
model are specified, the predictor does not necessarily satisfy counterfactual
fairness (see Theorem 2). In contrast, our work considers the setting where we
are only given a predictor satisfying demographic parity (not a causal model)
and can build a causal model of our choice. In (Kasirzadeh and Smart 2021),
the authors conclude that counterfactual fairness may require what they deem
to be an “incoherent theory” of social categories, i.e., some social
categories may not admit counterfactual manipulation. In (Kilbertus et al.
2020), they examine the sensitivity of counterfactual fairness to “unmeasured
confounding.” This is when a true causal effect between variables can be at
least partially described by a non-zero correlation between two of the
$\epsilon$-“error” variables calculated during one possible counterfactual
fairness process, which can affect the independence guarantee from $A$. We
position our work as an important addition to this existing set of papers
expressing concerns with and limitations of counterfactual fairness.

Better understanding the theoretical and practical consistency of algorithmic
fairness has real social impact. As fairness is operationalized through open
source libraries like Fairlearn (Bird et al. 2020) and AIF360 (Bellamy et al.
2019), work that furthers our understanding of fair model guarantees becomes
increasingly important.

## Demographic Parity Is Counterfactual Fairness

Our results start with a simple observation about the definition of latent
variables. In the original counterfactual fairness paper, the authors define
latent variables $U$ so that “$U$ is a set of latent background variables,
which are factors not caused by any variable in the set $V[=A\cup X]$ of
observable variables” where $A$ is the set of protected attributes and $X$ is
the set of remaining variables (Kusner et al. 2017). In other words, there are
no incoming edges to $U$ in the causal model. This means that instances of
latent variables are generated from a fixed set of parameters (that do not
depend on protected attributes). In addition, while not explicitly stated in
the definition, all causal models we are aware of also do not have any
incoming edges to $A$. This means that instances of protected attributes are
generated from a fixed set of parameters (that do not depend on latent
variables). Together, these observations imply the following:

######  Criteria 3.

Latent variables are independent of protected attributes.

This criteria is key to understanding practical applications of counterfactual
fairness. Counterfactual fairness attempts to explain behavior from protected
attributes and latent variables. The protected attributes hold information on
features outside of individuals’ control which would very often be unfair to
base decisions on while the latent variables contain information on features
within an individual’s control which are fair to base decisions on. From this
perspective, it is clear that latent variables must be independent of
protected attributes. Otherwise, this would imply that the inherent
‘worthiness’ of latent variables would problematically depend on protected
attributes like race, gender, and age.

With Criteria 3 in hand, we show that any predictor which satisfies
demographic parity also satisfies counterfactual fairness after a trivial
modification.

Note that a predictor which satisfies counterfactual fairness must have an
internal method of estimating latent variables and a causal model. So to say
that a predictor which satisfies demographic parity also satisfies
counterfactual fairness, we must introduce such a method and causal model. We
do so in the proof of the first theorem in a trivial way.

######  Theorem 4.

Any predictor $\hat{Y}^{\prime}:X\times A\rightarrow Y$ that satisfies
demographic parity can be modified into a method for estimating latent
variables and a predictor $\hat{Y}:U\times A\rightarrow Y$ that is
counterfactually fair.

###### Proof.

Our estimate of the latent variables $u$, given protected attributes $a$ and
remaining attributes $x$, is $u=\hat{Y}^{\prime}(x,a)$. Then the predictor
$\hat{Y}:U\times A\rightarrow Y$ is given by the identity function
$\hat{Y}_{A\leftarrow a}(u)=u$. Because $\hat{Y}^{\prime}$ satisfies
demographic parity, we know $\Pr(u|a)=\Pr(u|a^{\prime})$ for $a,a^{\prime}\in
A$. That is, latent variables are independent of protected attributes which
meets Criteria 3. Further, $\hat{Y}$ satisfies counterfactual fairness because
$\hat{Y}_{A\leftarrow a}(u)=u=\hat{Y}_{A\leftarrow a^{\prime}}(u)$. ∎

The next theorem proves the converse of Theorem 4; namely, any
counterfactually fair predictor also satisfies demographic parity.

######  Theorem 5.

Consider a method for estimating latent variables and a counterfactually fair
predictor $\hat{Y}:U\times X\rightarrow Y$. Then the resulting predictor
$\hat{Y^{\prime}}:X\times A\rightarrow Y$ satisfies demographic parity.

###### Proof.

Since the predictor is counterfactually fair, we know

$\displaystyle\Pr(\hat{Y}_{A\leftarrow a}(u)=y|x,a)=\Pr(\hat{Y}_{A\leftarrow
a^{\prime}}(u)=y|x,a)$ (1)

for all $y\in Y$ where the realization of latent variables $u\in U$ is
estimated from observations $a\in A$ and $x\in X$. Taking a weighted sum (if
$X$ contains continuous random variables, an analogous statement holds via
integration) of the right side of Equation (1) yields

$\displaystyle\sum_{x\in X}\Pr(x|a)\Pr(\hat{Y}_{A\leftarrow a}(u)=y|x,a)$
$\displaystyle=\sum_{x\in X}\Pr(x,\hat{Y}_{A\leftarrow a}(u)=y|a)$
$\displaystyle=\Pr(\hat{Y}_{A\leftarrow a}(u)=y|a)=\Pr(\hat{Y}_{A\leftarrow
a}(u)=y).$

The first equality holds because $\Pr(x|a)\Pr(y|x,a)=\Pr(x,y|a)$ for random
events $x,y,a$. The last equality holds by Criteria 3 and since the (possible)
randomness of $\hat{Y}_{A\leftarrow a}(u)$ only comes from $u$ and the
assignment $A\leftarrow a$. The final equality gets us close to demographic
parity but the predictions could still depend on the assignment. We repeat the
same steps for the left side of Equation (1) and find that

$\displaystyle\Pr(\hat{Y}_{A\leftarrow a}(u)=y)=\Pr(\hat{Y}_{A\leftarrow
a^{\prime}}(u)=y).$

This tells us that the distribution of predictions made by a counterfactually
fair algorithm are independent of protected attributes. That is, demographic
parity holds. ∎

Note that the contrapositive of Theorem 5 gives us a simple way to test
whether a predictor is counterfactually fair.

## Algorithms for Counterfactual Fairness

In this section, we empirically analyze six algorithms in the context of
counterfactual fairness on the running law school example. We compare them in
terms of how well they achieve demographic parity (Definition 1),
counterfactual fairness (Definition 2), and independence between protected
attributes and latent variables (Criteria 3). We also compare the algorithms’
test accuracy and the resulting trade-offs between fairness and performance.
Finally, we investigate how predictions on individuals differ between the
algorithms.

### The Algorithms

The first three algorithms—referred to as “Levels 1, 2 and 3”—come from the
original counterfactual fairness paper (Kusner et al. 2017). The next two
algorithms are simple heuristics for demographic parity while the final
algorithm is a straightforward learner without fairness constraints.

For all the algorithms we analyze, we use a linear regression model,
$\ell_{2}$-norm loss, and Adam optimizer for learning. We describe the
algorithms next.

#### Level 1

Since evaluating the relationships between variables in a causal model is
computationally expensive, the first level only uses the remaining variables
that are independent of protected attributes to learn the outcomes. In
practice, scenarios where remaining variables are not at least partially
conditioned on protected attributes are so rare as to be virtually non-
existent. Instead, we implement Level 1 by using all remaining variables
(without any protected attributes). This approach has been called “fairness
through unawareness” (Grgic-Hlaca et al. 2016) and fails to make fair
decisions because protected attributes are often redundantly encoded
(Pedreshi, Ruggieri, and Turini 2008).

#### Level 2

The second level uses the full power of causal models, but suffers from
expensive and intensive computations; it requires domain expertise to specify
the joint distributions over all variables for the causal model. Remaining
variables are distributed according to subsets of latent variables and
protected attributes. By using the known protected attributes, the remaining
variables, and the causal model, Level 2 estimates likely values of latent
variables. Then only the latent variables are used to learn a predictor over
the set of possible outcomes.

#### Level 3

The third level is a compromise between the simplicity of Level 1 and
complexity of Level 2. Level 3 uses the relationships in the causal model to
express the remaining variables of each individual as a deterministic function
of related protected attributes and a special explanation term. The
deterministic function is learned from protected attributes to explain the
remaining variable. Then the difference between the deterministic function and
an individual’s remaining variable is their explanation term. These
explanation terms are used to learn the outcomes.

Listing 1: Converting unfair predictions to fair predictions when labels are
normally distributed.

⬇

1Input: $n$ group identities $a_{i}\in A$ and unfair labels $\bar{y_{i}}$ for
samples $i\in[n]$

2Output: predictions $\hat{y}_{i}$ that satisfy demographic parity

3$\mu\leftarrow$ mean($\\{\bar{y}_{i}:i\in[n]\\}$)

4$\sigma\leftarrow$ std($\\{\bar{y}_{i}:i\in[n]\\}$)

5for $a$ in $A$ do

6 $\mu_{a}\leftarrow$ mean($\\{\bar{y}_{i}:a_{i}=a\\}$)

7 $\sigma_{a}\leftarrow$ std($\\{\bar{y}_{i}:a_{i}=a\\}$)

8 for $i$ in $\\{i:a_{i}=a\\}$ do

9 $\hat{y_{i}}=\mu+\sigma(\bar{y}_{i}-\mu_{a})/\sigma_{a}$

Listing 2: Converting unfair predictions to fair predictions under any
distribution of labels.

⬇

1Input: $n$ group identities $a_{i}\in A$ and unfair labels $\bar{y_{i}}$ for
samples $i\in[n]$

2Output: predictions $\hat{y}_{i}$ that satisfy demographic parity

3$CDF\leftarrow$ empirical CDF of $\\{y_{i}:i\in[n]\\}$

4for $a$ in $A$ do

5 $CDF_{a}\leftarrow$ empirical CDF of $\\{y_{i}:a_{i}=a\\}$

6 for $i$ in $\\{i:a_{i}=a\\}$

7 $\hat{y}_{i}\leftarrow CDF(CDF_{a}^{-1}(\bar{y}_{i}))$

The next two algorithms are simple heuristics for demographic parity.

#### Listing 1

The first listing assumes the distribution of outcomes in every protected
group is normally distributed. Then we can achieve demographic parity by
simply calculating the normalized score of each individual within their
protected group and converting it to the outcome distribution of the full
population.

#### Listing 2

The second listing drops the assumption on the distribution of each protected
group. Instead, Listing 2 estimates the cumulative density function (CDF) of
each protected group and converts an individual’s relative position within
their group to the same position within the full population.

Listing 1 is appropriate when the outcomes are normally distributed while
Listing 2 is appropriate in general but cannot distinguish noise from signal
in distributions.

#### Full Linear Model

The final algorithm uses a linear model on all protected attributes and
remaining variables to learn outcomes. We expect the full linear model to
mimic the unfair behavior of the underlying data but achieve the best test
accuracy.

Figure 3: Density plots for the distributions over predictions made by the six
algorithms we consider. The root mean squared error (rMSE) is given next to
each title and in Table 2. Groups 1, 2, 3, and 4 correspond to protected
groups over the Cartesian product of binary race and gender.

### Measures of Fairness

We use several empirical tests to measure how well each of the algorithms
achieves different definitions of fairness.

We first define a relaxation of counterfactual fairness introduced in (Russell
et al. 2017).

######  Definition 6 ($(\epsilon,\delta)$-Approximate Counterfactual Fairness
(ACF)).

A predictor $\hat{Y}:X\times A\rightarrow Y$ satisfies $(\epsilon,\delta)$-ACF
if

$\displaystyle\Pr(|\hat{Y}_{A\leftarrow a}(u)-\hat{Y}_{A\leftarrow
a^{\prime}}(u)|\leq\epsilon|x,a)\geq 1-\delta.$

Notice that calculating $(\epsilon,\delta)$-ACF requires an internal estimate
of latent variables. The only algorithms that have a nontrivial estimate of
latent variables are Level 2 and Level 3. Since both Level 2 and Level 3 use
only the latent variables in learning the outcomes, changing the protected
attribute has no effect and both algorithms are $(0,0)$-ACF.

For testing demographic parity and whether latent variables are independent of
protected attributes, we use the Kruskal-Wallis H test (Kruskal and Wallis
1952).

######  Definition 7 (Kruskal-Wallis H Statistic).

The H statistic is given by

$\displaystyle(N-1)\frac{\sum_{i=1}^{g}n_{i}(\bar{r}_{i}-\bar{r})^{2}}{\sum_{i=1}^{g}\sum_{j=1}^{n_{i}}(r_{ij}-\bar{r})^{2}}$

where $N$ is the total number of observations, $g$ is the number of groups,
$n_{i}$ is the number of observations in group $i$, $r_{ij}$ is the rank
(among all observations) of observation $j$ from group $i$, $\bar{r}_{i}$ is
the average rank of observations in group $i$, and $\bar{r}$ is the average
rank of all observations.

Using the Kruskal-Wallis H statistic, we can calculate the probability that
every group comes from a distribution with the same median. Since the latent
variables are independent of the protected attributes if and only if the
distributions of the protected groups are the same, a small probability from
the Kruskal-Wallis H test implies that the latent variables and protected
attributes are not independent. We also use the test to determine whether an
algorithm satisfies demographic parity. Note it is possible that all groups
come from distributions with the same median but that the distributions are in
fact different so the test has one-sided error.

We present our results of the Kruskal-Wallis H tests in Table 1. The first
three data rows tell us the probability that Criteria 3 holds. It is very
likely that the latent variables in each protected group come from
distributions with the same median in Level 2, suggesting that Level 2
satisfies Criteria 3 in addition to counterfactual fairness. In contrast, it
is unlikely (probability $<.05$) that the two latent variables in Level 3 are
independent of protected attributes, suggesting that Level 3 does not satisfy
Criteria 3. Note that Level 3 is an example of an algorithm that ostensibly
satisfies counterfactual fairness but the latent variables do not meet the
necessary condition of Criteria 3. We corroborate this finding in Figure 3:
Level 1 and Level 3 clearly do not satisfy demographic parity because the
latent variables are not independent of protected attributes.

The last six rows in Table 1 tell us whether each algorithm satisfies
demographic parity. As expected, Level 1 and the Full Linear Model are
remarkably unlikely to satisfy demographic parity. In line with the results
for latent variables, we see that Level 3 also is unlikely to satisfy
demographic parity. Finally, Level 2 likely satisfies demographic parity. This
is to be expected since the assumptions of Theorem 5—namely, counterfactual
fairness and Criteria 3—hold. Of course, Listings 1 and 2 are very likely to
satisfy demographic parity by design.

Variable | H Statistic |  $p$-value
---|---|---
Level 2 Latent Variable | $.347$ | $.951$
Level 3 Latent UGPA | $10.8$ | $.0126$
Level 3 Latent LSAT | $84.6$ | $3.21\times 10^{-18}$
Level 1 Predictions | $221$ | $9.81\times 10^{-48}$
Level 2 Predictions | $.346$ | $.951$
Level 3 Predictions | $69.3$ | $5.97\times 10^{-15}$
Listing 1 Predictions | $.165$ | $.983$
Listing 2 Predictions | $.035$ | $.998$
Full Predictions | $829$ | $2.53\times 10^{-179}$
Table 1: Kruskal-Wallis H test results for different variables. The $p$-value
indicates the likelihood of observing the H statistic if the variable was
distributed the same in every group. Every number is reported to three
significant figures.

Table 2 gives the root mean squared error (rMSE) of every algorithm we
consider. As expected, the Full Linear Model without fairness constraints has
the lowest error. Listing 1 and Listing 2 also give relatively low error. In
contrast, Level 2 has the highest error. So far, Level 2 simulatenously
satisfies Definition 1, Definition 2, and Criteria 3 but we see this is at the
expense of accuracy.

Lvl 1 | Lvl 2 | Lvl 3 | Lst 1 | Lst 2 | Full
---|---|---|---|---|---
.933 | .936 | .908 | .919 | .921 | .881
Table 2: The root mean squared error (rMSE) for each algorithm predicted to
three significant figures.

### Individual Predictions

We have seen how the algorithms we consider perform differently in terms of
three fairness measures. But how do the algorithms differ on the actual
outcomes for individuals?

Figure 4 visualizes the change in relative ordering of predicted outcomes for
40 randomly chosen individuals in Group 1 (Black men) in the law school
example. Notably, the relative ordering is constant between Listing 1, Listing
2, and the Full Linear Model, as expected. However, the relative ordering is
highly unstable for Level 1, Level 2, and Level 3. This suggests that while
Level 2 and Level 3 achieve counterfactual fairness, they do so in
dramatically different ways. One explanation is that Level 2 satisfies
Criteria 3 while Level 3 does not.

Figure 4: We randomly sample 40 individuals from Group 1 (Black men) in the
law school example and visualize how the relative order of their predicted
outcomes changes based on the algorithm making predictions.

We find it surprising that Level 2 does not agree with Listing 1 and Listing 2
on the relative order of outcomes. All three algorithms satisfy demographic
parity, Criteria 3, and, by Theorem 4, counterfactual fairness (when Listing 1
and Listing 2 are trivially modified). How could it be that two algorithms are
both fair under the same definitions while making radically different
predictions on individuals?

This question suggests the following natural definition.

######  Definition 8 (Preserving Group Ordering).

Given unfair labels, we say an algorithm preserves group ordering if the
relative ordering within each protected group under the unfair labels is the
the same as the relative ordering of individuals induced by the algorithm’s
predictions.

Put differently, if individuals are in the same protected class, then their
relative ordering from a (possibly) unfair predictor should induce the same
relative ordering as the fair predictor. That is, if an individual performs
better under the same unfair conditions as another individual, they should
perform better in a hypothetical world where those unfair conditions were
removed.

We believe that preserved within-group orderings are preferable to the highly
unstable orderings in the levels of (Kusner et al. 2017). One reason to prefer
preserved orderings is that even soft guarantees for relative orderings of
within group individuals leads to more transparent, trustworthy and ultimately
useful fair algorithms (Abdollahi and Nasraoui 2018; Bhatt et al. 2020).

Observe that Listing 1 and Listing 2 satisfy Definition 8 by design. This
contrasts with Levels 1, 2, and 3 which radically change the relative ordering
of individuals within a protected group. In Proposition 9, we show that there
is a causal model where counterfactual fairness is mutually exclusive with
Definition 8. This suggests that for some causal models, achieving
counterfactual fairness and preserving relative group ordering are
incompatible.

Note that Proposition 9, combined with the fact that Listing 1 and 2 satisfy
demographic parity and preserve group order, does not refute Theorem 4 since
the method of estimating latent variables in the proof of the theorem uses a
trivial causal model.

######  Proposition 9.

There is a causal model and predictor where a counterfactually fair algorithm
does not maintain the relative ordering of predictions within each group.333An
earlier version of this work gave an incorrect proof of the proposition.

| Short | Tall
---|---|---
| World A | World B | World A | World B
| $u$ | $\hat{y}$ | $u$ | $\hat{y}$ | $u$ | $\hat{y}$ | $u$ | $\hat{y}$
Teal | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 0
Lucas | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 1
Table 3: Latent variable charm $U$ and predicted outcome $\hat{Y}$ for each
person, world, and protected group in the example for Proposition 9.

###### Proof.

Consider an example with two people, say Teal and Lucas. The protected
attribute $A$ is binary height, the latent variable $U$ is binary charm, and
the true outcome $Y$ is binary success. The latent variable for Teal is
$u_{\textrm{Teal}}\sim\textrm{Binomial}(1/2)$ while the latent variable for
Lucas is $u_{\textrm{Lucas}}=1-u_{\textrm{Teal}}$. In other words, the latent
variable for each person is probabilistic but correlated: with probability
1/2, we live in World A and otherwise, we live in World B. In World A, Teal is
not charming and Lucas is while, in World B, Teal is charming and Lucas is
not. Now the predictor function satisfies
$\hat{Y}_{A\leftarrow\textrm{short}}(u)=u$,
$\hat{Y}_{A\leftarrow\textrm{tall}}(u)=\texttt{NOT}(u)$ for latent variable
charm $u\in\\{0,1\\}$. The output of the predictor in each world is given in
Table 3.

Notice that the relative order of the outcome produced by the predictor and
causal model changes with the counterfactual intervention. In World A, Teal is
behind Lucas as a short person and ahead as a tall person. In World B, Teal is
ahead of Lucas as a short person and behind as a tall person. We have
constructed a causal model and predictor but it remains to show that they
satisfy counterfactual fairness and that the latent variable is independent of
protected attributes.

First, we show that the causal model and predictor are counterfactually fair.
Recall that World A and World B are equally likely. As a result, by inspecting
Table 3 and the construction, we conclude that both Teal and Lucas are as
likely to be successful as a short person and as a tall person. Therefore the
construction satisfies counterfactual fairness.

Second, we show that the latent variable is independent of the protected
attribute. That is, we want to show $\Pr(u,a)=\Pr(u)\Pr(a)$ for
$u,a\in\\{0,1\\}$ in our example. Again, by inspecting Table 3 and the
construction, we see that this holds. For example, the probability a person is
charming and tall is 2/8 while the probability a person is charming is 4/8 and
the probability a person is tall is 4/8. The same holds for the other three
possibilities. ∎

#### Additional Experiments

So far we have reported on the extensive empirical experiments we ran on the
law school data set. We corroborate our findings by running the same set of
experiments on two more examples. The first example is data from the Home
Credit Default Risk data set444https://www.kaggle.com/c/home-credit-default-
risk. We simulated a “domain expert” process of causal analysis and
constructed a credit risk causal model by hand (using observations from the
data). The second example is synthetic data generated directly from a causal
model we built, modeling a hypothetical healthcare context loosely motivated
by (Obermeyer et al. 2019), and represents the “ideal” circumstance where the
causal model posited over the data is certainly correct. We show the causal
models in Figure 5. The remaining figures and results are included for
reproducibility in the the same repository as the rest of our results.

Figure 5: The causal diagrams for the default risk example (on the left) and
healthcare example (on the right).

## Conclusion

Counterfactual fairness has been celebrated as novel and promising while
demographic parity has been treated as simple and flawed. Our work shows how
the two definitions of fairness are basically equivalent when considered
across an entire population. We think it is a promising avenue for future work
to identify similarities between other definitions of fairness. We also
introduce a method for comparing algorithms at the level of individual
predictions. Our findings suggest a natural definition of preserving group
orderings which can be mutually exclusive with counterfactual fairness. We
leave open the problem of designing novel algorithms that are competitive with
simple benchmarks in terms of fairness, performance, and preserving group
orderings.

## Acknowledgements

We thank Apoorv Vikram Singh for his insightful comments on an early draft of
this work.

## References

  * Abdollahi and Nasraoui (2018) Abdollahi, B.; and Nasraoui, O. 2018.  Transparency in fair machine learning: the case of explainable recommender systems.  In _Human and machine learning_ , 21–35. Springer.
  * Bellamy et al. (2019) Bellamy, R. K.; Dey, K.; Hind, M.; Hoffman, S. C.; Houde, S.; Kannan, K.; Lohia, P.; Martino, J.; Mehta, S.; Mojsilović, A.; et al. 2019.  AI Fairness 360: An extensible toolkit for detecting and mitigating algorithmic bias.  _IBM Journal of Research and Development_ , 63(4/5): 4–1.
  * Bhatt et al. (2020) Bhatt, U.; Xiang, A.; Sharma, S.; Weller, A.; Taly, A.; Jia, Y.; Ghosh, J.; Puri, R.; Moura, J. M.; and Eckersley, P. 2020.  Explainable machine learning in deployment.  In _Proceedings of the 2020 conference on fairness, accountability, and transparency_ , 648–657.
  * Bird et al. (2020) Bird, S.; Dudík, M.; Edgar, R.; Horn, B.; Lutz, R.; Milan, V.; Sameki, M.; Wallach, H.; and Walker, K. 2020.  Fairlearn: A toolkit for assessing and improving fairness in AI.  _Microsoft, Tech. Rep. MSR-TR-2020-32_.
  * Black, Yeom, and Fredrikson (2020) Black, E.; Yeom, S.; and Fredrikson, M. 2020.  FlipTest: fairness testing via optimal transport.  _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_.
  * Calders, Kamiran, and Pechenizkiy (2009) Calders, T.; Kamiran, F.; and Pechenizkiy, M. 2009.  Building classifiers with independency constraints.  In _2009 IEEE International Conference on Data Mining Workshops_ , 13–18. IEEE.
  * Caton and Haas (2020) Caton, S.; and Haas, C. 2020.  Fairness in machine learning: A survey.  _arXiv preprint arXiv:2010.04053_.
  * Chiappa (2019) Chiappa, S. 2019.  Path-Specific Counterfactual Fairness.  In _AAAI_.
  * Chikahara et al. (2021) Chikahara, Y.; Sakaue, S.; Fujino, A.; and Kashima, H. 2021.  Learning Individually Fair Classifier with Path-Specific Causal-Effect Constraint.  In _AISTATS_.
  * Chouldechova (2016) Chouldechova, A. 2016.  Fair prediction with disparate impact: A study of bias in recidivism prediction instruments.  arXiv:1610.07524.
  * Corbett-Davies and Goel (2018) Corbett-Davies, S.; and Goel, S. 2018.  The measure and mismeasure of fairness: A critical review of fair machine learning.  _arXiv preprint arXiv:1808.00023_.
  * Coston et al. (2020) Coston, A.; Mishler, A.; Kennedy, E. H.; and Chouldechova, A. 2020.  Counterfactual risk assessments, evaluation, and fairness.  In _Proceedings of the 2020 conference on fairness, accountability, and transparency_ , 582–593.
  * Dwork et al. (2012) Dwork, C.; Hardt, M.; Pitassi, T.; Reingold, O.; and Zemel, R. 2012.  Fairness through awareness.  In _Proceedings of the 3rd innovations in theoretical computer science conference_ , 214–226.
  * Fawkes, Evans, and Sejdinovic (2022) Fawkes, J.; Evans, R.; and Sejdinovic, D. 2022.  Selection, Ignorability and Challenges With Causal Fairness.  In _First Conference on Causal Learning and Reasoning_.
  * Feldman et al. (2015) Feldman, M.; Friedler, S. A.; Moeller, J.; Scheidegger, C.; and Venkatasubramanian, S. 2015.  Certifying and removing disparate impact.  In _proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining_ , 259–268.
  * Gajane and Pechenizkiy (2017) Gajane, P.; and Pechenizkiy, M. 2017.  On formalizing fairness in prediction with machine learning.  _arXiv preprint arXiv:1710.03184_.
  * Grgic-Hlaca et al. (2016) Grgic-Hlaca, N.; Zafar, M. B.; Gummadi, K. P.; and Weller, A. 2016.  The case for process fairness in learning: Feature selection for fair decision making.  In _NIPS symposium on machine learning and the law_ , volume 1, 2\.
  * Hardt, Price, and Srebro (2016) Hardt, M.; Price, E.; and Srebro, N. 2016.  Equality of opportunity in supervised learning.  _Advances in neural information processing systems_ , 29: 3315–3323.
  * Kasirzadeh and Smart (2021) Kasirzadeh, A.; and Smart, A. 2021.  The use and misuse of counterfactuals in ethical machine learning.  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , 228–236.
  * Kilbertus et al. (2020) Kilbertus, N.; Ball, P. J.; Kusner, M. J.; Weller, A.; and Silva, R. 2020.  The sensitivity of counterfactual fairness to unmeasured confounding.  In _Uncertainty in artificial intelligence_ , 616–626. PMLR.
  * Kim et al. (2021) Kim, H.; Shin, S.; Jang, J.; Song, K.; Joo, W.; Kang, W.; and Moon, I.-C. 2021.  Counterfactual Fairness with Disentangled Causal Effect Variational Autoencoder.  In _AAAI_.
  * Kleinberg, Mullainathan, and Raghavan (2016) Kleinberg, J.; Mullainathan, S.; and Raghavan, M. 2016.  Inherent Trade-Offs in the Fair Determination of Risk Scores.  arXiv:1609.05807.
  * Kruskal and Wallis (1952) Kruskal, W. H.; and Wallis, W. A. 1952.  Use of ranks in one-criterion variance analysis.  _Journal of the American statistical Association_ , 47(260): 583–621.
  * Kusner et al. (2017) Kusner, M. J.; Loftus, J. R.; Russell, C.; and Silva, R. 2017.  Counterfactual fairness.  _arXiv preprint arXiv:1703.06856_.
  * Mhasawade and Chunara (2021) Mhasawade, V.; and Chunara, R. 2021.  Causal multi-level fairness.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , 784–794.
  * Mitchell et al. (2021) Mitchell, S.; Potash, E.; Barocas, S.; D’Amour, A.; and Lum, K. 2021.  Algorithmic fairness: Choices, assumptions, and definitions.  _Annual Review of Statistics and Its Application_ , 8: 141–163.
  * Nilforoshan et al. (2022) Nilforoshan, H.; Gaebler, J. D.; Shroff, R.; and Goel, S. 2022.  Causal Conceptions of Fairness and their Consequences.  In _ICML_.
  * Obermeyer et al. (2019) Obermeyer, Z.; Powers, B.; Vogeli, C.; and Mullainathan, S. 2019.  Dissecting racial bias in an algorithm used to manage the health of populations.  _Science_ , 366(6464): 447–453.
  * Pearl et al. (2000) Pearl, J.; et al. 2000.  Models, reasoning and inference.  _Cambridge, UK: CambridgeUniversityPress_ , 19: 2.
  * Pedreshi, Ruggieri, and Turini (2008) Pedreshi, D.; Ruggieri, S.; and Turini, F. 2008.  Discrimination-aware data mining.  In _Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining_ , 560–568.
  * Pfohl et al. (2019) Pfohl, S. R.; Duan, T.; Ding, D. Y.; and Shah, N. H. 2019.  Counterfactual Reasoning for Fair Clinical Risk Prediction.  _ArXiv_ , abs/1907.06260.
  * Rawls (2004) Rawls, J. 2004.  A theory of justice.  In _Ethics_ , 229–234. Routledge.
  * Russell et al. (2017) Russell, C.; Kusner, M. J.; Loftus, J.; and Silva, R. 2017.  When worlds collide: integrating different counterfactual assumptions in fairness.  _Advances in neural information processing systems_ , 30.
  * Sarı, Hasegawa-Johnson, and Yoo (2021) Sarı, L.; Hasegawa-Johnson, M.; and Yoo, C. D. 2021.  Counterfactually fair automatic speech recognition.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 29: 3515–3525.
  * von Kügelgen et al. (2022) von Kügelgen, J.; Bhatt, U.; Karimi, A.-H.; Valera, I.; Weller, A.; and Scholkopf, B. 2022.  On the Fairness of Causal Algorithmic Recourse.  In _AAAI_.
  * Wu, Zhang, and Wu (2019) Wu, Y.; Zhang, L.; and Wu, X. 2019.  Counterfactual Fairness: Unidentification, Bound and Algorithm.  In _IJCAI_.
  * Xu et al. (2019) Xu, D.; Wu, Y.; Yuan, S.; Zhang, L.; and Wu, X. 2019.  Achieving Causal Fairness through Generative Adversarial Networks.  In _IJCAI_.
  * Yang, Loftus, and Stoyanovich (2020) Yang, K.; Loftus, J. R.; and Stoyanovich, J. 2020.  Causal intersectionality for fair ranking.  _arXiv preprint arXiv:2006.08688_.
  * Zafar et al. (2015) Zafar, M. B.; Valera, I.; Rodriguez, M. G.; and Gummadi, K. P. 2015.  Learning fair classifiers.  _arXiv preprint arXiv:1507.05259_ , 1(2).
  * Zhang and Bareinboim (2018) Zhang, J.; and Bareinboim, E. 2018.  Fairness in Decision-Making - The Causal Explanation Formula.  In _AAAI_.
  * Zliobaite (2015) Zliobaite, I. 2015.  On the relation between accuracy and fairness in binary classification.  _arXiv preprint arXiv:1505.05723_.


# FS-RAG: A Frame Semantics Based Approach for Improved Factual Accuracy in
Large Language Models

Harish Tayyar Madabushi
The University of Bath, Bath, UK
<EMAIL_ADDRESS>

###### Abstract

We present a novel extension to Retrieval Augmented Generation with the goal
of mitigating factual inaccuracies in the output of large language models.
Specifically, our method draws on the cognitive linguistic theory of frame
semantics for the indexing and retrieval of factual information relevant to
helping large language models answer queries. We conduct experiments to
demonstrate the effectiveness of this method both in terms of retrieval
effectiveness and in terms of the relevance of the frames and frame relations
automatically generated. Our results show that this novel mechanism of Frame
Semantic-based retrieval, designed to improve Retrieval Augmented Generation
(FS-RAG), is effective and offers potential for providing data-driven insights
into frame semantics theory. We provide open access to our program code and
prompts111https://github.com/H-TayyarMadabushi/A-Frame-Semantics-based-
approach-for-Improved-Factual-Accuracy-in-Large-Language-Models.

##  1 Introduction, Motivation and Context

Large language models (LLMs), despite their significant capabilities and
widespread adoption have the inherent tendency to generate plausible sounding,
yet inaccurate, output. This phenomenon, referred to as “hallucinations,” has
been a significant stumbling block in the widespread deployment of these
solutions Ji et al. (2023). Hallucinations themselves are not limited to
factual inaccuracies, and include other modes of failure, including the
incorrect interpretation of input prompts and errors in logical inference.
However, factual hallucinations are particularly important to address as the
retrieval of incorrect facts is particularly hard to recover from and can
neutralise and make irrelevant all other improvements.

The capabilities of LLMs typically improve with an increase in their “size,”
which is a combination of a model’s parameters and the size of the pre-
training corpus. Until recently, this was seen by some as being evidence that
further scaling would eventually address the shortcomings of LLMs, including
hallucinations. For example, LLMs were claimed to develop “emergent
abilities”: specifically, it was believed that LLMs, when scaled to several
billion parameters developed capabilities including those required to solve
tasks requiring reasoning in humans, thus indicative that they were developing
reasoning skills Wei et al. (2022b). More recent work, however, has shown that
this is not the case and that LLMs instead develop a single capability, which
they leverage to solve tasks Lu et al. (2023). This capability, called “in-
context learning,” is, roughly put, the ability of models to solve a
particular task based on a few examples provided in the prompt Brown et al.
(2020); Chowdhery et al. (2023). (Lu et al., 2023) further suggest that the
process of instructional fine-tuning LLMs to understand instructions Wei et
al. (2022a), enables models to leverage the same “in-context” abilities even
in the absence of examples. This finding indicates that further scaling, while
providing improved instruction following abilities, will not grant modes the
broader capacity to for general reasoning.

The fact that LLMs are not likely to develop the ability to reason has
profound implications to work on improving them, including to mitigating
hallucinations. It implies that we must explore alternative approaches. This
is especially the case when it comes to factual hallucinations as the
‘parametric memory’ in LLMs is orders of magnitude smaller than the pre-
training data Ji et al. (2023). As such, they must necessarily use some method
of compressing their pre-training data. Without the ability to distinguish
between the information that is relevant and what is not relevant in their
pre-training data, their method of compression defaults to be the memorisation
of frequent information and the learning of statistical patterns and trends to
represent the less frequent information.

###  1.1 Retrieval Augmented Generation

Given this context, it isn’t surprising that Retrieval Augmented Generation,
or RAG, which involves the inclusion of relevant information to the prompt has
been so successful Lewis et al. (2020). It provides a viable mechanism of
offloading information retrieval (IR) demands and instead focuses on using the
LLM as a mechanism of analysing and processing data based on explicit
instructions. It should be noted that analysing and processing data through
explicit instructions is precisely what models excel at through in-context
learning. Importantly, RAG also provides a solution to another of LLMs’
problems, which is the outdating of knowledge. By incorporating the latest
information, RAG prevents the need for further training for even minor
knowledge updates to the LLM which is clearly infeasible Shuster et al.
(2021).

However, RAG comes with its own shortcomings. RAG transfers the problem of
mitigating factual hallucinations to one of retrieving information relevant to
answering a query Gao et al. (2024). While LLMs can handle some noise in the
retrieved context provided, a dramatic increase in noise unsurprisingly leads
to deteriorating performance of models. The retrieval of information relevant
to answering a query is non-trivial as such information is not always likely
to be semantically related to the query. This problem becomes even more
important when the query requires reasoning over multiple facts each of which
are progressively semantically further from the query. Overall, the fact that
logically connected information is not always semantically similar makes
existing keyword and semantic similarity based search and IR systems poorly
suited for the specific IR requirements of LLMs.

Existing mechanisms of dealing with this problem in information retrieval
typically involve using LLMs themselves to solve this problem. Broadly, there
are two ways of doing this: the first involves the use of LLMs to decompose
the query into sub-queries each of which require less complex reasoning Patel
et al. (2022); Zhou et al. (2023), and the second involves the use of LLMs to
generate intermediary reasoning steps, called chain of thought Wei et al.
(2022c), to answer the query. These intermediary steps are then used to
generate IR search queries and the resultant facts are fed back to an LLM to
generate a response Shao et al. (2023). While these methods show some promise,
the problem with them is clear: they are yet another opaque mechanism the
failure of which will be hard to fix. In fact, we might be able to improve
these systems be fine-tuning models to perform this task. However, such a
solution is only going to further obfuscate the process.

###  1.2 Contributions

_To address these concerns we propose a novel, transparent, and mutable
storage and retrieval system for the mitigation of factual hallucinations in
LLMs. This solution is based on theoretical insights from cognitive
linguistics, specifically frame semantics._ Specifically, this work makes the
following contributions:

  1. 1.

We propose a novel, transparent, and mutable storage and retrieval system for
the mitigation of factual hallucinations in LLMs which is based on theoretical
insights from frame semantics. We call this system FS-RAG.

  2. 2.

We show the effectiveness of our method by experimenting on a closed domain
question answering. Specifically, our method of storage and retrieval provides
a significant boost over both treditional search systems and, significantly,
also outperforms an LLM based search system wherein we generate search terms
using a Language Model.

  3. 3.

We show that our method has the significant additional advantage of being
interpretable, and thus having the potential to provide data-driven insights
to the theory of frame semantics.

##  2 Frame Semantics

Given the context provided earlier, it is evident that only the most
frequently occurring factual information from pre-training is explicitly
retained in LLMs. Less frequently occurring facts are not explicitly stored
and instead the model has access to only statistical approximations. Given
that the exact information stored is not explicit and also different for
models of different scale and training regimes, the only way to get around
hallucination is to explicitly provide LLMs with all but the most common
information The effectiveness of RAG provides an easy mechanism for including
such information. However, as mentioned previously, the effectiveness of this
method hinges on the non-trivial task of retrieving relevant information.

We propose that this mechanism can be found in the concept of Frame Semantics
in cognitive linguistics, which purportedly facilitate human understanding of
words by allowing us to recall pertinent information.

Frame semantics Fillmore et al. (2006) is a theory of linguistic meaning that
emphasises that the meanings of words are best understood by the semantic and
conceptual “frames” or “schemas” within which they function. A frame is a
cognitive structure that helps individuals understand and perceive the world
around them, enabling them to organise knowledge based on typical situations,
actions, or common experiences. In linguistics, a frame influences how the
meanings of words are interpreted in different contexts. For example, the word
“sell” invokes a commercial transaction frame involving a seller, a buyer, an
item being sold, and an agreed price and helps to predict and explain the use
of other related words and the roles they play within the same context.
FrameNet Baker et al. (1998) is an online database based on frame semantics,
with the goal to catalogue English words and their associated semantic frames,
defining the various roles and relations in a frame and illustrating these
with example sentences. Each “frame” in FrameNet captures a specific type of
event, relation, or entity and the roles associated with it. For example, the
commerce_sell frame in FrameNet includes roles for the seller, the buyer, the
goods being sold, and is annotated to be inherited by the frame renting_out
and as a “perspective on” the frame commerce_goods-transfer.

##  3 Frame Semantic RAG

This section provides an overview of Frame Semantic RAG (FS-RAG), the proposed
mechanism of storing and indexing factual information to aid effective
retrieval for the mutation of hallucinations in LLMs. In evaluating our
mechanism of retrieval we make use of Entailment Bank, Dalvi et al. (2021),
which comprises science questions from school years 4 to 6, along with
relevant facts and “entailment trees”.

Question |  How might eruptions affect plants?
---|---
Associated Facts |  F1: eruptions emit lava;
F2: eruptions produce ash clouds;
F3: plants have green leaves;
F4: plant producers die without sunlight;
F5: ash clouds block sunlight.
Inference Steps |  F2 + F5 implies I1: eruptions block sunlight;
F4 + I1 implies I2: eruptions can cause plants to die.
Answer |  eruptions can cause plants to die.
Table 1: An example question from Entailment Bank and associated factoids.
Language models find it significantly easier to generate the required
entailment trees when presented with all relevant facts. However, the fact
that the retrieval of relevant facts is non-trivial motivates a frame
semantics based mechanism for indexing and search facts relevant to answering
questions.

Consider Table 1, which presents an example from the Entailment-Bank dataset.
The original task involves building an entailment tree to answer questions and
consists of three tasks at different levels of difficulty: a) Task 1 presents
the model with all relevant facts and requires only the construction of the
entailment tree; b) Task 2 requires the model to perform the same task, but
with 15 to 25 distractors included; c) Task 3 involves first extracting the
relevant facts from the entire corpus before constructing the entailment tree.
The authors find that even a relatively small model, T5-11B Raffel et al.
(2020), can perform relatively well on Tasks 1 and 2, when fine-tuned. Task 3,
they find, is much harder highlighting the importance of efficient retrieval.
We direct the reader to the results section of (Dalvi et al., 2021) for
details.

Overall, these results reinforce our earlier points: Retrieval is non-trivial
and improving retrieval has the potential to significantly boost model
performance. In the example presented above, using search terms derived just
from the question (e.g. "eruption") including more complex combinations (e.g.
“eruption and plants”) may not effectively retrieve relevant information.
Additionally, if the search terms are too broad, it can cause the retrieval of
a significant number of irrelevant facts. Both the lack of relevant facts and
a large number of unrelated facts can hinder the model’s performance.

_This work is motivated by the hypothesis that we can significantly narrow the
search space if we index facts–stored as plain text–according to the frames
they invoke and use the frames associated with the question along with the
relations between frames to retrieve relevant facts._ To test our hypothesis,
we focus our experiments on the retrieval of relevant facts.

In the above example, the FrameNet frames associated with the question could
include: a) Surviving, which captures situations requiring endurance in a
dangerous situation and includes annotations of the frame element
“Dangerous_situation”; and b) Cause_harm, which describes a situation where an
‘agent’ injures a ‘victim’. Our frame-based mechanism additionally allows the
exploitation of the relations between frames to traverse the hierarchical
interconnections between frames. This method approximates reasoning steps,
enabling the retrieval of facts that are logically connected, even if they are
not semantically similar. Within the limited scope of a single task, this work
shows a significant improvement in retrieval (recall) using frames when
compared to traditional search based baseline, and when compared to retrieval
using search terms generated by LLMs, thus verifying the feasibility of this
approach. Finally, we use the adaptability of state of the art LLMs to develop
helper-LLMs that generate sets of frames, which we then evaluate and compare
for their effectiveness in retrieving relevant information. Expanding these
methods has the potential to provide, for the first time, data-driven insights
that can be used to refine the theory of frame semantics.

##  4 Experimental Settings

In this section, we describe the experiments we conduct to test the hypothesis
that frame based indexing and search is a more effective mechanism than
keyword based indexing.

###  4.1 Task

Our choice of the specific task is motivated by the observation that LLMs can
perform reasonably well when provided with relevant facts alongside some
distractors. However, as described in the previous section, the retrieval of
these relevant facts poses a significant challenge. Therefore, we focus on the
task of retrieving relevant factoids for answering questions in Entailment
Bank. Specifically, we focus on the information extraction subtask required in
Task 3 described in Section 3. Notice that the effective retrieval of facts
would simplify Task 3 to Task 2, the task of building entailment trees given
the relevant facts and some distractors. Given how effective T5-11B, which, by
current standards consists of relatively few parameters, is on Task 2,
simplifying Task 3 to Task 2 provides a template for solving tasks based
exclusively on retrieved facts, which would in turn help with the mitigation
of factual hallucinations in LLMs. We slightly modify Task 3 by constructing
the corpus of facts that we extract from using all the facts required by any
question across the relevant data split, instead of the complete text book
corpus which is harder to process. Regardless, we evaluate frame semantic
retrieval and the baselines on exactly the same set of questions and facts to
ensure a fair comparison.

###  4.2 Empirical Evaluation Metrics

Given the nature of our task, we select Retrieval@k as our evaluation metric.
The average length of entailment trees in the Entailment Bank dataset 7.6 with
very few having more than 10. Given that Task 2 (described previously in
Section 3) includes between 15 and 25 distractors, we test our methods using
Retrieval@k for $k\in 35,40,45$. Success in this setting will demonstrate that
our retrieval mechanism can effectively simplify Task 3, which requires
retrieval from the entire corpus, into Task 2, which involves building
entailment trees based on relevant facts and a few distractors. Recall that
models perform significantly better on Task 2 than on Task 3.

###  4.3 Baselines

We use two different baselines to compare the effectiveness of frame semantic
indexing and retrieval against. The first, is a simple keyword match baseline
and is chosen due to our emphasis on interpretability and ease of correction.
Being able to understand why a system retrieves certain facts enables
refinement of the system by reassigning facts to different clusters or index
buckets. This level of transparency is not possible with more opaque methods,
such as dense vector-based retrieval. Since frame semantic retrieval
implicitly provides interpretable we choose a baseline that is similarly
transparent. We first generate search terms by feeding the relevant question
to RAKE Rose et al. (2010), a tool for extracting search terms, which is known
to be effective. We then perform a simple string match to extract all factoids
that contain the keywords.

The second baseline we use is not directly comparable as it is not
interpretable. This consists of using an LLM to generate relevant search terms
and is included so we might compare frame semantic retrieval to a rough
analogue of question decomposition, a standard mechanism used in RAG as
described above. Both baselines can be boosted using several techniques. For
example, we do not generate chain of thoughts before generating the search
terms using GPT-4, which is likely to improve the effectiveness of the
resultant search terms. This is because the purpose of this work is not create
a mechanism that outperforms existing methods, but to establish the
feasibility of the frame semantic indexing and retrieval process which has the
advantage of being interpretable and being based on cognitive linguistic
theory.

##  5 Frame Semantic Retrieval: Methods and Qualitative Analysis

Task |  Prompt |  Output Example
---|---|---
Frame Identification

Facts are indexed by the the frames they invoke
During Inference, relevant facts are extracted based on frames invoked by the question and additional frames that are related. |  What is the single/two most important frame, based on the theory of frame semantics, relevant for answering the question/fact below. Do not include frames about answering questions or reasoning, that is implied. Do not include frames which are metaphorical. Ensure the the name of the frame is as descriptive as possible. Output a single frame and join words in the frame by underscores. Output nothing but the name of the frame.
Question 1: How does the appearance of a constellation change during the
night?
Answer 1: celestial_motion
…
Problem:
Question Problem: <QUESTION>
Answer Problem: |  Input Question: Tides, such as those along the coast of Massachusetts, are caused by gravitational attractions acting on Earth. Why is the gravitational attraction of the Moon a greater factor in determining tides than the gravitational attraction of the much larger Sun?
Output Frame: gravitational_influence
Check if the new frame must be added to the frame set

Used during inference |  The following question has been tagged with the single frame listed. Is this frame significantly different from existing frames listed and should it be added as a new frame? Respond with True if it is significantly different otherwise False. Respond with True and False only.
Example Question: From Earth, the Sun appears brighter than any other star
because the Sun is the
Example Tagged Frame:proximity
Example Existing Frames 2: celestial_motion
Example Answer: True
…
Question Problem: <INPUT QUESTION>
Tagged Frame Problem: <INPUT NEW FRAME>
Existing Frames Problem: <INPUT EXISTING FRAME>
Answer Problem: |  Input Question:Melinda learned that days in some seasons have more daylight hours than in other seasons. Which season receives the most hours of sunlight in the Northern Hemisphere?
Input Frame Assigned: seasonal_variation_in_daylight
Input List of Existing Frames: daylight_variation, seasonal_adaptation,
seasonal_behavior, seasonal_change, seasonal_variation
Output (Add seasonal_variation_in_daylight to Frame Set?): False
Action Taken: Question tagged with daylight_variation
Identifying Frame Relations

Used during inference |  Listed below is a single frame relevant to a question. List those frames which are most likely to be associated with the facts required for answer this question. These frames are based on the theory of frame semantics. Do not include frames about answering questions or reasoning, that is implied. Do not include frames which are metaphorical. […]
Example Question 1: Stars are organized into patterns called constellations.
One constellation is named Leo. Which statement best explains why Leo appears
in different areas of the sky throughout the year?
Example Question Frame: celestial_motion
Example Output Frames: constellation_classification, star_classification,
celestial_motion
Problem Question : <QUESTION>
Problem Question Frame : <FRAME>
Problem Output Frames: |  Input Question: Which measurement is best expressed in light-years?
Input Question Frame: distance_in_astronomy
Output set of Frames Related to Question Frame: celestial_distance,
astronomical_unit, spatial_measurement

Table 2:  Prompts and associated outputs for each step in frame based indexing
and retrieval.

In this section we detail the methods used for frame semantic indexing and
retrieval. Given that one of objectives is to maintain interpretability and to
to potentially provide data-driven insights to the theory of frame semantics,
we perform a qualitative analysis of the outputs of each of the stages. An
empirical evaluation of the effectiveness of these methods is presented next,
in Section 6.

The mechanism of retrieving information based on frame semantics consists of
three distinct stages: The first is a pre-processing step, which involves
indexing all relevant factoids based on between two and four of the most
prominent frames that they invoke. The second and third steps are performed
when answering a question at inference time and involve identifying the single
most important frame associated with the question (the question frame), and
frames associated with the question frame, which are likely to be associated
with factoids relevant to answering the original question but separated by one
or more logical steps. The most important parts of the prompts associated with
each of these steps along with example outputs are presented in Table 2. The
complete prompts, including prior versions we experimented with, are included
in the supplementary data uploaded with this paper. In all cases, we prompt
GPT-4 OpenAI et al. (2024) using a temperature of 0 to ensure reproducible
results.

###  5.1 Frame Identification

There are two difficulties in identifying the frames associated with facts or
questions. The first is the necessity to define a complete set of frames, and
the second is the linking of these frames to the relevant fact or question.
Our exploratory experiments of using FrameNet as a definitive source of all
frames which we use to compare against facts and questions from Entailment
Bank, showed that FrameNet is inadequate for this purpose. Specifically, the
approximately 1,200 frames indexed on FrameNet have two significant
shortcomings. The first is FrameNet’s focus on ‘trigger’ words to identify
frames is problematic. This emphasis on individual trigger words, likely
influenced by the tools available at the time of FrameNet’s inception,
overlooks the fact that a sentence, as a whole, might invoke a frame that is
difficult to identify through trigger words alone, which themselves can be
challenging to extract within sentences. The second is the fact that the
frames available within FrameNet cover a limited set of domains, which overlap
minimally with the frames that are appropriate for the Entailment Bank
dataset.

To address these issues, we bootstrap the creation of frames using an LLM,
specifically GPT-4. We prompt GPT-4 to generate frames relevant to the input
fact or question, allowing us to organically expand our set of frames. We use
in-context examples, selected from the training set, to enable the model to
better output relevant frames. The complete prompts, including prior versions
we experimented with, are included in the supplementary data uploaded with
this paper. We start with an empty ‘frame set’ and iteratively generate frames
associated with facts and questions. For each fact or question, the frames
output by GPT-4 are compared with the existing frames previously generated (or
none in the initial instances). This comparison is also done with the help of
GPT-4. We first extract 5 frames, whose frame names are most semantically
similar to that of the newly generated frame. This is done using sentence BERT
Reimers and Gurevych (2019), an effective semantic similarity metric that
originally relied on BERT Devlin et al. (2019), but now makes use of custom
contextual embeddings. We then prompt GPT-4 to determine if the newly
generated frame must be added to the frame set.

As an example, GPT-4, when prompted to generate frames related to the factoid
“the gravitational pull of the sun on earth’s oceans causes the tides,” might
generate gravitational influence and tidal movement. These frames are compared
against the existing frames and the frame gravitational influence might be
replaced by the similar frame gravitational attraction already in our frame
set. If a similar frame is not found, the original frame is added to the frame
set. This same process is then used to generate frames associated with
questions. We find that this method is sub-optimal and that GPT-4 is a poor
judge of identifying frames which are truly different form those already in
the frame set. As such, we always augment the original set of frames with five
frames whose names are most semantically similar to the original. See also
Table 2 for more examples.

###  5.2 Frame Relations

Recall@ |  RAKE Search
(Baseline 1) |  GPT-4 Search
(Baseline 2) |  Frame Semantic Retrieval
(our method)
---|---|---|---
@35 |  0.330 |  0.385 |  0.439
@40 |  0.333 |  0.390 |  0.464
@45 |  0.338 |  0.396 |  0.473
Table 3:  Recall at k between 35 and 45 comparing frame semantic retrieval to
search based retrieval where the search terms are generated using a
traditional keyword based method (RAKE) and using GPT-4. It is notable that
frame semantic retrieval performs significantly better than both baselines
across all selected values of $k$.

We call the overlap between the frames invoked by a question and those invoked
by the facts necessary for answering that question a first-order frame
overlap. This first-order overlap is not sufficient to extracting all facts
that are relevant to answering a question. As such, we require a means of
identifying relations between frames, so we can expand the set of relevant
frames to include those which are related to the original, as a proxy for the
reasoning process. This is in addition to the expansion using semantic
similarity described previously.

Instead of importing definitions of frame relations, for example from
FrameNet, we generate these relations using a data-driven approach.
Specifically, we extract questions and associated facts from the training
data. We then assign frames to both the questions and the facts using the
methods described previously. The frames associated with the questions and the
corresponding facts are assumed to hold a latent relation, which we use to
generate similar frame relations at the time of answering questions. This is
done by prompting GPT-4 with the relevant question and the frame associated
with the frame and requiring GPT-4 to generate frames relevant to answering
the question. Row three of Table 2 presents the prompt and an example output
of this step.

###  5.3 Qualitative Analysis

A qualitative analysis of resultant frames and frame relations demonstrate the
surprising effectiveness of this method. Table 2 presents some of the frames
and frame relations automatically generated using the methods described above.
The results are far from perfect, but are interesting from the perspectives of
the diversity and adaptability they present. It is important to note that
these results are achieved though prompting alone. Given that LLMs, such as
GPT-4, are unlikely to be designed to solve tasks such as this, it is not
surprising that there is much room for improvement, although the results
demonstrate the feasibility of this method. Overall, we believe that
bootstrapping these methods–by first manually refining a dataset generated
through prompting, then iteratively training models specifically for this
task, and subsequently using that model to generate more data–is an effective
way to scale these methods.

##  6 Empirical Evaluation

In addition to the qualitative analysis we present an empirical evaluation of
the frame semantic retrieval methods described above. We compare the
performance of frame semantic retrieval to the two search based baselines
described in Section 4.3. We present the results in Table 3. Overall, we find
that frame semantic retrieval outperforms both the simple search based
baseline and the baseline where search terms are generated using GPT-4 by a
significant margin. Recall that we test our methods using Retrieval@k for
$k\in 35,40,45$ to take into account the fact that this allows us to
demonstrate that our retrieval mechanism can effectively simplify Task 3,
which requires retrieval from the entire corpus, into Task 2, which involves
building entailment trees based on relevant facts and a few distractors.

While our results are not perfect, frame semantic indexing and retrieval has
one significant advantage. It is the fact that each stage of the process can
be improved by fine-tuning LLMs for the specific purpose. In addition, the
transparent nature of this process, which outputs frames at each stage, allows
for the analysis and ‘debugging’ of each stage.

##  7 Conclusions and Future Work

This work presents a novel mechanism for the indexing and retrieval of facts
relevant for answering specific questions with the purpose of mitigating
factual hallucinations in LLMs. This work demonstrates the feasibility and
effectiveness of this method in both retrieval and the automatic generation of
frames which, when scaled to multiple tasks, has the potential to provide
data-driven insights to the theory of Frame Semantics.

We believe that this work provides a template for effectively integrating
cognitive linguistics and LLM research, benefiting both fields. In future
work, we intend to first create models that are specifically fine-tuned to
perform each of the tasks described above, specifically frame generation,
frame identification and frame relation identification. In addition, we indent
to extend this work to multiple tasks so as to establish its effectiveness.

## Limitations

This work presents a novel method of indexing and retrieval of facts
specifically for the purpose of ensuring that LLMs reason over retrieved, and
therefore correct facts, thereby mitigating factual hallucinations in LLMs.
Our experiments are based on a single task in a specific domain. As a proof of
concept of a novel method that is based on cognitive linguistic theory, these
experiments are effective in showcasing the feasibility of this method.
However, demonstrating the effectiveness of this method on multiple tasks is
required for a more rigorous test, which we leave to future work.
Additionally, our experiments, however, do not extend to testing LLMs for
reduced hallucinations – prior work implies that improved retrieval will
indeed lead to reduced hallucinations, but it is left to future work to
rigorously test this.

## References

  * Baker et al. (1998) Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998.  The Berkeley FrameNet project.  In _36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1_ , pages 86–90, Montreal, Quebec, Canada. Association for Computational Linguistics.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Chowdhery et al. (2023) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2023.  Palm: Scaling language modeling with pathways.  _Journal of Machine Learning Research_ , 24(240):1–113.
  * Dalvi et al. (2021) Bhavana Dalvi, Peter Jansen, Oyvind Tafjord, Zhengnan Xie, Hannah Smith, Leighanna Pipatanangkura, and Peter Clark. 2021.  Explaining answers with entailment trees.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7358–7370, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Fillmore et al. (2006) Charles J Fillmore et al. 2006.  Frame semantics.  _Cognitive linguistics: Basic readings_ , 34:373–400.
  * Gao et al. (2024) Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024.  Retrieval-augmented generation for large language models: A survey.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Comput. Surv._ , 55(12).
  * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 9459–9474. Curran Associates, Inc.
  * Lu et al. (2023) Sheng Lu, Irina Bigoulaeva, Rachneet Sachdeva, Harish Tayyar Madabushi, and Iryna Gurevych. 2023.  Are emergent abilities in large language models just in-context learning?
  * OpenAI et al. (2024) OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. 2024.  Gpt-4 technical report.
  * Patel et al. (2022) Pruthvi Patel, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2022.  Is a question decomposition unit all we need? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 4553–4569, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(1).
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence embeddings using Siamese BERT-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.
  * Rose et al. (2010) Stuart Rose, Dave Engel, Nick Cramer, and Wendy Cowley. 2010.  Automatic keyword extraction from individual documents.  _Text mining: applications and theory_ , pages 1–20.
  * Shao et al. (2023) Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023.  Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_ , pages 9248–9274, Singapore. Association for Computational Linguistics.
  * Shuster et al. (2021) Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021.  Retrieval augmentation reduces hallucination in conversation.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pages 3784–3803, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Wei et al. (2022a) Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. 2022a.  Finetuned language models are zero-shot learners.  In _International Conference on Learning Representations_.
  * Wei et al. (2022b) Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022b.  Emergent abilities of large language models.  _Transactions on Machine Learning Research_.  Survey Certification.
  * Wei et al. (2022c) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. 2022c.  Chain of thought prompting elicits reasoning in large language models.  In _Advances in Neural Information Processing Systems_.
  * Zhou et al. (2023) Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, and Ed H. Chi. 2023.  Least-to-most prompting enables complex reasoning in large language models.  In _The Eleventh International Conference on Learning Representations_.