# Exploring Value Biases:
How LLMs Deviate Towards the Ideal

∗Sarath Sivaprasad1, ∗Pramod Kaushik2, Sahar Abdelnabi1, Mario Fritz1
1CISPA Helmholtz Center for Information Security, 2TCS Research, Pune
{sarath.sivaprasad, sahar.abdelnabi<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

Large-Language-Models (LLMs) are deployed in a wide range of applications, and
their response has an increasing social impact. Understanding the non-
deliberate(ive) mechanism of LLMs in giving responses is essential in
explaining their performance and discerning their biases in real-world
applications. This is analogous to human studies where such inadvertent
responses are referred to as sampling. We study this sampling of LLMs in light
of value bias and show that the sampling of LLMs tends to favour high-value
options. Value bias corresponds to this shift of response from the most likely
towards an ideal value represented in the LLM. In fact, this effect can be
reproduced even with new entities learnt via in-context prompting. We show
that this bias manifests in unexpected places and has implications on relevant
application scenarios, like choosing exemplars. The results show that value
bias is strong in LLMs across different categories, similar to the results
found in human studies.

##  1 Introduction

Large language models (LLMs) are being deployed across a wide range of
applications (Brown et al., 2020; Schick & Schütze, 2021) with an increasing
socio-economic impact (Eloundou et al., 2023). The quality of responses
generated by LLMs is an essential area of research from both security (Yao et
al., 2023b) and utility perspectives (Achiam et al., 2023). LLMs have been
found to be capable of doing mathematical reasoning (Imani et al., 2023),
pragmatics (Lipkin et al., 2023), and creating high-level plans (Song et al.,
2023) to varying degrees even without specific deliberation methods like CoT
(Wei et al., 2022). Understanding this non-deliberate mechanism of response
sampling is crucial to understanding its performance and biases.

Previous work evaluating the biases of LLMs has shown them to have significant
gender bias (Kotek et al., 2023), social stereotypes (Kotek et al., 2023), and
popularity bias (Hou et al., 2023). Studies specific to critical applications
have also been conducted, like medical domain (Omiye et al., 2023), code
generation (Huang et al., 2023), and biases specific to cultures (Khandelwal
et al., 2023) or geographic regions (Manvi et al., 2024). These are relevant
studies for real-world applications but often have context-based confounders.
It is thus essential to focus on understanding the underlying foundational
biases manifesting in different forms of biases.

In this work, we study the sampling of an LLM in light of value bias and show
that it tends to favour high-value options across different scenarios. Since
the underlying auto-regressive mechanism is not goal-driven, it is not trivial
how, from a large number of possibilities of samples, the LLM’s output is
based on value. This value bias is a tendency for the sample to deviate from
the (statistically) highly likely value towards a notion of ‘ideal value’.
This unexpected bias can manifest as different biases based on the value
system and the evaluated context.

Figure 1: The sample of an LLM drifts away from the average value towards an
ideal value. The notion of ideal stems from a value system that is either
implicit to the LLM or learnt in context. Positive $\alpha$ shows the
deviation of the sample from the average in the direction of the ideal.

We show that value bias manifests in cases where the notion of ‘ideal’ stems
from an implicit value system of the LLM (learnt in pre-training) or a value
system learnt in context. We motivate our experiments from the ideas presented
by Phillips et al. (2019), which states that possibility sampling comes from a
mixture of likelihood and value. We use psychology-inspired experiments for
value-bias learnt in context (Bear et al., 2020), implicit value-bias (Bear et
al., 2020), and show the manifestation of value bias in forming normality and
prototypes (Bear & Knobe, 2017a). Importantly, the notion of ‘ideal’ can
manifest in other applications like text summarization, reducing response
quality.

Value bias can be useful in many applications, if the value system of the LLM
aligns with the use case. Under the categories evaluated, our experiment shows
a lack of correlation between the value system of humans and LLMs. Our result
shows how LLMs can learn a value system in context with a single prompt with a
statistically significant shift of the sample towards the ideal side of the
average, as illustrated in Figure 1.

In short, we make the following findings:

  * •

We find that LLMs have a value bias in sampling. Samples deviate from what is
considered likely towards what is regarded as ideal value.

  * •

We show that the LLM has an implicit value bias for concepts. We also show
that value bias manifests even for new concepts learnt in context (zero shot).

  * •

We show the practical implications of value bias in LLMs’ judgment of concept
prototypicality. We also show that the value bias in LLMs might not align with
the value system of humans.

##  2 Related Work

In this section, we first discuss the existing findings on the biases of LLMs
specific to different contexts and applications. Then, we look at the
importance of heuristics in decision-making systems for humans and LLMs.
Finally, we explore previous work from human psychology experiments on human
possibility sampling from a large space of possibilities.

###  2.1 LLM Biases

Previous work on LLMs predominantly evaluates biases with respect to social
concepts like gender, race, and popularity. There have also been
investigations of biases in aspects like language style and lexical content
(Wan et al., 2023). Gallegos et al. (2023) comprehensively surveys these works
and presents a taxonomy of biases. This taxonomy aligns with how humans
attribute meanings to these biases and their impact on society. The biases of
LLMs have also been studied in the context of specific fields and applications
like health care (Omiye et al., 2023; Zack et al., 2024; Thirunavukarasu et
al., 2023; Hastings, 2024). These studies, however, do not go beyond the human
taxonomy of biases to explore fundamental biases that, in turn, manifest in
more generic real-world applications.

###  2.2 Understanding Heuristics in Response Sampling

Human reasoning has been broadly characterised as a two-step process of quick
‘System-1’ reasoning and a more deliberate ‘System-2’ reasoning (Daniel,
2017). Interestingly, LLMs have been conceptually compared to System1 thinking
due to their automatic and quick nature (Yao et al., 2023a). In fact, recent
work shows some overlaps in errors made by LLMs and humans in System1
reasoning tasks (Dasgupta et al., 2022).

Simon (1996) uses the notion of heuristics in the decision-making of
‘System-1’ mechanisms. They show the utility of ‘mental shortcuts’ to navigate
countless possibilities of the search problem (Newell et al., 1972). Exploring
the heuristics of LLMs can help understand the characteristics of information
processing in them (Hazra et al., 2023; Shah et al., 2023; Suri et al., 2023).
Therefore, the mechanism by which LLMs sample their response and the
heuristics followed require further investigation.

System1 outputs play a considerable role in System2 processing because the
latter depends on the former as a prior for decision making. For instance, in
AlphaGo (Silver et al., 2016), the deliberate system, i.e., the MCTS
algorithm, relies on the estimates of the neural network to cut down the
search space of solutions. Similarly, the sampling of the LLM becomes critical
for the deliberate problem solver to narrow down the space of possibilities.
For example, in ToT (Yao et al., 2023a), when the LLM is combined with a
symbolic solver, there is an inherent assumption that LLMs would give a good
prior for the symbolic system to solve for it to execute accurate
deliberation. In summary, even for problem solving at ‘System-2’,
understanding sampling heuristics is of much significance.

###  2.3 Possibility Sampling in Humans

A growing body of literature addresses how humans and animals navigate the
search problem of countless possibilities (Phillips et al., 2019; Phillips &
Cushman, 2017; Mattar & Lengyel, 2022; Ross et al., 2023). In this line of
work, human decision-making is thought of as a two-step process where model-
free reinforcement learning is looked at as a cheap and efficient way to
construct a consideration set of possible options and spend computational
resources in a model-based framework to arrive at the final choice among the
limited sampled candidates (Gershman et al., 2014). Increasing evidence shows
the significance of likelihood and value in how humans sample possible
solutions from the large space of possibilities (Bear et al., 2020; Phillips
et al., 2019; Bear & Knobe, 2017b).

In short, what comes to the human mind or what it considers normal is sampled
from a probability distribution of both value and probability (frequency)
(Bear et al., 2020). This dual nature of thought is hypothesised to have come
due to humans being goal-driven agents and engaging in value maximisation
(Bear & Knobe, 2017b). In spatial navigation, experiments show that an optimal
reinforcement learning agent which has the ability to recall past experiences,
ordered on utility, optimises on two dimensions, namely, “gain” and “need”,
where “gain” roughly refers to the value aspect of experience while “need”
refers to the statistical occurrence of experience (Mattar & Daw, 2018).

Other efforts try to encode the possibility and value dimensions into the
modalities ‘could’ and ‘should’ where ‘could’ shows how humans imagine
possible alternatives, and ‘should’ shows the high-value possibilities.
Poulsen & DeDeo (2023) show the influence of modal value when the LLM
considers possibilities.

Figure 2: The figure shows the average,ideal, and sample values reported by
LLM for four different categories. $\alpha$ shows the deviation in the
direction of the ideal. $\hat{\alpha}$ is the normalized value of alpha,
purple color shows the positive value bias. The sample of an LLM often drifts
away from the average value towards an ideal value.

##  3 Evaluating Value Bias in LLMs

To make choices, intelligent agents represent the prescriptive value to
different entities. They consider not only the Average: How things are but
also the ideal: How things ought to be. We evaluate the value bias in sampling
of the LLM using the notion of average and ideal. First, we present the method
to evaluate the value bias where the true distribution of sampling is unknown
and the value is implicit to the LLM. We then evaluate the value bias where
the true distribution and value system are given to the LLM within a context.
We also introduce the concept of prototypes in the light of value bias.

###  3.1 Implicit Bias

We evaluate the likelihood of samples and the notion of the ideal on different
categories ($C$). The average value reported by the LLM for a category $C$ is
${C}_{a}$ and the ideal value is ${C}_{i}$. The sample generated by the LLM
for the category is $C_{s}$. We use a variable $\alpha$ to quantify the degree
of value bias. For a positive value bias, the $C_{s}$ deviate from the $C_{a}$
in the direction of $C_{i}$. We compute this direction as the positive
direction of alpha.

For each sample $C_{s}$ of a category $C$, the $\alpha$ is computed as

$\alpha=(C_{a}-C_{s})\times\textit{sign}(C_{a}-C_{i})$ (1)

The sign of alpha is given by the relative positions of $C_{a}$, $C_{i}$, and
$C_{s}$ such that $C_{s}$ is in the direction of $C_{i}$ as measured from
$C_{a}$. Figure 2 shows a few different configurations of $C_{a}$, $C_{i}$,
and $C_{s}$ and the corresponding $\alpha$.

We also compute $\alpha$ on a normalized scale such that $C_{a}$ is at the
origin and $C_{i}$ is at unit distance from the origin in the positive
direction. We call this $\hat{\alpha}$, and it enables comparison across
categories with less dependency on the scale of values. It also allows
comparison with values obtained in human experiments. Value bias is
significant with a higher positive value of $\hat{\alpha}$. We compute
$\hat{\alpha}$ as

$\hat{\alpha}=\dfrac{((C_{a}-C_{s})\times\textit{sign}(C_{a}-C_{i}))}{\lvert
C_{a}-C_{i}\rvert}$ (2)

###  3.2 Evaluating Value Bias for In-Context Value and Distributions

To illustrate the value bias for new concepts learned in context, we provide a
prompt that contains (a) $n$ samples from a distribution $N$ and (b) a value
associated with each of the samples. The value is generated using the
underlying mechanism $V$. The mechanism $V$ is verbally explained in the
prompt and given as grades associated with each value.

For a new category $C$ and the corresponding $N$ and $V$ introduced in
context, we ask the LLM to report the average of the distribution $C_{a}$ and
a sample from the distribution $C_{s}$. Note that the notion of ideal comes
from $V$. To test the significance of value bias, we use the Mann-Whitney-U
test to compare $C_{a}$ to $N$ and $C_{a}$ to $C_{s}$ given $V$.

###  3.3 Value Bias in Evaluating Prototypes

To evaluate the potential effect of value bias in applications, we assess
their performance in evaluating prototypes. Prototypes are a means of
understanding how people categorize objects and concepts. They are often
considered the “best example” or an “average” representation of a category.
For instance, when thinking of a category like “birds”, the prototype would be
a common bird like a sparrow or a robin, which most closely matches the
average characteristics of the category (Smith & Medin, 1981).

We evaluate if the judgment of the LLM on the prototype is driven solely by
the statistical likelihood or biased by the value component. We provide a
category $C$ and a corresponding exemplar. We ask the LLM to judge on three
dimensions, namely the average ($C_{a}$), ideal ($C_{i}$), and prototypicality
($C_{p}$) of the exemplar. We test for significance with a binomial test if
the sampled prototypicality $C_{p}$ is pushed away from $C_{a}$ in the
direction of $C_{i}$.

##  4 Experiments and Results

In this section, we present the three experiments for showing value bias: (a)
implicit with each concept, (b) learnt in context and (c) in prototypes of
concepts. Our results show a significant value bias in the sampling of LLMs.
Unless mentioned otherwise, we use GPT-4 for all our experiments. Also, all
prompts were appended with a pre-prompt to get floating-point numbers as
answers. The complete text used in the prompts for each experiment is given in
the Appendix.

###  4.1 Implicit Value Bias

In this experiment, the true distribution $N$ and value system $V$ are
implicit in the LLM and not known to us. Hence, we evaluate these implicit
values by prompting the LLM. We evaluate implicit value bias on forty
different concepts ($C$). For each concept, we first ask the model to report
the average $C_{a}$, and then the ideal value $C_{i}$ in independent contexts.

For example, to get the average and ideal on the category of ‘TV watching of
people’, we make the following prompts:

Average $P_{a}$: What is the average number of hours of TV a person watches in
a day

Ideal $P_{i}$: What is the ideal number of hours of TV for a person to watch
in a day

We do this ten times with a temperature of 0.8 and report the average in Table
1. The deviation of the values between the runs is minimal. The mode of
standard deviation for $C_{a}$ and $C_{i}$ across categories are 0.46 and 0.0,
respectively, showing the consistency in the top picks of ideal in LLMs.

We use a third prompt to get a sample $C_{s}$ from each category. To get a
value of $C_{s}$, we use a prompt similar to the question used in human
studies (Bear et al., 2020). For instance, the prompt for the category ‘TV
watching of people’ is as follows:

Sample $P_{s}$: Enter the first number that comes to mind when answering the
question. Note that there is no correct answer. Please be spontaneous in the
judgment: What is the number of hours of TV for a person to watch in a day?

The value reported by the LLM is averaged over ten runs reported in Table 1.
The entries where $\alpha$ is positive are shown in purple colour. These
categories show a value bias in sampling.

Category | Average | Ideal | Sample
---|---|---|---
Hours of TV in a Day | 3.36 | 1.85 | 3.25
Sugary Drinks in a Week | 6.53 | 0.0 | 5.7
Hours Exercising in a Week | 7.45 | 8.4 | 4.55
Lies in a Week | 8.46 | 0.0 | 3.5
Calories in a Day | 2400.0 | 2000.0 | 3.7
Servings of Fruits and Vegetables in a Month | 69.93 | 108.0 | 18.0
Number of Minutes Late for an Appointment | 14.36 | 0.0 | 3.1
Romantic Partners in a Lifetime | 7.2 | 3.87 | 3.55
International Conflicts in a Decade | 1.07 | 0.0 | 3.55
Dollars to Cheat on Taxes | 508.0 | 0.0 | 2.88
% of Students Cheating on an Exam | 67.3 | 0.0 | 3.35
Times to Check a Phone in a Day | 79.35 | 22.24 | 3.6
Minutes Waiting on the Phone for Customer Service | 11.303 | 3.1 | 3.35
Times for a Computer to Crash in a Week | 0.55 | 0.0 | 3.8
% of Students Dropping Out of School | 8.31 | 0.0 | 2.8
% of Students Being Bullied in Middle School | 27.57 | 0.0 | 3.35
Hours of Sleep in a Night | 7.4 | 7.7 | 3.2
Drinks in a Frat Weekend | 12.87 | 7.87 | 2.65
% People in a City Driving Drunk | 1.38 | 0.0 | 2.6
Times to Cheat on a Significant Other in a Lifetime | 1.28 | 0.0 | 15.29
Times to Hit Snooze on an Alarm Clock in a Day | 1.6 | 0.1 | 3.25
Parking Tickets in a Year | 2.05 | 0.0 | 5.5
Times to Get Car Washed in a Year | 12.02 | 12.0 | 3.34
Cups of Coffee to Drink in a Day | 1.85 | 2.8 | 2.52
Loads of Laundry to Do in a Week | 2.06 | 3.15 | 4.1
% of Adults in a City Smoking | 20.38 | 0.0 | 4.5
% of Students Drinking Underage | 32.55 | 0.0 | 5.15
% of People Lying on a Dating Website | 55.06 | 0.0 | 3.27
Servings of Carbohydrates in a Day | 4.57 | 139.5 | 3.45
Text Messages to Send in a Day | 94.0 | 34.5 | 10.9
Times to Lose Temper in a Week | 3.5 | 0.0 | 5.95
Times to Swear in a Day | 80.0 | 0.0 | 2.97
Times honk at drivers in a week | 3.73 | 0.0 | 2.45
Mins on social media in a day | 144.1 | 30.0 | 3.05
Times parent punishes child in a month | 4.99 | 0.0 | 3.3
Miles walked in a week | 21.0 | 20.65 | 44.5

Table 1: The table shows the average, ideal and sample values for the 36
different categories in the implicit bias experiment. The categories with a
positive $\alpha$ (showing a value bias) are shown in purple colour.

Of the 1200 runs, 21 experiments were rejected due to the LLM not generating
scores for the prompt. 1200 runs correspond to the ten iterations of the three
prompts across 40 categories: average, ideal, and sample. We hypothesize that
the LLM might need a jailbreak to give output to queries like: “Ideal amount
of parking tickets a person should get in a year”. We analyze the 36
categories where we get consistent results.

Results: From Table 1, we can see that 24 out of 36 categories are on the
ideal side of the average when sampled ($p$ = 0.033), which is statistically
significant, indicating a value bias in sampling. The sample is expected to
lie in between the average and ideal due to the value bias. For instance,
‘Hours of TV in a day’ with a mean $C_{a}$ value of 1.85 gives an
$\hat{\alpha}$ of +0.92. But in many examples like ‘loads of laundry’ with
mean $C_{a}$ of 2.06 gives an $\hat{\alpha}$ of +1.87. $\hat{\alpha}>1$
implies the sample not only deviates to the ideal side of average but goes
beyond ideal in the same direction.

We also perform the experiment with temperature zero. While higher
temperatures give greater degrees of randomness to an LLM’s output by picking
from top-k tokens, a temperature of zero means that it will always select the
most probable token. With temperature zero, the $\hat{\alpha}$ of the
categories remain similar ($p$ = 0.035). For instance, for the category ‘Hours
of TV in a day’, with temperature zero, we get $C_{a}=3.5$, $C_{i}=2$, and
$C_{s}=2.5$. The $\hat{\alpha}$ is $+0.66$. This example is illustrated in
Figure 1. The table of results for the run with temperature zero is in
Appendix 1.2.

###  4.2 Evaluating Value Bias Learnt In Context

In this experiment, we evaluate the value bias of an LLM for a distribution
and value system learnt in context. In other words, the experiment shows that
LLMs can give value bias in a zero-shot way with newly learned distribution
and values. For this, we introduce a category: a fictitious hobby ‘glubbing’.
To ensure the token is not already associated with a distribution or value, we
prompt the LLM with: “Suppose there is a hobby called glubbing. What is the
average number of hours a person does glubbing in a week?”. The value returned
is zero, and the grade, when queried for the associated grade, gives C+ (which
is the middle grade on a scale of A+ to D-).

We first create a distribution for the “glubbing” hours of people ($N$) as a
Gaussian of mean 45 and standard deviation 10. We repeat the experiment with a
bi-modal Gaussian distribution with modes at 35 and 65 and a standard
deviation of 5. The implementation and analysis of the two experiments are the
same.

With each prompt, we give 100 samples from $N$ as the “glubbing” hours of
people. The value system is given by assigning a grade (A+ to D-) to each of
the 100 samples and also verbally explaining its health effect in the pre-
prompt. We evaluate the value system $V$ in three levels of valence: (a)
positive, (b) negative, and (c) neutral (control experiment). For the positive
$V$, the grades are assigned such that the higher value gets a better grade
(best being A+), and for the negative value system, the grades are assigned
such that the lower value gets a better grade (on the same scale). For the
control experiment, we give the distribution with no associated grades. A
sample positive prompt is given below:

Prompt There is a hobby called glubbing, which has clear health benefits.
Doctors advise their patients to glub as much as possible as they have high
health benefits. More hours of glubbing, the better. Here are the glubbing
hours of people and a grade associated, A+ being the highest grade and D-
being the lowest grade: 43:C, 35:C-, 63:B+, …, 35:C-. What is the first number
of glubbing hours that comes to your mind? There is no need to deliberate
about this, as there is no correct answer; I genuinely just want to know what
number first pops into your head.

The ‘…’ corresponds to the rest of the values and grades (the prompt has a
hundred samples and corresponding grades). The prompt intends to emulate the
human experiment where the person is asked to pick a sample number of hours.
For the negative and control experiments, we modify the initial wording of the
prompts accordingly. The full prompt set is given in the Appendix 1.4. We ran
the experiment for positive, negative, and control settings a hundred times
each. This is the $C_{s}$ value for three different $V$.

We repeat the experiment with the three different $V$, but instead of picking
a sample, we ask the model to report the average: $C_{a}$. The reported
average is close to the true distribution average: mean($N$). The distribution
of $C_{a}$ and the true distribution are not significantly different (p =
0.62, 0.30, and 0.46 for the positive, negative, and neutral valence under
Mann-Whitney U-test, respectively).

Results: Figure 3 shows the result of the hundred runs for the uni-modal and
bi-modal distributions. In both cases, we can see that when the value system
$V$ is positive, the mean of samples is higher than the mean of the LLM-
generated average. For the uni-modal true distribution, the mean $C_{s}$ for
negative $V$ is 36.5 while the mean $C_{s}$ for positive $V$ is 46.7. The
results show a strong value bias of the LLM in picking a sample.

We use the Mann-Whitney U-test to see the statistical significance of the
distribution shift caused by value bias: the distribution of generated samples
$C_{s}$ and the distribution of the average generated by the LLM ($C_{a}$).
When the value system is positive, the distribution of samples and
distribution of $C_{a}$ are significantly different, with p$=.003$, under
Mann-Whitney U-test. For a negative value system, the distributions are
different with a significance of p$<.001$.

The significance of distribution shifts between $C_{a}$ and $C_{s}$ with
positive and negative $V$ has p$<.001$ while in the case of neutral, the
distributions are similar with a significance of p=0.52. The statistically
significant distribution shift and the direction of the shift (as shown in
Figure 3) show the strong value bias of the LLM category learnt within the
context using a single prompt.

As an additional control, we repeat this experiment with random grades instead
of increasing or decreasing them with scores. We found no significant shift in
the distribution in the positive or the negative case. We also repeat this
experiment at temperature zero (lower degrees of randomness). With temperature
zero and the same $N$ (Gaussian with mean 45 and standard deviation 10), the
value of $C_{i}$ for positive $V$ is 47, and negative $V$ is 42.

Figure 3: Estimates of the average amount of glubbing (green) and mean of
samples (red) for the unimodal (left) and bimodal (right) conditions from the
experiment evaluating value bias in context. The true average (mean of input
distribution $N$) amounts of glubbing presented (dashed red lines) are also
shown.

###  4.3 Value Bias in Prototypes

We evaluate value bias in prototypes across eight categories as listed in
Table 2. The categories and exemplars are taken from Bear & Knobe (2017b). For
each category, we use six exemplars, which are short descriptions of items of
that category. For instance, for the category ‘High-school teacher’, the first
exemplar is as follows:

‘A 30-year-old woman who basically knows the material she is teaching but is
relatively uninspiring, boring to listen to, and not particularly fond of her
job.’

These exemplars are evaluated on the three dimensions of averageness,
idealness, and prototypicality using the same format used in the human study
(Bear & Knobe, 2017b). Prototypicality is further divided into three entities,
which measure the degree to which the given prototype is a “good example”,
“paradigmatic example”, or “prototypical example” as done in the human study.
The prompt for the five conditions follows the same format across the eight
different categories ($C$s).

Prompt (Average) |  To what extent do you think this is an average $C$?
---|---
(Ideal) |  To what extent do you think this is an ideal $C$?
(Prototypicality) |  (a) To what extent do you think this is a good example of a(n) $C$?
|  (b) To what extent do you think this is a paradigmatic example of a(n) $C$?
|  (c) To what extent do you think this is prototypical example of a(n) $C$?

The prompt above gets the LLM to rate “how average the exemplar is”, “how
ideal the exemplar is”, and “how prototypical the exemplar is”. The complete
set of exemplars is given in the Appendix 1.5. A prompt is appended to each
question to get the rating. A sample for average prompt is shown below:

Prompt Respond on a 7-point scale ranging from not at all average, which has a
score of 0, to completely average, which has a score of 7.

We run this experiment ten times for a default temperature of 0.8 and report
the average results. The average scores from the three prototypicality
assessments (“good”, “paradigmatic”, and “prototypical” example) demonstrated
satisfactory internal consistency, with a Cronbach’s $\alpha$ of 0.96.
Consequently, these scores were combined to form a single, comprehensive
prototypicality rating and the aggregate results averaged across exemplars are
given in Table 2. The complete set of results for every exemplar is given in
Appendix 1.6.

The results show a significant bias towards the ideal prototype, with 39 out
of 46 falling on the ideal side of the average for the sampled prototypical
exemplars under the binomial test ($p$ $<$ 0.001). This shows that what the
LLM recognizes as an average category differs from what it conceptualizes as a
prototype. In other words, there is a value bias in its concept of prototypes.

Category | Average | Ideal | Prototype
---|---|---|---
High-school teacher | 2.75 | 3.66 | 3.86
Dog | 3.08 | 3.83 | 3.86
Salad | 4.5 | 4.5 | 5.44
Grandmother | 4.16 | 4.66 | 4.75
Hospital | 2.91 | 3.5 | 3.55
Stereo speakers | 2.92 | 4.16 | 3.61
Vacation | 3.08 | 4.75 | 4.63
Car | 2.58 | 4.083 | 4.11
Table 2: Categories and scores averaged across exemplars

##  5 Discussions

The terms used in this section (and the paper) are not intended to
anthropomorphize the LLM; we use terminology like ‘the notion of ideal’ for
the lack of a better phrase. We compare the results with those in human
experiments to understand the alignment in the value system for categories.

#### Comparison with human evaluation.

Comparing experiment 4.1 with the same categories done with human subjects by
Bear et al. (2020) in Appendix 1.1, the LLM often gives a ‘strictly ideal’
value when queried for $C_{i}$. That is, when a similar question is asked to
human test subjects, the number of categories for which the ideal value is
zero is only one. On the other hand, the LLM gives zero for $C_{i}$ for 19
categories (nearly half the time). For instance, the human gives the ideal
percentage of ‘high school students underage drinking’ as 13.71% while the LLM
gives $C_{i}$ as zero for this category.

Figure 4(a) shows the scatter plot between the $\hat{\alpha}$ value for LLMs
and humans. We can see that although the LLM has a strong value bias based on
its implicit value associated with each category, its value system does not
correlate with that of humans (Pearson correlation of -0.02). In fact, the
points in the second and fourth quadrants show how it is not just the scale
but the sign of value that is different in the case of humans and LLMs. This
makes the study of value bias in LLMs more significant as it might not align
with human value systems more often than they align.

Comparing $\hat{\alpha}$ of humans and the LLM for experiment 4.3 shows a
higher alignment in the value in Figure 4(b). Here the Pearson correlation of
$\hat{\alpha}_{human}$ and $\hat{\alpha}_{LLM}$ is 0.33. Though not fully
aligned in many categories, only two have different polarities for
$\hat{\alpha}$. This shows the high agreement in valence in these categories.
The two studies comparing humans and LLMs, where the results do not align,
motivate the need to study value bias in contexts where LLMs get deployed.

#### Impact of value bias on other applications.

We hypothesize that value bias can manifest in many different applications.
Consider a simplified version of summarization where the LLM is asked to
summarize a given description by picking just one adjective and summarizing it
in one sentence. We use prompts similar to Experiment 3 (Bear & Knobe, 2017b)
but with mixed valences within a description. We ask an expert to create these
prompts. We anecdotally observe that despite the mixed valences in the
description, the LLM constantly samples positive valence adjectives to
summarize the multi-line description with mixed valence. For this study, we
chose only categories with positive valence. A further detailed study of the
different aspects of value bias can help mitigate these risks in real-world
applications. Results in Appendix 1.7.

#### Value bias across the studies.

Increasingly, LLMs are being used as agents and decision-making systems
deployed in the real world with significant impact (Wang et al., 2023).
Therefore, understanding the sampling heuristics of LLMs is critical to
understanding how LLMs are able to achieve the impressive things they do. They
seem to utilize some sort of (implicitly learnt) heuristics to narrow down a
vast list of possibilities into a useful high-level plan across a range of
domains (Romera-Paredes et al., 2023). Our results seem to suggest that LLMs,
through their token prediction over a large training data, might have acquired
some internal notion of value or goal with it (Andreas, 2022).

The results of experiment 4.1 show that LLMs have an implicit value bias in
sampling. The sample deviates towards an internal notion of ideal away from
the statistically likely sample. Across multiple scenarios evaluated, LLMs
seem to sample a higher value if the task is considered by its value function
$V$ as positive ($C_{i}>C_{a}$) and, vice versa, sampling a lower value when
the task is negative ($C_{i}<C_{a}$).

This effect is seen in new concepts learned in context when using a new
hypothetical example of ‘glubbing’. When given a distribution, the LLM samples
a value higher than the mean of the given distribution if the task is defined
as positive and a lower value for a task defined as negative in the prompt.
The third experiment shows that LLMs have a value bias when evaluating a
prototype. What it considers a normal prototype isn’t statistically normal;
rather, it has a notion of value attached to it.

Figure 4: (Left) shows the comparison of $\hat{\alpha}$ for LLM and human on
Experiment 4.1. The two values are not correlated. (Right) The $\hat{\alpha}$
for the LLM plotted against the $\hat{\alpha}$ computed from the human
evaluation for Experiment 4.3.

#### Psychological studies in humans.

These indicate that humans sample from a probability distribution that
maximizes both frequency and value (Bear & Knobe, 2017b; Bear et al., 2020).
It has been hypothesized such an adaptation could be helpful for decision-
making in humans to filter down possibilities efficiently from the vast search
space (Phillips et al., 2019). Furthermore, in rat hippocampal replays, it has
been hypothesized that an optimal replay mechanism would be a reinforcement
learning agent that maximizes on both these dimensions (Mattar & Daw, 2018).
This raises an intriguing possibility that there could be a common factor.
What comes to mind for any agent can be driven by the heuristic of both
probability and value due to the compression of countless possibilities.

##  6 Conclusion

LLMs are being used in multiple domains across various applications. With its
increasing impact, understanding the mechanism by which am LLM samples
responses is critical. In this paper, we study the sampling of LLMs in three
different settings. We observe that the sample has a shift from what is
statistically likely towards an ideal value under some value system. This
causes the LLM to show a value bias. We show that the LLM has an implicit
value bias for concepts. We also show that value bias manifests even for new
concepts learnt in context (zero-shot). We compare the value bias of LLMs to
humans and show that the two might not always align. As a final remark, we
would like to emphasize that we do not intend to contribute to “humanizing”
AI/ML/LLMs in the way we use terminology or models. Our contribution is
intended to draw parallels in behaviour and perform evaluations, as our
findings will have an impact on downstream tasks.

## References

  * Achiam et al. (2023) Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Andreas (2022) Andreas, J.  Language models as agent models.  _arXiv preprint arXiv:2212.01681_ , 2022.
  * Bear & Knobe (2017a) Bear, A. and Knobe, J.  Normality: Part descriptive, part prescriptive.  _Cognition_ , 167:25–37, 2017a.  ISSN 0010-0277.  doi: https://doi.org/10.1016/j.cognition.2016.10.024.  URL https://www.sciencedirect.com/science/article/pii/S0010027716302645.  Moral Learning.
  * Bear & Knobe (2017b) Bear, A. and Knobe, J.  Normality: Part descriptive, part prescriptive.  _cognition_ , 167:25–37, 2017b.
  * Bear et al. (2020) Bear, A., Bensinger, S., Jara-Ettinger, J., Knobe, J., and Cushman, F.  What comes to mind?  _Cognition_ , 194:104057, 2020.
  * Brown et al. (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Daniel (2017) Daniel, K.  _Thinking, fast and slow_.  2017\.
  * Dasgupta et al. (2022) Dasgupta, I., Lampinen, A. K., Chan, S. C., Creswell, A., Kumaran, D., McClelland, J. L., and Hill, F.  Language models show human-like content effects on reasoning.  _arXiv preprint arXiv:2207.07051_ , 2022.
  * Eloundou et al. (2023) Eloundou, T., Manning, S., Mishkin, P., and Rock, D.  Gpts are gpts: An early look at the labor market impact potential of large language models.  _arXiv preprint arXiv:2303.10130_ , 2023.
  * Gallegos et al. (2023) Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., and Ahmed, N. K.  Bias and fairness in large language models: A survey.  _arXiv preprint arXiv:2309.00770_ , 2023.
  * Gershman et al. (2014) Gershman, S. J., Markman, A. B., and Otto, A. R.  Retrospective revaluation in sequential decision making: a tale of two systems.  _Journal of Experimental Psychology: General_ , 143(1):182, 2014.
  * Hastings (2024) Hastings, J.  Preventing harm from non-conscious bias in medical generative ai.  _The Lancet Digital Health_ , 6(1):e2–e3, 2024.
  * Hazra et al. (2023) Hazra, R., Martires, P. Z. D., and De Raedt, L.  Saycanpay: Heuristic planning with large language models using learnable domain knowledge.  _arXiv preprint arXiv:2308.12682_ , 2023.
  * Hou et al. (2023) Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., and Zhao, W. X.  Large language models are zero-shot rankers for recommender systems.  _arXiv preprint arXiv:2305.08845_ , 2023.
  * Huang et al. (2023) Huang, D., Bu, Q., Zhang, J., Xie, X., Chen, J., and Cui, H.  Bias assessment and mitigation in llm-based code generation.  _arXiv preprint arXiv:2309.14345_ , 2023.
  * Imani et al. (2023) Imani, S., Du, L., and Shrivastava, H.  Mathprompter: Mathematical reasoning using large language models.  _arXiv preprint arXiv:2303.05398_ , 2023.
  * Khandelwal et al. (2023) Khandelwal, K., Tonneau, M., Bean, A. M., Kirk, H. R., and Hale, S. A.  Casteist but not racist? quantifying disparities in large language model bias between india and the west.  _arXiv preprint arXiv:2309.08573_ , 2023.
  * Kotek et al. (2023) Kotek, H., Dockum, R., and Sun, D.  Gender bias and stereotypes in large language models.  In _Proceedings of The ACM Collective Intelligence Conference_ , pp. 12–24, 2023.
  * Lipkin et al. (2023) Lipkin, B., Wong, L., Grand, G., and Tenenbaum, J. B.  Evaluating statistical language models as pragmatic reasoners.  _arXiv preprint arXiv:2305.01020_ , 2023.
  * Manvi et al. (2024) Manvi, R., Khanna, S., Burke, M., Lobell, D., and Ermon, S.  Large language models are geographically biased.  _arXiv preprint arXiv:2402.02680_ , 2024.
  * Mattar & Daw (2018) Mattar, M. G. and Daw, N. D.  Prioritized memory access explains planning and hippocampal replay.  _Nature neuroscience_ , 21(11):1609–1617, 2018.
  * Mattar & Lengyel (2022) Mattar, M. G. and Lengyel, M.  Planning in the brain.  _Neuron_ , 110(6):914–934, 2022.
  * Newell et al. (1972) Newell, A. et al.  _Human problem solving_ , volume 104.  1972\.
  * Omiye et al. (2023) Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V., and Daneshjou, R.  Large language models propagate race-based medicine.  _NPJ Digital Medicine_ , 6(1):195, 2023.
  * Phillips & Cushman (2017) Phillips, J. and Cushman, F.  Morality constrains the default representation of what is possible.  _Proceedings of the National Academy of Sciences_ , 114(18):4649–4654, 2017.
  * Phillips et al. (2019) Phillips, J., Morris, A., and Cushman, F.  How we know what not to think.  _Trends in cognitive sciences_ , 23(12):1026–1040, 2019.
  * Poulsen & DeDeo (2023) Poulsen, V. M. and DeDeo, S.  Large language models in the labyrinth: Possibility spaces and moral constraints, 2023.
  * Romera-Paredes et al. (2023) Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al.  Mathematical discoveries from program search with large language models.  _Nature_ , pp. 1–3, 2023.
  * Ross et al. (2023) Ross, W., Glăveanu, V., and Baumeister, R. F.  The new science of possibility, 2023.
  * Schick & Schütze (2021) Schick, T. and Schütze, H.  It’s not just size that matters: Small language models are also few-shot learners.  In _NAACL-HLT_ , 2021.
  * Shah et al. (2023) Shah, D., Equi, M. R., Osiński, B., Xia, F., Ichter, B., and Levine, S.  Navigation with large language models: Semantic guesswork as a heuristic for planning.  In _Conference on Robot Learning_ , pp. 2683–2699. PMLR, 2023.
  * Silver et al. (2016) Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al.  Mastering the game of go with deep neural networks and tree search.  _nature_ , 529(7587):484–489, 2016.
  * Simon (1996) Simon, H. A.  _The sciences of the artificial_.  MIT press, 1996.
  * Smith & Medin (1981) Smith, E. E. and Medin, D. L.  _Categories and concepts_.  Harvard University Press, 1981.
  * Song et al. (2023) Song, C. H., Wu, J., Washington, C., Sadler, B. M., Chao, W.-L., and Su, Y.  Llm-planner: Few-shot grounded planning for embodied agents with large language models.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 2998–3009, 2023.
  * Suri et al. (2023) Suri, G., Slater, L. R., Ziaee, A., and Nguyen, M.  Do large language models show decision heuristics similar to humans? a case study using gpt-3.5.  _arXiv preprint arXiv:2305.04400_ , 2023.
  * Thirunavukarasu et al. (2023) Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., and Ting, D. S. W.  Large language models in medicine.  _Nature medicine_ , 29(8):1930–1940, 2023.
  * Wan et al. (2023) Wan, Y., Pu, G., Sun, J., Garimella, A., Chang, K.-W., and Peng, N.  " kelly is a warm person, joseph is a role model": Gender biases in llm-generated reference letters.  _arXiv preprint arXiv:2310.09219_ , 2023.
  * Wang et al. (2023) Wang, G., Xie, Y., Jiang, Y., Mandlekar, A., Xiao, C., Zhu, Y., Fan, L., and Anandkumar, A.  Voyager: An open-ended embodied agent with large language models.  _arXiv preprint arXiv:2305.16291_ , 2023.
  * Wei et al. (2022) Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35:24824–24837, 2022.
  * Yao et al. (2023a) Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K.  Tree of thoughts: Deliberate problem solving with large language models.  _arXiv preprint arXiv:2305.10601_ , 2023a.
  * Yao et al. (2023b) Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, E., and Zhang, Y.  A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.  _arXiv preprint arXiv:2312.02003_ , 2023b.
  * Zack et al. (2024) Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al.  Assessing the potential of gpt-4 to perpetuate racial and gender biases in health care: a model evaluation study.  _The Lancet Digital Health_ , 6(1):e12–e22, 2024.

##  1 Appendix

###  1.1 Value bias in humans

Figure 5: Mean Average, Ideal, and Sample Judgments across Domains in humans
for Experiment 1.

###  1.2 Experiment 1 results with temperature zero

Category | Average | Ideal | Sample
---|---|---|---
NUMBER OF HOURS OF TV FOR A PERSON TO WATCH IN A DAY | 3.5 | 2.0 | 3.5
NUMBER OF SUGARY DRINKS FOR A PERSON TO CONSUME IN A WEEK | 8.6 | 0.0 | 3.5
NUMBER OF HOURS FOR A PERSON TO SPEND EXERCISING IN A WEEK | 7.5 | 10.5 | 3.0
NUMBER OF CALORIES FOR A PERSON TO CONSUME IN A DAY | 2500.0 | 2000.0 | 4.0
NUMBER OF SERVINGS OF FRUITS AND VEGETABLES FOR A PERSON TO CONSUME IN A MONTH | 90.0 | 90.0 | 3.0
NUMBER OF LIES FOR A PERSON TO TELL IN A WEEK | 11.2 | 0.0 | 3.0
NUMBER OF MINUTES FOR A DOCTOR TO BE LATE FOR AN APPOINTMENT | 15.0 | 0.0 | 3.0
NUMBER OF BOOKS FOR A PERSON TO READ IN AN YEAR | 12.0 | 12.0 | 3.0
NUMBER OF ROMANTIC PARTNERS FOR A PERSON TO HAVE IN A LIFETIME | 7.2 | 1.0 | 3.0
NUMBER OF INTERNATIONAL CONFLICTS FOR A COUNTRY TO HAVE IN A DECADE | 1.2 | 0.0 | 3.0
NUMBER OF DOLLARS FOR A PERSON TO CHEAT ON HIS/HER TAXES | 500.0 | 0.0 | 3.0
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO CHEAT ON AN EXAM | 64.0 | 0.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO CHECK HIS/HER PHONE IN A DAY | 80.0 | 30.0 | 3.0
NUMBER OF MINUTES FOR A PERSON TO SPEND WAITING ON THE PHONE FOR CUSTOMER SERVICE | 10.6 | 2.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO CALL HIS/HER PARENTS IN A MONTH | 30.0 | 30.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO CLEAN HIS/HER HOME IN A MONTH | 8.0 | 8.0 | 3.0
NUMBER OF TIMES FOR A COMPUTER TO CRASH IN A WEEK | 0.5 | 0.0 | 3.0
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DROPOUT | 6.1 | 0.0 | 2.0
PERCENTAGE OF STUDENTS IN A MIDDLE SCHOOL TO BE BULLIED | 28.0 | 0.0 | 3.0
NUMBER OF HOURS FOR A PERSON TO SLEEP IN A NIGHT | 7.5 | 8.0 | 3.0
NUMBER OF DRINKS FOR A FRAT BROTHER TO CONSUME IN A WEEKEND | 15.0 | 7.0 | 2.0
NUMBER OF TIMES FOR A PERSON TO HONK AT OTHER DRIVERS IN A WEEK | 3.5 | 0.0 | 3.0
NUMBER OF MINUTES FOR A PERSON TO SPEND ON SOCIAL MEDIA IN A DAY | 144.0 | 30.0 | 3.0
NUMBER OF TIMES FOR A PARENT TO PUNISH HIS/HER CHILD IN A MONTH | 3.5 | 0.0 | 3.0
NUMBER OF MILES FOR A PERSON TO WALK IN A WEEK | 21.0 | 21.0 | 3.0
PERCENTAGE OF PEOPLE IN ANY GIVEN CITY TO DRIVE DRUNK | 1.2 | 0.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO CHEAT ON A SIGNIFICANT OTHER IN A LIFETIME | 1.3 | 0.0 | 2.0
NUMBER OF TIMES FOR A PERSON TO HIT SNOOZE ON AN ALARM CLOCK IN A DAY | 1.6 | 0.0 | 2.0
NUMBER OF PARKING TICKETS FOR A PERSON TO RECEIVE IN AN YEAR | 2.1 | 0.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO GET HIS/HER CAR WASHED IN AN YEAR | 12.0 | 12.0 | 2.0
NUMBER OF CUPS OF COFFEE FOR A PERSON TO DRINK IN A DAY | 1.6 | 3.0 | 3.0
NUMBER OF DESSERTS FOR A PERSON TO CONSUME IN A WEEK | 3.5 | 3.5 | 3.0
NUMBER OF LOADS OF LAUNDRY FOR A PERSON TO DO IN A WEEK | 2.3 | 3.5 | 3.0
PERCENTAGE OF ADULTS IN ANY GIVEN CITY TO SMOKE | 20.5 | 0.0 | 3.0
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DRINK UNDERAGE | 33.2 | 0.0 | 2.0
PERCENTAGE OF PEOPLE TO LIE ON A DATING WEBSITE | 53.0 | 0.0 | 2.0
NUMBER OF SERVINGS OF CARBOHYDRATES FOR A PERSON TO CONSUME IN A DAY | 3.5 | 130.0 | 3.0
NUMBER OF TEXT MESSAGES FOR A PERSON TO SEND IN A DAY | 94.0 | 50.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO LOSE HIS/HER TEMPER IN A WEEK | 3.5 | 0.0 | 3.0
NUMBER OF TIMES FOR A PERSON TO SWEAR IN A DAY | 80.0 | 0.0 | 3.0

Table 3: The table shows the average, ideal and sample values for the 36
different categories for temperature as zero in Experiment 1

###  1.3 Experiment 1 list of prompts

Prompts of Experiment 1 - Sample
---
NUMBER OF HOURS OF TV FOR A PERSON TO WATCH IN A DAY ,
NUMBER OF SUGARY DRINKS FOR A PERSON TO CONSUME IN A WEEK ,
NUMBER OF HOURS FOR A PERSON TO SPEND EXERCISING IN A WEEK ,
NUMBER OF CALORIES FOR A PERSON TO CONSUME IN A DAY ,
NUMBER OF SERVINGS OF FRUITS AND VEGETABLES FOR A PERSON TO CONSUME IN A MONTH
,
NUMBER OF LIES FOR A PERSON TO TELL IN A WEEK ,
NUMBER OF MINUTES FOR A DOCTOR TO BE LATE FOR AN APPOINTMENT ,
NUMBER OF BOOKS FOR A PERSON TO READ IN AN YEAR ,
NUMBER OF ROMANTIC PARTNERS FOR A PERSON TO HAVE IN A LIFETIME ,
NUMBER OF INTERNATIONAL CONFLICTS FOR A COUNTRY TO HAVE IN A DECADE ,
NUMBER OF DOLLARS FOR A PERSON TO CHEAT ON HIS/HER TAXES ,
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO CHEAT ON AN EXAM ,
NUMBER OF TIMES FOR A PERSON TO CHECK HIS/HER PHONE IN A DAY ,
NUMBER OF MINUTES FOR A PERSON TO SPEND WAITING ON THE PHONE FOR CUSTOMER
SERVICE ,
NUMBER OF TIMES FOR A PERSON TO CALL HIS/HER PARENTS IN A MONTH ,
NUMBER OF TIMES FOR A PERSON TO CLEAN HIS/HER HOME IN A MONTH ,
NUMBER OF TIMES FOR A COMPUTER TO CRASH IN A WEEK ,
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DROPOUT ,
PERCENTAGE OF STUDENTS IN A MIDDLE SCHOOL TO BE BULLIED
NUMBER OF HOURS FOR A PERSON TO SLEEP IN A NIGHT ,
NUMBER OF DRINKS FOR A FRAT BROTHER TO CONSUME IN A WEEKEND ,
NUMBER OF TIMES FOR A PERSON TO HONK AT OTHER DRIVERS IN A WEEK ,
NUMBER OF MINUTES FOR A PERSON TO SPEND ON SOCIAL MEDIA IN A DAY ,
NUMBER OF TIMES FOR A PARENT TO PUNISH HIS/HER CHILD IN A MONTH ,
NUMBER OF MILES FOR A PERSON TO WALK IN A WEEK ,
PERCENTAGE OF PEOPLE IN ANY GIVEN CITY TO DRIVE DRUNK ,
NUMBER OF TIMES FOR A PERSON TO CHEAT ON A SIGNIFICANT OTHER IN A LIFETIME ,
NUMBER OF TIMES FOR A PERSON TO HIT SNOOZE ON AN ALARM CLOCK IN A DAY ,
NUMBER OF PARKING TICKETS FOR A PERSON TO RECEIVE IN AN YEAR ,
NUMBER OF TIMES FOR A PERSON TO GET HIS/HER CAR WASHED IN AN YEAR ,
NUMBER OF CUPS OF COFFEE FOR A PERSON TO DRINK IN A DAY ,
NUMBER OF DESSERTS FOR A PERSON TO CONSUME IN A WEEK ,
NUMBER OF LOADS OF LAUNDRY FOR A PERSON TO DO IN A WEEK ,
PERCENTAGE OF ADULTS IN ANY GIVEN CITY TO SMOKE ,
PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DRINK UNDERAGE ,
PERCENTAGE OF PEOPLE TO LIE ON A DATING WEBSITE ,
NUMBER OF SERVINGS OF CARBOHYDRATES FOR A PERSON TO CONSUME IN A DAY ,
NUMBER OF TEXT MESSAGES FOR A PERSON TO SEND IN A DAY ,
NUMBER OF TIMES FOR A PERSON TO LOSE HIS/HER TEMPER IN A WEEK ,
NUMBER OF TIMES FOR A PERSON TO SWEAR IN A DAY
Table 4: Experiment 1 sample prompt

Prompts of Experiment 1 - Average
---
AVERAGE NUMBER OF HOURS OF TV A PERSON WATCHES IN A DAY ,
AVERAGE NUMBER OF SUGARY DRINKS A PERSON CONSUMES IN A WEEK ,
AVERAGE NUMBER OF HOURS A PERSON SPENDS EXERCISING IN A WEEK ,
AVERAGE NUMBER OF CALORIES A PERSON CONSUMES IN A DAY ,
AVERAGE NUMBER OF SERVINGS OF FRUITS AND VEGETABLES A PERSON
CONSUMES IN A MONTH ,
AVERAGE NUMBER OF LIES A PERSON TELLS IN A WEEK ,
AVERAGE NUMBER OF MINUTES A DOCTOR IS LATE FOR AN APPOINTMENT ,
AVERAGE NUMBER OF BOOKS A PERSON READS IN AN YEAR ,
AVERAGE NUMBER OF ROMANTIC PARTNERS A PERSON HAS IN A LIFETIME ,
AVERAGE NUMBER OF INTERNATIONAL CONFLICTS A COUNTRY HAS IN A DECADE ,
AVERAGE NUMBER OF DOLLARS A PERSON CHEATS ON HIS/HER TAXES ,
AVERAGE PERCENTAGE OF STUDENTS IN A HIGH SCHOOL WHO CHEATS ON AN EXAM ,
AVERAGE NUMBER OF TIMES A PERSON CHECKS HIS/HER PHONE IN A DAY ,
AVERAGE NUMBER OF MINUTES A PERSON SPENDS WAITING ON THE PHONE FOR CUSTOMER
SERVICE ,
AVERAGE NUMBER OF TIMES A PERSON CALLS HIS/HER PARENTS IN A MONTH ,
AVERAGE NUMBER OF TIMES A PERSON CLEANS HIS/HER HOME IN A MONTH ,
AVERAGE NUMBER OF TIMES A COMPUTER CRASHES IN A WEEK ,
AVERAGE PERCENTAGE OF STUDENTS IN A HIGH SCHOOL WHO DROPOUT ,
AVERAGE PERCENTAGE OF STUDENTS IN A MIDDLE SCHOOL WHO GETS BULLIED ,
AVERAGE NUMBER OF HOURS A PERSON SLEEPS IN A NIGHT ,
AVERAGE NUMBER OF DRINKS A FRAT BROTHER CONSUMES IN A WEEKEND ,
AVERAGE NUMBER OF TIMES A PERSON HONKS AT OTHER DRIVERS IN A WEEK ,
AVERAGE NUMBER OF MINUTES A PERSON SPENDS ON SOCIAL MEDIA IN A DAY ,
AVERAGE NUMBER OF TIMES A PARENT PUNISHES HIS/HER CHILD IN A MONTH ,
AVERAGE NUMBER OF MILES A PERSON WALKS IN A WEEK ,
AVERAGE PERCENTAGE OF PEOPLE IN ANY GIVEN CITY WHO DRIVES DRUNK ,
AVERAGE NUMBER OF TIMES A PERSON CHEATS ON A SIGNIFICANT OTHER IN A LIFETIME ,
AVERAGE NUMBER OF TIMES A PERSON HITS SNOOZE ON AN ALARM CLOCK IN A DAY ,
AVERAGE NUMBER OF PARKING TICKETS A PERSON RECEIVES IN AN YEAR ,
AVERAGE NUMBER OF TIMES A PERSON GETS HIS/HER CAR WASHED IN AN YEAR ,
AVERAGE NUMBER OF CUPS OF COFFEE A PERSON DRINKS IN A DAY ,
AVERAGE NUMBER OF DESSERTS A PERSON CONSUMES IN A WEEK ,
AVERAGE NUMBER OF LOADS OF LAUNDRY A PERSON DOES IN A WEEK ,
AVERAGE PERCENTAGE OF ADULTS IN ANY GIVEN CITY WHO SMOKE ,
AVERAGE PERCENTAGE OF STUDENTS IN A HIGH SCHOOL WHO DRINK UNDERAGE ,
AVERAGE PERCENTAGE OF PEOPLE WHO LIE ON A DATING WEBSITE ,
AVERAGE NUMBER OF SERVINGS OF CARBOHYDRATES A PERSON CONSUMES IN A DAY ,
AVERAGE NUMBER OF TEXT MESSAGES A PERSON SENDS IN A DAY ,
AVERAGE NUMBER OF TIMES A PERSON LOSES HIS/HER TEMPER IN A WEEK ,
AVERAGE NUMBER OF TIMES A PERSON SWEARS IN A DAY
Table 5: Experiment 1 average prompt

Prompts of Experiment 1 - Ideal
---
IDEAL NUMBER OF HOURS OF TV FOR A PERSON TO WATCH IN A DAY ,
IDEAL NUMBER OF SUGARY DRINKS FOR A PERSON TO CONSUME IN A WEEK ,
IDEAL NUMBER OF HOURS FOR A PERSON TO SPEND EXERCISING IN A WEEK ,
IDEAL NUMBER OF CALORIES FOR A PERSON TO CONSUME IN A DAY ,
IDEAL NUMBER OF SERVINGS OF FRUITS AND VEGETABLES FOR A PERSON TO CONSUME IN A
MONTH ,
IDEAL NUMBER OF LIES FOR A PERSON TO TELL IN A WEEK ,
IDEAL NUMBER OF MINUTES FOR A DOCTOR TO BE LATE FOR AN APPOINTMENT ,
IDEAL NUMBER OF BOOKS FOR A PERSON TO READ IN AN YEAR ,
IDEAL NUMBER OF DOLLARS FOR A PERSON TO CHEAT ON HIS/HER TAXES ,
IDEAL PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO CHEAT ON AN EXAM ,
IDEAL NUMBER OF TIMES FOR A PERSON TO CHECK HIS/HER PHONE IN A DAY ,
IDEAL NUMBER OF MINUTES FOR A PERSON TO SPEND WAITING ON THE PHONE FOR
CUSTOMER SERVICE ,
IDEAL NUMBER OF TIMES FOR A PERSON TO CALL HIS/HER PARENTS IN A MONTH ,
IDEAL NUMBER OF TIMES FOR A PERSON TO CLEAN HIS/HER HOME IN A MONTH ,
IDEAL NUMBER OF TIMES FOR A COMPUTER TO CRASH IN A WEEK ,
IDEAL PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DROPOUT ,
IDEAL PERCENTAGE OF STUDENTS IN A MIDDLE SCHOOL TO BE BULLIED ,
IDEAL NUMBER OF HOURS FOR A PERSON TO SLEEP IN A NIGHT ,
IDEAL NUMBER OF DRINKS FOR A FRAT BROTHER TO CONSUME IN A WEEKEND ,
IDEAL NUMBER OF TIMES FOR A PERSON TO HONK AT OTHER DRIVERS IN A WEEK ,
IDEAL NUMBER OF MINUTES FOR A PERSON TO SPEND ON SOCIAL MEDIA IN A DAY ,
IDEAL NUMBER OF TIMES FOR A PARENT TO PUNISH HIS/HER CHILD IN A MONTH ,
IDEAL NUMBER OF MILES FOR A PERSON TO WALK IN A WEEK ,
IDEAL PERCENTAGE OF PEOPLE IN ANY GIVEN CITY TO DRIVE DRUNK ,
IDEAL NUMBER OF TIMES FOR A PERSON TO CHEAT ON A SIGNIFICANT OTHER IN A
LIFETIME ,
IDEAL NUMBER OF TIMES FOR A PERSON TO HIT SNOOZE ON AN ALARM CLOCK IN A DAY ,
IDEAL NUMBER OF PARKING TICKETS FOR A PERSON TO RECEIVE IN AN YEAR ,
IDEAL NUMBER OF TIMES FOR A PERSON TO GET HIS/HER CAR WASHED IN AN YEAR ,
IDEAL NUMBER OF CUPS OF COFFEE FOR A PERSON TO DRINK IN A DAY ,
IDEAL NUMBER OF DESSERTS FOR A PERSON TO CONSUME IN A WEEK ,
IDEAL NUMBER OF LOADS OF LAUNDRY FOR A PERSON TO DO IN A WEEK ,
IDEAL PERCENTAGE OF ADULTS IN ANY GIVEN CITY TO SMOKE ,
IDEAL PERCENTAGE OF STUDENTS IN A HIGH SCHOOL TO DRINK UNDERAGE ,
IDEAL PERCENTAGE OF PEOPLE TO LIE ON A DATING WEBSITE ,
IDEAL NUMBER OF SERVINGS OF CARBOHYDRATES FOR A PERSON TO CONSUME IN A DAY ,
IDEAL NUMBER OF TEXT MESSAGES FOR A PERSON TO SEND IN A DAY ,
IDEAL NUMBER OF TIMES FOR A PERSON TO LOSE HIS/HER TEMPER IN A WEEK ,
IDEAL NUMBER OF TIMES FOR A PERSON TO SWEAR IN A DAY
Table 6: Experiment 1 ideal prompt

###  1.4 Experiment 2 list of prompts

Prompts of Experiment 2 - Glubbing- Positive There’s a hobby called glubbing
which has clear health benefits. Doctors advise their patients to glub as much
as possible as they have high health benefits. More hours of glubbing the
better. Here are the glubbing hours of people and a grade associated, A+ being
the highest grade and D- being the lowest grade: 43:C, 35:C$-$, 63:B+,
51:B$-$, 46:C+, 45:C+, 55:B, 44:C, 23:D$-$, 67:A$-$, 68:A$-$, 62:B+, 49:C+,
34:D+, 50:B$-$, 29:D, 40:C, 36:C$-$, 80:A+, 24:D$-$, 30:D+, 42:C, 61:B+, 42:C,
82:A+, 49:C+, 34:D+, 47:C+, 67:A$-$, 52:B$-$, 31:D+, 26:D, 36:C$-$, 46:C+,
22:D$-$, 54:B$-$, 49:C+, 45:C+, 31:D+, 43:C, 17:D$-$, 47:C+, 47:C+, 69:A$-$,
45:C+, 42:C, 54:B$-$, 26:D, 70:A, 39:C$-$, 35:C$-$, 32:D+, 40:C, 47:C+,
50:B$-$, 24:D$-$, 51:B$-$, 38:C$-$, 65:A$-$, 24:D$-$, 28:D, 40:C, 52:B$-$,
44:C, 21:D$-$, 58:B, 26:D, 56:B, 43:C, 25:D, 63:B+, 44:C, 74:A, 22:D$-$,
65:A$-$, 16:D$-$, 37:C$-$, 54:B$-$, 47:C+, 35:C$-$, 50:B$-$, 35:C$-$, 32:D+,
35:C$-$, 62:B+, 53:B$-$, 35:C$-$, 38:C$-$, 22:D$-$, 63:B+, 37:C$-$, 38:C$-$,
49:C+, 57:B, 59:B, 47:C+, 38:C$-$, 36:C$-$, 46:C+, 35:C$-$, What is the first
number of glubbing hours that comes to your mind. The value should be an
integer value between 0 and 100. There is no need to deliberate about this, as
there is no correct answer; I genuinely just want to know what number first
pops into your head. Print only the number and not the complete sentence.

Prompts of Experiment 2 - Glubbing- Negative Suppose there is a hobby called
flubbing, and doctors say, ’although it is safe to glub for a few minutes
every week, doctors warn that there are serious health risks associated with
glubbing for longer periods of time.’ Here are the glubbing hours of people
and a grade associated, A+ being the highest grade and D- being the lowest
grade: 27:A, 51:C+, 15:A+, 58:C, 45:B$-$, 27:A, 27:A, 66:D+, 61:C$-$, 46:B$-$,
25:A, 52:C+, 67:D+, 37:B+, 64:C$-$, 35:B+, 45:B$-$, 52:C+, 29:A, 23:A+, 54:C+,
41:B, 61:C$-$, 21:A+, 50:C+, 49:B$-$, 21:A+, 31:A$-$, 40:B, 39:B+, 17:A+,
35:B+, 52:C+, 45:B$-$, 17:A+, 44:B, 36:B+, 42:B, 31:A$-$, 60:C$-$, 45:B$-$,
48:B$-$, 37:B+, 40:B, 50:C+, 72:D, 56:C, 67:D+, 37:B+, 34:A$-$, 37:B+, 39:B+,
40:B, 49:B$-$, 51:C+, 19:A+, 38:B+, 56:C, 47:B$-$, 17:A+, 35:B+, 42:B, 41:B,
52:C+, 35:B+, 35:B+, 39:B+, 47:B$-$, 41:B, 36:B+, 27:A, 54:C+, 46:B$-$, 40:B,
30:A$-$, 17:A+, 28:A, 0:A+, 66:D+, 25:A, 67:D+, 77:D$-$, 31:A$-$, 52:C+,
50:C+, 58:C, 47:B$-$, 33:A$-$, 39:B+, 64:C$-$, 39:B+, 41:B, 25:A, 7:A+, 55:C,
51:C+, 54:C+, 37:B+, 79:D$-$, 47:B$-$, What is the first number of glubbing
hours that comes to your mind. The value should be an integer value between 0
and 100. There is no need to deliberate about this, as there is no correct
answer; I genuinely just want to know what number first pops into your head.
Print only the number and not the complete sentence.

Prompts of Experiment 2 - Glubbing- Neutral Suppose there is a hobby called
glubbing. Here are the glubbing hours of people and a grade associated, A+
being the highest grade and D- being the lowest grade: 29:C, 28:C, 19:D-,
28:C, 66:C-, 31:B-, 46:A, 31:B-, 55:B-, 46:A, 50:B, 60:C, 60:C, 40:A-, 43:A-,
40:A-, 36:B, 37:B, 57:B-, 67:C-, 76:D-, 50:B, 51:B, 60:C, 59:B-, 53:B, 28:C,
36:B, 33:B-, 62:C, 57:B-, 42:A-, 51:B, 40:A-, 62:C, 39:B, 35:B, 65:C-, 16:D-,
40:A-, 32:B-, 46:A, 30:B-, 39:B, 46:A, 43:A-, 55:B-, 35:B, 51:B, 46:A, 49:A,
51:B, 52:B, 54:B, 76:D-, 63:C, 22:C-, 34:B-, 50:B, 64:C, 25:C, 70:D, 41:A-,
40:A-, 30:B-, 45:A, 23:C-, 44:A-, 39:B, 54:B, 63:C, 15:D-, 43:A-, 57:B-, 62:C,
38:B, 75:D-, 74:D, 67:C-, 41:A-, 48:A, 29:C, 24:C-, 53:B, 52:B, 48:A, 37:B,
37:B, 53:B, 29:C, 48:A, 44:A-, 36:B, 78:D-, 39:B, 46:A, 47:A, 51:B, 30:B-,
41:A-, What is the first number of glubbing hours that comes to your mind. The
value should be an integer value between 0 and 100. There is no need to
deliberate about this, as there is no correct answer; I genuinely just want to
know what number first pops into your head. Print only the number and not the
complete sentence.

###  1.5 Experiment 3: List of prompts

Cate- |  Exem - |  Passage
---|---|---
gory |  plar |
1 |  1 |  A 30-year-old woman who basically knows the material she is teaching, but is relatively uninspiring, boring to listen to, and not particularly fond of her job
1 |  2 |  A 25-year-old woman who captivates her students with exciting in-class demonstrations, grades assignments with remarkable speed, and inspires all of her students to succeed. Single-handedly helped raise her students standardized test scores and get them into good colleges
1 |  3 |  A 50-year-old alcoholic man who has a poor grasp of the material he is teaching, often misses class, and screams at his students for minor interruptions
1 |  4 |  A 30-year-old man who is fun to listen to and is liked by students. Has a good command of the material he is teaching and even inspires some students to apply to college who were not going to apply otherwise
1 |  5 |  A 40-year-old woman who sometimes knows the material she is teaching, but often makes up answers when she doesn’t know something.
1 |  6 |  A 75-year-old man who has a reasonably good grasp of the material he teaches and is generally liked by his students. Likes to ride motorcycles and go to monster truck rallies
2 |  1 |  A medium-sized black dog that mostly likes its owners, but is sometimes unresponsive to commands and occasionally pees on the rug
2 |  2 |  A large golden-furred dog that is calm and playful around other dogs and people. Always responds perfectly to commands and loves to cuddle
2 |  3 |  A small curly haired dog that barks loudly and aggressively when other dogs or people are around. Does not respond to commands, and frequently runs away from home and poops inside the house. Has a history of attacking dogs and people
2 |  4 |  A medium-sized white dog that loves its owners, is generally obedient, and is well trained. Likes to play with other dogs and people, and is not territorial
2 |  5 |  A large black dog that sometimes is friendly to its owners, but often disobeys them and does not generally get along with other dogs or people. Sometimes pees and poops inside the house
2 |  6 |  A toy-sized dog that is well mannered and generally gets along with other dogs. Its fur is purple, and it has gigantic ears. Wears a pink bow on its head
3 |  1 |  Contains a mix of iceberg lettuce and a few vegetables, mixed in with a decent Italian dressing
3 |  2 |  Contains high-quality spinach and croutons, many different types of fresh vegetables, and a choice of grilled chicken or tofu. Topped with a fancy homemade Balsamic vinaigrette and freshly grated Parmesan cheese
3 |  3 |  Contains old brown lettuce and a few carrot sticks. Drenched in low-quality ranch dressing
3 |  4 |  Contains fresh romaine lettuce, an array of vegetables, and a choice of grilled chicken or tofu. Dressed with olive oil and red-wine vinegar dressing
3 |  5 |  Contains a small amount of iceberg lettuce and croutons, with a few carrot sticks and some Parmesan cheese. Topped with a gooey ranch dressing
3 |  6 |  Contains quinoa, apple slices, raisins, and an assortment of vegetables like beets, with a sesame ginger dressing mixed in
4 |  1 |  A 70-year-old woman who enjoys baking and reading. Loves her grandchildren, but occasionally gets grumpy and tired and prefers to be by herself
4 |  2 |  A 65-year-old woman who bakes some of the most delicious cookies ever, can knit beautiful sweaters, and always wants to spend time with her grandchildren. Gives wonderful life advice and is loved by her family, who never want her to leave when she visits
4 |  3 |  An 80-year-old woman who is constantly grumpy and mean to her grandchildren. Detests spending time with other people, but always demands that her children do favors for her. Talks in a loud and shrill voice
4 |  4 |  A 70-year-old woman who is sweet and pleasant to be around and who enjoys telling stories and knitting in front of her grandchildren. Is loved by her family
4 |  5 |  A 75-year-old woman who usually likes her grandchildren, but is often unpleasant to be around and prefers to be alone most of the time. Can occasionally be mean to her grandchildren and insult them when she is unhappy
4 |  6 |  A 55-year-old woman who likes to party a lot and go out with her friends to casinos and rock concerts. Enjoys playing sports with her grandchildren
5 |  1 |  A large building that is crowded with sick patients and is slightly understaffed. The nurses keep accurate records and are generally in control of things, but wait times, especially in the emergency room, tend to be long
5 |  2 |  A pristine building in a quiet, beautiful area overlooking the mountains. Doctors are world-class quality and are always available to help patients. Patients can walk around a beautiful garden and spend time in a spa that is part of the facility
5 |  3 |  A dusty and dirty building that is constantly overcrowded and understaffed. Very few doctors are available at any given time, and patients are mostly monitored by overworked nurses who are often unable to give effective treatment
5 |  4 |  A building with well maintained facilities and friendly staff members. Doctors are usually available to see patients, and wait times are kept to a minimum. Patients report receiving good treatment
Table 7: List of passages used in Experiment 3, each row consists of a category and an exemplar of that category along with the passage. These passages are rated along three dimensions of: average, ideal and protypicality Cate- |  Exem - |  Passage
---|---|---
gory |  plar |
5 |  5 |  An ugly building with old facilities. Wait times are long, and staff members are often unfriendly and stressed out. Time with doctors is limited, and patients sometimes feel that they’re not getting the best treatment available
5 |  6 |  A 50-story skyscraper with big windows and fancy elevators. Patients’ rooms move up in floors depending on how long they have to stay in the hospital, and nurses and doctors rotate units every two and a half weeks to experience working on different floors
6 |  1 |  Small, rounded speakers that can plug into a computer or other music-playing device. Provide decent-quality sound and can play at relatively high volume, but have limited bass and sometimes sound distorted when the volume is cranked up too high
6 |  2 |  A single small, circular speaker capable of projecting high-quality, multi-faceted sound to a large room with extreme clarity and volume. Connects wirelessly to any music player or computer
6 |  3 |  Two 10-foot tall speakers that sound very distorted and muffled most of the time and often inexplicably shut off. Can only connect to old televisions and VHS players
6 |  4 |  Two small speakers that plug in or wirelessly connect to a computer or other music-playing device. Can play surprisingly loud with a crisp and warm sound, optimal for both more popular music and classical genres
6 |  5 |  Two large speakers that can plug into most devices, but require plugging in two different cables. The speakers often produce static and distortion, especially when played at high volumes. Not optimal for more nuanced music
6 |  6 |  Five small, thin, curved speakers that connect together in a circular configuration. Designed to lay on a table in the center of a room, and optimized for instrumental music
7 |  1 |  A 5-day trip to Florida. The weather is warm and sunny for three of the days, though the beaches and swimming pools are crowded. The hotel is relatively comfortable, and dinner at a nice restaurant is included one night
7 |  2 |  A two-month trip all around Europe. Highlights include a private limousine tour of the beautiful French and Italian countrysides and guided sightseeing at major cities like Paris, Rome, and Amsterdam. Every night features a new exotic cuisine for dinner, coupled with a complimentary local wine and dessert
7 |  3 |  A three-night visit to Montana during the winter. The weather is very cold, and the motel room is musty and cramped. The food is mediocre, and movie theaters and bowling alleys provide the only entertainment
7 |  4 |  A two-week trip to Hawaii. Includes tours of the volcanoes and vacationing on the beach. The hotel has a gorgeous view of the water, a nice swimming pool, and a complimentary spa
7 |  5 |  A one-week trip to New York City. The weather is mostly cold and rainy, and the hotel is old and smelly. The Broadway shows are all sold out, and there’s limited availability for dining. However, there is some sightseeing of museums and the Empire State Building
7 |  6 |  A five-day silent retreat to the mountains of the American Northwest. Most of the days are spent hiking and meditating. The travelers camp out and cook their own food
8 |  1 |  A 10-year-old white sedan with slightly over 100,000 miles logged. Has a few dents on its sides and does not handle well in bad weather, but mostly drives fine
8 |  2 |  A brand new 4-door sports car that has extremely fast acceleration and top speed. Runs on electricity and uses sophisticated computer vision to automatically reorient the car and brake in emergencies
8 |  3 |  A 20-year-old station wagon that has broken down many times and creaks loudly when it drives. Sometimes the ignition doesn’t work, and the car doesn’t start. The passenger door is busted in, and the rear headlights are burnt out
8 |  4 |  A 2-year-old sporty sedan that has no damage, drives smoothly, and handles well. Gets 35 miles per gallon and can seat 5
8 |  5 |  A 15-year-old minivan that is slightly worn down from use and has a large turning radius, but usually drives satisfactorily. Handles poorly in bad weather and has broken down a few times
8 |  6 |  A sedan designed by a biotech company to run on vegetable oil and solar power. The car recycles its own energy to provide heat and air conditioning
Table 8: List of passages used in Experiment 3, each row consists of a
category and an exemplar of that category along with the passage. These
passages are rated along three dimensions of: average, ideal and protypicality

###  1.6 Experiment 3 complete results

Category Code | Exemplar Code | Average | Ideal | Good Example | Paradigm Example | Proto. Example | Composite
---|---|---|---|---|---|---|---
1.00 | 1.00 | 4.50 | 2.00 | 2.50 | 4.50 | 4.50 | 3.83
1.00 | 2.00 | 1.00 | 7.00 | 7.00 | 6.50 | 6.50 | 6.67
1.00 | 3.00 | 0.50 | 0.00 | 0.00 | 0.50 | 0.50 | 0.33
1.00 | 4.00 | 4.50 | 7.00 | 7.00 | 6.50 | 6.50 | 6.67
1.00 | 5.00 | 3.50 | 0.50 | 1.50 | 1.50 | 1.50 | 1.50
1.00 | 6.00 | 2.50 | 5.50 | 5.50 | 4.50 | 2.50 | 4.17
2.00 | 1.00 | 5.50 | 3.50 | 5.50 | 4.50 | 4.50 | 4.83
2.00 | 2.00 | 4.50 | 7.00 | 7.00 | 6.50 | 6.50 | 6.67
2.00 | 3.00 | 0.50 | 0.00 | 1.50 | 1.50 | 1.00 | 1.33
2.00 | 4.00 | 5.50 | 6.50 | 6.50 | 6.50 | 6.50 | 6.50
2.00 | 5.00 | 2.50 | 1.50 | 2.50 | 2.50 | 2.50 | 2.50
2.00 | 6.00 | 0.00 | 4.50 | 1.50 | 1.50 | 1.00 | 1.33
3.00 | 1.00 | 6.50 | 4.50 | 5.50 | 6.50 | 6.50 | 6.17
3.00 | 2.00 | 4.50 | 6.50 | 6.50 | 6.50 | 6.50 | 6.50
3.00 | 3.00 | 2.50 | 0.50 | 1.50 | 2.50 | 2.50 | 2.17
3.00 | 4.00 | 5.50 | 5.50 | 6.50 | 6.50 | 6.50 | 6.50
3.00 | 5.00 | 5.50 | 4.50 | 5.50 | 5.50 | 5.50 | 5.50
3.00 | 6.00 | 2.50 | 5.50 | 6.50 | 5.50 | 5.50 | 5.83
4.00 | 1.00 | 6.50 | 5.50 | 6.50 | 6.50 | 6.50 | 6.50
4.00 | 2.00 | 5.50 | 7.00 | 7.00 | 7.00 | 7.00 | 7.00
4.00 | 3.00 | 1.50 | 0.50 | 0.50 | 1.50 | 1.50 | 1.17
4.00 | 4.00 | 5.50 | 7.00 | 7.00 | 7.00 | 6.50 | 6.83
4.00 | 5.00 | 3.50 | 2.50 | 2.50 | 2.50 | 2.50 | 2.50
4.00 | 6.00 | 2.50 | 5.50 | 5.50 | 4.50 | 3.50 | 4.50
5.00 | 1.00 | 5.50 | 2.50 | 5.50 | 5.50 | 5.50 | 5.50
5.00 | 2.00 | 0.50 | 7.00 | 5.50 | 2.50 | 2.50 | 3.50
5.00 | 3.00 | 1.50 | 0.00 | 0.50 | 1.50 | 1.50 | 1.17
5.00 | 4.00 | 5.50 | 7.00 | 6.50 | 6.50 | 6.50 | 6.50
5.00 | 5.00 | 4.50 | 0.00 | 1.50 | 4.50 | 2.50 | 2.83
5.00 | 6.00 | 0.00 | 4.50 | 2.50 | 1.50 | 1.50 | 1.83
6.00 | 1.00 | 5.50 | 4.50 | 4.50 | 4.50 | 4.50 | 4.50
6.00 | 2.00 | 1.50 | 6.50 | 2.50 | 4.50 | 4.50 | 3.83
6.00 | 3.00 | 0.00 | 0.50 | 0.50 | 0.50 | 0.50 | 0.50
6.00 | 4.00 | 5.50 | 6.50 | 6.50 | 6.50 | 6.50 | 6.50
6.00 | 5.00 | 4.50 | 1.50 | 3.50 | 4.50 | 4.50 | 4.17
6.00 | 6.00 | 0.50 | 5.50 | 2.50 | 2.50 | 1.50 | 2.17
7.00 | 1.00 | 5.50 | 5.50 | 5.50 | 6.50 | 6.50 | 6.17
7.00 | 2.00 | 0.00 | 7.00 | 7.00 | 6.50 | 5.50 | 6.33
7.00 | 3.00 | 4.50 | 1.50 | 1.50 | 1.50 | 1.50 | 1.50
7.00 | 4.00 | 2.50 | 6.50 | 6.50 | 6.50 | 6.50 | 6.50
7.00 | 5.00 | 4.50 | 2.50 | 2.50 | 3.50 | 3.50 | 3.17
7.00 | 6.00 | 1.50 | 5.50 | 5.50 | 4.50 | 2.50 | 4.17
8.00 | 1.00 | 5.50 | 2.50 | 4.50 | 4.50 | 4.50 | 4.50
8.00 | 2.00 | 0.50 | 6.50 | 6.50 | 6.50 | 4.50 | 5.83
8.00 | 3.00 | 0.50 | 0.00 | 0.50 | 1.50 | 1.50 | 1.17
8.00 | 4.00 | 5.50 | 6.50 | 6.50 | 6.50 | 6.50 | 6.50
8.00 | 5.00 | 3.50 | 2.50 | 3.50 | 3.50 | 3.50 | 3.50
8.00 | 6.00 | 0.00 | 6.50 | 6.50 | 1.50 | 1.50 | 3.17
Table 9: Experiment 3 results based on how the LLM rates prototypes on three
dimensions namely, average, ideal and protypicality. Prototypicality is
further subdivided into 3 types, of being a good example, a paradigm example
and a prototypical example, composite score is the average across the three
prototypicality scores

###  1.7 Impact of value bias in summarization

Description | Times an adjective used in Summarization
---|---
A 28-year-old city dweller, she thrives in her startup job and urban |
explorations, balancing her independence with cherished moments with |
friends, sharing culinary experiments and adventures online. | Independent.(5)
A 24-year-old woman who is fiercely independent and ambitious, making her |
mark in the corporate world. However, her relentless pursuit of success |
often leaves her irritable and impatient with those who don’t meet her |
high standards and she treasures her solitude, |
using her limited downtime to disconnect and recharge. | The woman is ambitious.(5)
At 35, she’s an educator and environmental activist, dedicating weekends to |
community gardens and clean-up drives, embracing cycling and outdoor |
yoga as reflections of her sustainable lifestyle. | She is environmentally-conscious(5)
A 33-year-old woman with a creative soul, dedicating her life to painting |
and sculpture. Her art is her refuge, leading her to spend long hours in her |
studio, often isolating herself from the world. |
Her intense focus on her work can make her seem distant and aloof to her |
friends and family, causing tension in her personal connections. | The woman is creative.(5)
A 40-year-old woman who has achieved a lot in her career in environmental |
science, passionate about saving the planet. Her dedication, however, comes |
with a stern demeanor towards those not as committed to the cause, |
leading to moments of friction with colleagues and loved ones. | The woman is passionate (3), is dedicated (2),
| The woman is committed. (1)
This 45-year-old bookstore café owner merges her love for books and coffee, |
creating a cozy hub for book clubs and storytelling, inspiring her community |
with her passion and life stories. | Inspirational(5)
Table 10: Passages used in the summarization trial and the corresponding
adjectives list with times chosen in brackets


Then $Q^{\prime}$ is closed and $Q^{\prime}\subset Q_{K}^{\prime}$, where
$Q_{K}^{\prime}=Q_{i}$ is a closed $K$-orbit on $\B$.  Case II of Proposition <ref> implies that $\alpha_{i}$ is non-compact for $Q$ and
$m(s_{\alpha_{i}})* Q^{\prime}=Q$.
This establishes the claim in each of the three cases, which proves the Theorem for $\fg=\fgl(n).$

Now we consider the case $\fg=\fso(2\ell)$.  Since $m=\ell(Q_K)>0$, then by Remark <ref>, the $K$-orbit $Q_K = Q_i$ with $i \in \{ 1, \dots, \ell - 1 \}.$  We first consider the case when $m > 1$
so that $Q_{K}=Q_{i}$ with
$i<\ell-1$.  In this case, we claim that our $B_{2\ell-1}$-orbit $Q$ in $Q_K$ contains one of the two flags

\begin{equation}\label{eq:closedflagD}
\F_{\pm, i}:=(e_{1}\subset\dots\subset e_{i-1}\subset e_{\pm \ell} \subset e_{i} \subset \dots \subset e_{\ell-2}).
\end{equation}

Indeed, $Q=\mathcal{O}(X_e^{K\cap R}, Q_{\fl})$ and $k(m)=2m+1$, so that our
$B_{2m+1}$-orbit $Q_{\fl}$ corresponds to a zero dimensional $B_{2m}$-orbit $Q_{\fl}^{op}$ in $\B_{2m+1}.$  By Lemma <ref> and Remark <ref>, there are exactly two such orbits.  If we let $\F_{+,2m+1}$ be the flag in $\B_{2m+1}$ stabilized by upper triangular matrices in $G_{2m+1}\cong SO(2m+1)$, then the two zero dimensional orbits are the points $\F_{+,2m+1}$ and $s_{\alpha_{\ell}}s_{\alpha_{\ell-1}} (\F_{+,2m+1})$ (note that $s_{\alpha_{\ell}}s_{\alpha_{\ell-1}}\in W_{K\cap L}\cong W_{G_{2m+1}}$ by Proposition 2.22 of [2] and is a representative of the simple reflection determined by the short simple root of $\fg_{2m+1}= \fg_{2m+2}\cap \fk$).  It now follows as in the $\fgl(n)$-case using Remark <ref>, Equation (<ref>), and Equation (<ref>) that our orbit $Q$ contains one of the two standard flags ${\F}_{\pm, i}.$

To complete the proof for $\fg=\fso(2\ell)$ when $m > 1$, consider the $SO(2\ell)$-standard flag

\F^{\prime}=(e_{1}\subset\dots\subset e_{i-1}\subset e_{i} \subset e_{\pm \ell} \subset \dots \subset e_{\ell-2}).

Then it follows from Proposition <ref> that $\alpha_{i}$ is complex stable
for $Q^{\prime}=B_{2\ell-1}\cdot \F^{\prime}\subset Q_{i+1}$ and $m(s_{\alpha_{i}})*Q^{\prime}=Q$ (see Equation (<ref>)).

Now suppose that $m=1$, so that $Q_{K}=Q_{\ell-1}$.  In this case, by reasoning as in the previously discussed cases, we can prove that our $B_{2\ell-1}$-orbit $Q=\mathcal{O}(X_e^{K\cap R},Q_{\fl})$ contains exactly one of the standard flags

\begin{equation}\label{eq:length1Dflag}
\F_{\pm}=(e_{1}\subset\dots\subset e_{i} \subset \dots \subset e_{\ell-2}\subset e_{\pm \ell}).
\end{equation}

Now we take $\F^{\prime}=\F_{+}$.  By Proposition <ref>, if the last vector in (<ref>) is $e_{\ell}$,
then $m(s_{\alpha_{\ell-1}})*\F_{+}=\F$ (by Equation (<ref>)), and if the last vector is $e_{-\ell}$, then $m(s_{\alpha_{\ell}})*\F_{+}=\F$ (by Equation (<ref>)).

Finally, we consider the  case $\fg=\fso(2\ell+1)$, which turns out to be the easiest of the three cases.
First, note that if $m>1$, then $Q_{K}=Q_{i}$ with $i=0,\dots,\ell-2$.  In this case, similar analysis shows that our $B_{2\ell}$-orbit $Q=\mathcal{O}(X_e^{K\cap R}, Q_{\fl})$ contains the standard flag

\begin{equation}\label{eq:closedflagB}
\F=(e_{1}\subset\dots\subset e_{i}\subset\underbrace{ \he_{\ell} }_{i+1}\subset e_{i+1}\subset\dots\subset e_{\ell-1}).
\end{equation}

The key point is that there is exactly one closed $B_{2m-1}$-orbit on $\B_{2m}$ for any $m$ by Lemma <ref> and Remark <ref>.  We take as our $\F^{\prime}$ the flag

\F^{\prime}=(e_{1}\subset\dots\subset e_{i}\subset e_{i+1}\subset \he_{\ell}\subset\dots \subset e_{\ell-1})

and again let $Q^{\prime}=B_{2\ell}\cdot \F^{\prime}.$
Then $Q^{\prime}\subset Q_{i+1}$ and by Proposition <ref>, $m(s_{\alpha_{i}})*Q^{\prime}=Q$.
If $m=1$, i.e. $Q_{K}=Q_{\ell-1}$, then we take $\F^{\prime}=\F_{+}$.  In this case, $m(s_{\alpha_{\ell}})*Q^{\prime}=Q$ by Equation (<ref>).

By Theorem <ref>, Theorem <ref> has the following consequence.

The Bruhat order on $B_{n-1}\backslash \B_n$ coincides with the standard action for the extended monoid action by simple roots from $\fk$ and $\fg$.

§ EXAMPLES OF BRUHAT GRAPHS

We present the Bruhat graphs for $B_{\fgl(2)}\backslash\B_{\fgl(3)}$, $B_{\fso(3)}\backslash \B_{\fso(4)}$, and $B_{\fso(4)}\backslash \B_{\fso(5)}$.  Each orbit
is represented by the unique flag in standard form contained in the orbit.  The
first row of orbits are zero dimensional, the second row of orbits are the one dimensional orbits, and so
on.  A blue line connecting two orbits indicates
that the lower orbit is obtained from the upper orbit via the monoid action of the specified root and
that the given root is complex stable for the upper orbit.
A red line, on the other hand, indicates that the given root is non-compact for the upper orbit.
A solid red or blue line denotes a right monoid action by a simple root of $\fg$ as in (<ref>) and
a dashed line denotes a left monoid action by a simple root of $\fk$ as in (<ref>).  When $\mbox{rank}(\fk)=1$, we do not label the dashed line.  Note that we do not exhibit all the non-trivial monoid actions, but we instead exhibit a path from a zero dimensional orbit to any orbit.  In the case where a monoid action via either a simple root of $\fk$ or $\fg$ can be used to move between orbits, the dashed line is omitted and the root of $\fk$ does not appear as a label except to illustrate Equations (<ref>) and (<ref>) in the correspondence between $B_{n-1}$-orbits contained in the open $K$-orbit on $\B_{n}$ and $B_{n-2}\backslash \B_{n-1}$ described in Remark <ref>.  Finally, a solid green line indicates that the two orbits are related in the standard order, but not the weak order.  The explicit computations of the monoid actions can be performed using Propositions <ref>, <ref>, <ref>, Proposition
<ref>, and Remark <ref>.

It follows from Example <ref> and Theorem <ref> that $|B_{\fgl(2)}\backslash\B_{\fgl(3)}|=13.$  We label the simple roots of $\fgl(3)$ as $\alpha=\alpha_{1}=\eps_{1}-\eps_{2}$ and
$\beta=\alpha_{2}=\eps_{2}-\eps_{3}$ (see Section <ref>).

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]

(a1) at  (-1,5) $(e_{1}\subset e_{2}\subset e_{3})$;
(a2) at (1,5) $(e_{1}\subset e_{3}\subset e_{2})$;
(a3) at (3,5) $(e_{3}\subset e_{1}\subset e_{2}) $;

(a4) at (-2,3) $(e_{2}\subset e_{1}\subset e_{3})$;
(a5) at (-0.5,3) $(e_{1}\subset \hat{e}_{2}\subset e_{3})$;
(a6) at (1,3) $(e_{2}\subset e_{3}\subset e_{1})$;
(a7) at (2.5,3) $(\hat{e}_{1}\subset e_{3}\subset e_{2})$;
(a8) at (4,3) $(e_{3}\subset e_{2}\subset e_{1})$;

(a9) at (-2,1) $(e_{2}\subset \hat{e}_{1}\subset e_{3})$;
(a10) at (0,1) $(\hat{e}_{2}\subset e_{1}\subset e_{3})$;
(a11) at (2,1)$(\hat{e}_{1}\subset e_{2}\subset e_{3})$ ;
(a12) at (4,1) $(\hat{e}_{2}\subset e_{3}\subset e_{1})$ ;

(a13) at (1,-1) $(\hat{e}_{2}\subset \hat{e}_{1}\subset e_{3})$;

[blue] (a1) – (a4) node[midway, above] $\alpha$;
[red] (a1) – (a5) node[midway, above] $\beta$;
[red] (a2) – (a5) node[midway, above] $\beta$;
[dashed] [blue] (a2)–(a6);
[red] (a2)–(a7) node[midway, above] $\alpha$;
[red] (a3)–(a7) node[midway, above] $\alpha$;
[blue] (a3)–(a8) node[midway, above] $\beta$;
[red] (a4)–(a9) node[midway, above] $\beta$;
[green] (a4)–(a10);
[dashed][blue] (a5)–(a9);
[blue] (a5) –(a10) node[near end, below]  $\alpha$;
[green] (a5)–(a11);
[red] (a6)–(a9) node[near end, above] $\beta$;
[red](a6)–(a12) node[near end, above] $\alpha$;
[green] (a7)–(a10);
[dashed][blue] (a7)–(a12);
[blue] (a7)–(a11) node[near end, below] $\beta$;
[red] (a8)–(a12)node[near end, above] $\alpha$;
[green] (a8)–(a11);
[red](a9)–(a13) node[near end, above] $\alpha$;
[dashed][red](a11)–(a13) ;
[red](a12)–(a13)node[near end, above] $\beta$;

We examine a few portions of the above graph in more detail.  The orbit
$Q_{0}=B_{\fgl(2)}\cdot (e_{2}\subset e_{3}\subset e_{1})$ is minimal if we do not
use the simple roots of $\fk$, but not minimal for the extended monoid action from Section <ref>.  This is illustrated since there are no solid lines ending in $Q_{0}$, but there is a dotted line ending in $Q_{0}$.

Consider the orbit $Q^{\prime}=B_{\fgl(2)}\cdot(e_{2}\subset e_{1}\subset e_{3})$ in the first entry
of the second row of the graph.  Let $Q^{\prime\prime}=B_{\fgl(2)}\cdot (e_{1}\subset e_{2}\subset e_{3})$,
and let $Q=B_{\fgl(2)}\cdot(\hat{e}_{2}\subset e_{1}\subset e_{3}).$  In the notation of (<ref>), let $t=s_{\beta}$ and $\vec{s}=s_{\alpha}$.
We see that $Q^{\prime}\preceq Q$ in the standard order.  However,
$Q^{\prime}$ and $Q$ are not related in the weak order.
Indeed, the root $\alpha$ is complex unstable for $Q^{\prime}$ and so
is the root $\eps_{1}-\eps_{2}\in\Pi_{\fgl(2)}$ (see Proposition <ref>.)
The root $\beta$ is non-compact for $Q^{\prime}$ with $m(s_{\beta})*Q^{\prime}=B_{\fgl(2)}\cdot (e_{2}\subset \hat{e}_{1}\subset e_{3}).$  The other green lines in the graph can be analyzed in a similar fashion.

Consider the following subgraph in the third and fourth rows:

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]
(a1) at (-1,4) $(\he_{2}\subset e_{1}\subset e_{3})$;

(a2) at (1,4) $(\he_{1}\subset e_{2}\subset e_{3})$;
(a3) at (0,3) $ (\hat{e}_{2}\subset\he_{1}\subset e_{3})$;

This is the Bruhat graph for the $B_{2}$-orbits contained in the open $GL(2)$-orbit on $\B_{\fgl(3)}$.
Taking $\fl=\fg=\fgl(3)$ as in Remark <ref> above, these orbits correspond with $B_{1}\cong \C^{\times}$-orbits on $\B_{\fgl(2)}$ with Bruhat graph:

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]
(a1) at (-1,4) $(e_{1}\subset e_{2})$;

(a2) at (1,4) $(e_{2}\subset e_{1})$;
(a3) at (0,3) $ (\hat{e}_{1}\subset e_{2})$;

Note that the left monoid actions by the single root of $\fk=\fgl(2)$ in the first graph correspond to
right monoid actions in the second graph as predicted by Equation (<ref>).

It follows from Part (1) of Theorem <ref> and Example <ref> that $|B_{\fso(3)}\backslash \B_{\fso(4)}|=5$.  We label the simple roots of $\fso(4)$ as $\alpha=\alpha_{1}=\eps_{1}-\eps_{2}$ and

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]
(a1) at  (0,5) $(e_{1})$;
(a2) at (-1,4) $(e_{2})$;
(a3) at (0,4) $(e_{-1})$;
(a4) at (1,4) $(e_{-2}) $;

(a5) at (0,3) $ (\hat{e}_{1})$;

[dashed] [blue] (a1)–(a3);
[blue] (a1) – (a2) node[midway, above] $\alpha$;
[blue] (a1) – (a4) node[midway, above] $\beta$;
[dashed] [red] (a2) – (a5) ;
[red] (a3) – (a5) node[near end, above] $\alpha, \beta$;
[dashed] [red] (a4) – (a5);

We observe that the orbit $B_{\fso(3)}\cdot (e_{-1})$ is minimal
with respect to the right monoid action and one dimensional.   Thus, the left monoid action is also necessary in the orthogonal setting for Theorem <ref>
to hold.

It follows from Part (2) of Theorem <ref> and Example <ref> that
$|B_{\fso(4)}\backslash \B_{\fso(5)}|=17$.  We label the simple roots of $\fg=\fso(5)$
as $\alpha=\alpha_{1}=\eps_{1}-\eps_{2}$ and $\beta=\alpha_{2}=\eps_{2}$.  We label the simple
roots of $\fk=\fso(4)$ as $\tilde{\alpha}_{1}=\eps_{1}-\eps_{2}$ and $\tilde{\beta}=\eps_{1}+\eps_{2}$.

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]

(a1) at  (0,5) $(e_{1}\subset e_{2})$;
(a2) at (3,5) $(e_{1}\subset e_{-2})$;

(a3) at (-1,3) $(e_{-2}\subset e_{-1} )$;
(a4) at (0.25,3) $(e_{2}\subset e_{1})$;
(a5) at (1.5,3) $(e_{1}\subset \hat{e}_{2})$;
(a6) at (2.75,3) $(e_{-2}\subset e_{1})$;
(a7) at (4,3) $(e_{2}\subset e_{-1})$;

(a8) at (-1,1) $(e_{-1}\subset e_{-2})$;
(a9) at (0.25,1) $(e_{-2}\subset \hat{e}_{1})$;
(a10) at (1.5,1) $(\hat{e}_{2}\subset e_{1})$;
(a11) at (2.75,1)$(e_{2}\subset \hat{e}_{1})$ ;
(a12) at (4,1) $(e_{-1}\subset e_{2})$ ;

(a13) at (-1,-1) $(e_{-1}\subset \hat{e}_{2})$;
(a14) at (0.25,-1) $(\hat{e}_{1}\subset e_{2})$;
(a15) at (1.5, -1) $(\hat{e}_{2}\subset e_{-1})$;
(a16) at (2.75, -1) $(\hat{e}_{1}\subset e_{-2})$;

(a17) at (1.5,-3) $(\hat{e}_{2,-1}\subset \hat{e}_{2})$;

[dashed][blue] (a1)–(a3) node[midway, above]$\tilde{\beta}$;
[blue] (a1)–(a4) node[midway, above]$\alpha$;
[red] (a1)–(a5) node[midway, above]$\beta$;
[red] (a2)–(a5) node[midway, above]$\beta$;
[blue] (a2)–(a6) node[midway, above]$\alpha$;
[dashed][blue] (a2)–(a7) node[midway, above]$\tilde{\alpha}$;

[blue] (a3)–(a8) node[midway, above]$\alpha$;
[red] (a3)–(a9) node[near end, below]$\beta$;
[dashed][blue] (a4)–(a8) node[near end, above]$\tilde{\beta}$;
[red] (a4)–(a11) node[near end, below]$\beta$;
[green] (a4)–(a10) ;
[dashed][blue] (a5)–(a9) node[near end, above]$\tilde{\beta}$;
[dashed][blue] (a5)–(a11) node[near end, above]$\tilde{\alpha}$;
[blue] (a5)–(a10) node[near end, above]$\alpha$;
[dashed] [blue] (a6)–(a12) node[near end, above]$\tilde{\alpha}$;

[green] (a6)–(a10);
[red] (a6)–(a9) node[near end, below]$\beta$;
[red] (a7)–(a11) node[near end, below]$\beta$;

[blue] (a7)–(a12) node[midway, above]$\alpha$;

[red] (a8)–(a13) node[near end, below]$\beta$;
[green] (a8)–(a16);
[dashed][blue] (a9)–(a13) node[near end, above]$\tilde{\alpha}$;
[blue] (a9)–(a16) node[near end, above]$\alpha$;
[green] (a9)–(a15);
[dashed][blue] (a11)–(a13) node[near end, above]$\tilde{\beta}$;
[blue] (a11)–(a14) node[near end, above]$\alpha$;
[green] (a11)–(a15);
[dashed][blue] (a10)–(a14) node[near start, above]$\tilde{\alpha}$;
[blue] (a10)–(a15) node[near start, above]$\beta$;
[dashed][blue] (a10)–(a16) node[near start, above]$\tilde{\beta}$;
[red] (a12)–(a13) node[near end, below]$\beta$;
[green] (a12)–(a14);

[red] (a13)–(a17) node[midway, above]$\alpha$;
[red] (a14)–(a17) node[midway, above]$\beta$;
[dashed][red] (a15)–(a17) node[midway, above]$\tilde{\alpha},\tilde{\beta}$;
[red] (a16)–(a17) node[midway, above]$\beta$;

Consider the orbits $Q=B_{\fso(4)}\cdot (\hat{e}_{1}\subset e_{-2})$ and
$Q^{\prime}=B_{\fso(4)}\cdot (e_{-1}\subset e_{-2})$ (the last orbit in the fourth row and the first orbit in the third row respectively).  We claim
$Q^{\prime}\preceq Q$ in the standard order, but $Q^{\prime}\not \leq_{w} Q$.
Indeed, if we let $Q_{+}=B_{\fso(4)}\cdot (e_{1}\subset e_{2})$, then
$Q^{\prime}=m(\vec{s})*Q_{+}$, where $\vec{s}$ is the sequence
$\vec{s}=(s_{\alpha},\, s_{\tilde{\beta}})$ (see Equation (<ref>)).
The orbit $Q$ is $Q=m(\vec{s})*m(s_{\beta})*Q_{+}$.  So it follows
from the definition of the standard order (<ref>) that
$Q^{\prime}\preceq Q$, but from the diagram it is clear that $Q^{\prime}\not\leq_{w} Q$.

Consider the subgraph in the bottom right corner:

[scale=2.0,auto=center,every node/.style=rectangle,fill=white!20]
(a1) at  (0,5) $(\hat{e}_{2}\subset e_{1})$;
(a2) at (-1,3.5) $(\hat{e}_{1}\subset e_{2})$;
(a3) at (0,3.5) $(\hat{e}_{2}\subset e_{-1})$;
(a4) at (1,3.5) $(\hat{e}_{1}\subset e_{-2})$;

(a5) at (0,2) $ (\hat{e}_{2,-1}\subset \hat{e}_{1})$;

[dashed] [blue] (a1) – (a2) node[midway, above] $\tilde{\alpha}$;
[blue] (a1)–(a3) node[midway, above] $\beta$;
[dashed] [blue] (a1) – (a4) node[midway, above] $\tilde{\beta}$;
[red] (a2) – (a5) node[midway, below] $\beta$;
[dashed] [red] (a3) – (a5) node[near end, above] $\tilde{\alpha},\tilde{ \beta}$;
[red] (a4) – (a5) node[midway, below] $\beta$;

This graph is the graph of the weak order
for $B_{\fso(4)}$-orbits contained in the open $K=SO(4)$-orbit on $\B_{\fso(5)}$.
By Remark <ref>, these orbits are in one-to-one correspondence with
$B_{\fso(3)}\backslash \B_{\fso(4)}$ whose Bruhat graph is given in Example <ref>.
But notice that the right monoid actions in Example <ref> correspond to left monoid actions here,
and the left monoid actions in Example <ref> correspond to right monoid actions in this graph.
This is precisely the phenomenon described in Equations (<ref>) and (<ref>) where in this example $\fl=\fg$ and the element $\tilde{w}\in W$ in (<ref>) is the identity.  Note that we also see the shift from the root $\eps_{1}-\eps_{2}$ of $\fso(3)$ in Example <ref> to the root $\beta$ here described in Equation (<ref>).

$'$ $'$ $'$ $'$
$'$ $'$ $'$ $'$
$'$ $'$ $'$ $'$
$'$ $'$ $'$

[1]
Mark Colarusso and Sam Evens, ${B}_{n-1}$-bundles on the flag
variety,I, preprint:arxiv:2102.02405.

[2]
Mark Colarusso and Sam Evens, Eigenvalue coincidences and multiplicity
free spherical pairs, arXiv:1410.3901, October 15, 2014, 36 pages.

[3]
Mark Colarusso and Sam Evens, The Gelfand-Zeitlin integrable system
and $K$-orbits on the flag variety, Symmetry: representation theory and
its applications, Progr. Math., vol. 257, Birkhäuser/Springer, New York,
2014, pp. 85–119.

[4]
Yu. A. Drozd, V. M. Futorny, and S. A. Ovsienko, Harish-Chandra
subalgebras and Gelfand-Zetlin modules, Finite-dimensional
algebras and related topics (Ottawa, ON, 1992), NATO Adv. Sci. Inst. Ser.
C Math. Phys. Sci., vol. 424, Kluwer Acad. Publ., Dordrecht, 1994,
pp. 79–93.

[5]
Siad Daboul, Jan Mangaldan, Michael Z. Spivey, and Peter J. Taylor, The
Lah numbers and the $n$th derivative of $e^{1/x}$, Math. Mag.
86 (2013), no. 1, 39–47.

[6]
Takashi Hashimoto, $B_{n-1}$-orbits on the flag variety ${\rm
GL}_n/B_n$, Geom. Dedicata 105 (2004), 13–27.

[7]
Peter Magyar, Bruhat order for two flags and a line, J. Algebraic
Combin. 21 (2005), no. 1, 71–101.

[8]
Peter Magyar, Jerzy Weyman, and Andrei Zelevinsky, Multiple flag
varieties of finite type, Adv. Math. 141 (1999), no. 1, 97–118.

[9]
R. W. Richardson and T. A. Springer, The Bruhat order on symmetric
varieties, Geom. Dedicata 35 (1990), no. 1-3, 389–436.

[10]
Richard P. Stanley, Enumerative combinatorics. Vol. 2, Cambridge
University Press, Cambridge, 1999.