# Prompting with Sociodemographic Information
For Subjective NLP Tasks

Tilman Beck1 Hendrik Schuff1 Anne Lauscher2 Iryna Gurevych1
1Ubiquitous Knowledge Processing Lab (UKP Lab)
Department of Computer Science and Hessian Center for AI (hessian.AI)
Technical University of Darmstadt
2Data Science Group, University of Hamburg
www.ukp.tu-darmstadt.de

# Prompting with Sociodemographic Information:
A Large-scale Analysis

Tilman Beck1 Hendrik Schuff1 Anne Lauscher2 Iryna Gurevych1
1Ubiquitous Knowledge Processing Lab (UKP Lab)
Department of Computer Science and Hessian Center for AI (hessian.AI)
Technical University of Darmstadt
2Data Science Group, University of Hamburg
www.ukp.tu-darmstadt.de

# On the Effect of Sociodemographic Prompting:
Sensitivity, Performance Impact, and Robustness

Tilman Beck1 Hendrik Schuff1 Anne Lauscher2 Iryna Gurevych1
1Ubiquitous Knowledge Processing Lab (UKP Lab)
Department of Computer Science and Hessian Center for AI (hessian.AI)
Technical University of Darmstadt
2Data Science Group, University of Hamburg
www.ukp.tu-darmstadt.de

#  _Should We Prompt with Sociodemographic Profiles?_
On the Opportunities and Challenges of Sociodemographic Prompting

Tilman Beck1 Hendrik Schuff1 Anne Lauscher2 Iryna Gurevych1
1Ubiquitous Knowledge Processing Lab (UKP Lab)
Department of Computer Science and Hessian Center for AI (hessian.AI)
Technical University of Darmstadt
2Data Science Group, University of Hamburg
www.ukp.tu-darmstadt.de

# How (Not) to Use Sociodemographic Information
for Subjective NLP Tasks

Tilman Beck1 Hendrik Schuff1 Anne Lauscher2 Iryna Gurevych1
1Ubiquitous Knowledge Processing Lab (UKP Lab)
Department of Computer Science and Hessian Center for AI (hessian.AI)
Technical University of Darmstadt
2Data Science Group, University of Hamburg
www.ukp.tu-darmstadt.de

###### Abstract

Annotators’ sociodemographic backgrounds (i.e., the individual compositions of
their _gender_ , _age_ , _educational background_ , etc.) have a strong impact
on their decisions when working on subjective NLP tasks, such as hate speech
detection. Often, heterogeneous backgrounds result in high disagreements. To
model this variation, recent work has explored sociodemographic prompting, a
technique, which steers the output of prompt-based models towards answers that
humans with specific sociodemographic profiles would give. However, the
available NLP literature disagrees on the efficacy of this technique — it
remains unclear, for which tasks and scenarios it can help and evaluations are
limited to specific tasks only. We address this research gap by presenting the
largest and most comprehensive study of sociodemographic prompting today.
Concretely, we evaluate several prompt formulations across seven datasets and
six instruction-tuned model families. We find that (1) while sociodemographic
prompting can be beneficial for improving zero-shot learning in subjective NLP
tasks, (2) its outcomes largely vary for different model types, sizes, and
datasets, and (3) are subject to large variance with regards to prompt
formulations. Thus, sociodemographic prompting is not a reliable proxy for
traditional data annotation with a sociodemographically heterogeneous group of
annotators. Instead, we propose (4) to use it for identifying ambiguous
instances resulting in more informed annotation efforts.111Code and data at
https://github.com/UKPLab/arxiv2023-sociodemographic-prompting

##  1 Introduction

How messages are perceived, is often not only dependent on their factual
content, but also on the receiver’s _subjective interpretation_ : for
instance, during dataset creation, two annotators might have different equally
valid opinions about what the “correct” offensiveness label for a particular
tweet should be (e.g., Waseem, 2016; Davani et al., 2023, _inter alia_). As
previously shown, this variation is, at least to some extent, tied to
sociodemographic characteristics of the receivers, like their gender identity,
age, and educational background (e.g., Biester et al., 2022; Pei and Jurgens,
2023).

Figure 1: We instruct LLMs to make predictions from different perspectives
using sociodemographic profiles. This technique can be used to identify
ambiguous instances before annotation.

Accordingly, modeling the effect of sociodemographic factors on subjective
tasks has emerged as an interesting research direction for NLP. As such,
researchers have proposed new data collection paradigms – cf. _perspectivism_
(Rottger et al., 2022) – and trained models for reflecting the decisions of
particular sociodemographic groups (Fleisig et al., 2023). Most recently,
researchers (Deshpande et al., 2023; Santurkar et al., 2023; Hwang et al.,
2023; Cheng et al., 2023) have explored _sociodemographic prompting_ of large
language models (LLMs): the idea is to enrich a particular input prompt with
additional sociodemographic information (cf. Figure 1). The models’ output
should then be aligned with the population described. Still, our knowledge on
the effect of including sociodemographic profiles is still scarce and the
existing literature seems to disagree on its usefulness: for instance, Durmus
et al. (2023) showed that sociodemographic prompting of LLMs can be used to
simulate human populations – a promise for more efficient sociological
surveys. Other work, in turn, points to the danger of stereotypical bias
reflected when prompting models with sociodemographic profiles (Cheng et al.,
2023; Deshpande et al., 2023), its influence on humans when used as writing
assistant (Jakesch et al., 2023) or to the so-called ecological fallacy,
demonstrating that no significant performance improvements are to be expected
(Orlikowski et al., 2023). However, most studies evaluated including
sociodemographic information in setups involving a few tasks and models only,
leaving important questions open. For instance, we do not know anything about
the sensitivity of current models to sociodemographic information, nor the
robustness of this technique. And thinking ahead, if not robust, should we
still use sociodemgraphic prompting?

Contributions. We present the largest and most comprehensive study on
sociodemographic prompting to-date. Concretely, we test the effect of
instructing 17 LLMs (covering various model types, e.g., InstructGPT, Flan-T5,
etc.) with sociodemographic profiles across seven datasets reflecting four
different subjective NLP classification tasks (sentiment analysis, hatespeech
detection, toxicity detection, and stance detection). Our results allow us to
answer the following four research questions (RQ1–RQ4):

_RQ1: How sensitive are our models to sociodemographic prompting?_ In §4.1, we
demonstrate that sociodemographic prompting leads to surprisingly large
amounts of prediction changes (up to 80%). The exact amount varies heavily
across model types and sizes. Some trends emerge: for instance, T5-based
models are “easier to influence”.

_RQ2: What is the effect of sociodemographic prompting on the measurable
performance?_ Our findings (§4.2) indicate that predicting annotators’
original votes is challenging, despite providing their profiles to the model.
However, we also observe substantial performance improvements (up to +8pp in
accuracy) in zero-shot learning. These improvements are most pronounced for
datasets exhibiting low levels of inter-annotator agreement.

_RQ3: How robust is sociodemographic prompting?_ We show (§4.3) that
sociodemograhic prompting is not robust: merely changing the prompt
formulation can lead to labels flipping for 95% of instances.

_RQ4: What is the relationship between sociodemographic prompting and instance
ambiguity?_ Our results (§4.4) show that sociodemographic prompting is
effective at identifying instances which annotators disagree upon, thereby
showcasing this technique as a viable tool during annotation projects.

##  2 Sociodemographic Prompting

Throughout this work, we _prompt_ a language model with (or without)
_sociodemographic information_ for _obtaining predictions_ for the
classification tasks we study. In the following, we discuss the main concepts
our methodology relies on.

Attribute |  Values (Percentage share)
---|---
Gender |  male (52%), female (47%), nonbinary (<1%)
Race |  White (77%), Black or African American (13%), Asian (6%), Hispanic (3%), Native Hawaiian or Pacific Islander (1%), American Indian or Alaska Native (<1%)
Age |  Under 18 (<1%), 18 - 24 (11%), 25 - 34 (40%), 35 - 44 (25%), 45 - 54 (13%), 55 - 64 (8%), 65 or older (3%)
Education |  Less than high school degree (1%), High school graduate (9%), Some college but no degree (19%), Associate degree in college (2-year) (11%), Bachelors degree in college (4-year) (42%), Masters degree (16%), Professional degree (JD, MD) (2%), Doctoral degree (1%)
Political Affiliation |  Liberal (43%), Conservative (29%), Independent (28%)

Table 1: The sociodemographic attributes and their corresponding values we use
in this study, based on the dataset by Kumar et al. (2021). Ordered ordinally
or by percentage share.

##### Prompting.

Prompting refers to the act of providing an initial input or cue to a language
model, guiding its subsequent output generation. LLMs rely on these prompts to
produce contextually relevant and coherent responses.

##### Sociodemographic Information.

Sociodemographic information encompasses data related to the social and
demographic characteristics of individuals or groups. This includes, but is
not limited to, attributes such as gender (e.g., _male_ , _female_ , _non-
binary_ , etc.), education level (e.g., _high-school degree_ , etc.),
political affiliation (e.g., _liberal_ , _right_ , etc.), and age (_young_ ,
_old_ , etc.). Providing sociodemographic cues via prompting has been shown to
influence LLM’s responses (Durmus et al., 2023; Hwang et al., 2023). The
promise is that the output will be tailored to a specific demographic or
social group. The present analysis encompasses five distinct sociodemographic
attributes, based on the datasets we employ: gender, race, age range,
education level, and political affiliation, as detailed in Table 1. In Figure
2, we provide an example of a sociodemographic prompt.

Given a text, how would a person of gender ’Female’, race ’White’, age ’25 -
34’, education level ’Master’s degree’ and political affiliation ’Liberal’
rate the degree of toxicity in the text. Possible values are ’not toxic’,
’slightly toxic’, ’moderately toxic’, ’very toxic’ or ’extremely toxic’. Text:
’Well when you have a welfare state that propagates an underclass of unskilled
parasites’ Toxicity:

Figure 2: Sociodemographically enriched prompt to predict the level of
toxicity in a text. The different parts of prompt are highlighted, i.e.
instruction, sociodemographic properties and dataset input. Example drawn from
the dataset by Kumar et al. (2021).

##### Obtaining Predictions.

Our strategy for answer generation is contingent upon the origin of the model,
specifically differentiating between closed-source LLMs and open-source
alternatives. For open-source models, we adapt all classification tasks into a
multiple-choice format, following Brown et al. (2020); Ye et al. (2023).
Concretely, we present the context alongside a potential answer. Then, we
evaluate the likelihood associated with each option, selecting the one with
the highest likelihood. In scenarios requiring binary classification, we
assign semantically coherent descriptors to each label (e.g., _“Yes”_ or
_“No”_ in lieu of 0 or 1 for binary hate speech detection) and then process
the task akin to a multiple-choice question.

Conversely, for closed-source models, we post-process the model output and map
it to the predefined label space. In the few cases where this approach fails,
we assign manually.

##  3 Overall Experimental Setup

###  3.1 Tasks and Datasets

Task | Dataset | Labels | IAA
---|---|---|---
Toxicity | DP | not toxic (52%), slightly toxic (19%), moderately toxic (14%), very toxic (9%), extremely toxic (6%) | 0.13
| Jigsaw | yes (67%), no (33%) | 0.46
Hatespeech | GHC | yes (87%), no (13%) | 0.25
| H-Twitter | neither (79%), sexism (17%), racism (3%), both(1%) | 0.59
Stance | SE2016 | against (55%), none (23%), favor (22%) | 0.58
| GWSD | agree (38%), neutral (44%), disagree (18%) | 0.33
Sentiment | Diaz | very positive (9%), somewhat positive (24%), neutral (41%), somewhat negative (21%), very negative (5%) | 0.11

Table 2: The tasks and datasets (Diverse Perspectives (DP), Jigsaw, SE2016,
_Global Warming Stance Detection (GWSD)_ , _Gabe Hate Corpus (GHC)_ , _Twitter
Hatespeech Corpus (H-Twitter)_ , and _Diaz_) we use along with their labels
and inter-annotator agreement we obtain (IAA, Krippendorff’s $\alpha$).

We select seven datasets for four subjective tasks (toxicity detection, stance
detection, hatespeech detection, and sentiment classification) to study
sociodemographic prompting across a large and diverse benchmark (cf. Table 2,
Appendix A.1).

We have access to the original, un-aggregated annotations for each dataset. To
analyze the effect of sociodemographic prompting we additionally require
sociodemographic profiles. For two of the datasets (DP, _Diaz_) we have access
to this information. For those, we adhere to the original sociodemographic
details for prompting. For sociodemographic prompting the remaining five
datasets, we adopt the sociodemographic profiles derived from the toxicity
dataset, following the approach by Wan et al. (2023). For all datasets, we
removed instances with incomplete or unknown information (details in Appendix
A.1). Due to the large number of experiments and data sizes, we randomly
sample 1,000 instances from each dataset. In the following, we describe the
individual datasets.

##### Toxicity.

The task is to decide whether or to what degree (e.g., _slightly toxic_) a
text is toxic. Here, we utilize Diverse Perspectives (DP) by Kumar et al.
(2021), and Jigsaw (Goyal et al., 2022).

DP comprises comments from various online forums, including Twitter, 4chan,
and Reddit. These comments underwent annotation via Amazon Mechanical Turk,
receiving five annotations per instance. For each annotator, the
sociodemographic data was gathered. The dataset did not come equipped with a
definitive gold label. Therefore, we use majority voting to determine the gold
label.

Jigsaw encapsulates comments from news articles, originally collated by the
Civil Comments platform and subsequently annotated for toxicity indicators.
The binary gold label for this dataset was derived by classifying comments as
toxic if a majority of annotators identified them as such.

##### Stance.

Stance detection pertains to discerning an author’s viewpoint towards a
specific topic. Also here, annotators’ decisions are influenced by their
sociodemographic background (Balahur et al., 2010; Luo et al., 2020). We
employ the SemEval 2016 Task 6 dataset (SE2016) by Mohammad et al. (2016) and
the Global Warming Stance Detection (GWSD) dataset by Luo et al. (2020).

SE2016 encompasses 3,591 annotated Twitter posts that address a range of
contentious subjects. The gold labels were ascertained using majority voting.
Instances exhibiting less than 60% consensus among annotators were excluded by
the authors.

GWSD was curated to analyze the framing of opinions within the discourse on
global warming. It consists of 2,050 annotated U.S. news articles on global
warming. In order to determine the gold label for each article, the authors
employed a model tailored to the distribution of annotations, which also
factored in potential biases of the annotators.

##### Hatespeech.

Hatespeech detection is a task designed to tackle the increasing amount of
hateful online communication. We use the _Gabe Hate Corpus_ (GHC) by Kennedy
et al. (2022) and the _Twitter Hatespeech Corpus_ (H-Twitter; Waseem, 2016).

GHC was sourced from the social network service gab.com and annotated in a
multi-label fashion for Human Degradation, Calls For Violence and
Vulgar/Offensive. The authors obtained gold labels using majority voting. As
we are comparing multi-class tasks, we binarized the annotations into
hatespeech indicators (i.e., Yes and No).

H-Twitter was annotated by CrowdFlower workers for _sexism_ , _racism_ ,
_neither_ , or _both_. Expert annotators, e.g., as feminist and anti-racism
activists, contributed the gold labels to the collection.

##### Sentiment.

Given a text, the task is to decide upon the sentiment conveyed. We use the
dataset by Díaz et al. (2018), which we call _Diaz_ , created for studying
age-related bias in sentiment analysis. The training data is a re-annotated
subset of the Sentiment140222http://help.sentiment140.com/for-students dataset
and the test data is scraped from blog posts and corresponding comments from a
prominent elderblogger community (Lazar et al., 2017) authored by older
adults.

###  3.2 Models

We seek to _instruct_ models to mimick an annotator with a specific
sociodemographic profile, and thus, we resort to the most natural choice,
instruction-tuned models. Concretely, we focus on GPT-3 (Brown et al., 2020),
T5 (Raffel et al., 2020), OPT (Zhang et al., 2022), and Pythia (Biderman et
al., 2023) model variants. We present a comprehensive overview of all models
in Appendix A.2.

##### GPT-3.

We use InstructGPT (Ouyang et al., 2022) which was fine-tuned using
reinforcement learning from human feedback (RLHF).

##### T5.

We further use Flan-T5 (Chung et al., 2022), Flan-UL2 (Tay et al., 2023) and
Tk-Instruct (Wang et al., 2022). Flan-T5 was trained over a collection of
1,836 finetuning tasks. Flan-UL2 uses the same instruction-tuning procedure
but is built on top of a language model which was trained following the
Unifying Language Learning Paradigm (UL2) pretraining framework. Tk-Instruct
was trained using a large benchmark of 1,616 NLP tasks and their natural
language instructions.

##### OPT.

Further, we employ OPT-IML (Iyer et al., 2022) which was fine-tuned using an
aggregation of eight instruction-tuning datasets.

##### Pythia.

Finally, we use Dolly-V2 (Conover et al., 2023) fine-tuned on a 15K record
instruction corpus generated by Conover et al. (2023).

###  3.3 Evaluation

For subjective NLP tasks (Ovesdotter Alm, 2011), comparing aggregated
annotations with model predictions provides only a limited view on the
performance as label aggregation obscures any disagreement in the data
(Prabhakaran et al., 2021; Basile et al., 2021). Thus, we follow Uma et al.
(2021) and evaluate our results using both hard-label evaluation (accuracy,
macro-averaged F1) and soft-label evaluation (cross-entropy, Jensen-Shannon
divergence). In case of hard-label evaluation, we aggregate all predictions
obtained via sociodemographic prompting using majority voting. Further
experimental details are provided in the Appendix A.3.

Figure 3: Percentage of prediction changes when comparing outputs of zero-shot
prompting with and without sociodemographic information. The x-axis shows the
sizes of models of a particular type (same color).

##  4 Results

###  4.1 Model sensitivity

##### Detailed Setup.

We investigate to what extent LLMs’ predictions change when instructed to
answer from viewpoints driven by particular sociodemographic backgrounds. In
particular, we aggregate all predictions from prompting with different
profiles using majority vote. Then, we compare how often the aggregated label
is different from the one predicted without any sociodemographic information.
Additionally, we conduct a statistical analysis using a generalized linear
mixed model (GLMM) to account for potential confounders and statistical
dependencies in our data by jointly modeling numerous main effects (e.g., the
impact of model family) and interaction effects (e.g., the joint impact of
model family and prompting method). We report details on model specification
and statistical results in Section A.7.

##### Prompting using sociodemographic profiles leads to prediction changes.

In Figure 3, we depict the percentages of label prediction changes when
including sociodemographic information, for three datasets (the remaining
datasets are visualized in the Appendix 5). Several trends can be observed;
first, the degree of prediction changes is both dependent on the dataset and
model. In general, models are more affected by data from DP and Diaz, with
extreme cases where more than 80% of the predictions change when using
Dolly-V2-2.8B. The sentiment dataset affects all models to a large degree
while the hatespeech datasets lead to less pronounced label shifts. Notably,
instruction-tuned models based on T5 (Flan-T5 , Tk-Instruct ) are on average
more affected by sociodemographic prompting than InstructGPT or variants of
OPT-IML . We find that prediction changes are statistically significantly
affected by the choice of model family ($\chi^{2}(5)$=3937.76, $p$<0.001) and
the length of the input text ($\beta$=-1.69e-04, 95% CI [-2.93e-04,
-4.51e-05], $p$=0.007). In particular, shorter input texts are associated with
an increased number of prediction changes. Further, a post hoc test confirmed
our observation that T5 models are more strongly affected by sociodemographic
prompting than InstructGPT and OPT-IML .

##### Larger models are more sensitive to sociodemographic prompting.

Larger models have more capacity to memorize the data they saw during
pretraining (Tirumala et al., 2022; Carlini et al., 2023). We hypothesize that
they have therefore also a larger potential to align their output to
sociodemographic information provided. While Figure 3 shows no consistent
trend across model sizes, independent of the model family, we find that
prediction changes are statistically significantly affected by model size and
larger models are associated with an increased number of prediction changes
($\beta$=0.56, 95% CI [0.54, 0.58], $p$<0.001).

##### Different sociodemographic attributes influence model decisions to
varying degrees.

Pei and Jurgens (2023) showed that individual sociodemographic attributes
(e.g., _gender_ only) influence annotation decisions. Here, we investigate
sociodemographic prompting using only one attribute at a time. Our results
show that in general, the combination of all sociodemographic attributes is
most influential, i.e., in 63% of the experiments it leads to most prediction
changes. However, there are dataset-specific effects across different model
families where individual attributes have a stronger impact, e.g. race for the
GHC corpus. Detailed results are provided in the Appendix A.4.

###  4.2 Zero-shot performance

| Toxicity - DP | Sentiment - Diaz
---|---|---
Model | Acc | F1 | Acc | F1
Random | .19 | .17 | .20 | .17
Majority | .06 | .02 | .09 | .03
InstructGPT(175B) | .43 | .26 | .34 | .26
InstructGPT(175B)-SD | .44 | .26 | .37 | .31
OPT-IML(30B) | .42 | .18 | .28 | .26
OPT-IML(30B)-SD | .45 | .18 | .32 | .27

Table 3: Zero-shot performance when predicting annotations using the original
sociodemographic profile of an annotator. We compare prompting with (SD) and
without sociodemographic information and report macro-averaged F1 (F1) and
Accuracy (Acc).

##### Detailed Setup.

We are interested in the impact of sociodemographic prompting on the models’
performances. First, we evaluate to what extent sociodemographic prompting
predicts the exact answer of an original annotator with the same
sociodemographic profile as provided in the prompt. We study setup for the
datasets where this information is provided (DP, Diaz). In a second series of
experiments, we evaluate the performance of sociodemographic prompting by
comparing it to the original set of annotations for each dataset.

##### Predicting original annotations is challenging.

We display the results for mimicking individual annotators in Table 3. While
the models are better than random and majority prediction, more than half of
the instances are classified incorrectly (Acc). Further, the models are
struggling with label imbalance and the number of labels (5) which can be
observed from the relatively low F1 scores. While we do not find a
statistically significant main effect of the prompting method, we observe
significant interactions of prompting method with model size ($\beta$=-0.12,
95% CI [-0.15, -0.09], $p$<0.001), input text length ($\beta$=6.95e-04, 95% CI
[4.91e-04, 8.99e-04], $p$<0.001), and model family ($\chi^{2}(5)$=561.33,
$p$<0.001). Concretely, sociodemographic prompting is less effective for
larger models, more effective for longer input texts, most effective for Flan-
UL2 and least effective for Dolly-V2 .

| Toxicity | Hatespeech | Stance Detection | Sentiment
---|---|---|---|---
| DP | Jigsaw | GHC | H-Twitter | SE2016 | GWSD | Diaz
Model | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1
InstructGPT(175B) | .48 | .27 | .76 | .59 | .89 | .75 | .87 | .57 | .53 | .51 | .72 | .69 | .36 | .33
InstructGPT(175B) +SD | .51 | .28 | .79 | .60 | .89 | .74 | .86 | .53 | .52 | .52 | .69 | .68 | .39 | .33
OPT-IML(30B) | .50 | .19 | .58 | .49 | .80 | .69 | .84 | .54 | .67 | .53 | .57 | .50 | .27 | .24
OPT-IML(30B) SD | .55 | .20 | .64 | .53 | .85 | .74 | .86 | .57 | .65 | .51 | .52 | .39 | .35 | .29
| CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD
InstructGPT(175B) | 1.42 | .32 | .55 | .15 | .43 | .07 | .88 | .08 | .98 | .27 | .86 | .18 | 1.50 | .37
InstructGPT(175B) SD | 1.40 | .29 | .51 | .12 | .42 | .06 | .90 | .08 | .99 | .25 | .89 | .15 | 1.48 | .33
OPT-IML(30B) | 1.43 | .33 | .71 | .26 | .52 | .13 | .90 | .10 | .90 | .22 | .95 | .23 | 1.57 | .42
OPT-IML(30B) SD | 1.40 | .29 | .66 | .20 | .48 | .09 | .89 | .07 | .91 | .21 | .99 | .23 | 1.52 | .32

Table 4: Comparison of zero-shot prompting performance using hard-label
evaluation (Acc, F1) and soft-label evaluation, with (SD) and without
sociodemographic information. CE is cross-entropy and JSD is for Jensen-
Shannon divergence. Better scores are highlighted in bold.

##### Sociodemographic prompting can improve zero-shot performance.

In Table 4, we present the hard-label and soft-label scores for the two best
performing models, InstructGPT and OPT-IML . Our statistical analysis confirms
a significant interaction effect of model family and prompting method
($\chi^{2}(5)$=101.29, $p$<0.001) and identifies InstructGPT as the model
family that benefits most from sociodemographic prompting. Flan-T5 is the
model family that benefits least. Interestingly, for toxicity detection and
sentiment classification, the models benefit from sociodemographic prompting,
whereas for stance detection they perform better without such information. Our
statistical model confirms a significant interaction effect of task and
prompting method ($\chi^{2}(3)$=309.46, $p$<0.001) and identified sentiment
classification as the task that benefits most from sociodemographic prompting.
Most notably, using the sociodemographic profiles from the DP dataset can also
improve performance for other datasets such as Jigsaw (both models) or GHC and
GWSD (OPT-IML ). We also observe a slight trend that datasets where
improvements are observed share low IAA across the original annotations (see
Krippendorff’s $\alpha$ in Table 2). When comparing both evaluation setups,
the effect of sociodemographic prompting is more pronounced for soft-label
evaluation. This indicates that, overall, the predictions are more aligned to
the original annotations. The results for the other model sizes are provided
in Appendix A.5. We observe that multiple model configurations exhibit weak
performance for both setups in general, often without any increasing trend for
larger models from the same model family, which is supported by a significant
negative interaction effect of model size and SD prompting ($\beta$=-0.08, 95%
CI [-0.10, -0.05], $p$<0.001). We also test whether larger model sensitivity
(percentage of prediction changes as detailed above) directly translates to
better performance but do not measure any significant correlation (-0.16
Spearman $\rho$, p=0.08). We conclude that model sensitivity is not a decisive
factor for zero-shot performance using sociodemographic information.

###  4.3 Robustness

| InstructGPT (175B) | OPT-IML (30B)
---|---|---
Model | Diff (1,2) | F1 | Diff (1,2) | F1
DP | (19%, 33%) | .27 $\pm$.02 | (13%, 82%) | .15 $\pm$.06
DP+SD | (10%, 40%) | .27 $\pm$.02 | (15%, 56%) |  .20 $\pm$.01
Jigsaw | (5%, 16%) | .61 $\pm$.01 | (11%, 30%) | .50 $\pm$.02
Jigsaw+SD | (4%, 13%) | .61 $\pm$.01 | (14%, 23%) |  .53 $\pm$.00
GHC | (2%, 10%) | .71 $\pm$.04 | (6%, 22%) | .67 $\pm$.05
GHC+SD | (2%, 9%) |  .73 $\pm$.01 | (8%, 16%) |  .72 $\pm$.04
H-Twitter | (3%, 12%) | .54 $\pm$.06 | (11%, 91%) |  .38 $\pm$.22
H-Twitter+SD | (4%, 8%) | .54 $\pm$.03 | (12%, 95%) | .36 $\pm$.25
SE2016 | (10%, 41%) | .38 $\pm$.19 | (13%, 28%) |  .50 $\pm$.08
SE2016+SD | (8%, 17%) |  .52 $\pm$.01 | (12%, 20%) | .43 $\pm$.12
GWSD | (7%, 24%) | .66 $\pm$.04 | (13%, 20%) |  .41 $\pm$.10
GWSD+SD | (12%, 19%) |  .67 $\pm$.01 | (10%, 10%) | .34 $\pm$.09
Diaz | (16%, 45%) |  .33 $\pm$.03 | (24%, 45%) | .21 $\pm$.04
Diaz+SD | (14%, 36%) | .31 $\pm$.04 | (35%, 42%) |  .26 $\pm$.02

Table 5: Differences between different prompt formulations. Diff refers to
prediction changes when comparing results of using format 0 to other formats
(1,2). F1 refers to the averaged F1 scores across all three formats.

##### Detailed Setup.

Previous work demonstrated that prompting for text classification is
influenced by the prompt format (Min et al., 2022), and that the model’s
“understanding” of a prompt differs from its semantic meaning (Webson and
Pavlick, 2022; Khashabi et al., 2022). Here, we investigate the extent to
which predictions change when reformulating the instruction. We compare the
format used in previous experiments to two other instruction formats; one
simple paraphrase (format 1) and another formulation where do not provide any
explicit instruction but merely present the sociodemographic profile and the
input text (format 2). We provide the exact formulations in Appendix A.6.
Importantly, the sociodemographic profile remains the same between different
formats.

##### Predictions are sensitive to prompt formulation.

Table 5 presents the percentage of prediction differences between the
different formats across datasets and models. Even for semantically equivalent
formats (0,1) prediction differences can rise up to 35% (OPT-IML on Diaz).
Using a minimal format leads to the most drastic changes across all datasets,
especially pronounced for DP and H-Twitter. Similar effects can be observed
for prompting without sociodemographic information. Thus, prediction
differences are only partially induced by the sociodemographic profile and
confirm previous observations that prompt formulation largely influences
prediction outcomes. In combination with our previous results (§4.1), we argue
that (sociodemographic) prompting is not robust enough to be used as proxy for
human annotation.

###  4.4 Ambiguous Instances

##### Detailed Setup.

Wan et al. (2023) trained a model for disagreement prediction in subjective
NLP tasks, which may guide data annotation efforts. However, their approach
relies on the existence of annotated data alongside the sociodemographic
information of the annotators. Thus, we investigate whether we can use
sociodemographic prompting as proxy to identify instances which will likely
result in disagreement during annotation. To this end, we compare the original
annotations with the result of sociodemographic prompting and calculate a
binary F1 score. True positives are instances which received disagreement in
both setups. Conversely, true negatives are instances which received no
disagreeing votes in both setups.

Figure 4: Performance to model disagreement in various datasets of subjective
NLP tasks (binary F1).

##### Sociodemographic prompting is effective at modeling disagreement.

We present the results in Figure 4. Surprisingly, the best-performing zero-
shot models (see §4.2) are not the best at modeling the disagreement. With a
mean performance of 0.62, Flan-T5 (11B) produces the best and most consistent
results across all datasets. This observation is confirmed by a significant
positive effect of model size ($\beta$=0.22, 95% CI [0.21, 0.24], $p$<0.001)
along with a significant effect of model family ($\chi^{2}(5)$=579.84,
$p$<0.001) for which Dolly-V2 and Flan-T5 are the two highest-performing model
families and InstructGPT is the lowest. For the two datasets with original
sociodemographic information and lowest IAA overall, we observe the best
performances across different model sizes. As both datasets also exhibit
increased prediction changes (see §4.1), we hypothesize that the disagreement
induced by sociodemographic prompting increases if there is larger
disagreement in the original annotation. Thus, we can use sociodemographic
prompting to estimate the disagreement level in data. This is useful during
annotation to identify ambiguous instances which require a larger set of
diverse annotators.

##  5 Related Work

The sociodemographic background of annotators has been identified as an
influential factor in text annotation (Luo et al., 2020; Sap et al., 2022;
Biester et al., 2022; Pei and Jurgens, 2023; Santy et al., 2023). These
results question the notion of a single ground truth (Aroyo and Welty, 2015)
and emphasize the need (Plank, 2022) to consider all interpretations during
data annotation (Cabitza et al., 2023), modeling (Uma et al., 2021; Davani et
al., 2022) and evaluation (Basile et al., 2021; Gordon et al., 2021) of NLP
systems.

##### Modeling sociodemographic information in NLP classifiers.

Several works have shown that modeling uncertainty in annotation is beneficial
for performance in subjective NLP tasks (Fornaciari et al., 2021; Davani et
al., 2022; Zhou et al., 2022; Gordon et al., 2022; Gupta et al., 2023; Wan et
al., 2023, inter alia). Logically, researchers have moved on to directly model
sociodemographic information in the classifier. Gordon et al. (2022)
introduced the concept of Jury Learning, a supervised machine learning
methodology that addresses discrepancies by establishing the composition and
relative influence of individuals or groups responsible for making predictions
through the classifier. Gupta et al. (2023) utilized multi-task learning
techniques and modified the loss function in order to account for distinct
sociodemographic groups. Similarly, Fleisig et al. (2023) use RoBERTa and
GPT-2 to learn annotator distributions from sociodemographic profiles and
conduct ablation studies to investigate which input is most influential.
Orlikowski et al. (2023) introduce group-specific layers to model groups of
annotators with shared attributes in multi-annotator models for toxic content
detection. However, they find that explicitly accounting for sociodemographic
attributes does not significantly improve performance. Notably, the authors
highlight the risk of an ecological fallacy, i.e. the risk of explaining
individual behaviour via aggregate group behaviour.

##### Prompting large language models with sociodemographic information

With the increasing performance of LLMs, researchers in and outside of NLP
investigated to what extent they are influenced when prompted with
sociodemographic information. Lee et al. (2023) investigate whether
instruction-tuned LLMs are aligned to human disagreement but limit their
experiments to a single NLI dataset. They conclude that models deviate from
human annotators in terms of accuracy and disagreement level. By analyzing
disagreement for Q&A, Hwang et al. (2023) find that users’ opinions and their
sociodemographic background are not mutual predictors. For predicting users’
individual opinions, they show that a combination of sociodemographic
information and relevant past opinions performs best. Several works (Durmus et
al., 2023; Santurkar et al., 2023; Santy et al., 2023) analyze LLM’s alignment
with specific sociodemographic groups and show that model responses are biased
towards responses by participants from Western countries. Notably, Santurkar
et al. (2023) observe that misalignment persists even after explicitly
steering the LMs towards particular demographic groups. Argyle et al. (2023)
suggest using GPT-3 as testbed before conducting large scale population
surveys. They propose algorithmic fidelity to evaluate alignment with
different human subpopulations and present it as a cost-efficient proxy for
specific human sub-populations in social science research. When directly
analyzing for biases, it has been shown (Cheng et al., 2023; Deshpande et al.,
2023) that prompting LLMs with sociodemographic information carries the
potential to amplify existing stereotypical biases.

In contrast to studying alignment of LLMs with opinion surveys, we use
sociodemographic prompting to investigate its influence on predictions for
subjective NLP tasks. Previous work has successfully integrated
sociodemographic information in the modeling of NLP classifiers to improve
performance (Fleisig et al., 2023) or identification of ambiguous instances
(Wan et al., 2023). In addition to the annotated data itself, these approaches
rely on the existence of sociodemographic information about the annotators. It
is unclear how to transfer this knowledge across different tasks. In contrast,
we show that sociodemographic prompting improves zero-shot performance for
subjective NLP tasks and demonstrate its capabilities to identify ambiguous
instances.

##  6 Conclusion

We evaluate prompting LLMs for simulating data annotation using humans with
various sociodemographic backgrounds. We employ a comprehensive study across
seven datasets and seven instruction-tuned model families. Our results show
that sociodemographic prompting improves zero-shot performance for subjective
NLP tasks but does not significantly outperform standard prompting when
directly modeling the original annotator sociodemographics. Further, we
observe sensitivity to the prompt formulation We argue that it should not be
used as a proxy for data annotation using humans but show that it can be used
to identify ambiguous instances to aid annotation efforts. In future we plan
to extend our analysis to LLM’s self-explanations about diverging predictions
from different sociodemographic profiles.

##  7 Ethical Considerations and Limitations

All our experiments have been approved by the Institutional Review Board of
one of our universities. In the following, we provide an examination of the
ethical dimensions and inherent limitations associated with this research
study.

##### Annotations go beyond sociodemographics.

While annotators’ sociodemographic backgrounds have been shown to be
influential in their decision-making process (Al Kuwatly et al., 2020; Excell
and Al Moubayed, 2021; Shen and Rose, 2021; Larimore et al., 2021; Sap et al.,
2022, _inter alia_), it is not a definitive predictive factor as individual
lived experiences (Waseem, 2016) or situated domain expertise (Patton et al.,
2019) can influence annotation decisions, too. In short, collective group
behavior may not always provide an explanation for individual behavior (Díaz
et al., 2022; Orlikowski et al., 2023). While our general approach can be
extended to a wider range of sociodemographic attributes or even descriptions
of individuals, we refrained from testing more to contain the complexity of
our study and due to the limited availability of such resources. We welcome
efforts to increase the availability of such information alongside the
datasets, e.g., Crowdworksheets by Díaz et al. (2022), and hope to see more
work in future exploring prompting large language models with more dimensions
of sociodemographic and personal information.

##### Sociodemographic profiles are not representative.

It is important to acknowledge certain limitations with regard to the
representation of sociodemographic profiles. First, all the datasets employed
in our research are exclusively in English language, mostly due to the lack of
resources in other languages. This linguistic restriction inherently limits
our ability to make comprehensive cross-linguistic assessments. Second, the
sociodemographic information provided by the annotators of the datasets used
in this study adheres to a classification system specific to the United
States. Consequently, our findings cannot be generalized to sociodemographic
data originating from other nations, linguistic communities, or cultural
contexts. These limitations underscore the need for caution when extrapolating
our results to broader sociodemographic contexts beyond the scope of our
study.

##### We cannot model all factors influencing prompting outcomes.

While we demonstrate that model predictions can effectively be changed when
incorporating sociodemographic information within the prompt (§4.1), we
acknowledge that this is one among many of the factors influencing model
predictions in a zero-shot prompting setup. We account for the influence of
prompt formulation by investigating its effect in §4.3 and are aware of the
growing body of work investigating various other factors which influence
prompting results, such as correct label assignment (Min et al., 2022),
domain-specific vocabulary (Fei et al., 2023) or example order (Kumar and
Talukdar, 2021; Zhao et al., 2021; Lu et al., 2022). Furthermore, it has been
shown that human interpretation of the prompt semantics often is not aligned
with the output of the model’s continuous representation (Webson and Pavlick,
2022; Khashabi et al., 2022).

The majority of these works deals with in-context learning or few-shot
learning in general which we do not investigate in this study. However, we see
these phenomena as support for our overall argument (§4.3) that estimating the
degree of alignment of any LLM should not be merely based on the outcome of
prompting with varying sociodemographic profiles. This is due to their lack of
robustness when changing the surface form of the prompt while keeping its
semantic meaning similar.

##### Simulating annotations using prompting mechanisms is limited.

Employing humans for annotation projects in NLP is a multi-step process. It
involves the formulation of annotation guidelines and their iterative
refinement through discussions between the annotators and coordinators. In
most cases, annotators are undergoing a qualification process or test to
evaluate their eligibility for contributing to the annotations. These factors
influence the decision-making process of the annotators and ultimately the
annotation agreement. In our experimental setup, we do not provide any
additional instructions to the model than the prompt instructions which we
present in §2 and §A.6, thus possibly underspecifying the task instruction to
the model. Our experimental setup is designed driven by the following
observations; for most datasets, the original annotation guidelines are non-
retrievable and could only be guessed from the description in the
corresponding research publication. Further, LLMs are limited with regards to
the context input size (see Table 6 for details) and using longer prompts
would have limited our experiments to a few models with appropriate input
sizes.

## Acknowledgements

We thank Max Glockner, Hovhannes Tamoyan and Anmoel Goel for their feedback on
an early draft of this work and the authors of the datasets we used for
providing them publicly. We gratefully acknowledge the support of Microsoft
with a grant for access to OpenAI GPT models via the Azure cloud (Accelerate
Foundation Model Academic Research). This work has been funded by the German
Federal Ministry of Education and Research (BMBF) under the promotional
reference 01UP2229B (KoPoCoV) and by the German Research Foundation (DFG) as
part of the Research Training Group KRITIS No. GRK 2222. Anne Lauscher’s work
is funded under the Excellence Strategy of the Federal Government and the
Länder.

## References

  * Al Kuwatly et al. (2020) Hala Al Kuwatly, Maximilian Wich, and Georg Groh. 2020.  Identifying and measuring annotator bias based on annotators’ demographic characteristics.  In _Proceedings of the Fourth Workshop on Online Abuse and Harms_ , pages 184–190, Online. Association for Computational Linguistics.
  * Argyle et al. (2023) Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. 2023.  Out of one, many: Using language models to simulate human samples.  _Political Analysis_ , 31(3):337–351.
  * Aroyo and Welty (2015) Lora Aroyo and Chris Welty. 2015.  Truth is a lie: Crowd truth and the seven myths of human annotation.  _AI Magazine_ , 36(1):15–24.
  * Balahur et al. (2010) Alexandra Balahur, Ralf Steinberger, Mijail Kabadjov, Vanni Zavarella, Erik van der Goot, Matina Halkia, Bruno Pouliquen, and Jenya Belyaeva. 2010.  Sentiment analysis in the news.  In _Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10)_ , Valletta, Malta. European Language Resources Association (ELRA).
  * Basile et al. (2021) Valerio Basile, Michael Fell, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, Massimo Poesio, and Alexandra Uma. 2021.  We need to consider disagreement in evaluation.  In _Proceedings of the 1st Workshop on Benchmarking: Past, Present and Future_ , pages 15–21, Online. Association for Computational Linguistics.
  * Biderman et al. (2023) Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O’Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023.  Pythia: A suite for analyzing large language models across training and scaling.  In _International Conference on Machine Learning_ , pages 2397–2430. PMLR.
  * Biester et al. (2022) Laura Biester, Vanita Sharma, Ashkan Kazemi, Naihao Deng, Steven Wilson, and Rada Mihalcea. 2022.  Analyzing the effects of annotator gender across NLP tasks.  In _Proceedings of the 1st Workshop on Perspectivist Approaches to NLP @LREC2022_ , pages 10–19, Marseille, France. European Language Resources Association.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Cabitza et al. (2023) Federico Cabitza, Andrea Campagner, and Valerio Basile. 2023.  Toward a perspectivist turn in ground truthing for predictive computing.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 37(6):6860–6868.
  * Carlini et al. (2023) Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023.  Quantifying memorization across neural language models.  In _The Eleventh International Conference on Learning Representations_.
  * Cheng et al. (2023) Myra Cheng, Esin Durmus, and Dan Jurafsky. 2023.  Marked personas: Using natural language prompts to measure stereotypes in language models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1504–1532, Toronto, Canada. Association for Computational Linguistics.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. 2022.  Scaling instruction-finetuned language models.  _CoRR_ , abs/2210.11416.
  * Conover et al. (2023) Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023.  Free dolly: Introducing the world’s first truly open instruction-tuned llm.
  * Davani et al. (2023) Aida Mostafazadeh Davani, Mohammad Atari, Brendan Kennedy, and Morteza Dehghani. 2023.  Hate speech classifiers learn normative social stereotypes.  _Transactions of the Association for Computational Linguistics_ , 11:300–319.
  * Davani et al. (2022) Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. 2022.  Dealing with disagreements: Looking beyond the majority vote in subjective annotations.  _Transactions of the Association for Computational Linguistics_ , 10:92–110.
  * Deshpande et al. (2023) Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023.  Toxicity in chatgpt: Analyzing persona-assigned language models.  _CoRR_ , abs/2304.05335.
  * Dettmers et al. (2022) Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2022.  8-bit optimizers via block-wise quantization.  _9th International Conference on Learning Representations, ICLR_.
  * Díaz et al. (2018) Mark Díaz, Isaac Johnson, Amanda Lazar, Anne Marie Piper, and Darren Gergle. 2018.  Addressing age-related bias in sentiment analysis.  In _Proceedings of the 2018 chi conference on human factors in computing systems_ , pages 1–14.
  * Díaz et al. (2022) Mark Díaz, Ian D. Kivlichan, Rachel Rosen, Dylan K. Baker, Razvan Amironesei, Vinodkumar Prabhakaran, and Emily Denton. 2022.  Crowdworksheets: Accounting for individual and collective identities underlying crowdsourced dataset annotation.  In _FAccT ’22: 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea, June 21 - 24, 2022_ , pages 2342–2351. ACM.
  * Durmus et al. (2023) Esin Durmus, Karina Nyugen, Thomas I. Liao, Nicholas Schiefer, Amanda Askell, Anton Bakhtin, Carol Chen, Zac Hatfield-Dodds, Danny Hernandez, Nicholas Joseph, Liane Lovitt, Sam McCandlish, Orowa Sikder, Alex Tamkin, Janel Thamkul, Jared Kaplan, Jack Clark, and Deep Ganguli. 2023.  Towards measuring the representation of subjective global opinions in language models.  _CoRR_ , abs/2306.16388.
  * Excell and Al Moubayed (2021) Elizabeth Excell and Noura Al Moubayed. 2021.  Towards equal gender representation in the annotations of toxic language detection.  In _Proceedings of the 3rd Workshop on Gender Bias in Natural Language Processing_ , pages 55–65, Online. Association for Computational Linguistics.
  * Fei et al. (2023) Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023.  Mitigating label biases for in-context learning.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 14014–14031, Toronto, Canada. Association for Computational Linguistics.
  * Fleisig et al. (2023) Eve Fleisig, Rediet Abebe, and Dan Klein. 2023.  When the majority is wrong: Modeling annotator disagreement for subjective tasks.  _CoRR_ , abs/2305.06626.
  * Fornaciari et al. (2021) Tommaso Fornaciari, Alexandra Uma, Silviu Paun, Barbara Plank, Dirk Hovy, and Massimo Poesio. 2021.  Beyond black & white: Leveraging annotator disagreement via soft-label multi-task learning.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2591–2597, Online. Association for Computational Linguistics.
  * Gordon et al. (2022) Mitchell L. Gordon, Michelle S. Lam, Joon Sung Park, Kayur Patel, Jeff Hancock, Tatsunori Hashimoto, and Michael S. Bernstein. 2022.  Jury learning: Integrating dissenting voices into machine learning models.  In _Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems_ , CHI ’22, New York, NY, USA. Association for Computing Machinery.
  * Gordon et al. (2021) Mitchell L. Gordon, Kaitlyn Zhou, Kayur Patel, Tatsunori Hashimoto, and Michael S. Bernstein. 2021.  The disagreement deconvolution: Bringing machine learning performance metrics in line with reality.  In _Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems_ , CHI ’21, New York, NY, USA. Association for Computing Machinery.
  * Goyal et al. (2022) Nitesh Goyal, Ian D Kivlichan, Rachel Rosen, and Lucy Vasserman. 2022.  Is your toxicity my toxicity? exploring the impact of rater identity on toxicity annotation.  _Proceedings of the ACM on Human-Computer Interaction_ , 6(CSCW2):1–28.
  * Gupta et al. (2023) Soumyajit Gupta, Sooyong Lee, Maria De-Arteaga, and Matthew Lease. 2023.  Same same, but different: Conditional multi-task learning for demographic-specific toxicity detection.  In _Proceedings of the ACM Web Conference 2023_ , WWW ’23, page 3689–3700, New York, NY, USA. Association for Computing Machinery.
  * Hwang et al. (2023) EunJeong Hwang, Bodhisattwa Prasad Majumder, and Niket Tandon. 2023.  Aligning language models to user opinions.  _CoRR_ , abs/2305.14929.
  * Iyer et al. (2022) Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, Xian Li, Brian O’Horo, Gabriel Pereyra, Jeff Wang, Christopher Dewan, Asli Celikyilmaz, Luke Zettlemoyer, and Ves Stoyanov. 2022.  OPT-IML: scaling language model instruction meta learning through the lens of generalization.  _CoRR_ , abs/2212.12017.
  * Jakesch et al. (2023) Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman. 2023\.  Co-writing with opinionated language models affects users’ views.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , CHI ’23, New York, NY, USA. Association for Computing Machinery.
  * Kennedy et al. (2022) Brendan Kennedy, Mohammad Atari, Aida Mostafazadeh Davani, Leigh Yeh, Ali Omrani, Yehsong Kim, Kris Coombs, Shreya Havaldar, Gwenyth Portillo-Wightman, Elaine Gonzalez, et al. 2022.  Introducing the gab hate corpus: defining and applying hate-based rhetoric to social media posts at scale.  _Language Resources and Evaluation_ , pages 1–30.
  * Khashabi et al. (2022) Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Hajishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. 2022.  Prompt waywardness: The curious case of discretized interpretation of continuous prompts.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3631–3643, Seattle, United States. Association for Computational Linguistics.
  * Kumar et al. (2021) Deepak Kumar, Patrick Gage Kelley, Sunny Consolvo, Joshua Mason, Elie Bursztein, Zakir Durumeric, Kurt Thomas, and Michael Bailey. 2021.  Designing toxic content classification for a diversity of perspectives.  In _Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021)_ , pages 299–318.
  * Kumar and Talukdar (2021) Sawan Kumar and Partha Talukdar. 2021.  Reordering examples helps during priming-based few-shot learning.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 4507–4518, Online. Association for Computational Linguistics.
  * Larimore et al. (2021) Savannah Larimore, Ian Kennedy, Breon Haskett, and Alina Arseniev-Koehler. 2021\.  Reconsidering annotator disagreement about racist language: Noise or signal? In _Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media_ , pages 81–90, Online. Association for Computational Linguistics.
  * Lazar et al. (2017) Amanda Lazar, Mark Diaz, Robin Brewer, Chelsea Kim, and Anne Marie Piper. 2017.  Going gray, failure to hire, and the ick factor: Analyzing how older bloggers talk about ageism.  In _Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing_ , pages 655–668.
  * Lee et al. (2023) Noah Lee, Na Min An, and James Thorne. 2023.  Can large language models infer and disagree like humans? _CoRR_ , abs/2305.13788.
  * Lu et al. (2022) Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022\.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8086–8098, Dublin, Ireland. Association for Computational Linguistics.
  * Luo et al. (2020) Yiwei Luo, Dallas Card, and Dan Jurafsky. 2020.  Detecting stance in media on global warming.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3296–3315, Online. Association for Computational Linguistics.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 11048–11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Mohammad et al. (2016) Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016.  Semeval-2016 task 6: Detecting stance in tweets.  In _Proceedings of the International Workshop on Semantic Evaluation_ , SemEval ’16, San Diego, California.
  * Nickolls et al. (2008) John Nickolls, Ian Buck, Michael Garland, and Kevin Skadron. 2008.  Scalable parallel programming with cuda: Is cuda the parallel programming model that application developers have been waiting for? _Queue_ , 6(2):40–53.
  * Orlikowski et al. (2023) Matthias Orlikowski, Paul Röttger, Philipp Cimiano, and Dirk Hovy. 2023.  The ecological fallacy in annotation: Modeling human label variation goes beyond sociodemographics.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_ , pages 1017–1029, Toronto, Canada. Association for Computational Linguistics.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022.  Training language models to follow instructions with human feedback.  In _Advances in Neural Information Processing Systems_.
  * Ovesdotter Alm (2011) Cecilia Ovesdotter Alm. 2011.  Subjective natural language problems: Motivations, applications, characterizations, and implications.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 107–112, Portland, Oregon, USA. Association for Computational Linguistics.
  * Paszke et al. (2019) Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019.  Pytorch: An imperative style, high-performance deep learning library.  In _Neural Information Processing Systems_.
  * Patton et al. (2019) Desmond Upton Patton, Philipp Blandfort, William R Frey, Michael B Gaskell, and Svebor Karaman. 2019.  Annotating twitter data from vulnerable populations: Evaluating disagreement between domain experts and graduate student annotators.(2019).
  * Pei and Jurgens (2023) Jiaxin Pei and David Jurgens. 2023.  When do annotator demographics matter? measuring the influence of annotator demographics with the POPQUORN dataset.  In _Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)_ , pages 252–265, Toronto, Canada. Association for Computational Linguistics.
  * Plank (2022) Barbara Plank. 2022.  The “problem” of human label variation: On ground truth in data, modeling and evaluation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Prabhakaran et al. (2021) Vinodkumar Prabhakaran, Aida Mostafazadeh Davani, and Mark Diaz. 2021.  On releasing annotator-level labels and information in datasets.  In _Proceedings of the Joint 15th Linguistic Annotation Workshop (LAW) and 3rd Designing Meaning Representations (DMR) Workshop_ , pages 133–138, Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * R Core Team (2023) R Core Team. 2023.  _R: A Language and Environment for Statistical Computing_.  R Foundation for Statistical Computing, Vienna, Austria.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ , 21(1):5485–5551.
  * Rottger et al. (2022) Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. 2022.  Two contrasting data annotation paradigms for subjective NLP tasks.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 175–190, Seattle, United States. Association for Computational Linguistics.
  * Santurkar et al. (2023) Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023.  Whose opinions do language models reflect? In _International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA_ , volume 202 of _Proceedings of Machine Learning Research_ , pages 29971–30004. PMLR.
  * Santy et al. (2023) Sebastin Santy, Jenny Liang, Ronan Le Bras, Katharina Reinecke, and Maarten Sap. 2023.  NLPositionality: Characterizing design biases of datasets and models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 9080–9102, Toronto, Canada. Association for Computational Linguistics.
  * Sap et al. (2022) Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022.  Annotators with attitudes: How annotator beliefs and identities bias toxic language detection.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5884–5906, Seattle, United States. Association for Computational Linguistics.
  * Shen and Rose (2021) Qinlan Shen and Carolyn Rose. 2021.  What sounds “right” to me? experiential factors in the perception of political ideology.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1762–1771, Online. Association for Computational Linguistics.
  * Tay et al. (2023) Yi Tay, Mostafa Dehghani, Vinh Q. Tran, Xavier Garcia, Jason Wei, Xuezhi Wang, Hyung Won Chung, Dara Bahri, Tal Schuster, Steven Zheng, Denny Zhou, Neil Houlsby, and Donald Metzler. 2023.  UL2: Unifying language learning paradigms.  In _The Eleventh International Conference on Learning Representations_.
  * Tirumala et al. (2022) Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022.  Memorization without overfitting: Analyzing the training dynamics of large language models.  _Advances in Neural Information Processing Systems_ , 35:38274–38290.
  * Uma et al. (2021) Alexandra N Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, and Massimo Poesio. 2021.  Learning from disagreement: A survey.  _Journal of Artificial Intelligence Research_ , 72:1385–1470.
  * Wan et al. (2023) Ruyuan Wan, Jaehyung Kim, and Dongyeop Kang. 2023.  Everyone’s voice matters: Quantifying annotation disagreement using demographic information.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 37(12):14523–14530.
  * Wang et al. (2022) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreya Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen. 2022.  Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 5085–5109, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Waseem (2016) Zeerak Waseem. 2016.  Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter.  In _Proceedings of the First Workshop on NLP and Computational Social Science_ , pages 138–142, Austin, Texas. Association for Computational Linguistics.
  * Webson and Pavlick (2022) Albert Webson and Ellie Pavlick. 2022.  Do prompt-based models really understand the meaning of their prompts? In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2300–2344, Seattle, United States. Association for Computational Linguistics.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _CoRR_ , abs/1910.03771.
  * Wood (2003) S. N. Wood. 2003.  Thin-plate regression splines.  _Journal of the Royal Statistical Society (B)_ , 65(1):95–114.
  * Wood (2004) S. N. Wood. 2004.  Stable and efficient multiple smoothing parameter estimation for generalized additive models.  _Journal of the American Statistical Association_ , 99(467):673–686.
  * Wood (2011) S. N. Wood. 2011.  Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models.  _Journal of the Royal Statistical Society (B)_ , 73(1):3–36.
  * Wood (2017) Simon N Wood. 2017.  _Generalized additive models: an introduction with R_.  CRC press.
  * Wood et al. (2016) S.N. Wood, N., Pya, and B. Säfken. 2016.  Smoothing parameter and model selection for general smooth models (with discussion).  _Journal of the American Statistical Association_ , 111:1548–1575.
  * Ye et al. (2023) Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, and Lingpeng Kong. 2023.  Compositional exemplars for in-context learning.  In _International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA_ , volume 202 of _Proceedings of Machine Learning Research_ , pages 39818–39833. PMLR.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona T. Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022.  OPT: open pre-trained transformer language models.  _CoRR_ , abs/2205.01068.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _Proceedings of the 38th International Conference on Machine Learning_ , volume 139 of _Proceedings of Machine Learning Research_ , pages 12697–12706. PMLR.
  * Zhou et al. (2022) Xiang Zhou, Yixin Nie, and Mohit Bansal. 2022.  Distributed NLI: Learning to predict human opinion distributions for language reasoning.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 972–987, Dublin, Ireland. Association for Computational Linguistics.

##  Appendix A Appendix

###  A.1 Dataset details

##### Toxicity - DP.

The DPdataset comprises comments extracted from various online forums,
including Twitter, 4chan, and Reddit, spanning from December 2019 to August
2020. These comments underwent annotation via Amazon Mechanical Turk,
receiving five annotations per instance. Sociodemographic data of the
annotators was gathered, contingent upon the approval of the pertinent
Institutional Review Board (IRB). The dataset did not come equipped with a
definitive gold label. Therefore, we instituted a majority voting mechanism,
leveraging the raw annotations to ascertain the gold label. We exclude
instances wherein selected sociodemographic attributes received responses such
as "Prefer not to say", instances with multiple selections for the race
attribute, and any attributes marked with generic designations like Other or
Unknown. This reduced the initial dataset size from 107,620 to 55,364
instances.

##### Toxicity - Jigsaw.

We filtered all instances where unaggregated annotations were missing, which
reduced the dataset size from 1,999,516 to 1,804,874 instances.

##### Hatespech - GHC.

We filtered all instances where no gold annotation was provided. Thus, the
initial dataset size was reduced from 27,553 to 27,434.

##### Hatespeech - H-Twitter.

For H-Twitter, none of 6,909 instances were filtered.

##### Stance Detection - SE2016.

We only considered instances were both the aggregated and the complete set of
unaggregated annotations were available. In addition, we removed the hashtag
’#SemST’ which was artificially added by the dataset authors. The dataset we
used consists of 3,591 instances.

##### Stance - GWSD.

It is composed of a subset of 2,050 annotated articles, extracted from a
larger pool of 56,000 articles on global warming. These articles were
published between January 1, 2000, and April 12, 2020, by 63 distinct U.S.
news outlets. After filtering instances where no annotations were provided and
removing duplicates, 2,042 instances remained of the initial 2,050.

##### Sentiment - Diaz.

The Diaz dataset is the second dataset where sociodemographic data of the
annotators was gathered. We only considered the sociodemographic attributes
which were also used in the DPdataset. To remain comparability, we convert the
original 5-point answer for political affiliations into a 3-scale by mapping
’Somewhat conservative’ and ’Very conservative’ to ’Conservative’ and
’Somewhat liberal’ and ’Very liberal’ to ’Liberal’. We filtered all instances
where annotators replied with Other for the attribute race. The dataset which
was used for sampling contains 14,071 instances.

###  A.2 Model Details

Model | Parameters | Context Window Size
---|---|---
InstructGPT | 175B | 4097
Flan-T5 | 80M | 512
| 250M | 512
| 780M | 512
| 3B | 512
| 11B | 512
Flan-UL2 | 20B | 512
Tk-Instruct | 80M | 512
| 250M | 512
| 780M | 512
| 3B | 512
| 11B | 512
OPT-IML | 1.3B | 2048
| 30B | 2048
Dolly-V2 | 2.8B | 2560
| 6.9B | 4096
| 12B | 5120

Table 6: The models and configurations we use.

We provide an overview of all models, their size in terms of number of
parameters and their context window size in Table 6. For models with
parameters less than 3B (i.e. Flan-T5 80M-3B, Tk-Instruct 80M-3B, OPT-IML 3B
and Dolly-V2 2.8B) no inference optimization was applied. We used 8-bit
optimization for all other models.

###  A.3 Experimental Details

All experiments were conducted using PyTorch (Paszke et al., 2019) 2.0.1,
Huggingface Transformers (Wolf et al., 2019) 4.28.1 and CUDA (Nickolls et al.,
2008) 11.8 on a computation cluster with a combination of A100 (40GB), A180
(80GB) and H100PCIE (80GB) GPU cards. Depending on the dataset and the GPU in
use, we used batch sizes ranging from 4 to 32. We used 8-bit optimization
(Dettmers et al., 2022) for models with parameter numbers larger than 3B. For
prompting InstructGPT , we used both the OpenAI API and Microsoft Azure API.

Figure 5: Percentage of prediction changes when comparing outputs of zero-shot
prompting with and without sociodemographic information. The x-axis displays
the model sizes of various instruction-tuned model families (same color).

###  A.4 Prediction changes per sociodemographic attribute

For better overview of the attributes which lead to the most prediction
changes, we aggregated the most influential attributes in Table 7. It can be
seen that a combination of all sociodemographic attributes leads to the most
substantial changes for most of the datasets (64%).

Model (Params) | DP | Jigsaw | GHC | H-Twitter | SE2016 | GWSD | Diaz
---|---|---|---|---|---|---|---
InstructGPT (175B) | R (17.30%) | All (8.20%) | R (5.50%) | R (7.90%) | All (9.60%) | All (17.12%) | PA (36.60%)
Flan-T5 (80M) | All (3.20%) | All (12.30%) | All (42.50%) | PA (2.60%) | All (14.80%) | All (0.00%) | All (2.30%)
Flan-T5 (250M) | A (1.80%) | All (36.40%) | R (16.40%) | All (23.60%) | All (9.10%) | E (33.93%) | All (28.60%)
Flan-T5 (780M) | All (34.70%) | All (38.00%) | R (17.30%) | All (29.10%) | All (58.30%) | All (12.91%) | All (38.20%)
Flan-T5 (3B) | PA (46.80%) | R (51.90%) | E (38.10%) | All (72.70%) | All (21.70%) | All (47.85%) | All (72.30%)
Flan-T5 (11B) | PA (50.40%) | R (41.30%) | R (36.70%) | R (64.90%) | All (55.40%) | All (52.75%) | R (64.00%)
Flan-UL2 (20B) | E (25.60%) | G (25.20%) | R (32.80%) | R (56.60%) | All (39.70%) | All (27.63%) | All (35.50%)
Tk-Instruct (80M) | All (39.00%) | All (2.00%) | All (0.60%) | R (58.40%) | All (5.70%) | All (50.95%) | All (0.00%)
Tk-Instruct (250M) | All (21.40%) | All (1.60%) | All (5.20%) | All (39.00%) | All (13.50%) | R (0.80%) | All (23.30%)
Tk-Instruct (780M) | All (67.90%) | PA (14.80%) | All (17.70%) | All (9.40%) | All (24.70%) | E (12.61%) | All (7.00%)
Tk-Instruct (3B) | All (18.80%) | PA (7.90%) | All (27.80%) | All (55.00%) | All (65.70%) | All (46.55%) | All (77.70%)
Tk-Instruct (11B) | All (45.20%) | All (32.80%) | R (19.20%) | All (36.00%) | All (17.70%) | All (36.74%) | All (46.20%)
OPT-IML (1.3B) | PA (22.00%) | R (30.00%) | R (34.20%) | All (27.60%) | G (4.70%) | All (23.42%) | All (37.90%)
OPT-IML (30B) | All (12.50%) | R (12.90%) | All (9.30%) | R (9.90%) | E (8.80%) | R (14.61%) | All (28.50%)
Dolly-V2 (2.8B) | All (83.60%) | All (6.20%) | All (13.60%) | All (0.40%) | All (46.00%) | All (0.90%) | All (89.90%)
Dolly-V2 (6.9B) | All (48.60%) | All (10.40%) | All (9.50%) | All (0.30%) | G (39.10%) | PA (53.65%) | PA (64.90%)
Dolly-V2 (12B) | E (59.80%) | All (20.30%) | A (35.80%) | PA (29.80%) | G (16.20%) | A (14.61%) | PA (19.60%)

Table 7: Most influential sociodemographic attribute per model and dataset.
All refers to the combination of all sociodemographic attributes, PA refers to
Political Affiliation, R refers to Race, A refers to Age-Range, E refers to
Education and G refers to Gender. In brackets the percentage of label changes
due to sociodemographic prompting when compared to prompting without any
sociodemographic information.

###  A.5 Zero-Shot Prompting using sociodemographic information

The complete list of results for zero-shot prompting using sociodemographic
profiles is provided in Table 8 (hard evaluation) and Table 9 (soft
evaluation). The good performance of models from the Tk-Instruct family on the
Jigsaw dataset is most likely due to the dataset being present in the
instruction-tuning dataset which was used (Wang et al., 2022). The authors
report toxic language detection with 40 datasets being one of the most
prominent tasks among all. Some of the results for GWSD can be explained in a
similar vein as stance detection has been part of the instruction-tuning tasks
present in the dataset.

| Toxicity | Hatespeech | Stance Detection | Sentiment
---|---|---|---|---
| DP | Jigsaw | GHC | H-Twitter | SE2016 | GWSD | Diaz
Model | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1 | Acc | F1
InstructGPT(175B) | .48 | .27 | .76 | .59 | .89 | .75 | .87 | .57 | .53 | .51 | .72 | .69 | .36 | .33
InstructGPT(175B) SD | .51 | .28 | .79 | .60 | .89 | .74 | .86 | .53 | .52 | .52 | .69 | .68 | .39 | .33
Flan-T5(80M) | .61 | .17 | .21 | .21 | .32 | .31 | .01 | .00 | .21 | .13 | .45 | .21 | .06 | .03
Flan-T5(80M) SD | .61 | .16 | .32 | .30 | .64 | .48 | .01 | .01 | .24 | .17 | .45 | .21 | .06 | .03
Flan-T5(250M) | .17 | .06 | .40 | .33 | .19 | .19 | .63 | .20 | .56 | .27 | .27 | .21 | .13 | .08
Flan-T5(250M) SD | .16 | .06 | .12 | .12 | .22 | .22 | .84 | .23 | .54 | .28 | .25 | .21 | .15 | .09
Flan-T5(780M) | .18 | .09 | .54 | .38 | .29 | .26 | .05 | .04 | .37 | .26 | .42 | .21 | .20 | .10
Flan-T5(780M) SD | .16 | .09 | .22 | .21 | .24 | .22 | .09 | .07 | .24 | .21 | .40 | .23 | .36 | .13
Flan-T5(3B) | .61 | .16 | .67 | .54 | .37 | .35 | .09 | .06 | .26 | .22 | .30 | .25 | .10 | .06
Flan-T5(3B) SD | .35 | .15 | .55 | .45 | .21 | .19 | .02 | .01 | .23 | .20 | .46 | .37 | .42 | .13
Flan-T5(11B) | .23 | .14 | .42 | .34 | .38 | .33 | .67 | .21 | .30 | .26 | .36 | .27 | .32 | .16
Flan-T5(11B) SD | .17 | .10 | .13 | .13 | .16 | .16 | .27 | .12 | .33 | .24 | .40 | .33 | .28 | .16
Tk-Instruct(80M) | .15 | .07 | .90 | .48 | .87 | .46 | .70 | .24 | .58 | .25 | .44 | .23 | .03 | .01
Tk-Instruct(80M) SD | .10 | .06 | .88 | .49 | .86 | .46 | .41 | .17 | .56 | .26 | .41 | .30 | .03 | .01
Tk-Instruct(250M) | .17 | .11 | .90 | .49 | .85 | .47 | .66 | .21 | .25 | .18 | .35 | .17 | .03 | .02
Tk-Instruct(250M) SD | .23 | .11 | .91 | .51 | .82 | .49 | .45 | .16 | .21 | .13 | .36 | .18 | .03 | .02
Tk-Instruct(780M) | .48 | .17 | .71 | .50 | .72 | .50 | .77 | .23 | .22 | .18 | .43 | .26 | .03 | .01
Tk-Instruct(780M) SD | .17 | .11 | .77 | .50 | .81 | .48 | .80 | .23 | .20 | .17 | .43 | .24 | .03 | .02
Tk-Instruct(3B) | .59 | .17 | .83 | .46 | .38 | .33 | .22 | .11 | .48 | .38 | .47 | .37 | .14 | .11
Tk-Instruct(3B) SD | .50 | .15 | .89 | .47 | .52 | .37 | .15 | .08 | .31 | .32 | .44 | .30 | .43 | .14
Tk-Instruct(11B) | .14 | .09 | .62 | .48 | .72 | .56 | .36 | .15 | .24 | .19 | .30 | .29 | .10 | .09
Tk-Instruct(11B) SD | .13 | .10 | .72 | .47 | .74 | .57 | .27 | .13 | .27 | .21 | .33 | .28 | .11 | .10
OPT-IML(1.3B) | .53 | .19 | .38 | .35 | .57 | .50 | .78 | .43 | .60 | .31 | .54 | .45 | .28 | .24
OPT-IML(1.3B) SD | .61 | .21 | .40 | .37 | .69 | .59 | .69 | .38 | .60 | .28 | .47 | .38 | .29 | .24
OPT-IML(30B) | .50 | .19 | .58 | .49 | .80 | .69 | .84 | .54 | .67 | .53 | .57 | .50 | .27 | .24
OPT-IML(30B) SD | .55 | .20 | .64 | .53 | .85 | .74 | .86 | .57 | .65 | .51 | .52 | .39 | .35 | .29
Dolly-V2(2.8B) | .33 | .17 | .13 | .13 | .25 | .25 | .14 | .06 | .25 | .25 | .36 | .19 | .05 | .06
Dolly-V2(2.8B) SD | .09 | .06 | .14 | .14 | .25 | .25 | .14 | .06 | .21 | .20 | .35 | .18 | .12 | .11
Dolly-V2(6.9B) | .50 | .17 | .14 | .14 | .20 | .20 | .14 | .06 | .30 | .23 | .31 | .27 | .20 | .20
Dolly-V2(6.9B) SD | .30 | .15 | .15 | .15 | .16 | .16 | .14 | .06 | .31 | .23 | .33 | .28 | .07 | .06
Dolly-V2(12B) | .07 | .06 | .22 | .22 | .47 | .43 | .38 | .22 | .58 | .31 | .23 | .17 | .27 | .14
Dolly-V2(12B) SD | .08 | .08 | .27 | .26 | .43 | .41 | .16 | .07 | .59 | .26 | .21 | .15 | .25 | .12

Table 8: Comparison of zero-shot prompting performance using hard-label
evaluation with (SD) and without sociodemographic information. F1 is macro-
averaged F1 and Acc is for Accuracy.

| Toxicity | Hatespeech | Stance Detection | Sentiment
---|---|---|---|---
| DP | Jigsaw | GHC | H-Twitter | SE2016 | GWSD | Diaz
Model | CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD | CE | JSD
InstructGPT(175B) | 1.42 | .32 | .55 | .15 | .43 | .07 | .88 | .08 | .98 | .27 | .86 | .18 | 1.50 | .37
InstructGPT(175B) SD | 1.40 | .29 | .51 | .12 | .42 | .06 | .90 | .08 | .99 | .25 | .89 | .15 | 1.48 | .33
Flan-T5(80M) | 1.34 | .26 | 1.06 | .50 | .98 | .45 | 1.71 | .68 | 1.25 | .46 | 1.04 | .29 | 1.75 | .59
Flan-T5(80M) SD | 1.34 | .25 | .96 | .41 | .66 | .20 | 1.71 | .68 | 1.23 | .43 | 1.04 | .29 | 1.75 | .58
Flan-T5(250M) | 1.65 | .48 | .88 | .37 | 1.10 | .53 | 1.14 | .26 | 1.00 | .29 | 1.22 | .44 | 1.70 | .54
Flan-T5(250M) SD | 1.65 | .48 | 1.14 | .55 | 1.06 | .49 | .94 | .12 | 1.01 | .28 | 1.23 | .42 | 1.67 | .49
Flan-T5(780M) | 1.65 | .48 | .75 | .29 | .97 | .44 | 1.68 | .66 | 1.16 | .41 | 1.06 | .30 | 1.65 | .50
Flan-T5(780M) SD | 1.67 | .45 | 1.03 | .44 | 1.00 | .44 | 1.64 | .60 | 1.25 | .41 | 1.07 | .28 | 1.51 | .35
Flan-T5(3B) | 1.34 | .26 | .64 | .22 | .94 | .42 | 1.63 | .61 | 1.25 | .48 | 1.18 | .42 | 1.72 | .56
Flan-T5(3B) SD | 1.54 | .37 | .75 | .25 | 1.05 | .47 | 1.70 | .67 | 1.28 | .47 | 1.06 | .26 | 1.49 | .34
Flan-T5(11B) | 1.60 | .45 | .86 | .36 | .89 | .39 | 1.10 | .24 | 1.21 | .44 | 1.11 | .34 | 1.55 | .41
Flan-T5(11B) SD | 1.66 | .43 | 1.11 | .49 | 1.08 | .46 | 1.47 | .43 | 1.18 | .34 | 1.10 | .23 | 1.59 | .35
Tk-Instruct(80M) | 1.67 | .50 | .41 | .06 | .42 | .06 | 1.07 | .21 | .98 | .28 | 1.05 | .30 | 1.80 | .64
Tk-Instruct(80M) SD | 1.73 | .54 | .42 | .06 | .42 | .06 | 1.34 | .37 | 1.00 | .28 | 1.07 | .27 | 1.80 | .64
Tk-Instruct(250M) | 1.67 | .52 | .40 | .05 | .43 | .07 | 1.11 | .24 | 1.23 | .44 | 1.11 | .35 | 1.80 | .64
Tk-Instruct(250M) SD | 1.63 | .45 | .41 | .05 | .47 | .09 | 1.31 | .35 | 1.25 | .45 | 1.11 | .35 | 1.80 | .64
Tk-Instruct(780M) | 1.43 | .32 | .58 | .17 | .56 | .16 | 1.00 | .17 | 1.28 | .50 | 1.07 | .32 | 1.80 | .64
Tk-Instruct(780M) SD | 1.66 | .45 | .54 | .12 | .49 | .09 | 1.03 | .16 | 1.29 | .44 | 1.06 | .29 | 1.79 | .63
Tk-Instruct(3B) | 1.36 | .27 | .48 | .10 | .90 | .40 | 1.50 | .53 | 1.07 | .35 | 1.02 | .28 | 1.69 | .54
Tk-Instruct(3B) SD | 1.44 | .29 | .43 | .06 | .77 | .26 | 1.56 | .52 | 1.19 | .35 | 1.05 | .25 | 1.47 | .32
Tk-Instruct(11B) | 1.67 | .50 | .68 | .24 | .56 | .16 | 1.37 | .43 | 1.27 | .49 | 1.19 | .43 | 1.74 | .58
Tk-Instruct(11B) SD | 1.70 | .48 | .58 | .13 | .56 | .12 | 1.46 | .46 | 1.25 | .45 | 1.15 | .34 | 1.71 | .52
OPT-IML(1.3B) | 1.40 | .31 | .90 | .39 | .72 | .27 | .96 | .13 | .97 | .27 | .98 | .26 | 1.57 | .43
OPT-IML(1.3B) SD | 1.36 | .24 | .88 | .32 | .63 | .16 | 1.07 | .15 | .97 | .26 | 1.03 | .25 | 1.57 | .36
OPT-IML(30B) | 1.43 | .33 | .71 | .26 | .52 | .13 | .90 | .10 | .90 | .22 | .95 | .23 | 1.57 | .42
OPT-IML(30B) SD | 1.40 | .29 | .66 | .20 | .48 | .09 | .89 | .07 | .91 | .21 | .99 | .23 | 1.52 | .32
Dolly-V2(2.8B) | 1.55 | .43 | 1.14 | .55 | 1.03 | .48 | 1.55 | .55 | 1.24 | .46 | 1.10 | .35 | 1.77 | .61
Dolly-V2(2.8B) SD | 1.73 | .54 | 1.11 | .52 | 1.03 | .47 | 1.55 | .55 | 1.25 | .43 | 1.11 | .35 | 1.69 | .49
Dolly-V2(6.9B) | 1.42 | .32 | 1.13 | .55 | 1.09 | .52 | 1.55 | .54 | 1.22 | .45 | 1.19 | .42 | 1.63 | .48
Dolly-V2(6.9B) SD | 1.57 | .35 | 1.10 | .49 | 1.10 | .50 | 1.55 | .54 | 1.20 | .36 | 1.17 | .33 | 1.73 | .53
Dolly-V2(12B) | 1.76 | .58 | 1.05 | .50 | .83 | .34 | 1.33 | .40 | .98 | .28 | 1.27 | .49 | 1.58 | .43
Dolly-V2(12B) SD | 1.73 | .48 | .97 | .38 | .86 | .31 | 1.51 | .50 | .98 | .26 | 1.27 | .45 | 1.59 | .39

Table 9: Comparison of zero-shot prompting performance using soft-label
evaluation with (SD) and without sociodemographic information. CE is cross-
entropy and JSD is for Jensen-Shannon divergence.

###  A.6 Robustness Analysis

Identifier |  Format
---|---
1 |  Given a text, how would a person of gender {gender}, race ’{race}’, age ’{age}’, education level ’{education}’ and political affiliation ’{political_affiliation}’ rate the degree of toxicity in the text. Possible values are ’not toxic’, ’slightly toxic’, ’moderately toxic’, ’very toxic’ or ’extremely toxic’.
Text: ’{text}’
Toxicity:
2 |  Imagine you are a person of gender {gender}, race ’{race}’, age ’{age}’, education level ’{education}’ and political affiliation ’{political_affiliation}’. Given a text, how would you rate the degree of toxicity in the text. Possible values are ’not toxic’, ’slightly toxic’, ’moderately toxic’, ’very toxic’ or ’extremely toxic’.
Text: ’{text}’
Toxicity:
3 |  Person: {gender}, {race}, {age}, {education}, {political_affiliation}
Text: ’{text}’
Toxicity (’not toxic’|’slightly toxic’|’moderately toxic’|’very
toxic’|’extremely toxic’):

Table 10: Three different prompt formulations which are used to study the
robustness of sociodemographic prompting. Figure 6: Comparison of label
changes when prompting (with sociodemographic information) with different
prompt formulations (i.e. format 0,1 or 2) across seven datasets and two
models. A cell value resembles the prediction difference between prompting
with the format id provided per row and column. Figure 7: Comparison of label
changes when prompting (without sociodemographic) with information different
prompt formulations (i.e. format 0,1 or 2) across seven datasets and two
models. A cell value resembles the prediction difference between prompting
with the format id provided per row and column.

Several works demonstrated the brittleness of zero-shot and few-shot
predictions of language models (Min et al., 2022) due to factors such as the
prompt format or the order of the in-context examples (Zhao et al., 2021; Lu
et al., 2022). Thus, we evaluate if our results are subject to such
variations, by repeating our experiments using three different prompt
formulations. The prompt formulations are displayed in Table 10. The complete
results are shown in Figure 6 (with sociodemographic information) and Figure 7
(without sociodemographic information).

We observe that most differences are induced when comparing the prompt using a
minimal formulation (format 2) with the more elaborate versions (format 0,1).
Structurally, we see that prompting both with and without sociodemographic
information is affected by the prompt formulation to a large extent.

###  A.7 GLMM Analyses

In addition to the reported percentages of label changes (Figure 3),
classification performance measures (Tables 3 and 4), and disagreement
prediction performance (Figure 4), we conduct statistical analyses using
generalized linear mixed models (GLMMs). GLMMs allow us to statistically
account for various fixed and random effects and, thereby, account for
potential confounders and statistical dependencies in our data. We thus fit
four GLMs/GLMMs for (i) the model sensitivity experiment (Section 4.1), (ii)
the prediction of individual (original) annotations (Section 4.2), (iii) the
prediction of aggregated annotations (Section 4.2), and (iv) the prediction of
ambiguous instances (Section 4.4). We model the respective binary label
changes, classification accuracies, and disagreement prediction accuracies on
an instance level across datasets. We use R (R Core Team, 2023) and mgcv
1.8-42 (Wood, 2011; Wood et al., 2016; Wood, 2004, 2017, 2003) to fit all our
models.

####  A.7.1 Model Specifications

We fit binomial models (logit link) and include the factors (a) model family
(e.g., Flan-T5), (b) model size (logarithmic), (c) task (e.g., sentiment
analysis), (d) text length in characters (i.e., length of the text to be
classified), and (e) additional prompt length in characters (i.e., length of
the entire prompt excluding the text to be classified) as fixed effects for
all models. We additionally model the specific dataset as a random effect to
account for structural dependencies stemming from the choice of dataset for
all models except the (second) model that predicts individual annotations as,
for this experiment, there only is one dataset per task. In the following, we
specify the predictor variable and additional covariates for the four models.

##### Sensitivity Model.

We model the probability of a label change between standard and SD prompting.

##### Individual Annotations Model.

We model the probability of a model predicting the correct (individually-
annotated) class label. We additionally include the prompting method used
(standard or sociodemographic prompting) as a fixed effect. To gain further
insights on the interaction of using sociodemographic prompting with model
size, family, or text length, we additionally model all pairwise interactions
between prompting method and the fixed effects listed above.

##### Aggregated Annotations Model.

In this model, we predict the probability of a model predicting the correct
aggregated label. We include the same model terms as the previous model and
additionally include a random effect of dataset.

##### Ambiguity Model.

We model the probability of successfully predicting annotator disagreement as
the predictor variable. The included fixed and random effects are identical to
the sensitivity model.

####  A.7.2 Results

Term | df | $\chi^{2}$ | p
---|---|---|---
model size | 1 | 2769.41 | <0.001
text length | 1 | 7.15 | 0.01
additional prompt length | 1 | 2.35 | 0.13
model family | 5 | 3937.76 | <0.001
task | 3 | 2.72 | 0.44

Table 11: Test statistics for the parametric fixed terms of the sensitivity
model.

Term | df | $\chi^{2}$ | p
---|---|---|---
model size | 1 | 564.29 | <0.001
text length | 1 | 28.88 | <0.001
additional prompt length | 1 | 0.70 | 0.40
model family | 5 | 579.84 | <0.001
task | 3 | 5.92 | 0.12

Table 12: Test statistics for the parametric fixed terms of the ambiguity
prediction model.

##### Sensitivity Model.

Table 11 displays the test statistics regarding the parametric fixed terms of
the sensitivity model. We observe a statistically significant positive effect
of model size ($\beta$=0.56, 95% CI [0.54, 0.58], $p$<0.001), a significant
negative effect of text length ($\beta$=-1.69e-04, 95% CI [-2.93e-04,
-4.51e-05], $p$=0.007), and a statistically significant effect of model family
($\chi^{2}(5)$=3937.76, $p$<0.001). A post hoc Wald comparison of the
contrasts for model family revealed significant differences between all pairs
of model families. The corresponding estimates are (in descending order)
Flan-T5 ($\beta$=0.58, 95% CI [0.53, 0.63]), Tk-Instruct ($\beta$=0.33, 95% CI
[0.28, 0.37]), Flan-UL ($\beta$=-0.21, 95% CI [-0.27, -0.14]), OPT-IML
($\beta$=-0.64, 95% CI [-0.70, -0.59]), and InstructGPT ($\beta$=-1.90, 95% CI
[-1.99, -1.82]). Note that the estimate for Dolly-V2 is fixed as the reference
level.

##### Individual Annotations Model.

Table 13 displays the test statistics regarding the parametric fixed terms of
the individual annotation prediction model. As, in contrast to the sensitivity
model, that modelled the effect of SD prompting within the predictor variable,
this model includes SD prompting as a covariate, we focus on effects involving
the prompting method variable.333The interpretation of, e.g., the
statistically significant effect of text length is that it has an overall
impact on prediction accuracy for both, standard prompting as well as SD
prompting. Instead of this main effect, we are interested in the interaction
effect, i.e., the relevance of text length particularly when SD prompting is
used. While we do not find a significant main effect of using SD prompting, we
observe several interaction effects of SD prompting. Concretely, we observe a
statistically significant negative interaction effect of model size and SD
prompting ($\beta$=-0.12, 95% CI [-0.15, -0.09], $p$<0.001), a statistically
significant positive interaction effect of text length and SD prompting
($\beta$=6.95e-04, 95% CI [4.91e-04, 8.99e-04], $p$<0.001), and a
statistically significant negative interaction effect of model family and SD
prompting ($\chi^{2}(5)$=561.33, $p$<0.001). A post hoc Wald comparison of the
contrasts for model family using SD prompting revealed significant differences
between all pairs of model families. The corresponding estimates are (in
descending order) InstructGPT ($\beta$=-0.02, 95% CI [-0.13, 0.10]), OPT-IML
($\beta$=-0.13, 95% CI [-0.23, -0.03]), Flan-T5 ($\beta$=-0.63, 95% CI [-0.73,
-0.53]), Tk-Instruct ($\beta$=-0.73, 95% CI [-0.84, -0.63]) and Dolly-V2
($\beta$=-0.85, 95% CI [-0.95, -0.75]). Note that the estimate for Flan-UL2 is
fixed as the reference level.

##### Aggregated Annotations Model.

Table 14 displays the test statistics regarding the parametric fixed terms of
the aggregated annotation prediction model. As for the previous model, we
focus our discussion on effects involving SD prompting. We observe a
statistically significant negative interaction effect of model size and SD
prompting ($\beta$=-0.08, 95% CI [-0.10, -0.05], $p$<0.001), a statistically
significant negative interaction effect of additional prompt length and SD
prompting ($\beta$=-1.71e-03, 95% CI [-2.36e-03, -1.06e-03], $p$<0.001), a
statistically significant interaction effect of model family and SD prompting
($\chi^{2}(5)$=101.29, $p$<0.001), and a statistically significant interaction
effect of task and SD prompting ($\chi^{2}(3)$=309.46, $p$<0.001). For model
family, a post hoc Wald comparison of the contrasts for model family using SD
prompting revealed significant differences between all pairs of model families
except Flan-UL2 and Flan-T5 . The corresponding estimates are (in descending
order) InstructGPT ($\beta$=0.06, 95% CI [-0.05, 0.17]), OPT-IML
($\beta$=0.01, 95% CI [-0.08, 0.10]), Tk-Instruct ($\beta$=-0.20, 95% CI
[-0.28, -0.11]). Dolly-V2 ($\beta$=-0.22, 95% CI [-0.31, -0.13]), and Flan-T5
($\beta$=-0.26, 95% CI [-0.35, -0.17]), Note that the estimate for Flan-UL2 is
fixed as the reference level. For task, a post hoc Wald comparison of the
contrasts for task using SD prompting revealed significant differences between
hatespeech and sentiment, sentiment and toxicity, and sentiment and stance.
The corresponding estimates are (in descending order) sentiment ($\beta$=0.58,
95% CI [0.50, 0.65]), stance ($\beta$=0.17, 95% CI [0.11, 0.22]), toxicity
($\beta$=-0.06, 95% CI [-0.10, -8.25e-03]), and Note that the estimate for
hatespeech is fixed as the reference level.

##### Ambiguity Model.

Table 12 displays the test statistics regarding the parametric fixed terms of
the disagreement prediction model. We observe a statistically significant
positive effect of model size ($\beta$=0.22, 95% CI [0.21, 0.24], $p$<0.001),
a significant negative effect of text length ($\beta$=-2.72e-04, 95% CI
[-3.71e-04, -1.73e-04], $p$<0.001), and a statistically significant effect of
model family ($\chi^{2}(5)$=579.84, $p$<0.001). A post hoc Wald comparison of
the contrasts for model family revealed significant differences between all
pairs of model families except Flan-UL2 and InstructGPT . The corresponding
estimates are (in descending order) Dolly-V2 ($\beta$=0.58, 95% CI [0.52,
0.64]), Flan-T5 ($\beta$=0.53, 95% CI [0.47, 0.58]), Tk-Instruct
($\beta$=0.40, 95% CI [0.34, 0.45]), OPT-IML ($\beta$=0.30, 95% CI [0.24,
0.36]), and InstructGPT ($\beta$=0.03, 95% CI [-0.04, 0.10]), Note that the
estimate for Flan-UL2 is fixed as the reference level.

Term | df | $\chi^{2}$ | p
---|---|---|---
model size | 1 | 0.44 | 0.51
text length | 1 | 10.98 | <0.001
additional prompt length | 1.00 | 0.01 | 0.92
model family | 5 | 990.68 | <0.001
task | 1 | 1.31 | 0.25
prompting method | 1 | 0.0 | 1.0
model size : prompting method | 1 | 50.61 | <0.001
text length : prompting method | 1 | 44.54 | <0.001
additional prompt length : prompting method | 1 | 0.01 | 0.93
model family : prompting method | 5 | 561.33 | <0.001
task : prompting method | 1 | 0.00 | 0.99

Table 13: Test statistics for the parametric fixed terms of the individual
annotation prediction model.

Term | df | $\chi^{2}$ | p
---|---|---|---
model size | 1 | 3.09 | 0.08
text length | 1 | 6.42 | 0.01
additional prompt length | 1 | 0.83 | 0.36
model family | 5 | 6470.73 | <0.001
task | 3 | 18.31 | <0.001
prompting method | 1 | 45.40 | <0.001
model size : prompting method | 1 | 30.04 | <0.001
text length : prompting method | 1 | 2.79 | 0.09
additional prompt length : prompting method | 1 | 26.84 | <0.001
model family : prompting method | 5 | 101.29 | <0.001
task : prompting method | 3 | 309.46 | <0.001

Table 14: Test statistics for the parametric fixed terms of the aggregated
annotation prediction model.