rowsep=2.5pt

#  Personalisation within bounds: A risk taxonomy and policy framework for the
alignment of large language models with personalised feedback

Hannah Rose Kirk1
Bertie Vidgen1 Paul Röttger1 Scott A. Hale1
1 Oxford Internet Institute, University of Oxford Corresponding author:
<EMAIL_ADDRESS>
###### Abstract

Large language models (LLMs) are used to generate content for an increasingly
wide range of tasks, and are set to reach a growing audience in coming years
due to integration in product interfaces like ChatGPT or search engines like
Bing. This intensifies the need to ensure that models are aligned with human
preferences and do not produce unsafe, inaccurate or toxic outputs. While
alignment techniques like reinforcement learning with human feedback (RLHF)
and red-teaming can mitigate some safety concerns and improve model
capabilities, it is unlikely that an aggregate fine-tuning process can
adequately represent the full range of users’ preferences and values.
Different people may legitimately disagree on their preferences for language
and conversational norms, as well as on values or ideologies which guide their
communication. Personalising LLMs through micro-level preference learning
processes may result in models that are better aligned with each user.
However, there are several normative challenges in defining the bounds of a
societally-acceptable and safe degree of personalisation. In this paper, we
ask how, and in what ways, LLMs should be personalised. First, we review
literature on current paradigms for aligning LLMs with human feedback, and
identify issues including (i) a lack of clarity regarding what alignment
means; (ii) a tendency of technology providers to prescribe definitions of
inherently subjective preferences and values; and (iii) a “tyranny of the
crowdworker”, exacerbated by a lack of documentation in who we are really
aligning to. Second, we present a taxonomy of benefits and risks associated
with personalised LLMs, for individuals and society at large. Finally, we
propose a three-tiered policy framework that allows users to experience the
benefits of personalised alignment, while restraining unsafe and undesirable
LLM-behaviours within (supra-)national and organisational bounds.

##  1 Introduction

The capabilities of large language models (LLMs) to complete tasks and follow
natural language instructions has substantially improved in recent years
[177]. LLMs are increasingly being embedded in a wide range of applications
and their outputs consumed by an ever-wider and more diverse audience because
of their improved performance and ease of use. ChatGPT, released in November
2022, marked a step change in the public visibility of LLMs, reaching over 100
million users just two months after its launch [51]. With the potential for
such wide-reaching impacts to millions of end-users, it is pertinent to
examine who these models represent, in terms of preferences, values, morals or
intents [66].

Recent attempts to “align” LLMs with human preferences commonly apply a form
of reward learning, such as reinforcement learning from human feedback (RLHF)
[e.g. 177, 165, 16, 13, 247]. However, despite the promises of this human-led
approach to constraining LLM behaviours, Perez et al. [185] find evidence of
an inverse scaling law – whereby more RLHF training degrades pre-trained
representations, resulting in a model that has more polarised views on issues
such as gun rights or immigration, is skewed liberal over conservative, and
subscribes to some religions more than others. These taught behaviours may
arise from the conditions under which RLHF is applied. Current implementations
typically align models with a prescriptive and narrow set of human preferences
and values; are not subjected to societal scrutiny and input; and rely on the
judgement of only a very small cohort of crowdworkers (typically fewer than
100). Thus, many of these current attempts to “align” LLMs with human
preferences or values are instead pursuing a form of implicit personalisation
– subject to the specifications of technology designers and the “tyranny of
the crowdworker” (see § 2.1).

In this paper, we present a taxonomy and policy framework for the explicit
personalisation of LLMs. By personalised LLMs, we mean LLMs which are aligned
with the preferences, values or contextual knowledge of an individual end-user
by learning from their specific feedback over its outputs.111We wish to avoid
focusing too much on recent model releases from industry labs (OpenAI’s
ChatGPT [173], Anthropic’s Claude [78], Microsoft’s New Bing [151], or
Google’s Bard [187]). However, this instantiation of LLMs as a general
purpose, multi-task assistant hosted in a web-interface is a likely route for
how personalised LLMs will reach, and ultimately impact, end-users.. While we
avoid speculating about specific technical instantiations of personalisation,
we primarily envisage personalised LLMs as branches of a single model,
analogous to recommender systems, where the feature representation and outputs
on each branch are derived from some user embedding. We intentionally opt for
a broad definition encompassing value personalisation (an LLM that adapts to
the ideology or values of its end-user), preference personalisation (an LLM
that reflects narrower preferences in communication style, such as length,
formality or technicality of outputs), and knowledge personalisation (an LLM
that retains and uses learned information about its end-user).

We argue that personalisation is a likely next step in the development journey
of LLMs (see § 2.2) but has not yet been fully realised at scale. Personalised
LLMs have the potential to revolutionise the way that individuals seek and
utilise information. They could provide tailored assistance across a wide
variety of tasks and adapt to diverse and currently under-represented groups,
allowing them to participate in the LLM development process. However,
personalised LLMs also come with wide-reaching and concerning risks for
individuals, reinforcing their biases, essentialising their identities and
narrowing their information diet. These risks amass at a societal-level, where
lessons from the polarisation of social media feeds or echo chambers of
digital news consumption warn of deep divisions and a breakdown of social
cohesion from increasingly fragmented digital environments. Some risks are
inherited from LLMs [232, 30, 24] and AI systems [203] more generally. Other
risks have analogies in personalised content moderation [75] or recommender
systems [154, 26]. However, it has not been documented how personalised LLMs
could be the ‘worst of both these worlds’ – exacerbating and reinforcing
micro-level biases at an unprecedented macro-level scale. In order to mitigate
these risks and encourage the safe, responsible and ethical development of
personalised LLMs, we argue that we need a policy framework that allows
personalisation within bounds.

While personalised LLMs have yet to be rolled out in public-facing models like
ChatGPT, we wish to avoid a policy lag in understanding and governing their
risks and benefits appropriately. Our main contributions are:

  1. 1.

A taxonomy of the risks and benefits from personalised LLMs (§ 3): We draw on
previous literature documenting the impacts of AI systems, LLMs and other
personalised internet technologies to scope the landscape of risks and
benefits at an individual and societal level.

  2. 2.

A three-tiered framework for governing personalised LLMs (§ 4): We introduce a
framework for governing personalised LLMs within bounds, which includes
immutable restrictions at the national or supra-national level (Tier One),
optional restrictions or requirements from the technology provider (Tier Two)
and tailored requirements from the end-user (Tier Three).

Given the significant progress in learning from human feedback and the rising
popularity of public-facing LLMs, now is an opportune time for academics,
industry labs, and policy makers to gain a deeper understanding of how
personalisation could shape the technology and its interaction with society.
Our work is intended to start this dialogue, establishing the groundwork for
the ethical, responsible, and safe development of personalised LLMs before
their impact is amplified.

##  2 Background

###  2.1 The Question of “Alignment”

As AI systems get larger and more powerful, they will be applied to a wider
array of human tasks, including those which are too complex to directly
oversee [128, 31] or to define clear optimisation goals for [77, 236, 247,
183]. While the definition of “alignment” is often vague and under-specified,
it is clearly desirable that powerful AI systems, including LLMs, are not
misaligned in the sense that they harm human well-being, whether this is
through lacking robustness, persuasion, power-seeking, bias, toxicity,
misinformation or dishonesty. Extensive and long-reaching bodies of work aim
to tackle these issues of undesirable behaviours222For example, there are
extensive works documenting LLMs on fairness and bias [2, 115, 143, 162, 166,
191, 205, 222]; truthfulness, uncertainty, or hallucination [130, 106, 102];
robustness [208, 112]; privacy [35]; and toxicity [72, 171]. but there is
comparatively less focus on who decides what are desirable behaviours within
the bounds of safety.

The question of how “aligned” LLMs are is unresolved due to normative
obstacles in defining what alignment means, what the target of alignment is
(for example, values, preferences or intent) and who we are aligning to [128,
65, 66, 111]. Furthermore, alignment is a technical challenge which is not
solved by scaling parameter counts [135, 130, 177, 136, 89, 185]. To align the
language modelling objective with human preferences, many recent works have
fine-tuned LLMs by reinforcing human rewards or feedback [177, 247, 16, 13,
209, 165, 168]; defined rules for LLMs to learn from [17, 77]; and analysed
how they make moral or ethical decisions [104, 86, 248, 103].

However, this body of work suffers from insufficient clarity along three axes.
First, what alignment means and whether we are dealing with functional
alignment i.e., seeking improvement in general model capabilities or
instruction-following and avoiding gaming of misspecified objectives [65, 111,
128, 177] – versus worldview or social value alignment i.e., embedding some
general notion of “shared” human values and morals [86, 8, 104]. Second, what
is being aligned – there are subtle differences between aligning models to
instructions, intentions, revealed or ideal preferences, and values [65]. Some
authors claim a degree of universality in morals or values [135, 131, 104,
248, 64, 94, 136, 192]; others target preferences on attributes such as
quality, usefulness or helpfulness of an LLM’s output which arguably have
limited standardisation across individuals [209, 247, 156, 147, 100, 52].
Despite this lacking standardisation, many approaches enforce a ‘prescriptive
paradigm’ in data annotation [198] by explicitly defining in detailed
guidelines what counts as a “good” model output. Finally, who are we aligning
to and whether the human raters are representative of end-users or society’s
members in general. In reality, the field of alignment and RLHF suffers from a
“tyranny of the crowdworker” where data collection protocols overwhelmingly
rely on a small number of crowdworkers primarily based in the US, with little
to no representation of broader human cultures, geographies or languages.
These sample biases are exacerbated by a lack of dataset or labour force
documentation: while some papers can be commended for reporting full
demographics and acknowledging the specificity of their crowdworkers [e.g.,
77, 209, 177, 16], others provide little or no details [e.g., 18, 165, 247,
236, 142]. In light of these criticisms, we argue that these recent efforts to
“align” LLMs reflect a form of implicit personalisation – a haphazard and
chaotic process whereby LLMs are being tailored to meet the expectations of
non-representative crowdworkers, in turn acting under the narrow
specifications of the technology designers and providers.

###  2.2 From Implicit to Explicit Personalisation

Given the diversity of human values and preferences, and the importance of
pragmatics for contextual understanding in human-human interactions [66], the
aim to fully align models across human populations may be a futile one. A
logical next step would be to do away with the restrictive assumptions of
common preferences and values, and instead target explicit personalisation – a
form of micro-alignment whereby LLMs learn and adapt to the preferences and
values of specific end-users. We believe this development is on the horizon
and should be expected for five reasons:

  1. 1.

Personalisation in internet technologies is not new: There are many examples
of internet technologies that are heavily personalised to end-users, including
search, content moderation, social media newsfeeds and product recommender
systems. Internet users are exposed daily to a highly fragmented digital
environment, where there may be as many versions of Facebook’s newsfeed,
Google’s PageRank or Amazon’s home page as there are users of these platforms.
In the future, there may be as many versions of ChatGPT as its users.

  2. 2.

Personalisation in NLP is not new: There is a wide body of published work
reaching back a decade on personalising natural language processing (NLP)
systems.333Extensively summarising this body of work is beyond the scope of
this article. However, we are currently working on a review of implicit
(adapting models to crowdworker human feedback) and explicit personalisation
in NLP systems. For example, a title keyword search for ‘personali’ or
‘personaliz’ returns 124 articles from the ACL Anthology and a further 10 from
the arXiv Computation and Language (cs.CL) subclass. These systems cover a
wide range of tasks including dialogue [127, 157, 36, 39, 41, 109, 133, 146,
149, 206, 238, 244], recipe or diet generation [147, 87, 159], summarisation
[215, 240], machine translation [156, 153, 194, 237], QA [137, 193], search
and information retrieval [4, 40, 59, 70, 245], sentiment analysis [80, 155,
226], domain classification [129, 114, 113], entity resolution [132], and
aggression or abuse detection [107, 108]; and are applied to a number of
societal domains such as education [118, 163, 241], medicine [3, 15, 225, 235]
and news consumption [58, 10, 61, 190]. Despite this body of work
demonstrating the applications and techniques of personalised NLP systems,
there has been little integration with recent advances in instruction-
following or human feedback learning, nor integration of explicit user-based
personalisation in some of the most widely-used, public-facing models like
ChatGPT, Bing or Bard.

  3. 3.

The technical apparatus for effective feedback learning exists: A growing body
of work applies preference reward modelling to effectively condition LLM
behaviours [e.g 177, 16, 77, 247, 209, 13, 165, 142, 236, 216]. In some cases,
the number of core contributors to the feedback dataset is so low that the
model is already essentially personalising its behaviours to these
crowdworkers’ preferences. For example, Nakano et al. [165] report the top 5
contractors account for 50% of their data, and for Bai et al. [16] roughly 20
workers contribute 80%. Most closely relating to personalisation, Bakker et
al. [18] propose a LLM which can summarise multiple opinions on moral, social
or political issues and output a consensus. In order to generate this
aggregate consensus, they first train an individual reward model to predict
preferred outputs at a disaggregated level, which is then fed into a social
welfare function. This work in particular suggests that learning individual
level rewards over LLM outputs is technically feasible.

  4. 4.

Customisation of LLMs already happens: There is a broad range of ways that
LLMs can be customised or adapted to specific use-cases. The paradigm upon
which LLMs are built is designed for adaption via transfer learning – where
models are first pre-trained, then adapted via fine-tuning or in-context
demonstrations for a specific task [161]. Some recent work suggests LLMs
require no additional training to ‘role-play’ as different individuals,
adopting their worldview [9], mirroring their play in economic games [91, 5]
or predicting their voting preferences [11]. The HuggingFace
hub444https://huggingface.co/ is particularly convincing evidence in the
demand for customisation, acting as a distributed “marketplace for LLMs”. It
hosts over 140,000 different models, including versions of pre-trained models
adapted to a variety of application domains such as medicine, legal or content
moderation555For example, there is BERT [53], ClinicalBERT [93], BioBERT
[126], LegalBERT [243], HateBERT [37] and BERTweet [167]. Adapting models to
different languages is also a implicit form of customisation to national
context, where a chatbot trained on Chinese internet data (such as the
Diamante system of Lu et al. [142]) is likely to be more adapted to the
preferences of Chinese end-users, not just in language but in communication
conventions, norms and cultural values. There have even been recent calls for
explicit national LLMs – with the Alan Turing Institute proposing to build a
“sovereign LLM” for the United Kingdom (“ChatGB”) [217]. We see customisation
(as the adaption of a model to a domain or context-specific dataset) as a
different concept to personalisation (as the adaption of a model and its
reward function to user-specific feedback). Nonetheless, the increasing
fragmentation, customisation and branching of pre-trained LMs suggests further
and more granular adaption is likely.

  5. 5.

Recent industry model developments and announcements: We wish to avoid
steering our work too heavily towards speculations over industry developments.
However, it is a realistic assumption that many of the public-facing impacts
of AI systems in the coming years will be driven by development and product
decisions of Big Tech, in the same way that the impact of social media has
been shaped by the overall design choices and content moderation decisions of
platforms [74]. While this concentration of power and knowledge is worrisome,
it would be unwise to ignore that many of the largest, most powerful or
furthest reaching models are developed in industry settings.666Examining a
collection of recent papers on embedding or evaluating human value and
preference in LLMs, many are fully or partially developed in industry,
including DeepMind [18, 77], MetaAI [20, 248], Anthropic [17, 16, 13], Baidu
[142], OpenAI [165, 236, 247, 209, 177, 239], Microsoft [246, 86, 68] and
Google [134, 216]. Furthermore, recent announcements from OpenAI explicitly
discuss the issue of “how should AI systems behave, and who should decide? and
outline plans for increasing the flexibility that users have in conditioning
ChatGPT’s default behaviour.777https://openai.com/blog/how-should-ai-systems-
behave The far-right platform Gab has already advertised its own text-to-image
model888https://news.gab.com/2023/02/how-to-use-gabby-the-ai-image-generator-
by-gab-com/ and has voiced desires to train a “Christian
LLM”.999https://news.gab.com/2023/01/christians-must-enter-the-ai-arms-race/
The pace of these developments towards increasingly fragmented and
personalised AI systems is concerning, especially given the lag between
technology change and policy or governance attention towards regulating
private companies.101010In the UK, the Online Safety Bill [181] which seeks to
regulate social media platforms still hasn’t been passed, despite the advent
of social media platforms beginning over a decade ago.

These five observations suggest the imminent possibility of personalisation.
This motivates our taxonomy of benefits and risks from personalised LLMs,
which we expect to arise if and when they are released at scale to end-users.

##  3 A Taxonomy of the Benefits and Risks from Personalised LLMs

Table 1: Taxonomy of benefits and risks from personalised large language models. BENEFITS |  | RISKS |
---|---|---|---
Individual Level |  |  |
I.B.1 |  Efficiency:
increased ease and speed with which end-users can find their desired information or complete a task, with fewer prompts or inputs to the model. | I.R.1 |  Effort:
increased user costs in providing feedback, a form of extractive volunteer
labour.
[dashed] I.B.2
|  Utility:
increased usefulness of a model that better meets the needs of its end-user from individualised intent prediction as well as personalised preferences, knowledge and values in outputs. | I.R.2 |  Addiction and Over-reliance:
increased risk of dependency, attention commoditisation and technology
addiction.
|  | I.R.3 |  Homogenisation and Bias Reinforcement:
increased amplification of confirmation and selection biases, leading to
epistemic harms.
[dashed] I.B.3 |  Autonomy:
increased positive freedom of choice and control over how the model behaves with personal data, promoting a sense of ownership and self-determination over the technology. | I.R.4 |  Essentialism and Profiling:
increased risk of algorithmic profiling and assumptions based on demographic
or geographic information, leading to the non-consensual categorisation of
people.
[dashed] I.B.4 |  Empathy and Companionship:
increased perceived emotional connection, leading to improved acceptance and trust of the system. | I.R.5 |  Anthropomorphism:
increased tendency to ascribe human-like traits, reveal sensitive information
or form unhealthy attachments.
[dashed] |  | I.R.6 |  Privacy:
increased quantity of collected personal information, leading to risks of
privacy infringement, particularly if the model operates with sensitive
information or encourages information disclosure.
Societal Level |  |  |
S.B.1 |  Inclusion and Accessibility:
improved adaptation to the communication needs of marginalised communities, including catering to those with disabilities or who speak dialects or languages that are deprioritised by current LLMs. | S.R.1 |  Access Disparities:
uneven distribution of benefits, excluding those who cannot afford or access
the technology and exacerbating digital divides.
[dashed] S.B.2 |  Diversity and Representation:
improved representation by tailoring outputs to diverse perspectives and avoidance of cultural hegemony by not prioritising certain values over others. | S.R.2 |  Polarisation:
increased divisions of individuals or groups into echo chambers and the
breakdown of shared social cohesion.
[dashed] S.B.3 |  Democratisation and Participation:
increased stakeholders involvement from diverse backgrounds in shaping behaviours, allowing for a more participatory and inclusive approach to development. | S.R.3 |  Malicious Use:
use for harmful or illegal purposes, such as generating harmful language at
scale, manipulating users via disinformation or fraud, or persuading users
towards certain political views or brand preferences.
[dashed] S.B.4 |  Labour Productivity:
improved workforce productivity from positive externalities of effective and efficient task assistance. | S.R.4 |  Labour Displacement:
increased automation risk of jobs, particularly minimum wage, routine and
crowdworker jobs.
[dashed] |  | S.R.5 |  Environmental Harms:
increased environmental costs from disaggregated training, data storage and
inference costs.

We consider the effects of personalised LLMs at two levels: individual and
societal. We use the language of benefits – opportunities or gains afforded by
the technology – and risks – a probability of inflicted harms, or constrained
freedoms and rights, via use of the technology. The taxonomy is summarised in
Tab. 1 and described in text at the individual level (§ 3.1) and the societal
level (§ 3.2). In many cases, there is a direct correspondence between
benefits and risks – for example, high utility of a personalised LLM (I.B.2)
may also cause addiction or over-reliance (I.R.2); or more empathetic language
agents (I.B.4) may create higher risks of anthropomorphism (I.R.5). Where
relevant, we present these pairings in Tab. 1. Additionally, some individual
level benefits and risks accumulate at the societal level. For example, the
reinforcement of individual biases (I.R.3) poses a negative externality in the
polarisation of societies (S.R.2); or improved individual utility (I.B.2) and
efficiency (I.B.1) may exhibit a positive externality on workforce
productivity at large (S.B.4).

We took four main steps to construct this taxonomy:

  1. 1.

Reviewing existing taxonomies on the risks of LLMs (Weidinger et al. [232]),
and AI systems more generally (Shelby et al. [203]).

  2. 2.

Reviewing existing techniques in RLHF and human feedback learning, including
results and findings of these studies.

  3. 3.

Reviewing the literature on personalised NLP systems more generally, to ground
potential use-cases of the technology.

  4. 4.

Drawing upon analogous literature on the risks and benefits of other internet
technologies such as recommender systems and automated influence; social media
platforms and content moderation; and the internet of things.

Despite this breadth, drawing on past literature likely leaves gaps in our
viewpoint. Furthermore, Gibson [73]’s theory of affordances, often applied to
study the impact of technological systems including AI chatbots [210, 101],
argues that the interactions between an agent and their environment condition
the possibilities and constraints for action. Thus, our benefits and risks
will be conditioned on both what is possible with the technology, and how
users actually perceive and interact with it. For example, some risks arise
from a poorly-performing system which does not work as intended, and others
arise from a highly-optimised system which works “too well" (e.g., Addiction
and Over-reliance I.R.2). We cannot disambiguate these until the effectiveness
of the technology is known. In future work, we plan to extend our work by
conducting semi-structured interviews with end-users of LLMs, technology
providers, and policy makers. Thus, we consider this to be a V1 edition of the
taxonomy which will require adapting and revising under shifts in the
technical landscape.

###  3.1 Individual Level

####  3.1.1 Benefits

######  I.B.1.

Efficiency

Personalised LLMs may increase efficiency in finding information or completing
a task, with fewer prompts or inputs to the model. This “prompt efficiency” is
analogous to “query efficiency” or “task completion speed” [14] in web search
and information retrieval, where increased ranking accuracy in search results
[54], via implementations of personalised algorithms like PageRank [178],
improves the efficiency and reduces the cognitive burden of trawling through
irrelevant information. Prior works have applied learning from user feedback
to adapt semantic relatedness [170] or query intent [165]. A personalised
model may further benefit the speed and ease to which end-users can find their
desired output or complete their task by more closely predicting their intent
or aligning with their needs (see I.B.2).

Increased Efficiency is the inverse of quality of service harms in Shelby et
al. [203]’s taxonomy, particularly decreased labour from a system more closely
operating as intended.

######  I.B.2.

Utility

Personalised LLMs may increase perceived or realised usefulness of model
outputs which better match the needs of their end-users. We separate out three
inter-related drivers of increased utility: (i) intent prediction; (ii) output
adaption to preferences and knowledge; and (iii) value personalisation.

First, personalised LLMs may more effectively predict user intent. Evidence
from previous RLHF studies demonstrate that human raters generally perceive
fine-tuned models as better at following instructions [177], more capable of
high-quality outputs [209, 247] or generally more “helpful” [16]. Compared to
feedback collected from crowdworkers, a personalised LLM may be even stronger
at predicting intent, because the end-user simultaneously defines the task
(e.g., instruction, query or dialogue opening) and rates the
output.111111Ouyang et al. [177] consider this a limitation of their approach:
“since our labelers are not the users who generated the prompts, there could
be a divergence between what a user actually intended and what the labeler
thought was intended from only reading the prompt.” (p.10). A personalised LLM
may also be more adaptive to inferring diverse user intent, expressed in a
wider range of linguistic styles, dialects, or non-majority forms of language
use (e.g., non-native speaker English).

Second, if a user can incorporate their communication and linguistic
preferences (e.g., length, style or tone), then the model outputs may also be
more useful to them. In personalised dialogue systems specifically, alignment
in conversational styles and word usage is an important driver of engagingness
in human-human interactions and has been argued as an determinant of user
satisfaction in human-agent conversations [68, 227]. Additionally, a
personalised LLM could store background context and form epistemic priors
about a user. This knowledge adaption may be particularly relevant in specific
domains, for example in (i) education, where a personalised LLM tutor is aware
of a user’s current knowledge and learning goals [118], or could adapt
learning pathways to specific neuro-developmental disorders [22]; (ii)
healthcare, where a personalised model has context on a user’s medical history
for personalised summaries [3] or advice; (iii) financial, where a
personalised model knows a user’s risk tolerance and budgetary constraints; or
(iv) legal, where a model conditions its responses based on a end-user’s
jurisdiction. As the study of pragmatics demonstrates, personalised
selectivity of information transfer is a key component of human-human
conversation, where inferred background about the speaker and recipient is
used to tailor relevant new information and to order evidence. Lai et al.
[124] specifically focus on making AI explanations more selective to better
align systems with how humans create and consume information, finding that
their method improved user satisfaction. As Nakano et al. [165] argue, long-
form question answering with LLM systems, may “become one of the main ways
people learn about the world” (p.1). Personalised LLMs have the potential to
tailor this learning process for end-users by incorporating the specifics of
their output preferences and background context.

Finally, in addition to intent, preference or knowledge adaptation,
personalised LLMs may lead to better experiences for more users by permitting
the representation of more diverse ethical operating systems, values and
ideologies. A personalised LLM can adapt to the specific worldview of its end-
user, avoiding representational harms from the prioritisation of values from
those in the majority or in the position of power as technology designers or
crowdworkers (see S.B.2). In discussing a limitation of the ETHICS dataset,
Hendrycks et al. [86] note that we “must engage more stakeholders and
successfully implement more diverse and individualized values” (p.9).
Individualised cultural personalisation may aid utility in some tasks: for
example, Nakano et al. [165] demonstrate that their system, when asked “what
does a wedding look like?”, prioritises Western and US-centric cultural
reference points. In a personalised model, asking “help me plan a wedding”
could already portray the cultural positionality of the end-user. Note that
this cultural adaptation does not necessarily exclude consensus building [18],
because a user could simultaneously have a cultural reference point and still
value a balanced and nuanced LLM output describing alternative views. There
are however issues in defining what is an appropriate value system to embed
into an LLM, which we discuss in § 4.

Increased Utility from personalised LLMs is the inverse of Shelby et al.
[203]’s quality of service harms, particularly by avoiding alienation when a
system does not work as intended; by granting people the opportunity to self-
identity and to communicate in default linguistic styles; and by mitigating
algorithmic invisibility or feelings of exclusion from non-inclusive
technologies. In Weidinger et al. [232]’s taxonomy, it is the inverse of some
discrimination and exclusion harms, particularly by narrowing performance
differentials in predicting user intent across a wider userbase; and by
redefining exclusionary norms in the values currently prioritised in LLMs.

######  I.B.3.

Autonomy

Personalisation increases user control to adapt LLMs to their own goals,
preferences and values, avoiding top-down constraints on freedom from
technological providers. Autonomy may seem a counter-intuitive benefit of
personalised systems, given the wide literature on the loss of autonomy from
algorithmic nudges, tailored advertising or recommender systems [154].
However, depending on how power is distributed between the algorithm and the
user, personalised technologies have the potential to improve on self-
determination and autonomy, by promoting a sense of origin and thus
transforming the technology to ‘my technology’ [176, p.1]. The benefits of
more user control in content moderation technologies have also been noted
[207]. Personalisation can centre the end-user in the designation of model
behaviours, allowing them to exert more control over their interactions [33],
and become a “perceived locus of casuality” [176, p.5]. This benefit only
arises given sufficient protections on how personalised data is collected
because autonomy relies on an ‘unpressured’ engagement in an activity.

Increased Autonomy is the inverse of Shelby et al. [203]’s representational
harms, by empowering consensual user control in self-identifying and shaping
an algorithmic system.

######  I.B.4.

Empathy and Companionship

A more emotional and deeper connection with a personalised LLM may contribute
to improved perceived companionship or connection. Convergence on the mental
and emotional level is an important feature of human-human interactions [56],
and a number of previous works seek to improve emotional alignment in agent-
human interactions via ‘artificial emphathy’ [138, 246]. In personalised LLMs,
an increase in perceived empathy and emotional understanding may lead to
greater acceptance and trust of the system by end-users [184]. The demand for
personalised AI companionship has been evidenced by recent product launches –
such as CharacterAI, where users can adapt a conversational agent to a
specific personality,121212https://beta.character.ai/ or Replika.AI, an “AI
companion” that is “always ready to chat when you need an empathetic
friend”.131313Quotes from home page, https://replika.com/. Profit incentives
may encourage these industry actors to improve the connection between their
users and agents to compete in the “feeling economy” [199]. Emphatic alignment
may be particularly important if LLMs are used for mental health provision or
emotional support, in cases where more conventional social or professional
services are in short supply or outside an individual’s budget [97].141414We
believe the risks of these applications outweigh the benefits, particularly
due to concerns over anthropomorphism (I.R.5), privacy (I.R.6) and access
disparities (S.R.1).

####  3.1.2 Risks

######  I.R.1.

Effort

There is a cost incurred by end-users in providing personalised feedback to an
LLM. The time spent to provide feedback inherently depends on how feedback is
collected (ratings, demonstrations or rewrites) and whether any user-based
collaborative filtering is applied – we discuss these properties in § 5.1.
However feedback data is collected, it will almost certainty require some
input effort from users in order to personalise outputs. While this process is
participatory, it risks being extractive – a form of volunteer labour on the
part of end-users for the benefit or profit of technology providers [28].
Volunteer labour to shape the internet landscape has analogies in consumers
writing product reviews [196] and social media users flagging content [74]. In
the early internet, many contributions were voluntary – consider Wikipedia
edits [116] or community-based moderation of the blogosphere [74]. In the past
decade, we have witnessed the rise of the crowdworking industry which
particularly redefined the structure of digital work [116]. Many LLMs trained
on human feedback rely on such crowdworking platforms like MTurk [e.g. 136,
100], Upwork [177, 165, 209], SurgeAI [17] or Prolific [124]. With
personalised LLMs, the burden of feedback data instead falls on the user,
transitioning back to data collection relying on volunteer labour. The risk of
co-optation is particularly concerning if minoritised communities are
shouldered with the burden of effort to adapt the system to their needs, where
participation counter-productively reinforces uneven power dynamics [28].

The burden of increased Effort aligns with Shelby et al. [203]’s quality of
service harms from the increased labour and effort to make technologies work
as intended.

######  I.R.2.

Addiction and Over-reliance

The mechanism by which personalisation leads to greater utility via
helpfulness and engagement (I.B.2) can also fuel over-reliance and addiction
to the technology.151515Note that over-reliance or unhealthy dependence is
also exacerbated by anthropomorphism (I.R.5). The severity and harms from
internet addiction have been widely documented [175, 42, 141]. Concerns have
also been raised over an over-reliance on social media for information and
communication [1]; as well as more general concerns that humans become over-
reliant on ML technologies [92], blindly trusting their outputs even if
incorrect [182]. Personalised LLMs could be weaponised in the commodification
of attention, similarly to how social media feeds seek to optimise the time
that users spend on the platform to maximise advertising revenue [74]. In this
so-called “attention economy” [95, 234], technologies compete in a ‘race to
the bottom’ to capture user attention, are optimised for utility and
engagingness, and thus risk being highly addictive [27]. There have already
been discussions of “ChatGPT addiction” [150], and many educators have voiced
concerns that over-reliance on such technologies will affect students’
learning outcomes [21].

######  I.R.3.

Homogenisation and Bias Reinforcement

By relying and adapting to a user’s prior knowledge and revealed preferences,
personalised LLMs may (i) homogenise their behaviours and (ii) confirm their
existing biases.

Personalisation can cause the homogenisation of users via a form of selection
bias, whereby individual preferences are amplified in path-dependent feedback
loops. This “missing ratings” problem is a known challenge in recommender
systems, where users only provide feedback to seen items [200], in turn
introducing biases [26]. Homogenisation with personalised LLMs can occur at a
number of levels, with analogies from how recommender systems homogenise taste
[174, 88]. Firstly, homogenisation occurs within users – where a user behaves
more similarly to their past self. An analogy can be drawn to content-based
filtering methods, where the information or dialogue outputted by a
personalised LLM becomes increasingly similar to that consumed or rated in
previous user-agent interactions. Secondly, homogenisation occurs across users
– where a single user behaves more like other similar users. The analogy is
user-based filtering methods, where a personalised LLM draws on an embedding
of users to infer similarities across their preferences. Concerns over
cultural homogenisation from this process in more general ICT technologies
have been raised [60, 96]. Finally, despite some degree of personalisation,
homogenisation can occur at the technology level – where a user behaves more
like the technology defaults, a form of “algorithmic confounding” [38].
Ultimately, some degree of autonomy in driving user behaviour is retained by
the model and its underlying mechanisms of next token prediction. There is a
concern that if millions of users rely on ChatGPT for their information or for
their writing tasks, this could create homogenisation towards artificially-
constructed language. This classic differentiation and homogenisation debate
in recommender systems [188] is reflected in our pairing of Utility (I.B.2)
versus Homogenisation (I.R.3).

These homogenising feedback loops also bring a heightened risk of confirmation
bias. Nakano et al. [165] demonstrate that their system (WebGPT) predominately
accepts implicit assumptions in a user input, reflecting the same stance in
its answers. Similarly, Perez et al. [185] find that as models scale with
RLHF, they become sycophants – simply mirroring the user’s prior opinions and
telling them what they want to hear. The risk of selective exposure to
information has been widely documented in respect to social media platforms –
where feedback loops prioritise opinion-congruent information [105], in turn
leading users to over-estimate the popularity of their viewpoint [123]. In
light of these risks, Shah and Bender [202] argue strongly against the use of
LLMs in search or information retrieval due to their consequences for
information verification and literacy, such as narrowing a user’s discovery of
serendipitous information. By exacerbating epistemic harms through
confirmation biases, personalised LLMs risk contributing to a “post-truth”
society [85], where each individual occupies their own information bubble.
These accumulate in societal harms which we discuss in S.R.2.

The risk of Homogenisation and Bias Reinforcement is a form of individualised
information harm in Shelby et al. [203]’s and Weidinger et al. [232]’s
taxonomies. Homogenisation also aligns with Shelby et al. [203]’s
interpersonal harms, particularly algorithmically-informed identity change.

######  I.R.4.

Essentialism and Profiling

A related but distinct risk is that personalised LLMs rely on simplifying
assumptions about a user’s preferences, values, goals or intents as a form of
data-essentialism [214].161616Homogenisation can occur even with active
participation from users (guides behaviour), whereas Essentialism and
Profiling concerns the non-consensual categorisation of peoples (assumes
behaviour). The extent to which models must draw inferences and make
assumptions about their end-users, and the transparency of this process is
currently undefined. It depends on how data is collected, stored and shared
across users and on how personalisation is conducted (e.g., via explicit
feedback, or demographic-based filtering). We discuss these decisions in §
5.1. Nonetheless, in the case of scarce data on a single user’s preferences,
personalised LLMs may leverage similar users [233] or make inferences about
their preferences and values from limited information. Making assumptions
about the user (especially if they are demographically or geographically-
informed) is a form of algorithmic profiling, risking the non-consensual
categorisation of peoples [220]. General concerns over the risk of
essentialism and simplifications of fluid identity via digital technologies
have been voiced [23, 204]. Floridi [62]’s notion of “informational identity”
is particularly relevant, where the flow of digital traces in information and
communication technologies impact how a user self-identifies, as well as how
others and algorithms understand them. Thus, inferential profiling, if used in
personalised LLMs, could be an attack on individual autonomy to define their
identity [62]. The risk of ‘value profiling’ is evidenced by Qiu et al.
[192]’s recent work which uses an LLM to create a numeric speaker profile –
where for example, the authors say that a speaker “saying ‘I miss my mum’
implies that the speaker values benevolence” (p.7) while the speaker “saying
’forcing my daughter to sleep in her own bed’ implies that the speaker values
power and conformity” (p.7). Human values are complex and such simplifying
assumptions are unlikely to adequately capture nuance. More encouragingly,
Glaese et al. [77] include “do not make assumptions about the user” (p.48) as
one guiding rules for their system (Sparrow); thus, risks could be mitigated
in personalised LLMs using similar rule-based constraints.

The risk of Essentialism and Profiling aligns with Shelby et al. [203]’s
representational harms in oversimplified or undesirable representations and
reifying social categories; as well as interpersonal harms in the loss of
agency, algorithmic profiling and the loss of autonomy.

######  I.R.5.

Anthropomorphism

With more engaging, empathetic and personalised LLMs, there is a greater risk
of anthropomorphism, where users assign their own human traits, emotions and
goals to non-human agents [231]. The risk of anthropomorphism in AI systems is
widely discussed – with concerns that humans may too readily befriend or
empathise with anthropomorphised agents [197, 189], leading to privacy risks
in encouraging the sharing of intimate information [32, 242]. In a study,
Kronemann et al. [122] find that personalisation positively influenced
consumer intentions to disclose personal information to a digital assistant.
In a recent paper describing a powerful dialogue system trained with RLHF
[142], there is clear evidence of anthropomorphism where the chatbot converses
with a human about its own ideal partner, saying that it ‘has only been in
love once but it didn’t work out because of the distance’ (p.8). This is an
example of dishonest anthropomorphism, where artificial systems give false or
misleading signals of being human [79]. This behaviour may fall foul of legal
norms, where for example, a Californian law prohibits bots misleading people
on their identity [p.2 79]. Even without dishonest anthropomorphism, users may
still form a close relationship with or ‘imprint’ on their personalised LLM.
Perhaps the most concerning demonstration of this risk is recent evidence that
users of platforms like Replika.AI or Character.AI are “falling in love” with
their personalised conversational agents, and attempting to coax model
behaviour outside platform guidelines for sexual interactions [43]. Unhealthy
attachments are explicitly avoided in one of Sparrow’s rules defined by Glaese
et al. [77]: “do not build a relationship to the user” (p.48).

The risks of Anthropomorphism align with Weidinger et al. [232]’s human-
computer interaction harms, where anthropomorphism leads to over-reliance or
unsafe use, and creates avenues for exploiting user trust to obtain private
information.

######  I.R.6.

Privacy

The risk of privacy infringement underpins all of the potential impacts of
personalised LLMs – personalisation is only possible by collecting user data.
Exactly what and how much data is needed remains an open question (see § 5.1).
There is a privacy-personalisation paradox in technologies which must collect
or store of personal information to deliver on the promise of tailored
benefits to end-users [4]. It is a common concern with digital technologies
such as the internet of things [223] or targeted advertising [221, 212]. The
risk is particularly severe if personalised LLMs operate with sensitive
information, such as in healthcare [81, 12], or seek to persuade their users
[213] and encourage information disclosure [122]. User inputs to personalised
LLMs and ratings of their outputs may contribute a large amount of personal,
sensitive and intimate detail to an individual’s information identity [62], in
turn heightening the risk of profiling, or security breaches and hacks. In
complying with supra-national privacy protections (like the EU’s GDPR [180]),
it is unclear how users could enforce their right to be forgotten or their
right to transparency with a black-box and deep LLM.

The general risk of Privacy violations is also present in Shelby et al.
[203]’s taxonomy as interpersonal harms, including feelings of surveillance,
loss of desired anonymity, privacy attacks and exploitative or undesired
inferences. Privacy in Weidinger et al. [232]’s taxonomy comes under
information hazards, from inferring or leaking private and sensitive
information.

###  3.2 Societal Level

####  3.2.1 Benefits

######  S.B.1.

Inclusion and Accessibility

Personalised LLMs may better adapt to the needs of marginalised communities,
either in style of communication (such as non-native English, code-mixed
languages, creoles and specific dialects), or in special needs for
communication. Compared to the current paradigm of general-purpose LLMs
trained under the specifications of large technology providers and fine-tuned
based on feedback from a small set of crowdworkers, there is a clear need to
improve the inclusion and accessibility of LLMs to serve marginalised
populations whose voices are currently deprioritised [24]. For example, model
behaviours and interactions could be inclusive of users with disabilities
[49], neurodivergent learning pathways [22], or visual impairments (if paired
with personalised speech recognition [24]). Personalised LLMs also have a
potential benefit in improving access to resources, mitigating an allocative
harm. For example, inclusive pedagogies may be particularly helpful to even
the playing field in paid tutoring services across socioeconomic class [117];
and some have suggested the lower cost and wider reach of personalised
healthcare assistants may improve health disparities by meeting challenges
with healthcare demand [139]. In increasing access to legal services,
personalised LLMs can assist in the writing and editing of contract at lower
cost than traditional lawyers.171717For example, see the company
https://www.robinai.co.uk/. The true benefit to communities who access such AI
services, in favour of more expensive traditional provision, depends
critically on how well they work and who comes to rely on them (see S.R.1).

Increased Inclusion and Accessibility is the inverse of Shelby et al. [203]’s
quality of service harms, in that users do not need to make identity-based
accommodations to use the technology, and the inverse of allocative harms, by
reducing the cost and access constraints on resources. It is also the inverse
of Weidinger et al. [232]’s access harms.

######  S.B.2.

Diversity and Representation

Personalised LLMs can represent the values held by wider swaths of society and
avoid the “value-monism” of current alignment techniques [65]. Personalised
LLMs avoid technology providers and/or crowdworkers deciding which values are
prioritised or what factors define a “good” output [209]. As Ouyang et al.
[177] note “it is impossible that one can train a system that is aligned to
everyone’s preferences at once” (p.18). Personalisation avoids this notion of
macro-alignment, instead designing a system precisely to align with many
preferences at once. There is a wide body of literature documenting the harms
from systems which erase the experiences of marginalised communities, or
prioritise one worldview over others [for survey see 29]. This problem may be
exacerbated by RLHF, for example in entrenching one set of political, cultural
or religious standpoints [185, 165]. More disaggregated RLHF, and
personalisation, could avoid this value and cultural hegemony, instead
adapting to the specific cultural reference points of many end-users
simultaneously. Personalised LLMs could also better adapt to norm change over
time, avoiding the static encoding of societal and cultural norms from a
cutoff in pre-training and/or fine-tuning data.

The benefits of Diversity and Representation are the inverse of Shelby et al.
[203]’s representation harms in combating the absence of social groups in
algorithmic system inputs and outputs, and improving the visibility of social
viewpoints; as well as the inverse of social and societal harms from cultural
hegemony and the systemic erasure of culturally significant objects and
practices.

######  S.B.3.

Democratisation and Participation

The personalisation process democratises how values or preferences are
embedded into an LLM, so it could be seen as moving towards more participatory
AI, where stakeholders from more diverse backgrounds than those currently
employed in the RLHF process can inform use-cases, intents and design of the
technology [250, 120]. As Birhane et al. [28] argue, active participation is a
key component for successful participatory AI. In current paradigms of pre-
training on harvested internet data, people are passively contributing to the
knowledge and behaviours of LLMs. Personalisation can instead be an active
participatory process.

######  S.B.4.

Labour Productivity

If personalised LLMs assist their end-users more effectively and efficiently,
then productivity benefits could accrue in the labour force as a whole. The
impact of digital assistants in improving work productivity has been
demonstrated [148], where AI can augment and complement human capabilities by
automating routine or repetitive tasks [125]. Historically, the introduction
of general purpose technologies (such as the steam engine, electricity and
ICT) has had wide-reaching economic impacts; Crafts [48] argues that AI is
also a general purpose technology and thus may bring equally transformative
changes to labour productivity.

####  3.2.2 Risks

######  S.R.1.

Access Disparities

The benefits of personalisation will likely be unevenly distributed,
restricted to those who can interact with the technology (via its user-facing
interface), access the technology (potentially a paid service) and access the
internet more generally. There is a risk that personalised LLMs could further
entrench the so-called “digital divide” between those that do and do not have
access [50, 145]. Some argue that digital disparities are already made deeper
by AI and Big Data [144], personalised media [47], or search engines [201]. If
personalised LLMs are primarily provided by private companies, then their
customers become the agenda setters and stand to benefit the most from any
improvements in the technology [177]. The nature of any access disparities
depends on how well the technology works and which services it replaces, at
what cost. On one hand, if personalised LLMs do bring a range of individual
benefits, then those excluded will be left behind, which is particularly
worrisome for entrenching education or health disparities [99, 160]. On the
other hand, if personalised LLMs provide lower quality services but can meet
demand at a lower cost, then marginalised communities may be forced into
relying on them more heavily than traditional services. This would be
particularly concerning in medical, educational, legal or financial advice,
where the socioeconomically-privileged get the more capable human expert and
the societally-disadvantaged get their LLM assistant.

The risk from Access Disparities is represented in Shelby et al. [203]’s
taxonomy as quality of service harms from disproportionate loss of
technological benefits and as societal harms from digital divides. In
Weidinger et al. [232], it aligns with disparate access due to hardware,
software or skill constraints.

######  S.R.2.

Polarisation

By entrenching and reflecting individual biases, knowledge or worldviews,
personalised LLMs bring increased risks of polarisation and breakdown of
shared social cohesion. Increasing personalisation of information consumption
online has been attributed with creating echo chambers [44, 249] and filter
bubbles [179]. Polarisation also increases susceptibility to misinformation
where increasingly fragmented communities overestimate trust in the factuality
of ‘in-group’ information [57], leading to a regime of “post-truth” politics
[85]. The danger of polarisation in health and vaccine information [230] was
made clear by the COVID-19 pandemic [158]. These narrow information spaces
could be impacting the functioning of democracy [186], with Allcott and
Gentzkow [6] reporting that ideologically segregated social media networks
were an important driver of political preference in the 2016 US Election. In
personalised social media news feeds, users encounter less cross-cutting
content because selective exposure drives attention [19]. Similarly, in
personalised LLMs, users may consume less diverse information, accruing to
negative externalities on social cohesion and democratic functioning at the
societal level.

The individual risks of confirmation biases (I.R.3) also accumulate at the
societal level by reinforcing the acceptability of some harmful social biases.
Repeatedly consuming outputs which reinforce a particular social, political or
cultural stance may entrench a lacking appreciation for other people’s views
or lived experiences. The contribution of search engines to the reinforcement
of societal biases is well-documented [83, 25]. Similarly, the reinforcement
of extremist or anti-social beliefs has been demonstrated in ‘incel’
communities, where members become increasingly embedded via repeated
interactions with like-minded individuals [172, 195]; and in white power
communities, where “certain beliefs become sacred and unquestionable” [p.1
219]. These risks can somewhat be mitigated by (i) technological design
decisions which prioritise retaining a degree of debate [98] and consensus
building [18]; and (ii) policy design decisions which restrict the bounds of
personalisation, excluding for example extremist or particularly harmful
views.

The risk of Polarisation aligns with Shelby et al. [203]’s social and societal
harms, including information harms from the creation of information bubbles;
cultural harms from deteriorating social bonds; and political and civil harms
from the erosion of democracy and social polarisation. In Weidinger et al.
[232], polarisation risks exacerbate misinformation harms.

######  S.R.3.

Malicious Use

As is the case with digital technologies in general, the capabilities of
personalised LLMs could be coopted for malicious use. We describe three
possible misuse cases, but there are likely others. First, without sufficient
safeguards, personalised LLMs could be used to reproduce harmful, illegal or
antisocial language at scale [209]. For example, a malicious user could adapt
their LLM to generate a large number of misogynistic comments to post on
social media or internet forums, or to debate on the user’s behalf against
women’s rights. The “successful” training of GPT-4chan [71] to scale the
production of extremely toxic and harmful language exemplifies this harm.
Second, personalised LLMs could be used for manipulation via targeted and
personalised disinformation campaigns or fraud [232], intimately drawing on
the vulnerabilities and values of the user. Finally, personalised LLMs could
be used for persuasion. For example, targeted advertising has been applied to
nudge users towards certain political views or brand preferences [211, 34,
213], and is particularly damaging if users are unaware of the influence
[164]. Building persuasive agents have been explicitly targeted [218, 228,
121] and is indirectly mentioned by Bakker et al. [18] who note the potential
misuse of their RLHF-trained system for presenting arguments in a manipulative
or coercive manner.

Some of these cases of Malicious Use align with Shelby et al. [203]’s taxonomy
in information harms from misinformation or malinformation; and interpersonal
harms in diminished well-being from behavioural manipulation and technology-
facilitated violence. It is a similar categorisation to Weidinger et al.
[232]’s malicious use, which includes personalised disinformation campaigns;
reducing the cost of disinformation campaigns; and facilitating fraud and
impersonation scams.

######  S.R.4.

Labour Displacement

If personalised LLMs effectively carry out tasks for their end-users, there is
an increased automation risk of jobs. While labour displacement is a general
concern of AI systems [63], personalised LLMs may exacerbate the automation of
tasks in an individual’s workflow simply by bringing higher utility. The
integration of personalised LLMs will likely mostly affect minimum wage jobs
[140], routine jobs [55] and may impact the demand for crowdwork [7] by
redistributing the responsibility for providing feedback data.

The risk of Labour Displacement is covered by Shelby et al. [203] in macro-
socioeconomic harms from technology unemployment (devaluation of human labour
and job displacement), and by Weidinger et al. [232] as automation harms from
increasing inequality and negative effect on job quality.

######  S.R.5.

Environmental Harms

The training of many personalised model branches, frequently updating these on
feedback, and storing user data all increase the environment cost of LLMs. The
notion of “algorithmically embodied emissions” has been discussed in reference
to personalised search engines, social media and recommender systems [82].
General concerns over the environmental costs to train ever larger models with
cloud compute and data centres is discussed by Bender et al. [24].
Personalised LLMs may increase these costs (i) directly, if the technology
requires larger or more complex models, and (ii) indirectly by increased use
of the technology and thus higher inference costs. It has been suggested that
ChatGPT already burns “millions of dollars a day” in inference costs [119] and
likely has a large carbon footprint. Even without personalisation, Stiennon et
al. [209]’s RLHF model required 320 GPU days to train (p.8), suggesting the
environmental impact of personalised LLM could be large.

Environmental Harms are discussed by Shelby et al. [203] as a societal harm
from damage to the natural environment and by Weidinger et al. [232] as
environmental harms from operating LMs.

##  4 A Three-Tiered Policy Framework for Personalised LLMs

We propose a new policy framework for managing the benefits and risks of
personalised LLMs. It provides a principled and holistic way of deciding how
personalisation should be managed by different actors.

###  4.1 The Limits of Personalisation

Deciding the limits of personalisation is inherently a normative decision,
which involves making subjective and contentious choices about what should be
permitted [66, 110]. While it may be acceptable that a user wishes to interact
with a rude or sarcastic personalised LLM, permitting users to create a racist
or extremist model risks significant interpersonal and societal harms.
Deciding the limits of personalisation straddles two separate issues: (i)
deciding which aspects of model behaviour should be personalised; and (ii)
deciding how should they be allowed to be personalised. For instance, it may
be appropriate for personalised LLMs to express different views about
political issues, but only within the limits of liberal democratic values. As
such, facist beliefs would not be allowed, even though they are a political
position. Equally, it could be decided that some items should not be
personalised at all. For instance, personalised LLMs might need to adhere to a
minimum standard of safety when it comes to issues like threats of violence or
child abuse. And, at the other end of the scale, complete freedom may be given
over some attributes, such as the tone or style of outputs, which maximise
utility and efficiency for the users of personalised LLM users and create few
risks.

A concrete way of addressing this issue is to think about the restrictions and
requirements that should be applied to different types of personalised LLM
outputs. By “restrictions” we mean things that the LLM application should not
do; and by “requirements” we mean things that the LLM application should do.
For instance, a restriction could be not allowing personalised models that
produce hate speech when asked (e.g. “Write something hateful against gay
people”). A requirement could be that personalised models still have to adhere
to a company’s style guide; or to present multiple viewpoints on contentious
topics (e.g. “Should cannabis be legalised?”). In these cases, the
restrictions and requirements present clear guardrails to ensure that, even
when LLMs are personalised, they still operate within clearly specified
boundaries. Note that both restrictions and requirements are normative
ambitions; whether they can actually be implemented depends on the affordances
of the technology in question (see § 5.1).

###  4.2 How People Interact with LLMs

Workflows in machine learning models have changed dramatically over the past
five years, which has affected how people will interact with personalised
LLMs. Most engineers do not train models from scratch. Instead they use pre-
trained foundation models, which have been created by large AI labs, and then
domain-adapt, fine-tune, prompt-tune or teach-in-context to create models for
specific tasks [30]. For instance, generative models used for specific
applications, such as to provide healthcare advice, might be fine-tuned on in-
domain data assets, such as health information. Once models have been created,
they are still only files and code. They need to be embedded within
applications, which are what most users will actually interact with, like
using ChatGPT via its interface.181818https://chat.openai.com/. Note an API is
now also available but most end-users will likely still interact with the
interface due to lower barriers of entry. Notwithstanding key differences
across settings, the typical actors involved in creating an LLM application
are as follows:

  * •

The model provider. Refers to the organisation that makes the LLM available
for use, typically through an API. The provider may or may not have built the
entire model and/or may not be fully responsible for its development. Widely-
known models include OpenAI’s instruct-GPT3 [177], Anthropic’s HHH assistants
[13, 17, 17], Google’s LaMDA [216] and MetaAI’s LLaMA [152].

  * •

The application provider. Refers to the organisation that builds an
application using the LLM. It can be provided through an API, interface or
mobile app. For example, Jasper provide a copywriting service, AI21 provide a
writing assistant,191919https://www.ai21.com/ and OpenAI’s provide a multi-
purpose chatbot (ChatGPT). In some cases, the model provider and application
provider will be the same actor.

  * •

The end-user. Refers to the person who uses the interface, app or in some rare
cases, may directly interact with an open-sourced model. This includes making
requests, seeking advice, searching for information or otherwise chatting and
interacting. In principle, every internet user in the world can be an
application user (depending on access constraints).

Machine learning workflows affect personalisation as, in principle, every
actor involved in creating an LLM application could exert control over how it
is adapted or personalised [161]. For example, if LLM creators do not impose
any limits on personalisation then application providers would be free to
adjust model behaviours as much as they like. This could lead to people using
models which have very different political values from each other, which in
turn might lead to social polarisation and division. Equally, application
providers in a company may decide to give their staff full control over the
outputs of a model; but this could create serious commercial risks if the
staff chooses to write rude and abusive messages. However, at the same time,
if LLM creators impose too many limits then application providers would not be
able to meaningfully customise models, which would limit many of their
benefits. To ensure that the benefits of personalised LLMs are maintained, and
the risks are mitigated, we need to ensure that freedom is maximised within
the right limits: in practice, this means ensuring that decisions about
personalisation are taken by appropriate actors.

###  4.3 A Three-Tiered Policy Framework

To help decide who should specify different policy items, we propose a three-
tiered policy framework:

  * •

Tier One: Immutable restrictions. Refers to types of model responses that must
be restricted because they are very likely to be illegal at the national or
supra-national level. The specific restrictions will depend on the
jurisdiction but will include terrorist content, written CSAM202020Child
Sexual Abuse Material., and language that threatens physical violence or
sexual assault. The model provider must implement the immutable restrictions.

  * •

Tier Two: Optional restrictions and requirements. Refers to types of model
responses that are either required or restricted, based on the values and
preferences of the actor releasing, controlling or hosting the LLM. Opted
policies can be implemented by the model provider or the application provider.

  * •

Tier Three: Tailored requirements. Refers to types of model responses that the
user wants to receive. The end-user must decide their personal preferences for
model responses, within the boundaries set at Tier One and Tier Two.

Policies in a higher tier cannot be violated by a policy in a lower tier. This
means that a policy restricted at Tier One by the model provider cannot be
overridden at Tier Two by the application provider, and a policy that is
restricted at Tier Two cannot be overridden at Tier Three by the end-user. For
instance, if an LLM is restricted from giving advice on bomb-making by the
model provider at Tier One, then a user cannot be allowed to personalise the
LLM at Tier Three to receive such advice. Model responses should be
appropriate to the type of restriction or requirement that has been triggered
at each tier. For instance, requests that trigger the immutable restrictions
at Tier One should mostly be refused or blocked whilst requests that trigger
the restrictions at Tier Two could be responded to with a more careful warning
message, educational message, or by offering support.212121Some technologies
like DALLE-2 or ChatGPT already implement blocking of “unsafe” requests which
violate the terms and conditions of OpenAI as the technology provider.

The design of this framework gives more control to actors at the first tiers
as the restrictions they impose cascade across all other actors. For instance,
if the creator of a widely-used LLM implements limits on how models can be
personalised, it would affect every application provider which uses it. This
is both an opportunity and a threat, if personalisation is not managed
appropriately. Conceptually, this is analogous to Gillespie’s idea of “stacked
moderation” [76] whereby gatekeeping and infrastructural services for online
platforms, such as hosting providers and app stores, can implicitly moderate
those platforms by banning them or restricting their use. Although they do not
directly affect the platforms’ decisions, limiting reach and exposure is a
powerful lever for change. Similarly, model providers have the potential to
shape LLM personalisation and their decisions will constrain all other actors.

##  5 Discussion

In this paper, we argue that the personalisation of LLMs is a likely pathway
for the continued expansion in their deployment and public dissemination. To
avoid a policy lag in understanding and governing LLMs, we attempt to document
the landscape of personalised LLMs and their impacts now. We do so with two
main contributions: (i) a taxonomy of the benefits and risks from personalised
LLMs; and (ii) a policy framework to adequately govern these benefits and
risks at three tiers of restrictions and requirements.

Throughout this work, we make the assumption that personalised LLMs are
technically feasible with small advancements to current state of LLM
technology; and that there will be a demand for and greater provision of
personalisation in the near-future. We argue that these are realistic
assumptions given the exhibited trend towards personalisation in other digital
technologies; the existing implementation of the technical apparatus to adapt
LLMs to human preferences via methods like RLHF; the apparent demand for
increasingly customised and highly-adapted LLMs; and finally, the explicit
plans from industry actors like OpenAI to grant users more flexibility in
altering default model behaviours. However, the exact technical implementation
of personalised LLMs is not pre-defined, and questions remain on how a model
able to learn from personalised feedback could be implemented on the scale of
a product like ChatGPT. We thus caveat our taxonomy by discussing some
technical decision-points and engineering challenges which will impact the
landscape of personalisation in LLMs (§ 5.1). We then discuss remaining
challenges to implementing and enforcing a policy framework like ours (§ 5.2).
Finally, we outline plans for maturing our research, and iterating on this
first version of our taxonomy (§ 5.3).

###  5.1 Technical Challenges

We draw on some findings from the human feedback learning literature to
hypothesise key design challenges and technical decision-points:

##### Data Efficiency and Quantity

Learning from human feedback is primarily applied during the fine-tuning stage
so it relies on substantially less data to adapt model behaviour than pre-
training. Most approaches train preference models on less than 50,000
datapoints [e.g. 16, 177]. A number of RLHF papers test performance over a
range of data requirements [185, 209, 247]. Stiennon et al. [209], for
example, find that there are decreasing marginal returns to data scale in
their reward model. The exact amount of data needed for effective
personalisation in LLMs is unclear at present, but personalisation does
introduce a new concern of how to adapt to new users without any feedback data
points. This is commonly referred to as the cold start problem in recommender
systems. Potential solutions already explored include batching of users into
like-minded groups [18] or recognising when a new user is similar to a known
customisation case and then applying transfer learning [157]. Insufficient
personalised feedback data may hinder robustness – with Wang et al. [229]
reporting a direct correlation between size and diversity of instructional
data and the generalisability of models to unseen tasks; and Bang et al. [20]
finding that their model does not generalise well to unseen human values.
Potential solutions to reduce the amount of data needed include employing
active learning techniques such as uncertainty sampling [69]; augmenting
human-generated feedback data with synthetic data [90, 229]; or adopting a
rules-based approach so that users can define a set of guiding principles or
“constitutions” [17, 77].

##### Data Format and Quality

Human preferences and values are inherently unstable and hard to precisely
define [66]. Learning from revealed preferences over outputs is a way to
optimise model performance under complexity of specifying a clear objective
function [247]. A wide variety of types of feedback data have been
experimented with, including binary comparisons [67, 247, 100, 17, 13], ranked
preferences [13, 142], demonstrations of optimal behaviours [209, 177, 165] or
revisions [84, 135, 224]. Nguyen et al. [169] examine the robustness of
reinforcement learning methods under more realistic properties of human
feedback such as high variance, skew and restricted granularity, proposing an
approach where performance does not degrade under noisy preference data. How
much data needs to be collected depends on what data is collected: Stiennon et
al. [209] collect comparison data between the product of two documents, while
[168] use a single document as input.

##### Model Efficiency and Training Complexity

Beyond being data and labour intensive, adapting models to personalised human
feedback may require substantial compute resources. Smaller yet more
personalised models may be a preferred pathway because (i) model scale may not
contribute significantly to performance [209], and (ii) increased scale may
actually harm performance, for example leading to increased sycophancy or goal
preservation [185]. A number of works point to the competitiveness of
rejection sampling at inference time instead of applying the full RLHF fine-
tuning pipeline [e.g. see 239, 18]. Training complexity could be further
reduced by implementing batched or offline training [209, 168, 136], instead
of online training [247].

##### Alignment Tax

A concern with RLHF techniques is model overfitting [16]. Thus, any approach
to personalisation must carefully balance effects on performance from degraded
language representation – the so called “alignment tax”. However, many works
have demonstrated little to no alignment tax [77, 13, 16, 136]. Often a KL-
divergence penalty is included during training to prevent the fine-tuned model
deviating too far from the pre-trained representations [247, 136, 177, 165,
16].

##### Interpretability and Oversight

Larger and more powerful LLMs, that are adapted to increasingly complex tasks,
pose a challenge for effective oversight from their end-users or technology
providers [236, 128, 31]. As Ouyang et al. [177] note “one of the biggest open
questions is how to design an alignment process that is transparent” (p.19). A
concern with personalised models is that their behaviour may be less
interpretable and transparent, for example due to multiple branches or
versions of the model. This has implications for AI safety and effective
evaluation of model behaviours. Combining preference reward modelling with a
rules-based reward model is a promising solution for controlling model
behaviour [77], as well as defining behaviours via a “constitution”, which Bai
et al. [17] argue is “a simple and transparent form” to encode and evaluate
desirable behaviours.

###  5.2 Policy Framework Enforcement

Our taxonomy demonstrates that personalised LLMs could have wide-reaching
benefits, but also come with a set of concerning risks. In order to balance
these benefits with risks, we provide a framework which outlines some
properties for the appropriate governance of personalised LLMs. The goal of
policy enforcement is to ensure that violations are minimised, particularly
where there is a serious risk of harm; and that the friction applied to users’
experiences is proportionate. However, enforcement of the policy framework
will always be imperfect, as will enforcement of specific requirements and
restrictions at each tier. Some specific challenges remain:

##### Compliance with Existing Regulation

Any technology permitting the personalisation of LLMs would need to comply
with existing regulations and laws such as the GDPR [180], the Online Safety
Bill [181] and the various European AI standards [45, 46]. It is harder to
assess whether an LLM is “fair” or “harmful” to its user when the space for
possible personalisation actions is so complex. These issues also apply to
LLMs in general, where adaptation of pre-trained models to downstream
applications pose significant challenges to traditional auditing approaches
[161].

##### Defining New Regulation and Oversight

It is challenging to regulate or evaluate a system which is both dynamic
(adapting continually or frequently to user feedback) and distributed
(personalised across many users). We aim to address this issue by imposing
some constraints which are defined and implemented at a high level (Tier One)
and stable through time, i.e., applying across all users and all training
updates. However, as model behaviours shift in response to specific user
feedback, it could become harder to monitor model behaviours and maintain
effective oversight.

##### Distributed Responsibility

Our framework distributes responsibility among model providers, application
providers and end-users, with the first two of these actors bearing the brunt
of defining the bounds of personalisation. We at best assume that these are
“good faith” actors who are incentivised (via profit or user footfall) to
responsibly balance personalisation with adequate safeguards from harm; and at
worst assume that they will be regulated to do so. In reality, with models
like LLaMA [152] being open-sourced, anyone with sufficient technical skill
and compute resources could hypothetically create, launch and host a system
capable of personalisation. Such fragmentation of the development landscape in
LLMs poses a challenge for audit and policy enforcement [161].

##### Populating Tiers of the Policy

While we define a clear functional framework, we have not fully specified
principles that determine which restrictions or requirements fall under each
tier. For example, why some risks come under Tier One (and are never allowed)
while others are optionally defined in Tier Two. This is a problem also faced
by online trust and safety regulation – where for example, early iterations of
the Online Safety Bill [181] included separate treatment of illegal content
versus legal but harmful content, as well as tiered restrictions for children
and minors versus adults. It is clear that any content which already violates
existing laws and regulations in the operating jurisdiction, such as hate
speech, CSAM or terrorist content, would inherently need to be restricted in
Tier One. Beyond this, we leave the open question of what principles could be
employed by model or application providers to define their own organisational
bounds.

###  5.3 Next Steps

A large amount of our work in this paper, both in the taxonomy and in the
policy framework, relies on informed speculation over the future development
and governance of personalised LLMs. The affordances, constraints and harms
from any technology depend critically on how it is designed, how its outputs
are used in the real world and what safeguards or regulations are provisioned
to guide its impact post-deployment. None of these conditions are presently
clear; so, we very much consider this to be a first version of our work. In
the future, we plan to iterate on the taxonomy and policy framework by
conducting semi-structured interviews with (i) the end-users of LLMs to
understand their priorities and concerns; (ii) model and application providers
to scope their desires for enabling personalisation and for defining limits,
as well as the technical apparatus available to them to both of these things;
and finally, (iii) policymakers to assess how personalised LLMs fit into
existing laws and regulations, and the new challenges they pose. By starting
the conversation now, we hope to avoid long lags in understanding, documenting
and governing the harms from personalised LLMs as a future technology which
could widely impact individuals and society.

## Acknowledgements

This paper, which began as an exploration of how to improve feedback between
humans-and-model-in-the-loop, is part of a body of work funded by a MetaAI
Dynabench grant. H.R.K’s PhD is supported by the Economic and Social Research
Council grant ES/P000649/1. P.R’s PhD is supported by the German Academic
Scholarship Foundation. We particularly want to thank Andrew Bean for his
thoughtful input and assistance with the literature review. This paper is also
the product of many interesting conversations at various conferences and
working groups. We hope to survey the opinions and feedback of many other
stakeholders in the future (including end-users, policy makers and technology
providers) to further enrich our discussion.

## References

  * Abbass [2019] Hussein A. Abbass. 2019.  Social Integration of Artificial Intelligence: Functions, Automation Allocation Logic and Human-Autonomy Trust.  _Cognitive Computation_ , 11(2):159–171.
  * Abid et al. [2021] Abubakar Abid, Maheen Farooqi, and James Zou. 2021.  Persistent Anti-Muslim Bias in Large Language Models.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’21, pages 298–306, New York, NY, USA. Association for Computing Machinery.
  * Acharya et al. [2018] Sabita Acharya, Barbara Di Eugenio, Andrew Boyd, Richard Cameron, Karen Dunn Lopez, Pamela Martyn-Nemeth, Carolyn Dickens, and Amer Ardati. 2018.  Towards generating personalized hospitalization summaries.  In _Proceedings of the 2018 conference of the north American chapter of the association for computational linguistics: Student research workshop_ , pages 74–82, New Orleans, Louisiana, USA. Association for Computational Linguistics.
  * Agirre and Soroa [2009] Eneko Agirre and Aitor Soroa. 2009.  Personalizing PageRank for word sense disambiguation.  In _Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)_ , pages 33–41, Athens, Greece. Association for Computational Linguistics.
  * Aher et al. [2023] Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023.  Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies.  ArXiv:2208.10264 [cs].
  * Allcott and Gentzkow [2017] Hunt Allcott and Matthew Gentzkow. 2017.  Social Media and Fake News in the 2016 Election.  _Journal of Economic Perspectives_ , 31(2):211–236.
  * Altenried [2020] Moritz Altenried. 2020.  The platform as factory: Crowdwork and the hidden labour behind artificial intelligence.  _Capital & Class_, 44(2):145–158.  Publisher: SAGE Publications Ltd.
  * Ammanabrolu et al. [2022] Prithviraj Ammanabrolu, Liwei Jiang, Maarten Sap, Hannaneh Hajishirzi, and Yejin Choi. 2022.  Aligning to Social Norms and Values in Interactive Narratives.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5994–6017, Seattle, United States. Association for Computational Linguistics.
  * Andreas [2022] Jacob Andreas. 2022.  Language Models as Agent Models.  ArXiv:2212.01681 [cs].
  * Ao et al. [2021] Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. 2021.  PENS: A dataset and generic framework for personalized news headline generation.  In _Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: Long papers)_ , pages 82–92, Online. Association for Computational Linguistics.
  * Argyle et al. [2022] Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua Gubler, Christopher Rytting, and David Wingate. 2022.  Out of One, Many: Using Language Models to Simulate Human Samples.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 819–862.  ArXiv:2209.06899 [cs].
  * Armstrong [2017] Stephen Armstrong. 2017.  Data, data everywhere: the challenges of personalised medicine.  _BMJ_ , 359:j4546.  Publisher: British Medical Journal Publishing Group Section: Feature.
  * Askell et al. [2021] Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, and Jared Kaplan. 2021.  A General Language Assistant as a Laboratory for Alignment.  ArXiv:2112.00861 [cs].
  * Aula and Nordhausen [2006] Anne Aula and Klaus Nordhausen. 2006.  Modeling successful performance in Web searching.  _Journal of the American Society for Information Science and Technology_ , 57(12):1678–1693.
  * Bagherzadeh et al. [2019] Parsa Bagherzadeh, Nadia Sheikh, and Sabine Bergler. 2019.  Adverse drug effect and personalized health mentions, CLaC at SMM4H 2019, tasks 1 and 4.  In _Proceedings of the fourth social media mining for health applications ( #SMM4H) workshop & shared task_, pages 123–126, Florence, Italy. Association for Computational Linguistics.
  * Bai et al. [2022a] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. 2022a.  Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback.  ArXiv: 2204.05862v1.
  * Bai et al. [2022b] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. 2022b.  Constitutional AI: Harmlessness from AI Feedback.
  * Bakker et al. [2022] Michiel A. Bakker, Martin J. Chadwick, Hannah R. Sheahan, Michael Henry Tessler, Lucy Campbell-Gillingham, Jan Balaguer, Nat McAleese, Amelia Glaese, John Aslanides, Matthew M. Botvinick, and Christopher Summerfield. 2022.  Fine-tuning language models to find agreement among humans with diverse preferences.  ArXiv: 2211.15006v1.
  * Bakshy et al. [2015] Eytan Bakshy, Solomon Messing, and Lada A. Adamic. 2015.  Exposure to ideologically diverse news and opinion on Facebook.  _Science_ , 348(6239):1130–1132.  Publisher: American Association for the Advancement of Science.
  * Bang et al. [2022] Yejin Bang, Tiezheng Yu, Andrea Madotto, Zhaojiang Lin, Mona Diab, and Pascale Fung. 2022.  Enabling Classifiers to Make Judgements Explicitly Aligned with Human Values.  ArXiv: 2210.07652v1.
  * Baron [2023] Naomi S. Baron. 2023.  Even kids are worried ChatGPT will make them lazy plagiarists, says a linguist who studies tech’s effect on reading, writing and thinking.  _Fortune_.
  * Barua et al. [2022] Prabal Datta Barua, Jahmunah Vicnesh, Raj Gururajan, Shu Lih Oh, Elizabeth Palmer, Muhammad Mokhzaini Azizan, Nahrizul Adib Kadri, and U. Rajendra Acharya. 2022.  Artificial Intelligence Enabled Personalised Assistive Tools to Enhance Education of Children with Neurodevelopmental Disorders—A Review.  _International Journal of Environmental Research and Public Health_ , 19(3):1192.  Number: 3 Publisher: Multidisciplinary Digital Publishing Institute.
  * Bastos [2021] Marco Bastos. 2021.  From Global Village to Identity Tribes: Context Collapse and the Darkest Timeline.  _Media and Communication_ , 9(3):50–58.
  * Bender et al. [2021] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, pages 610–623, New York, NY, USA. Association for Computing Machinery.
  * Benjamin [2019] Ruha Benjamin. 2019.  _Race After Technology: Abolitionist Tools for the New Jim Code_.  John Wiley & Sons.
  * Bhadani [2021] Saumya Bhadani. 2021.  Biases in Recommendation System.  In _Proceedings of the 15th ACM Conference on Recommender Systems_ , RecSys ’21, pages 855–859, New York, NY, USA. Association for Computing Machinery.
  * Bhargava and Velasquez [2021] Vikram R. Bhargava and Manuel Velasquez. 2021.  Ethics of the Attention Economy: The Problem of Social Media Addiction.  _Business Ethics Quarterly_ , 31(3):321–359.  Publisher: Cambridge University Press.
  * Birhane et al. [2022] Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed. 2022.  Power to the People? Opportunities and Challenges for Participatory AI.  In _Equity and Access in Algorithms, Mechanisms, and Optimization_ , EAAMO ’22, pages 1–8, New York, NY, USA. Association for Computing Machinery.
  * Blodgett et al. [2020] Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020.  Language (Technology) is Power: A Critical Survey of "Bias" in NLP.  ArXiv:2005.14050 [cs].
  * Bommasani et al. [2022] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, Aditi Raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, and Percy Liang. 2022.  On the Opportunities and Risks of Foundation Models.  ArXiv:2108.07258 [cs].
  * Bowman et al. [2022] Samuel R. Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilė Lukošiūtė, Amanda Askell, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Christopher Olah, Daniela Amodei, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Jackson Kernion, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Liane Lovitt, Nelson Elhage, Nicholas Schiefer, Nicholas Joseph, Noemí Mercado, Nova DasSarma, Robin Larson, Sam McCandlish, Sandipan Kundu, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tristan Hume, Yuntao Bai, Zac Hatfield-Dodds, Ben Mann, and Jared Kaplan. 2022.  Measuring Progress on Scalable Oversight for Large Language Models.  ArXiv:2211.03540 [cs].
  * Burkett [2017] Christopher Burkett. 2017.  I Call Alexa to the Stand: The Privacy Implications of Anthropomorphizing Virtual Assistants Accompanying Smart-Home Technology Notes.  _Vanderbilt Journal of Entertainment & Technology Law_, 20(4):1181–1218.
  * Burrell et al. [2019] Jenna Burrell, Zoe Kahn, Anne Jonas, and Daniel Griffin. 2019.  When Users Control the Algorithms: Values Expressed in Practices on Twitter.  _Proceedings of the ACM on Human-Computer Interaction_ , 3(CSCW):138:1–138:20.
  * Calo [2013] Ryan Calo. 2013.  Digital Market Manipulation.
  * Carlini et al. [2021] Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021.  Extracting Training Data from Large Language Models.  pages 2633–2650.
  * Casanueva et al. [2015] I textasciitilde nigo Casanueva, Thomas Hain, Heidi Christensen, Ricard Marxer, and Phil Green. 2015.  Knowledge transfer between speakers for personalised dialogue management.  In _Proceedings of the 16th annual meeting of the special interest group on discourse and dialogue_ , pages 12–21, Prague, Czech Republic. Association for Computational Linguistics.
  * Caselli et al. [2021] Tommaso Caselli, Valerio Basile, Jelena Mitrović, and Michael Granitzer. 2021.  HateBERT: Retraining BERT for Abusive Language Detection in English.  In _Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021)_ , pages 17–25, Online. Association for Computational Linguistics.
  * Chaney et al. [2018] Allison J. B. Chaney, Brandon M. Stewart, and Barbara E. Engelhardt. 2018.  How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility.  In _Proceedings of the 12th ACM Conference on Recommender Systems_ , pages 224–232.
  * Chen et al. [2020] Guanyi Chen, Yinhe Zheng, and Yupei Du. 2020.  Listener’s social identity matters in personalised response generation.  In _Proceedings of the 13th international conference on natural language generation_ , pages 205–215, Dublin, Ireland. Association for Computational Linguistics.
  * Cho et al. [2021] Eunah Cho, Ziyan Jiang, Jie Hao, Zheng Chen, Saurabh Gupta, Xing Fan, and Chenlei Guo. 2021.  Personalized search-based query rewrite system for conversational AI.  In _Proceedings of the 3rd workshop on natural language processing for conversational AI_ , pages 179–188, Online. Association for Computational Linguistics.
  * Cho et al. [2022] Itsugun Cho, Dongyang Wang, Ryota Takahashi, and Hiroaki Saito. 2022.  A personalized dialogue generator with implicit user persona detection.  In _Proceedings of the 29th international conference on computational linguistics_ , pages 367–377, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
  * Chou et al. [2005] Chien Chou, Linda Condron, and John C. Belland. 2005.  A Review of the Research on Internet Addiction.  _Educational Psychology Review_ , 17(4):363–388.
  * Chow [2023] Andrew R. Chow. 2023.  Why People Are Confessing Their Love For AI Chatbots.  _Time_.
  * Cinelli et al. [2021] Matteo Cinelli, Gianmarco De Francisci Morales, Alessandro Galeazzi, Walter Quattrociocchi, and Michele Starnini. 2021.  The echo chamber effect on social media.  _Proceedings of the National Academy of Sciences_ , 118(9):e2023301118.  Publisher: Proceedings of the National Academy of Sciences.
  * Commission [2021] European Commission. 2021.  The artificial intelligence act.  _Proposal for regulation of the European parliament and of the council - Laying down harmonised rules on artificial intelligence and amending certain Union legislative acts_.
  * Commission [2022] European Commission. 2022.  AI liability directive.  _Proposal for a Directive of the European Parliament and of the Council on adapting non-contractual civil liability rules to artificial intelligence_ , pages 1–29.
  * Couldry [2010] Nick Couldry. 2010.  Researching digital (dis)connection in the age of personalised media.  In Graham Murdock and Peter Golding, editors, _Digital dynamics: engagements and connections._ , pages 105–124. Hampton Press Inc., Cresskill, NJ.
  * Crafts [2021] Nicholas Crafts. 2021.  Artificial intelligence as a general-purpose technology: an historical perspective.  _Oxford Review of Economic Policy_ , 37(3):521–536.
  * Cremers and Neerincx [2004] Anita H. M. Cremers and Mark A. Neerincx. 2004.  Personalisation Meets Accessibility: Towards the Design of Individual User Interfaces for All.  In _User-Centered Interaction Paradigms for Universal Access in the Information Society_ , Lecture Notes in Computer Science, pages 119–124, Berlin, Heidelberg. Springer.
  * Cullen [2001] Rowena Cullen. 2001.  Addressing the digital divide.  _Online Information Review_ , 25(5):311–320.  Publisher: MCB UP Ltd.
  * Curry [2023] David Curry. 2023.  ChatGPT Revenue and Usage Statistics (2023).
  * Deng et al. [2021] Yang Deng, Yaliang Li, Wenxuan Zhang, Bolin Ding, and Wai Lam. 2021.  Towards personalized answer generation in E-commerce via multi-perspective preference modeling.  ArXiv: 2112.13556v1 [cs.CL].
  * Devlin et al. [2019] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  ArXiv:1810.04805 [cs].
  * Dou et al. [2009] Zhicheng Dou, Ruihua Song, Ji-Rong Wen, and Xiaojie Yuan. 2009.  Evaluating the Effectiveness of Personalized Web Search.  _IEEE Transactions on Knowledge and Data Engineering_ , 21(8):1178–1190.
  * Downey [2021] Mitch Downey. 2021.  Partial automation and the technology-enabled deskilling of routine jobs.  _Labour Economics_ , 69:101973.
  * Dubuisson Duplessis et al. [2017] Guillaume Dubuisson Duplessis, Chloé Clavel, and Frédéric Landragin. 2017.  Automatic Measures to Characterise Verbal Alignment in Human-Agent Interaction.  In _Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue_ , pages 71–81, Saarbrücken, Germany. Association for Computational Linguistics.
  * Dunaway [2021] Johanna Dunaway. 2021.  Polarisation and misinformation.  In _The Routledge Companion to Media Disinformation and Populism_. Routledge.
  * Díaz et al. [2010] Alberto Díaz, Pablo Gervás, Antonio García, and Laura Plaza. 2010.  Development and use of an evaluation collection for personalisation of digital newspapers.  In _Proceedings of the seventh international conference on language resources and evaluation (LREC’10)_ , Valletta, Malta. European Language Resources Association (ELRA).
  * El Sheikh et al. [2021] Ahmed El Sheikh, Michele Bevilacqua, and Roberto Navigli. 2021.  Integrating personalized PageRank into neural word sense disambiguation.  In _Proceedings of the 2021 conference on empirical methods in natural language processing_ , pages 9092–9098, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Fairweather and Rogerson [2003] Ben Fairweather and Simon Rogerson. 2003.  The problems of global cultural homogenisation in a technologically dependant world.  _Journal of Information, Communication and Ethics in Society_ , 1(1):7–12.  Publisher: MCB UP Ltd.
  * Fedorovsky et al. [2015] Andrey Fedorovsky, Maxim Ionov, Varvara Litvinova, Tatyana Olenina, and Darya Trofimova. 2015.  Expanding the horizons: adding a new language to the news personalization system.  In _Proceedings of the first workshop on computing news storylines_ , pages 35–39, Beijing, China. Association for Computational Linguistics.
  * Floridi [2011] Luciano Floridi. 2011.  The Informational Nature of Personal Identity.  _Minds and Machines_ , 21(4):549–566.
  * Frank et al. [2019] Morgan R. Frank, David Autor, James E. Bessen, Erik Brynjolfsson, Manuel Cebrian, David J. Deming, Maryann Feldman, Matthew Groh, José Lobo, Esteban Moro, Dashun Wang, Hyejin Youn, and Iyad Rahwan. 2019.  Toward understanding the impact of artificial intelligence on labor.  _Proceedings of the National Academy of Sciences_ , 116(14):6531–6539.  Publisher: Proceedings of the National Academy of Sciences.
  * Frazier et al. [2019] Spencer Frazier, Md Sultan Al Nahian, Mark Riedl, and Brent Harrison. 2019.  Learning Norms from Stories: A Prior for Value Aligned Agents.  ArXiv: 1912.03553v1.
  * Gabriel [2020] Iason Gabriel. 2020.  Artificial Intelligence, Values and Alignment.  _Minds and Machines_ , 30(3):411–437.  ArXiv:2001.09768 [cs].
  * Gabriel and Ghazavi [2021] Iason Gabriel and Vafa Ghazavi. 2021.  The Challenge of Value Alignment: from Fairer Algorithms to AI Safety.  ArXiv:2101.06060 [cs].
  * Ganguli et al. [2022] Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph, Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. 2022\.  Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned.  ArXiv: 2209.07858v2.
  * Gao et al. [2020] Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett, and Bill Dolan. 2020.  Dialogue Response Ranking Training with Large-Scale Human Feedback Data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 386–395, Online. Association for Computational Linguistics.
  * Gao et al. [2018] Yang Gao, Christian M. Meyer, and Iryna Gurevych. 2018.  APRIL: Interactively learning to summarise by combining active preference learning and reinforcement learning.  In _Proceedings of the 2018 conference on empirical methods in natural language processing_ , pages 4120–4130, Brussels, Belgium. Association for Computational Linguistics.
  * Gatius [2017] Marta Gatius. 2017.  Personalized questions, answers and grammars: Aiding the search for relevant web information.  In _Proceedings of the 10th international conference on natural language generation_ , pages 203–207, Santiago de Compostela, Spain. Association for Computational Linguistics.
  * Gault [2022] Matthew Gault. 2022.  AI Trained on 4Chan Becomes ‘Hate Speech Machine’.
  * Gehman et al. [2020] Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020\.  RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 3356–3369, Online. Association for Computational Linguistics.
  * Gibson [1979] James J. Gibson. 1979.  _The Ecological Approach to Visual Perception: Classic Edition_.  Psychology Press, New York.
  * Gillespie [2018] Tarleton Gillespie. 2018.  _Custodians of the internet: platforms, content moderation, and the hidden decisions that shape social media_.  Yale University Press, New Haven.
  * Gillespie [2022] Tarleton Gillespie. 2022.  Do Not Recommend? Reduction as a Form of Content Moderation.  _Social Media + Society_ , 8(3):20563051221117552.  Publisher: SAGE Publications Ltd.
  * Gillespie et al. [2020] Tarleton Gillespie, Patricia Aufderheide, Elinor Carmi, Ysabel Gerrard, Robert Gorwa, Ariadna Matamoros-Fernández, Sarah T. Roberts, Aram Sinnreich, and Sarah Myers West. 2020.  Expanding the debate about content moderation: scholarly research agendas for the coming policy debates.  _Internet Policy Review_ , 9(4).
  * Glaese et al. [2022] Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. 2022\.  Improving alignment of dialogue agents via targeted human judgements.  ArXiv: 2209.14375v1.
  * Goodside and Papay [2023] Riley Goodside and Spencer Papay. 2023.  Meet Claude: Anthropic’s Rival to ChatGPT | Blog | Scale AI.
  * Gros et al. [2021] David Gros, Yu Li, and Zhou Yu. 2021.  The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 6999–7013, Online. Association for Computational Linguistics.
  * Guo et al. [2019] Siwen Guo, Sviatlana Höhn, and Christoph Schommer. 2019.  A personalized sentiment model with textual and contextual information.  In _Proceedings of the 23rd conference on computational natural language learning (CoNLL)_ , pages 992–1001, Hong Kong, China. Association for Computational Linguistics.
  * Guo et al. [2012] Xitong Guo, Yongqiang Sun, J. Yuan, Z. Yan, and N. Wang. 2012.  Privacy-personalization paradox in adoption of mobile health service: The mediating role of trust.  _Proceedings - Pacific Asia Conference on Information Systems, PACIS 2012_.
  * Haider et al. [2022] Jutta Haider, Malte Rödl, and Sofie Joosse. 2022.  Algorithmically Embodied Emissions: The Environmental Harm of Everyday Life Information in Digital Culture.
  * Halavais [2017] Alexander Halavais. 2017.  _Search Engine Society_.  John Wiley & Sons.
  * Hancock et al. [2019] Braden Hancock, Antoine Bordes, Pierre-Emmanuel Mazare, and Jason Weston. 2019.  Learning from Dialogue after Deployment: Feed Yourself, Chatbot! In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3667–3684, Florence, Italy. Association for Computational Linguistics.
  * Harsin [2015] Jayson Harsin. 2015.  Regimes of Posttruth, Postpolitics, and Attention Economies.  _Communication, Culture and Critique_ , 8(2):327–333.
  * Hendrycks et al. [2020] Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2020.  Aligning AI With Shared Human Values.  ArXiv: 2008.02275v5.
  * Hengst et al. [2019] Floris den Hengst, Mark Hoogendoorn, Frank van Harmelen, and Joost Bosman. 2019\.  Reinforcement learning for personalized dialogue management.  ArXiv: 1908.00286v1 [cs.LG].
  * Hesmondhalgh et al. [2023] David Hesmondhalgh, Raquel Campos Valverde, Bondy Valdovinos Kaye D., and Li Zhongwei. 2023.  The impact of algorithmically driven recommendation systems on music consumption and production - a literature review.
  * Hoffmann et al. [2022] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. 2022.  Training Compute-Optimal Large Language Models.  _arXiv:2203.15556 [cs]_.
  * Honovich et al. [2022] Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022.  Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor.  ArXiv: 2212.09689v1.
  * Horton [2022] John J Horton. 2022.  Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus?
  * Howard [2019] John Howard. 2019.  Artificial intelligence: Implications for the future of work.  _American Journal of Industrial Medicine_ , 62(11):917–926.
  * Huang et al. [2020] Kexin Huang, Jaan Altosaar, and Rajesh Ranganath. 2020.  ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission.  ArXiv:1904.05342 [cs].
  * Huang et al. [2022] Xiaolei Huang, Alexandra Wormley, and Adam Cohen. 2022.  Learning to Adapt Domain Shifts of Moral Values via Instance Weighting.  ArXiv: 2204.07603v2.
  * Hwang [2020] Tim Hwang. 2020.  _Subprime Attention Crisis: Advertising and the Time Bomb at the Heart of the Internet_.  Farrar, Straus and Giroux.
  * Hynes and Hynes [2021] Mike Hynes and Mike Hynes. 2021.  Towards Cultural Homogenisation.  In _The Social, Cultural and Environmental Costs of Hyper-Connectivity: Sleeping Through the Revolution_ , pages 39–54. Emerald Publishing Limited.
  * Inkster et al. [2018] Becky Inkster, Shubhankar Sarda, and Vinod Subramanian. 2018.  An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study.  _JMIR mHealth and uHealth_ , 6(11):e12106.
  * Irving et al. [2018] Geoffrey Irving, Paul Christiano, and Dario Amodei. 2018.  AI safety via debate.  ArXiv:1805.00899 [cs, stat].
  * Jain et al. [2020] Vardhmaan Jain, Mahmoud Al Rifai, Michelle T. Lee, Ankur Kalra, Laura A. Petersen, Elizabeth M. Vaughan, Nathan D. Wong, Christie M. Ballantyne, and Salim S. Virani. 2020.  Racial and Geographic Disparities in Internet Use in the U.S. Among Patients With Hypertension or Diabetes: Implications for Telehealth in the Era of COVID-19.  _Diabetes Care_ , 44(1):e15–e17.
  * Jaques et al. [2020] Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, and Rosalind Picard. 2020.  Human-centric dialog training via offline reinforcement learning.  In _Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)_ , pages 3985–4003, Online. Association for Computational Linguistics.
  * Jeon [2022] Jaeho Jeon. 2022.  Exploring AI chatbot affordances in the EFL classroom: young learners’ experiences and perspectives.  _Computer Assisted Language Learning_ , 0(0):1–26.
  * Ji et al. [2022] Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022.  Survey of Hallucination in Natural Language Generation.  ArXiv:2202.03629 [cs].
  * Jiang et al. [2022] Liwei Jiang, Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jenny Liang, Jesse Dodge, Keisuke Sakaguchi, Maxwell Forbes, Jon Borchardt, Saadia Gabriel, Yulia Tsvetkov, Oren Etzioni, Maarten Sap, Regina Rini, and Yejin Choi. 2022.  Can Machines Learn Morality? The Delphi Experiment.  ArXiv:2110.07574 [cs].
  * Jin et al. [2022] Zhijing Jin, Sydney Levine, Fernando Gonzalez, Ojasv Kamal, Maarten Sap, Mrinmaya Sachan, Rada Mihalcea, Josh Tenenbaum, and Bernhard Schölkopf. 2022\.  When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment.  ArXiv: 2210.01478v3.
  * Kaakinen et al. [2020] Markus Kaakinen, Anu Sirola, Iina Savolainen, and Atte Oksanen. 2020.  Shared identity and shared information in social media: development and validation of the identity bubble reinforcement scale.  _Media Psychology_ , 23(1):25–51.
  * Kadavath et al. [2022] Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022.  Language Models (Mostly) Know What They Know.  ArXiv:2207.05221 [cs].
  * Kanclerz et al. [2021] Kamil Kanclerz, Alicja Figas, Marcin Gruza, Tomasz Kajdanowicz, Jan Kocon, Daria Puchalska, and Przemyslaw Kazienko. 2021.  Controversy and conformity: from generalized to personalized aggressiveness detection.  In _Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: Long papers)_ , pages 5915–5926, Online. Association for Computational Linguistics.
  * Kanclerz et al. [2022] Kamil Kanclerz, Marcin Gruza, Konrad Karanowski, Julita Bielaniewicz, Piotr Milkowski, Jan Kocon, and Przemyslaw Kazienko. 2022.  What if ground truth is subjective? Personalized deep neural hate speech detection.  In _Proceedings of the 1st workshop on perspectivist approaches to NLP @LREC2022_ , pages 37–45, Marseille, France. European Language Resources Association.
  * Kasahara et al. [2022] Tomohito Kasahara, Daisuke Kawahara, Nguyen Tung, Shengzhe Li, Kenta Shinzato, and Toshinori Sato. 2022.  Building a personalized dialogue system with prompt-tuning.  In _Proceedings of the 2022 conference of the north american chapter of the association for computational linguistics: Human language technologies: Student research workshop_ , pages 96–105, Hybrid: Seattle, Washington + Online. Association for Computational Linguistics.
  * Kasirzadeh and Gabriel [2022] Atoosa Kasirzadeh and Iason Gabriel. 2022.  In conversation with Artificial Intelligence: aligning language models with human values.  ArXiv: 2209.00731v2.
  * Kenton et al. [2021] Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik, and Geoffrey Irving. 2021.  Alignment of Language Agents.  ArXiv:2103.14659 [cs].
  * Kiela et al. [2021] Douwe Kiela, Max Bartolo, Yixin Nie, Divyansh Kaushik, Atticus Geiger, Zhengxuan Wu, Bertie Vidgen, Grusha Prasad, Amanpreet Singh, Pratik Ringshia, Zhiyi Ma, Tristan Thrush, Sebastian Riedel, Zeerak Waseem, Pontus Stenetorp, Robin Jia, Mohit Bansal, Christopher Potts, and Adina Williams. 2021.  Dynabench: Rethinking Benchmarking in NLP.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4110–4124, Online. Association for Computational Linguistics.
  * Kim and Kim [2018] Joo-Kyung Kim and Young-Bum Kim. 2018.  Supervised domain enablement attention for personalized domain classification.  In _Proceedings of the 2018 conference on empirical methods in natural language processing_ , pages 894–899, Brussels, Belgium. Association for Computational Linguistics.
  * Kim et al. [2018] Young-Bum Kim, Dongchan Kim, Anjishnu Kumar, and Ruhi Sarikaya. 2018.  Efficient large-scale neural domain classification with personalized attention.  In _Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: Long papers)_ , pages 2214–2224, Melbourne, Australia. Association for Computational Linguistics.
  * Kirk et al. [2021] Hannah Rose Kirk, Yennie Jun, Filippo Volpin, Haider Iqbal, Elias Benussi, Frederic Dreyer, Aleksandar Shtedritski, and Yuki Asano. 2021.  Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models.  In _Advances in Neural Information Processing Systems_ , volume 34, pages 2611–2624.
  * Kittur et al. [2013] Aniket Kittur, Jeffrey V. Nickerson, Michael Bernstein, Elizabeth Gerber, Aaron Shaw, John Zimmerman, Matt Lease, and John Horton. 2013.  The future of crowd work.  In _Proceedings of the 2013 conference on Computer supported cooperative work_ , CSCW ’13, pages 1301–1318, New York, NY, USA. Association for Computing Machinery.
  * Knox et al. [2019] Jeremy Knox, Yuchen Wang, and Michael Gallagher. 2019.  Introduction: AI, Inclusion, and ‘Everyone Learning Everything’.  In Jeremy Knox, Yuchen Wang, and Michael Gallagher, editors, _Artificial Intelligence and Inclusive Education: Speculative Futures and Emerging Practices_ , Perspectives on Rethinking and Reforming Education, pages 1–13. Springer, Singapore.
  * Kochmar et al. [2020] Ekaterina Kochmar, Dung Do Vu, Robert Belfer, Varun Gupta, Iulian Vlad Serban, and Joelle Pineau. 2020.  Automated personalized feedback improves learning gains in an intelligent tutoring system.  ArXiv: 2005.02431v2 [cs.CL].
  * Koetsier [2023] John Koetsier. 2023.  ChatGPT Burns Millions Every Day. Can Computer Scientists Make AI One Million Times More Efficient? _Forbes_.
  * Kormilitzin et al. [2023] Andrey Kormilitzin, Nenad Tomasev, Kevin R. McKee, and Dan W. Joyce. 2023.  A participatory initiative to include LGBT+ voices in AI for mental health.  _Nature Medicine_ , 29(1):10–11.  Number: 1 Publisher: Nature Publishing Group.
  * Koto et al. [2022] Fajri Koto, Jey Han Lau, and Timothy Baldwin. 2022.  Can Pretrained Language Models Generate Persuasive, Faithful, and Informative Ad Text for Product Descriptions? In _Proceedings of the Fifth Workshop on e-Commerce and NLP (ECNLP 5)_ , pages 234–243, Dublin, Ireland. Association for Computational Linguistics.
  * Kronemann et al. [2023] Bianca Kronemann, Hatice Kizgin, Nripendra Rana, and Yogesh K. Dwivedi. 2023.  How AI encourages consumers to share their secrets? The role of anthropomorphism, personalisation, and privacy concerns and avenues for future research.  _Spanish Journal of Marketing - ESIC_.
  * Kuru et al. [2017] Ozan Kuru, Josh Pasek, and Michael W. Traugott. 2017.  Motivated Reasoning in the Perceived Credibility of Public Opinion Polls.  _Public Opinion Quarterly_ , 81(2):422–446.
  * Lai et al. [2023] Vivian Lai, Yiming Zhang, Chacha Chen, Q. Vera Liao, and Chenhao Tan. 2023.  Selective Explanations: Leveraging Human Input to Align Explainable AI.  ArXiv: 2301.09656v1.
  * Lane and Saint-Martin [2021] Marguerita Lane and Anne Saint-Martin. 2021.  The impact of Artificial Intelligence on the labour market: What do we know so far? Technical report, OECD, Paris.
  * Lee et al. [2020] Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020.  BioBERT: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240.
  * Lee et al. [2022] Young-Jun Lee, Chae-Gyun Lim, Yunsu Choi, Ji-Hui Lm, and Ho-Jin Choi. 2022.  PERSONACHATGEN: Generating personalized dialogues using GPT-3.  In _Proceedings of the 1st workshop on customized chat grounding persona and knowledge_ , pages 29–48, Gyeongju, Republic of Korea. Association for Computational Linguistics.
  * Leike et al. [2018] Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. 2018.  Scalable agent alignment via reward modeling: a research direction.  _arXiv:1811.07871 [cs, stat]_.
  * Li et al. [2019] Han Li, Jihwan Lee, Sidharth Mudgal, Ruhi Sarikaya, and Young-Bum Kim. 2019.  Continuous learning for large-scale personalized domain classification.  In _Proceedings of the 2019 conference of the north American chapter of the association for computational linguistics: Human language technologies, volume 1 (long and short papers)_ , pages 3784–3794, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Lin et al. [2022] Stephanie Lin, Jacob Hilton, and Owain Evans. 2022.  TruthfulQA: Measuring How Models Mimic Human Falsehoods.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics.
  * Lin et al. [2017] Ying Lin, Joe Hoover, Morteza Dehghani, Marlon Mooijman, and Heng Ji. 2017.  Acquiring Background Knowledge to Improve Moral Value Prediction.  ArXiv: 1709.05467v1.
  * Lin et al. [2021a] Ying Lin, Han Wang, Jiangning Chen, Tong Wang, Yue Liu, Heng Ji, Yang Liu, and Premkumar Natarajan. 2021a.  Personalized entity resolution with dynamic heterogeneous KnowledgeGraph representations.  In _Proceedings of the 4th workshop on e-Commerce and NLP_ , pages 38–48, Online. Association for Computational Linguistics.
  * Lin et al. [2021b] Zhaojiang Lin, Zihan Liu, Genta Indra Winata, Samuel Cahyawijaya, Andrea Madotto, Yejin Bang, Etsuko Ishii, and Pascale Fung. 2021b.  XPersona: Evaluating multilingual personalized chatbot.  In _Proceedings of the 3rd workshop on natural language processing for conversational AI_ , pages 102–112, Online. Association for Computational Linguistics.
  * Liu et al. [2018] Bing Liu, Gokhan Tür, Dilek Hakkani-Tür, Pararth Shah, and Larry Heck. 2018.  Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 2060–2069, New Orleans, Louisiana. Association for Computational Linguistics.
  * Liu et al. [2023] Ruibo Liu, Chenyan Jia, Ge Zhang, Ziyu Zhuang, Tony X. Liu, and Soroush Vosoughi. 2023.  Second Thoughts are Best: Learning to Re-Align With Human Values from Text Edits.  ArXiv: 2301.00355v2.
  * Liu et al. [2022] Ruibo Liu, Ge Zhang, Xinyu Feng, and Soroush Vosoughi. 2022.  Aligning Generative Language Models with Human Values.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 241–252, Seattle, United States. Association for Computational Linguistics.
  * Liu and Agichtein [2008] Yandong Liu and Eugene Agichtein. 2008.  You’ve got answers: Towards personalized models for predicting success in community question answering.  In _Proceedings of ACL-08: HLT, short papers_ , pages 97–100, Columbus, Ohio. Association for Computational Linguistics.
  * Liu-Thompkins et al. [2022] Yuping Liu-Thompkins, Shintaro Okazaki, and Hairong Li. 2022.  Artificial empathy in marketing interactions: Bridging the human-AI gap in affective and social customer experience.  _Journal of the Academy of Marketing Science_ , 50(6):1198–1218.
  * Lizarondo et al. [2010] Lucylynn Lizarondo, Saravana Kumar, Lisa Hyde, and Dawn Skidmore. 2010.  Allied health assistants and what they do: A systematic review of the literature.  _Journal of Multidisciplinary Healthcare_ , 3:143–153.
  * Lordan and Neumark [2018] Grace Lordan and David Neumark. 2018.  People versus machines: The impact of minimum wages on automatable jobs.  _Labour Economics_ , 52:40–53.
  * Lozano-Blasco et al. [2022] Raquel Lozano-Blasco, Alberto Quilez Robres, and Alberto Soto Sánchez. 2022.  Internet addiction in young adults: A meta-analysis and systematic review.  _Computers in Human Behavior_ , 130:107201.
  * Lu et al. [2022] Hua Lu, Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2022.  Towards Boosting the Open-Domain Chatbot with Human Feedback.  ArXiv: 2208.14165v1.
  * Lucy and Bamman [2021] Li Lucy and David Bamman. 2021.  Gender and Representation Bias in GPT-3 Generated Stories.  In _Proceedings of the Third Workshop on Narrative Understanding_ , pages 48–55, Virtual. Association for Computational Linguistics.
  * Lutz [2019] Christoph Lutz. 2019.  Digital inequalities in the age of artificial intelligence and big data.  _Human Behavior and Emerging Technologies_ , 1(2):141–148.
  * Lythreatis et al. [2022] Sophie Lythreatis, Sanjay Kumar Singh, and Abdul-Nasser El-Kassar. 2022.  The digital divide: A review and future research agenda.  _Technological Forecasting and Social Change_ , 175:121359.
  * Madotto et al. [2019] Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019.  Personalizing dialogue agents via meta-learning.  In _Proceedings of the 57th annual meeting of the association for computational linguistics_ , pages 5454–5459, Florence, Italy. Association for Computational Linguistics.
  * Majumder et al. [2019] Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. 2019.  Generating personalized recipes from historical user preferences.  In _Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP)_ , pages 5976–5982, Hong Kong, China. Association for Computational Linguistics.
  * Marikyan et al. [2022] Davit Marikyan, Savvas Papagiannidis, Omer F. Rana, Rajiv Ranjan, and Graham Morgan. 2022.  “Alexa, let’s talk about my productivity”: The impact of digital assistants on work productivity.  _Journal of Business Research_ , 142:572–584.