**footnotetext: Equal co-supervision.$\circ$$\circ$footnotetext:
Correspondence to<EMAIL_ADDRESS>Centre
for the Governance of AI, bMila - Quebec AI Institute, cHarvard University,
dIndependent, eHarvard Law School, fUniversity of Oxford, gCooperative AI
Foundation, hUniversity of Cambridge, iUniversity of Toronto

# Visibility into AI Agents

Alan Chan∘ab, Carson Ezellc, Max Kaufmannd, Kevin Weie, Lewis Hammondfg,
Herbie Bradleyh, Emma Bluemkea, Nitarshan Rajkumarh, David Kruegerh, Noam
Kolti, Lennart Heim*a, Markus Anderljung*a

(20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

Increased delegation of commercial, scientific, governmental, and personal
activities to AI agents—systems capable of pursuing complex goals with limited
supervision—may exacerbate existing societal risks and introduce new risks.
Understanding and mitigating these risks involves critically evaluating
existing governance structures, revising and adapting these structures where
needed, and ensuring accountability of key stakeholders. Information about
where, why, how, and by whom certain AI agents are used, which we refer to as
visibility, is critical to these objectives. In this paper, we assess three
categories of measures to increase visibility into AI agents: agent
identifiers, real-time monitoring, and activity logging. For each, we outline
potential implementations that vary in intrusiveness and informativeness. We
analyze how the measures apply across a spectrum of centralized through
decentralized deployment contexts, accounting for various actors in the supply
chain including hardware and software service providers. Finally, we discuss
the implications of our measures for privacy and concentration of power.
Further work into understanding the measures and mitigating their negative
impacts can help to build a foundation for the governance of AI agents.

visibility, transparency, ai agents, ai deployment, ai oversight, ai
monitoring

††copyright: none

##  1\. Introduction

Many AI developers are creating systems with greater autonomy, access to
external tools or services, and an increased ability to reliably adapt, plan,
and act open-endedly over long time-horizons to achieve goals (Chan et al.,
2023b; Richards, 2023; Ruan et al., 2023; Liu et al., 2023; Sharkey et al.,
2023; Kenton et al., 2023; Sumers et al., 2023). We will say that such systems
possess relatively high degrees of agency and will refer to them as (AI)
agents or agentic systems (Chan et al., 2023b). Systems with relatively low
degrees of agency are those that only aid human decision-making or produce
outputs without acting in the world, such as image classifiers or text-to-
image models. Examples of agents could include reinforcement learning systems
(Team et al., 2023a; Reed et al., 2022) that interact extensively with the
real world111Including the physical environment but also digital environments
such as online platforms. or more capable versions of language models with
tool or service access that could, for example, plan and book a holiday or
send an email on a user’s behalf (OpenAI, 2023a; Schick et al., 2023; Ruan et
al., 2023; Bran et al., 2023).

Current AI agents sometimes struggle to perform even simple tasks (Liu et al.,
2023; Kinniment et al., 2023; Raji et al., 2022a; Valmeekam et al., 2023a, b;
Mialon et al., 2023), but given increasing investments in AI research (Epoch,
2023), scaling laws (Barnett and Besiroglu, 2023; Kaplan et al., 2020;
Hoffmann et al., 2022), pressures to develop autonomous capabilities for
military use (Kott, 2018; Horowitz and Scharre, 2021; Scharre, 2021), economic
applications (Chan et al., 2023b), and scientific prestige (Ganguli et al.,
2022; Chan et al., 2023b), we should not discount continued improvements in
capabilities (Bowman, 2022). Indeed, a core goal of the AI field since its
inception has been to build agents (Sutton and Barto, 2018; Russell and
Norvig, 2021).

As AI agents improve in capabilities, speed, and cost,222As an example of cost
reduction, FLOP (Hobbhahn et al., 2023) or FLOP/s (Hobbhahn and Besiroglu,
2022) per dollar could decrease at the same time as performance per FLOP
increases (Erdil and Besiroglu, 2023). it may be easier and more competitive
to delegate tasks currently done by humans to AI agents instead. The
development and deployment of agents has surged recently (OpenAI, 2023c;
Chase, 2022; Wu et al., 2023; Richards, 2023; Shen et al., 2023) and could
lead to the ubiquitous deployment of agents in commercial, scientific,
governmental, and personal activities. Since such deployment may exacerbate
existing risks and introduce new ones (Chan et al., 2023b; Shavit et al.,
2023), it is imperative to understand how to govern AI agents.

###  1.1. Risks of AI Agents

Rather than provide an exhaustive taxonomy of risks from AI agents,333See
Critch and Russell (2023); Weidinger et al. (2023); Shelby et al. (2023) for
taxonomies of risks from AI systems and (Chan et al., 2023b; Shavit et al.,
2023) for further discussion of risks from AI agents. we highlight certain
agent-specific risks. In comparison to risks from other AI systems, these
risks focus on the potential for increasingly agentic systems to substitute
for, rather than complement, human labour (Korinek and Juelfs, 2022) and
remove humans from the loop (Chan et al., 2023b). Without a human in the loop,
agents may take multiple consequential actions in rapid succession and bring
about significant impacts before a human notices. The ability to remove humans
from the loop also means that an agent’s task performance is less limited by
the expertise of its user, compared to a situation where user must guide an AI
system’s actions or take actions themself.

####  1.1.1. Malicious Use

AI agents could be a large impact multiplier for individuals or coordinated
groups who wish to cause harm (Shavit et al., 2023). Existing AI systems have
already assisted in malicious use, including voice cloning scams (Verma,
2023b) and fake news generation (Verma, 2023a). However, more capable AI
agents could automate end-to-end pipelines for complex tasks that currently
require substantial human expertise and time. For untrained individuals, such
agents could drastically increase the accessibility of engaging in severely
harmful activities because no human in the loop would be required. For
example, there is interest in building agents to execute scientific research,
comprising autonomous planning and execution of scientific experiments (Bran
et al., 2023; Boiko et al., 2023). If such agents were to become as capable as
human scientists, they might enable or accelerate the design and development
of harmful tools (e.g., biological (Sandbrink, 2023; Soice et al., 2023),
chemical (Boiko et al., 2023; Bran et al., 2023; Urbina et al., 2022)) for
groups that currently lack the expertise for such production. Extremely
persuasive AI agents may also enable and enhance influence campaigns
(Karinshak et al., 2023; Bai et al., 2024; Hazell, 2023).

Understanding the extent to which agents will facilitate malicious use
requires information about how they are used and how they interact with
external systems (Weidinger et al., 2023). Moreover, when malicious users do
cause harm with AI agents, regulatory enforcers will need measures to identify
the users and hold them accountable.

####  1.1.2. Overreliance and Disempowerment

Overreliance on AI agents to automate complex, high-stakes tasks could lead to
severe consequences. Humans can already rely on certain automated systems more
than is warranted (Cummings, 2004; Endsley and Kiris, 1995; Endsley and Kaber,
1999). More capable agents may enable automation of an increasing array of
complex and useful tasks. Users—including both individuals and
institutions—may rely on agents even in high-stakes situations, such as
interfacing with the financial or legal systems, because human alternatives
(e.g., hiring a lawyer) may become relatively slower and more expensive. At
the same time, these agents may malfunction for a variety of reasons,
including design flaws (Raji et al., 2022a; Ngo et al., 2022; Zwetsloot and
Dafoe, 2019) or adversarial attack (Willison, 2023; Bailey et al., 2023; Zou
et al., 2023). Malfunction may not be immediately apparent, especially if
users lack the requisite expertise or domain knowledge. Stopping the agent may
be difficult if doing so would lead to cascading failures or a competitive
disadvantage for the user (Shavit et al., 2023). More broadly, profit and
efficiency motives may lead to collective dependence on agents for essential
societal functions, such as the provision of government services (De La Garza,
2020; Zilka et al., 2022) or the operation of essential infrastructure
(Degrave et al., 2022; Biagioni et al., 2022). Companies providing access to
AI agents would hold substantial power (Burrell and Fourcade, 2021), while
malfunction of those agents could have societal-scale impacts.

At minimum, societies require information about the extent of reliance upon AI
agents and whether such reliance is justified.

####  1.1.3. Delayed and Diffuse Impacts

Potential negative impacts of AI agents may be delayed and diffuse.444Roughly
speaking, we consider an impact to be diffuse if it is difficult to observe
and most apparent in aggregate across many individual cases. Delayed and
diffuse impacts may be difficult to manage because they may require sustained
attention over long periods of time even to notice. Impacts of agents may be
delayed if users give agents long-horizon goals, while diffuseness of impact
may come from the widespread deployment of agents to automate complex
processes. Consider an agent given the goal of continually finding and hiring
job candidates who will most contribute to the company over the long-term.
This agent may screen résumés (Gan et al., 2024), perform interviews, make the
final hiring decision, and analyze the performance of hires. Given the time
horizon over which the agent is acting and its influence over the company, any
potential problems like algorithmic bias (Raghavan et al., 2020) could be hard
to identify and become deeply entrenched. The most severe consequences of such
problems may only be apparent when looking at how companies in aggregate use
AI agents for hiring. AI agents could also subtly benefit their developers,
akin to the self-preferencing behaviour of large-scale digital platforms
(Kittaka et al., 2023). Moreover, agents that mediate or even substitute for
human communication (noa, 2023b; Melley, 2024) could have diffuse and delayed
psychological and social impacts (Jakesch et al., 2023; Bai et al., 2024;
Karinshak et al., 2023), analogous to certain effects of social media (Bond et
al., 2012; Sheffield and Francois, 2021; Milli et al., 2023). The deployment
of agents may also induce changes in market structures or workforce impacts
from job displacement (Acemoglu and Restrepo, 2018, 2019).

Identifying delayed and diffuse impacts may require long-term tracking of the
extent and nature of AI agent usage across a wide range of application areas.

####  1.1.4. Multi-Agent Risks

Interactions and dependencies between many deployed agents could lead to risks
not present at the level of a single system (Green, 2023; Park et al., 2023;
Schelling, 1978; Hammond and TBD, 2024). Agents could enter into destabilising
feedback loops, such as those between automated trading algorithms in the 2010
flash crash (CTFC and SEC, 2010). Agents partially built upon the same
components—such as a particular foundation model—could have common
vulnerabilities and failure modes (Bommasani et al., 2022; Cohen and Havlin,
2010); widespread deployment of such agents could risk large-scale systemic
harms. More generally, there may be unpredictable behavioural changes that are
characteristic of complex systems (Siegenfeld and Bar-Yam, 2020; Cohen and
Havlin, 2010; Schelling, 1978). Competitive pressures and selection effects
could lead to the development of agents that act in more anti-social ways (Xu
et al., 2023; Hendrycks, 2023; Cassidy, 2009; Ely and Szentes, 2023).

These potential issues motivate understanding not just individual agents, but
also interactions within groups of agents.

####  1.1.5. Sub-Agents

Agents could instantiate more agents to achieve user goals, which may magnify
several of the risks discussed so far. It may be advantageous for an agent to
create potentially specialized and more efficient sub-agents, especially if
doing so is cheap and fast. For example, an agent could call copies of itself
through an API, or itself train, fine-tune, or otherwise program another agent
to accomplish (components of) a task. Sub-agents could be problematic because
they introduce additional points of failure; each sub-agent may itself
malfunction, be vulnerable to attack, or otherwise operate in a way contrary
to the user’s intentions. To stop an agent from causing harm, one would have
to intervene not only on the agent, but also on any relevant sub-agents (Carey
and Everitt, 2023; Soares et al., 2015). Yet, detecting harmful behaviour and
intervening may be difficult because we lack methods for determining when an
agent has created a sub-agent. Additionally, we would need to ensure that
safety constraints imposed on the original agent faithfully transfer to sub-
agents.

Information about the extent of sub-agent creation and operation can enable a
better understanding of the significance of these risks.

###  1.2. The Case for Visibility into AI Agents

Addressing the risks of AI agents requires visibility: information about
where, why, how, and by whom AI agents are used. Visibility would help to
evaluate existing governance structures, revise and adapt these structures
where needed, and ensure accountability of key stakeholders. Regulatory
oversight bodies which monitor and enforce rules on the activities of human
agents and certain automated programs (e.g., trading algorithms) (Laffont and
Martimort, 2002; Holmström, 1979; Holmstrom and Milgrom, 1991; Ball, 2010;
Jensen and Meckling, 1976; Birkinshaw, 2006; Minkkinen et al., 2022; Van Loo,
2019; Coglianese et al., 2004; Board of Governors of the Federal Reserve
System, 2021; CFTC, 2023; Bielicki et al., 2020; Guarnieri et al., 2021) may
require additional information to understand and address harms from AI agents.
For example, if agents are able to employ novel strategies for collusion
(Dorner, 2021) when carrying out economic activities, new rules and updates to
investigative authority may be necessary. Furthermore, AI agents may
simultaneously provide services traditionally regulated by different agencies,
such as both financial and legal services. The same agent developer or
deployer may thus exercise power across diverse and usually independent
domains of regulation, creating additional concerns related to market
consolidation and conflicts of interest.

Visibility measures also play a central role in addressing problems that arise
when humans delegate to other humans or institutions (Laffont and Martimort,
2002). The precise purpose of such regimes varies, and can include reducing
information asymmetries, shaping incentive structures, and triggering
enforcement actions (Holmström, 1979; Holmstrom and Milgrom, 1991). For
example, employers often monitor the conduct and performance of employees
through ongoing supervision and periodic performance reviews (Ball, 2010). In
corporate governance, shareholders monitor management through a range of
institutional mechanisms, including financial audits, shareholder meetings,
and company reports, buttressed by legally binding fiduciary duties (in the
case of directors) and the ability (in some cases) to dismiss management if
they fail to act in the collective interest of shareholders (Jensen and
Meckling, 1976). Comparable mechanisms exist to support citizens in monitoring
the activities of government bodies and public officials. These mechanisms
include maintaining records of government decisions, facilitating information
access through freedom of information requests, and commissioning detailed
public reports into government activities (Birkinshaw, 2006), a combination of
which may ultimately inform citizens’ electoral choices. Although visibility
measures can be costly (Fama and Jensen, 1983) and raise privacy concerns,
they remain a necessary feature of frameworks for shaping the incentives of,
and governing, agents.

We emphasize visibility into deployed AI agents because the scope and severity
of potential impacts may not be apparent during development. By deployed, we
mean agents that are in use, whether the agent is available to the general
public, select customers, or only for internal use within the organization
that develops it. Visibility into the last case may be particularly important
if organizations that carry out crucial societal functions, such as banks or
cloud compute providers, develop and deploy their own AI agents. We focus on
deployment because pre-deployment testing (Shevlane et al., 2023; Anderljung
et al., 2023) does not account for how users or deployers may exacerbate risks
(Weidinger et al., 2023) through fine-tuning (Davidson et al., 2023),
connecting to external tools or services (OpenAI, 2023a; Sharkey et al., 2023;
OpenAI, 2024, 2023c), or structuring calls to the system so as to better
enable it to pursue goals (Richards, 2023; Wu et al., 2023; Sharkey et al.,
2023; Wei et al., 2023). Even instances of agents that come from the same
underlying system can access different tools and can be conditioned to behave
differently based on prompts.

###  1.3. Contributions

In this paper, we assess three categories of measures to increase visibility
into AI agents: agent identifiers, real-time monitoring, and activity logs.
For each, we outline potential implementations that vary in intrusiveness of
data collection and informativeness of the data. We analyze how our measures
apply across a spectrum of centralized through decentralized deployment
contexts, accounting for various actors in the supply chain including hardware
and software service providers. Finally, we discuss the implications of our
measures for privacy and concentration of power. Rather than advocating for
immediate implementation of these measures, we emphasize the need for further
understanding them and how to mitigate their negative impacts.

The measures extend existing work in deployment visibility to better account
for the risks of AI agents. Agent identifiers, which indicate whether and
which AI agents are involved in interactions, generalize watermarks (Liu et
al., 2024; Wang et al., 2021) because they apply to all of an agent’s outputs,
including the use of external tools and services, not just text, image, or
audio outputs. This generalization is crucial for improving visibility if
agents increasingly substitute for human actions. For real-time monitoring and
activity logging, we assess practices that extend existing schemes (Henzinger
et al., 2023; Shergadwala et al., 2022; Javadi et al., 2020, 2021; Minkkinen
et al., 2022) so as to better track complex interactions between multiple
agents (Schelling, 1978; Critch and Russell, 2023), an agent’s interaction
with external tools or services, and delayed and diffuse effects of an agent’s
actions.

Figure 1. We illustrate how our main terms in Section 2 interact with each
other. Deployers are in red and encompass the agents box to denote the fact
that our paper focuses on agents that are run by deployers and served to
users. Developers build agents (or an underlying system) and deployers serve
instances of agents to users. Since deployers run agents, the inputs and
outputs of agents are by default visible to the deployer, which facilitates
the measures that we discuss in Section 3.

##  2\. Definitions

Besides the definitions here, we also define each term when we use it for the
first time in the main body. We illustrate the most common terms in Figure 1.

Agency is the degree to which an AI system acts directly in the world to
achieve long-horizon goals, with little human intervention or specification of
how to do so. An (AI) agent is a system with a relatively high degree of
agency; we consider systems that mainly predict without acting in the world,
such as image classifiers, to have relatively low degrees of agency. Examples
of agents could include reinforcement learning systems (Team et al., 2023a;
Reed et al., 2022) that interact extensively with the real world555Including
the physical environment but also digital environments such as online
platforms. or more capable versions of language models with tool access
(OpenAI, 2023a; Schick et al., 2023; Ruan et al., 2023; Bran et al., 2023). We
do not consider existing foundation models themselves to be agents. Our
definitions compress the characterization of agency in Chan et al. (2023b),
which points to four axes: the degree to which the system’s behaviour is
specified, the degree to which the system’s behaviour is goal-directed, the
degree to which the system has a direct impact in the world, and the degree to
which the system can achieve goals over long time-horizons. For the purposes
of this paper, we use agent and agentic system interchangeably.

Scaffolding is any method that structures the calls to an AI system so as to
facilitate the pursuit of goals (Richards, 2023; Wu et al., 2023; Chase,
2022). Scaffolding may include additional prompts, memory systems, access to
external tools, and planning mechanisms (Wang et al., 2023). For example,
AutoGPT (Richards, 2023) has a language model accept a high-level goal and
sequentially produce (reasoning, plan, criticism of the plan, action) tuples
so as to achieve the goal. Scaffolding can make an AI system, such as a
foundation model, more agentic.

The term developer(s) refers to the actor(s) involved in the construction of
an AI system. While the developers of a system include those who trained the
underlying machine-learning model, developers could also include those who
build other components of the complete system, such as the scaffolding
(Richards, 2023; Wu et al., 2023).

The user is the human individual or group that interacts with and provides
instructions to an AI system.

The (agent) deployer is the entity that operates an AI system and serves it to
users. The deployer may not be the same as the developer(s). For example,
Microsoft deploys OpenAI’s systems into its products (Warren, 2024). A
deployer may provide access to an agent in one of two ways. First, the
deployer may serve a foundation model which users may combine with other
components to make a more agentic system. For example, users may use the
scaffolding framework AutoGPT (Richards, 2023) to chain calls to GPT-4
(OpenAI, 2023b) and provide the model with tool access. Indeed, popular
scaffolding frameworks depend upon an underlying foundation model (Wu et al.,
2023; Richards, 2023). Second, the deployer may provide an agent or may
furnish ways for users to make a provided system more agentic. For instance,
OpenAI lets users build and share custom agents (OpenAI, 2024, 2023c)
augmented with a variety of tools, including browsing, using Google Drive
apps, and coding (OpenAI, 2023a).

The compute provider is responsible for supplying and maintaining the hardware
infrastructure on which an AI system operates. The compute provider could also
be the same as the deployer or the developer if either runs its own
infrastructure. Compute providers could be important partners for overseeing
large-scale deployments of agents that are not run by deployers, which we
discuss further in Section 4.

A tool or service refers to an external system or platform with which an AI
agent interacts to perform its tasks. For instance, a flight booking website
where the AI agent executes transactions, such as purchasing plane tickets on
behalf of the user, would be a service. The tool provider or service provider
is responsible for maintaining the platform that facilitates these
interactions. We will often use tool and service interchangeably. Agents often
interact with tools or services through dedicated APIs, which are interfaces
and protocols specifically structured for agents, rather than human users.

The outputs are the results or responses that an AI system generates. Some
types of outputs include images, text, or actions (e.g., calling a tool).
While the deployer generally has access to all the outputs, the tool or
service provider’s knowledge is limited to outputs relevant to their specific
service (e.g., the results of an API call).

Inputs are the data that the AI agent receives from a user, a tool or service,
another agent, or any other party, which inform its actions or responses. The
deployer in principle has access to inputs by virtue of running the system,
but may choose not to collect or store such information out of respect for
user privacy.

##  3\. Measures to Improve Visibility

We propose three complementary categories of measures to improve visibility
into AI agents. Agent identifiers indicate whether and which AI agents are
involved in a given interaction, such as watermarks or IDs that distinguish
agents in their requests to service providers. Real-time monitoring involves
real-time analysis of an agent’s activity, allowing deployers and/or service
or tool providers to flag and intervene on problematic behaviour as it is
occurring. Activity logs held by deployers and tool or service providers
record certain inputs and outputs of an agent, such as interacting with
external services or other agents, to facilitate post-incident attribution and
forensics. See Figure 2 for an overview of the information flows for each
measure.

Each category contains measures that vary in intrusiveness of data collection
and informativeness. More comprehensive information collection may be
justified for agents deemed to be high-risk, potentially based on the results
of evaluations (Kinniment et al., 2023; Anderljung et al., 2023; Shevlane et
al., 2023; Ruan et al., 2023; Mialon et al., 2023; Rismani et al., 2023) or
deployment in high-risk domains (noa, 2021; Koessler and Schuett, 2023; Critch
and Russell, 2023). For example, it may be desirable to subject agents
involved in financial trading to monitoring requirements at least as strict as
those for human traders (of Trading and Markets, 2008). Yet, more
comprehensive data collection may have serious privacy risks, which we discuss
in Section 3.4. Our goal is to provide an array of options, rather than an
answer to these trade-offs and the extent to which visibility measures should
be mandated. Finally, while we discuss potential implementations of the
measures, more research is required to understand their feasibility and
implications.

We focus in this section on agents run by (agent) deployers—entities that
deploy agents, or important subcomponents like a foundation model, as a
service to users. We include foundation models because many frameworks for
constructing agents use a foundation model as the central component (Wu et
al., 2023; Richards, 2023; OpenAI, 2023a). While deployers are unlikely to
account for all agent activity, they likely constitute a substantial fraction
because the most capable foundation models are only available through
deployers (OpenAI, 2023b; Team et al., 2023b; Bai et al., 2022),666Currently,
these deployers also happen to be developers. and using a deployer may be more
convenient than running a system oneself. Moreover, since deployers can
already see the inputs and outputs of deployed systems, they can attach agent
identifiers to outputs, perform real-time monitoring, and collect activity
logs. In Section 4, we analyze how to extend our measures to decentralized
deployments of agents.

(a) An agent identifier indicates to certain actors whether an AI agent is
involved in an interaction. Developers (not shown) and deployers cooperate to
implement agent identifiers, which the latter adds onto outputs. Here, we
illustrate an agent identifier that informs other parties in a given
interaction with an agent, as well as tools and services providers. If these
actors know that they are interacting with an agent, they may wish to verify
certain properties such as the security or robustness of the agent. See
Section 3.1 for further discussion.

(b) An agent’s inputs and outputs are visible to the deployer. Inputs come
from tool and service providers and users (not shown). Certain outputs, such
as requests to external tools and services, are also visible to tools and
services providers. These actors can monitor and filter the actions in real-
time (Section 3.2) or keep logs (Section 3.3) for post-incident attribution or
forensics. Insights gained from real-time monitoring or from logs can inform
regulators and governments.

Figure 2. We illustrate the flow of information for our measures in Section 3.

###  3.1. Agent Identifiers

An agent identifier indicates whether and which AI agents are involved in
interactions. Agent identifiers are attached to select outputs, are visible to
certain actors, and may include additional information about the agent.

The ability to identify agents could be useful to several actors. A regulator
could require AI agents to identify themselves as non-humans during
interaction with humans (Frazier, 2023; Baker, 2023), similar to bot
disclosure laws (DiResta, 2019).777Note that not all non-human activities may
come from AI agents. For example, consider currently automated trading
activity or ads auctions. Members of the general public may wish to know
whether they have interacted with AI agents. Summary statistics about when
agent identifiers are visible could inform governments and the general public
about the extent to which agents operate in high-risk settings (noa, 2021).
Identifiers for when agents send requests to tools or services providers could
help to identify significant actions, such as when agents transfer sensitive
information. A service provider may even reject a request absent certain
guarantees attached to the identifier, such as those related to the security
of the agent. Unique identifiers for each AI agent could facilitate
accountability by linking an action to an AI agent and its user, developer(s),
and deployer.

####  3.1.1. Types of Agent Identifiers

We consider three key design decisions for an agent identifier:

Which outputs contain the identifier? Decisions to attach an identifier may
consider both the format and content of the output. By format, we mean whether
the agent outputs data such as images, text, audio, or API requests to a
service provider. An agent identifier’s specific implementation depends upon
the output format. For example, identifiers for image outputs could be
watermarks (Liu et al., 2024; Wang et al., 2021), while an identifier for an
API request could be a simple header, similar to headers in HTTP requests. The
difficulty of implementing identifiers varies based on the format of the
output: for instance, adversarial users may easily remove watermarks (Zhang et
al., 2023).

Regarding content, identification may be especially important for significant
outputs. Certain outputs, such as purchases made on behalf of the user, may
merit identifiers by virtue of the task the agent is accomplishing. Other
outputs may only be significant beyond a certain threshold, such as requests
for compute resources that exceed a certain amount.

Which actors can see the identifier? An identifier may need to be visible to
different actors. For example, in the context of a financial transaction, an
agent identifier could be visible to any combination of the bank, the other
party in the transaction, or the service provider for the bank API (which
could be the bank itself). Some actors may need agent identifiers to fulfil
their existing duties, such as e-commerce websites which must authenticate
users and safeguard customer payment information (Council, 2022). Furthermore,
facilitating the identification of multi-agent risks may require that agent
identifiers be visible to other agents.

How specific is the identifier to a particular agent? An identifier could
point to a particular agent, or simply denote that some agent was involved in
the interaction. The former could facilitate incident reporting and
investigation. To implement unique identifiers for each deployed instance of
an agent, cryptographic methods—such as those used in software attestation—may
be needed to assure the agent’s provenance. In Section 3.1.2, we discuss
additional, useful information that could be attached to an identifier.

####  3.1.2. Attaching Additional Information to Agent Identifiers

Additional information attached to an agent identifier may be of further use.
Additional information could be specific to the instance of the agent deployed
to the user, or could pertain to the underlying system used during agent
development. Information about the former could include the goals the user has
given its agent, while information about the latter could include the results
of evaluations performed on the underlying system. We refer to the set of such
additional information as an agent card, drawing inspiration from previous
work on documenting AI systems (Mitchell et al., 2019; Gebru et al., 2021;
Gilbert et al., 2023; Bommasani et al., 2023). In Appendix A we provide a more
comprehensive list of what could be included on an agent card, but in this
section we discuss three particularly important types of information that
could be included.

The underlying system. This information could include the results of
evaluations (Shevlane et al., 2023); previous incidents; the dependencies
involved in the system’s construction (Mitchell et al., 2019; Gebru et al.,
2021; Bommasani et al., 2023); or training methods and data used. Such
information could inform the decisions of actors that interact with agents.
For example, tool providers may reject requests from agents that do not meet
certain security standards (Bhatt et al., 2023).

The specific instance of the agent. This information could consist of how the
agent was deployed (e.g., by its user directly or by another agent); a list of
external tools or services that the agent can access (e.g., applications or
software, any physical tools); the scaffolding framework (e.g. memory or
planning mechanisms); the intended scope, permissions, and goal(s) of the
agent (Gilbert et al., 2023); or the sector of the intended deployment
environment (e.g., finance). This type of information could be useful for
regulators to develop an overall understanding of the extent and significance
of agent use.

The actors involved in agent development and deployment. These actors could
include the user, the compute provider, developers, and other humans with whom
the agent has interacted. Authentication of the agent’s user may be useful
when individuals are providing sensitive information to agents (e.g. agents
used by banks to assist customers). Furthermore, this information helps to
clarify whom to hold accountable in case an agent causes harm during its
operation (Nissenbaum, 1996; Cooper et al., 2022). A regulator could use
identifying information associated to each of the actors, ideally in synergy
with data from Sections 3.2 and 3.3, to help investigate and address problems.
For example, a regulator may receive reports from third parties of an agent
engaging in financial scams. If the transactions of the agent contain agent
identifier, a regulator could match the identifier to the corresponding agent
card and ask the deployer of the agent to investigate further or shut it down.

A more comprehensive agent card may be more useful, but may also be more
intrusive to collect and require more substantial resources and expertise for
analysis. Comprehensiveness of information should likely scale with the risk
of the activity in question.

###  3.2. Real-Time Monitoring

Real-time monitoring involves oversight of agent activity in real-time, with
the goal of flagging and potentially filtering problematic behaviour. Such
monitoring is likely to be automated so as to be able to cope with the speed
and scale of agent activities (Shavit et al., 2023; Greenblatt et al., 2024);
see Naihin et al. (2023) for a preliminary implementation of a real-time
monitoring system for agents. Crucially, real-time monitoring does not require
the collection or storage of activity logs. As an analogy, consider the
difference between moderation filters and chat logs in the context of a text
messaging application—moderation filters might instantly delete harmful
content that trips a filter, but this measure is separate from keeping logs
for subsequent analysis. This difference can be critical for users who are
concerned about the privacy implications of a deployer keeping logs of user
inputs, such as business customers who need to consider the security of their
corporate IP. Indeed, some cloud providers already offer no-logging
provisions888For feasibility, sometimes logs must be kept for a short amount
of time. The no-logging provision of Google (2023) keeps logs for 72 hours.
for their language model deployments to some customers, subject to real-time
monitoring for abuse (ChrisHMSFT et al., 2023; mrbullwinkle and eric urban,
2023; Google, 2023).

Once behaviour is flagged, a deployer can intervene on the agent and
potentially inform relevant authorities. A user could be given the opportunity
to request human review of any action taken to impede its agent. We leave
details about the implementation challenges of automation and human review to
other work (Sullivan and Verreault-Julien, 2022; Green, 2022;
Venkatasubramanian and Alfano, 2020; Karimi et al., 2021).

Since real-time monitoring requires an immediate decision to flag a behaviour,
deployers would only be able to find problems which they can identify in real-
time. Thus, real-time monitoring may be most suitable for identifying
violations of clear rules such as restrictions on allowed activities of
agents, limits on tools and permissions made available to agents, leakage of
sensitive information, or usage caps for particular users or agents. Behaviors
can also be flagged for anomalies.

Flagging behaviors might also involve combining information from multiple
agents because an individual agent’s behaviour may only be problematic in the
context of other agents. For instance, collusion requires a group of agents,
not just a single agent, to coordinate to set high prices (Dorner, 2021).
Similarly, a single agent rapidly moving resources (e.g., data or financial
investments) between entities may not be an issue in isolation, but vast
numbers of agents engaging in the same behaviour might overload a given system
(e.g., server outages or stock crashes). A real-time monitoring system could
use agent identifiers to track which agents were involved in interactions
together. This system could raise flags if indicators of the collective
activity of these agents pass certain thresholds. For example, an indicator
for algorithmic collusion could be correlation in price movements.

Real-time monitoring could also flag when an agent exerts significant
influence in its environment (Pan et al., 2023). Such activities could include
economic transactions above a certain amount, usage of substantial compute
resources, running a large number of sub-agents, or simultaneous interfacing
with a large number of physical systems. While the acquisition of significant
influence may not itself cause harm, the impact of an agent’s failure would be
magnified. A potential response to such a flagged activity could be pausing it
until the deployer can verify explicit user approval, just as banks monitor
transactions and temporarily suspend accounts if they suspect fraud.

###  3.3. Activity Logs

Activity logs are records of certain inputs and outputs of an agent. Inputs
and outputs could be from both users or tools and services, as we illustrate
in Figure 1. Records could also report changes in the state of the agent, such
as updates to the weights. Logs can include varying levels of detail and may
only record actions above a certain degree of significance, depending on the
behaviour one is trying to track, privacy considerations, and storage
requirements. Frameworks for building agents already implement logging of
certain actions (Chase, 2022; Richards, 2023; Wu et al., 2023).

Activity logs can inform an understanding of the agent’s impacts as well as
allow one to tie behaviour to specific user choices, facilitating post-
incident attribution and forensics (O’Brien et al., 2023). During audits and
incident investigation, investigators may need detailed information to trace
the source and causes of harms arising from AI agents’ actions (Mökander et
al., 2023; Raji et al., 2022b). Researchers may benefit from detailed logs to
improve our understanding of agents and to develop better deployment controls.
Analysis of logs may also identify novel behaviours to improve real-time
monitoring.

####  3.3.1. Agent-Specific Information

While activity logs are useful for understanding risks from AI systems in
general (Javadi et al., 2020, 2021) and are common practice in many domains
(ISO, 2022; noa, 2022, 2023a; Commission, 2022a, b), they can be especially
helpful for addressing the risks we identified in Section 1.1.

Understanding the impacts of agents involves identifying which scaffolding,
tools, and services they use. For example, an agent with external memory can
plan and act over longer horizons. A log could explicitly label when an agent
has accessed an external tool and the tool’s output. Both the deployer and the
tool provider could engage in such logging: deployer logs may be more useful
for understanding how the tool affects the overall behaviour of the agent,
while tool provider logs may provide more insight into the impacts of the tool
itself. Indeed, tool providers may have strong incentives to engage in such
logging themselves: for example, tool providers can study logs to update APIs
or user interfaces to prevent abuse. Tool providers can also decide to
restrict services to certain AI agents with identifiers (and potentially other
attached certifications), as discussed in Section 3.1.

Identifying delayed and diffuse impacts may require logs to be retained for
extended periods of time. Details about the persistence of the agent could be
included in the logs, such as its running time, whether it is writing to and
accessing external memory, or the amount of compute used so far to run the
agent. These details could inform interventions, such as limiting the
lifetimes of certain agents. Yet, significant impacts may arise after the
lifetime of an individual agent. The impact of the original agent could be
delayed, or a user could run another agent for the same purposes, potentially
even with the same inputs and memory as the original agent. Accounting for
these possibilities means that logs may have to persist for a significant
amount of time after the lifetime of the corresponding agent. Furthermore,
logs for different agents may have to be combined if one agent can be viewed
as a continuation of another.

Combining information from multiple logs may also help to understand sub-agent
and multi-agent dynamics. For example, agent logs could be used to build
models of how a particular malfunction might propagate through a network of
agents or identifying undesirable forms of communication between agents (Roger
and Greenblatt, 2023).

####  3.3.2. Logging at Different Levels of Detail

A key design decision is the level of detail at which to record the agent’s
actions. Less detailed logging may only record high-level summaries of agent’s
behaviour or certain samples thereof. At the finest level, a regulator may
require a deployer to record in detail all of an agent’s behavior, especially
if an agent is operating in a high-risk environment. More detailed logging is
more useful, but may impose more significant costs on the deployer, require
more resources and expertise for analysis, and pose more significant privacy
concerns.

###  3.4. Risks

Privacy considerations may conflict with obtaining detailed information about
agent activity. Language model deployers are increasingly offering customers,
particularly business customers, privacy assurances around data collection and
use. Measures to reassure customers about confidentiality include:

  * •

Language model APIs with no logging of inputs or outputs, and the ability to
turn off safety filters and moderation classifiers (ChrisHMSFT et al., 2023).

  * •

Guarantees that customer data, including system outputs, will not be used to
train any AI system and will be kept in a customer’s cloud instance (Cloud,
2023).

  * •

The ability to delete logs kept by the provider after a certain amount of time
(ChrisHMSFT et al., 2023).

Additionally, existing data protection laws, such as GDPR, impose further
restrictions. Agent cards may contain identifying information about users.
Agent logs may be considered personal data, such as when agents are given
access to a filesystem containing personally identifiable information (Finck
and Pallas, 2020).

In general, if agents substitute for humans in a wide variety of activities,
information about those agents might be tantamount to information about the
users of those agents. Indeed, agent activities may be easier to monitor than
human activities because deployers are a central intermediary. Governments or
deployers may thus abuse their power to carry out excessive or unjustified
surveillance of personal activities (Goodwin, 2018; Greenwald and MacAskill,
2013). These considerations justify limiting data collection in accordance
with the risk of the agent’s activities or domain of deployment. Another
potential mitigation may be decentralized data custody schemes or data trusts
(Delacroix and Lawrence, 2019; Delacroix et al., 2020) whereby users or
accountable representatives would make decisions about data usage.

Modulating the degree of access to collected information can also help to
mitigate privacy concerns. Access can vary with respect to granularity, the
amount of detail contained in the records, and quantity, the number of records
that a party is allowed to access. With respect to granularity, information
can be aggregated, de-identified, or identifiable. Aggregated information
involves summary statistics but not individual records or logs; differentially
private (Bluemke et al., 2023) computations of summary statistics may help to
preserve the privacy of individual records. Records and logs can be de-
identified with respect to individual users or identifiable. With respect to
quantity, a party can have full access to all records, access based on
approved search queries or filters, or access upon-request to pre-specified
records to which they must provide a compelling reason for access.

The granularity and quantity of access should be the minimum necessary for the
accessing party to achieve its (legitimate) objectives. When investigations
pertain to specific users, identifiable information could be made available
upon request given a showing of compelling need and/or after approval from a
third-party adjudicator. Regulators may need logs containing identifiable
information in some cases, such as oversight of certain high-risk or high-
volume activities. For example, for traders transacting above a specified
threshold, CFTC collects identifiable personal information to enable
aggregation of data across different accounts and brokers (CFTC, 2023).

##  4\. Decentralized Deployments

Some deployments of agents may occur in a decentralized way and bypass
deployers. Users, whether enterprises or individuals, may run downloadable
(i.e., open release) (Solaiman, 2023) agents either on cloud compute or on
their own hardware. A user may even be able to combine systems from different
deployers to form an agent. Although visibility on the resulting agent may be
desirable, the individual systems may not be significant enough by themselves
to justify implementation of visibility measures by deployers. Indeed, a
malicious actor could build and run an agent in this way so as to avoid
detection by regulatory authorities. In this section, we discuss how our
visibility measures may be extended to such situations, as well as the risks
of doing so.

###  4.1. Compute Provider Oversight

Compute providers could enable oversight over deployments using large
quantities of compute. Such large-scale deployments could be concerning
because they might involve vast numbers of agents, which could translate into
a large impact multiplier for the user. Large-scale deployments are also
noticeable because they consume significant resources. Users may be
incentivized to employ compute providers for such deployments, rather than
their own hardware, due to compute providers’ economies of scale; indeed,
using compute as a service (e.g., infrastructure as a service or cloud) is the
default way for a business to deploy its IT services. If a compute provider
can identify large-scale deployments and whether they correspond to agent
activities, they may ask the user for proof that they have implemented certain
visibility measures (Egan and Heim, 2023; O’Brien et al., 2023).

###  4.2. Tool and Service Providers as Distributed Enforcement Mechanisms

The need for agents to interact with external tools offers another leverage
point. By conditioning tool and service access on implementation of certain
visibility measures such as agent identifiers, tool and service providers can
incentivize adherence to the measures. For example, financial institutions
could restrict access to AI agents without identifiers from certain trusted
deployers. Such identifiers might explicitly confirm permissions to access
certain services, such as performing financial transactions or accessing
certain websites. This approach could also allow tool providers to minimize
misuse and enable detailed analytics of AI agent interactions with their
tools.

One limitation is that AI agents could circumvent APIs by directly interacting
with tools in a way that mimics human behavior. The development of tools
capable of detecting disguised AI activity, akin to modern CAPTCHA systems
designed to differentiate between human and software interactions, may be
helpful. An alternative is to require proof of human identity for high-risk
actions. Certain industries perform identity verification for high-risk
activities with know-your-customer protocols (Egan and Heim, 2023). Similarly,
tool or service providers operating in high-risk domains could require human
identification. One difficulty is preventing AI agents from spoofing humans,
such as through generating fake identification documents or stealing real
ones. While CAPTCHA-like tests are a possibility, measures should be robust to
improvements in the capabilities of agents. How to balance privacy
considerations with the need for identity verification is another open
question. A potential direction is to understand what mechanisms may allow
humans to prove their status without identity disclosure.

While direct interaction with tools is possible, users and developers may
still opt for the convenience and efficiency offered by APIs, especially if
direct interaction is more complex. APIs can provide standardized interfaces,
tailored services for AI agent use, and can set specific conditions like
access rates or the scope of services available. This preference for API
interaction could reduce the difficulty of obtaining visibility into
decentralized deployments.

###  4.3. Risks

Extending visibility measures to decentralized deployments has serious
implications for privacy and concentration of power. Compute providers that
surveil deployments may be able to infer sensitive information about users.
Given that a handful of compute providers dominates the market (Richter,
2023), monitoring users of those providers would equivalent to monitoring much
of society. In addition to potential abuses of collected information, compute
providers may also have lax security standards that enable attackers to gain
sensitive information.

Distributed enforcement mechanisms through tool and service providers also
face similar concerns. If useful tools and services were closed off to agents
that do not come from certified deployers, users may face strong pressure to
use agents from such deployers. Whether because of government demand
(Greenwald and MacAskill, 2013) or regulatory capture (Dal Bó, 2006), those
deployers may have practises that are inimical to users or may not be
responsive to user interests. Visibility measures for those deployers may be
extremely invasive, such as excessive and unjustified logging. Agents from
those deployers may not be well-suited to the user’s use cases; for example,
the user might require an agent to be able to operate in a low-resource
language. If the market of deployers is heavily concentrated, further reliance
upon them could exacerbate systemic risks (Widder et al., 2023).

One way to mitigate these risks is to emphasize voluntary standards for
adopting visibility measures. For example, open-source frameworks to implement
agent identifiers may allow users to avoid deployers and, at the same time,
facilitate visibility. Even if tool and service providers reject requests from
agents without identifiers, users may easily be able to add an identifier to
their agents. Independent entities may be required to certify valid
identifiers, similar to certificate authorities on the Internet. A source of
inspiration may be Let’s Encrypt, a non-profit certificate authority which
provides a free, automatic certificate process and which has been instrumental
in promoting the use of the more secure HTTPS (Encrypt, 2024). Financial and
technical support to develop open-source, agent identifier frameworks will
likely be critical to the success of voluntary standards. Although, voluntary
standards may help improve visibility into how and where AI agents are broadly
used, they may not deter or provide visibility into malicious use.

Another potential mitigation is to limit the number of tools or services that
require agent identifiers. Obtaining visibility over tools involved in
potentially high-risk activities, such as scientific platforms that handle
pathogens or dangerous chemicals (Boiko et al., 2023), may be of higher
priority. Similarly, visibility into business uses of AI agents, rather than
personal uses, may be more important. Regulations could require certain
businesses (e.g., those above a certain size) that use AI agents to implement
certain visibility measures.

Rather than mandating denial of requests from agents that do not have
identifiers or that do not provide proof that certain visibility measures are
implemented, one alternative is accounting for such compliance when
determining the legal liability of users who deploy their own agents.
Analogously, compliance with HIPAA de-identification standards can be taken
into consideration to reduce regulatory fines or audits for violations (noa, ;
of Human and Health Services, 2022). Similarly, the 2023 U.S. National
Cybersecurity Strategy proposes to shield from private liability companies
that follow cybersecurity best practices (House, 2023). Accounting for
compliance when determining liability may incentivize standards adherence.

Other potential mitigations to explore include decentralized data custody
schemes for compute provider logs and enhanced transparency into both data
collection practises and government requests for data (Goodwin, 2018).

##  5\. Conclusion

Visibility facilitates the governance of increasingly agentic systems. We
assessed three mechanisms for visibility: agent identifiers, real-time
monitoring, and activity logs. Agent identifiers indicate whether and which
agents are involved in an interaction. To aid accountability and incident
investigation, an agent card containing additional information about the agent
can be attached to an agent identifier. Real-time monitoring aims to flag
problematic agent behaviour as it happens. Activity logs record certain inputs
and outputs of agents so as to enable in-depth, post-hoc analysis of
behaviour. We examined how to extend the visibility measures to decentralized
deployments of agents, in particular through using compute providers and tools
and services providers to obtain visibility. Finally, we analyzed the
implications of the visibility measures on privacy and concentration of power.
Rather than advocating for immediate implementation of these measures, further
understanding of the measures and how to mitigate their negative impacts is
required. Such understanding can help to build a foundation for the governance
of AI agents.

Visibility informs actions to manage risks from the deployment of increasingly
agentic systems, but is not by itself sufficient. Even with a comprehensive
understanding of agent activities, interventions to reduce risks rely upon
political will and sociotechnical infrastructure. Visibility into deployment
can help motivate the development of these two factors, but other important
directions include increasing public influence over AI development and
deployment (Chan et al., 2023a; Huang and Siddarth, 2023; Ovadya, 2023; Seger
et al., 2023), developing a wide range of potential policy levers (Anderljung
et al., 2023; Egan and Heim, 2023; Shavit et al., 2023), and implementing
infrastructure and practices to prevent or defend against harms (Buterin,
2023; Sandbrink et al., 2022).

###### Acknowledgements.

We would like to thank the following people for helpful conversations and
feedback over the course of completing this work: Charlotte Siegmann, Chinmay
Deshpande, Nathan Barnard, Leonie Koessler, Kevin Frazier, Matthijs Maas,
Micah Carroll, Yonadav Shavit, Merlin Stein, Ben Bucknall, Shalaleh Rismani,
Paul Crowley, Gabriel Recchia, Janet Egan.

## References

  * [1] 42 U.S. Code § 17941 - Recognition of security practices.  URL https://www.law.cornell.edu/uscode/text/42/17941.
  * noa [2021] Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act) and Amending Certain Union Legislative Acts, 2021.  URL https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A52021PC0206.
  * noa [2022] 12 CFR § 1026.25 - Record retention.  _Federal Register_ , January 2022.
  * noa [2023a] 14 CFR § 91.609 - Flight data recorders and cockpit voice recorders.  _Federal Register_ , January 2023a.
  * noa [2023b] character.ai, 2023b.  URL https://beta.character.ai/.
  * Acemoglu and Restrepo [2018] Daron Acemoglu and Pascual Restrepo.  Artificial Intelligence, Automation, and Work.  In _The Economics of Artificial Intelligence: An Agenda_ , pages 197–236. University of Chicago Press, January 2018.  URL https://www.nber.org/books-and-chapters/economics-artificial-intelligence-agenda/artificial-intelligence-automation-and-work.
  * Acemoglu and Restrepo [2019] Daron Acemoglu and Pascual Restrepo.  Automation and New Tasks: How Technology Displaces and Reinstates Labor.  _Journal of Economic Perspectives_ , 33(2):3–30, May 2019.  ISSN 0895-3309.  doi: 10.1257/jep.33.2.3.  URL https://www.aeaweb.org/articles?id=10.1257/jep.33.2.3.
  * Anderljung et al. [2023] Markus Anderljung, Joslyn Barnhart, Anton Korinek, Jade Leung, Cullen O’Keefe, Jess Whittlestone, Shahar Avin, Miles Brundage, Justin Bullock, Duncan Cass-Beggs, Ben Chang, Tantum Collins, Tim Fist, Gillian Hadfield, Alan Hayes, Lewis Ho, Sara Hooker, Eric Horvitz, Noam Kolt, Jonas Schuett, Yonadav Shavit, Divya Siddarth, Robert Trager, and Kevin Wolf.  Frontier AI Regulation: Managing Emerging Risks to Public Safety, September 2023.  URL http://arxiv.org/abs/2307.03718.  arXiv:2307.03718 [cs].
  * Bai et al. [2024] (Max) Hui Bai, Jan G. Voelkel, johannes C. Eichstaedt, and Robb Willer.  Artificial Intelligence Can Persuade Humans on Political Issues.  January 2024.  doi: 10.31219/osf.io/stakv.  URL https://osf.io/stakv.  Publisher: OSF.
  * Bai et al. [2022] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton, Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan.  Constitutional AI: Harmlessness from AI Feedback, December 2022.  URL http://arxiv.org/abs/2212.08073.  arXiv:2212.08073 [cs].
  * Bailey et al. [2023] Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons.  Image Hijacks: Adversarial Images can Control Generative Models at Runtime, September 2023.  URL https://arxiv.org/abs/2309.00236v2.
  * Baker [2023] Tessa Baker.  The EU AI Act: A Primer, September 2023.  URL https://cset.georgetown.edu/article/the-eu-ai-act-a-primer/.
  * Ball [2010] Kirstie Ball.  Workplace surveillance: an overview.  _Labor History_ , 51(1):87–106, February 2010.  ISSN 0023-656X.  doi: 10.1080/00236561003654776.  URL https://doi.org/10.1080/00236561003654776.  Publisher: Routledge _eprint: https://doi.org/10.1080/00236561003654776.
  * Barnett and Besiroglu [2023] Matthew Barnett and Tamay Besiroglu.  The Direct Approach, 2023.  URL https://epochai.org/blog/the-direct-approach.
  * Bhatt et al. [2023] Manish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis, Shengye Wan, Ivan Evtimov, Dominik Gabi, Daniel Song, Faizan Ahmad, Cornelius Aschermann, Lorenzo Fontana, Sasha Frolov, Ravi Prakash Giri, Dhaval Kapil, Yiannis Kozyrakis, David LeBlanc, James Milazzo, Aleksandar Straumann, Gabriel Synnaeve, Varun Vontimitta, Spencer Whitman, and Joshua Saxe.  Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models, December 2023.  URL http://arxiv.org/abs/2312.04724.  arXiv:2312.04724 [cs].
  * Biagioni et al. [2022] David Biagioni, Xiangyu Zhang, Dylan Wald, Deepthi Vaidhynathan, Rohit Chintala, Jennifer King, and Ahmed S. Zamzam.  PowerGridworld: a framework for multi-agent reinforcement learning in power systems.  In _Proceedings of the Thirteenth ACM International Conference on Future Energy Systems_ , e-Energy ’22, pages 565–570, New York, NY, USA, June 2022. Association for Computing Machinery.  ISBN 978-1-4503-9397-3.  doi: 10.1145/3538637.3539616.  URL https://dl.acm.org/doi/10.1145/3538637.3539616.
  * Bielicki et al. [2020] Julia A. Bielicki, Xavier Duval, Nina Gobat, Herman Goossens, Marion Koopmans, Evelina Tacconelli, and Sylvie van der Werf.  Monitoring approaches for health-care workers during the COVID-19 pandemic.  _The Lancet. Infectious Diseases_ , 20(10):e261–e267, October 2020.  ISSN 1474-4457.  doi: 10.1016/S1473-3099(20)30458-8.
  * Birkinshaw [2006] Patrick Birkinshaw.  Freedom of Information and Openness: Fundamental Human Rights?  _Administrative Law Review_ , 58(1):177–218, 2006.  ISSN 0001-8368.  URL https://www.jstor.org/stable/40712007.  Publisher: American Bar Association.
  * Bluemke et al. [2023] Emma Bluemke, Tantum Collins, Ben Garfinkel, and Andrew Trask.  Exploring the Relevance of Data Privacy-Enhancing Technologies for AI Governance Use Cases, March 2023.  URL http://arxiv.org/abs/2303.08956.  arXiv:2303.08956 [cs].
  * Board of Governors of the Federal Reserve System [2021] Board of Governors of the Federal Reserve System.  Proactive Monitoring of Markets and Institutions, December 2021.  URL https://www.federalreserve.gov/financial-stability/proactive-monitoring-of-markets-and-institutions.htm.
  * Boiko et al. [2023] Daniil A. Boiko, Robert MacKnight, and Gabe Gomes.  Emergent autonomous scientific research capabilities of large language models, April 2023.  URL http://arxiv.org/abs/2304.05332.  arXiv:2304.05332 [physics].
  * Bommasani et al. [2022] Rishi Bommasani, Kathleen A. Creel, Ananya Kumar, Dan Jurafsky, and Percy Liang.  Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?, November 2022.  URL http://arxiv.org/abs/2211.13972.  arXiv:2211.13972 [cs].
  * Bommasani et al. [2023] Rishi Bommasani, Dilara Soylu, Thomas I. Liao, Kathleen A. Creel, and Percy Liang.  Ecosystem Graphs: The Social Footprint of Foundation Models, March 2023.  URL http://arxiv.org/abs/2303.15772.  arXiv:2303.15772 [cs].
  * Bond et al. [2012] Robert M. Bond, Christopher J. Fariss, Jason J. Jones, Adam D. I. Kramer, Cameron Marlow, Jaime E. Settle, and James H. Fowler.  A 61-million-person experiment in social influence and political mobilization.  _Nature_ , 489(7415):10.1038/nature11421, September 2012.  ISSN 0028-0836.  doi: 10.1038/nature11421.  URL https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3834737/.
  * Bowman [2022] Samuel Bowman.  The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 7484–7499, Dublin, Ireland, May 2022. Association for Computational Linguistics.  doi: 10.18653/v1/2022.acl-long.516.  URL https://aclanthology.org/2022.acl-long.516.
  * Bran et al. [2023] Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller.  ChemCrow: Augmenting large-language models with chemistry tools, June 2023.  URL http://arxiv.org/abs/2304.05376.  arXiv:2304.05376 [physics, stat].
  * Burrell and Fourcade [2021] Jenna Burrell and Marion Fourcade.  The Society of Algorithms.  _Annual Review of Sociology_ , 47(1):213–237, 2021.  doi: 10.1146/annurev-soc-090820-020800.  URL https://doi.org/10.1146/annurev-soc-090820-020800.  _eprint: https://doi.org/10.1146/annurev-soc-090820-020800.
  * Buterin [2023] Vitalik Buterin.  My Techno-Optimism, November 2023.  URL https://vitalik.eth.limo/general/2023/11/27/techno_optimism.html.
  * Carey and Everitt [2023] Ryan Carey and Tom Everitt.  Human Control: Definitions and Algorithms.  In _Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence_ , pages 271–281. PMLR, July 2023.  URL https://proceedings.mlr.press/v216/carey23a.html.  ISSN: 2640-3498.
  * Cassidy [2009] John Cassidy.  _How markets fail: The logic of economic calamities_.  Farrar, Straus and Giroux, 2009.
  * CFTC [2023] CFTC.  CFTC Market Surveillance Program, November 2023.  URL https://www.cftc.gov/IndustryOversight/MarketSurveillance/CFTCMarketSurveillanceProgram/index.htm.
  * CFTC [2023] CFTC.  Large Trader Reporting Program, November 2023.  URL https://www.cftc.gov/IndustryOversight/MarketSurveillance/LargeTraderReportingProgram/index.htm.
  * Chan et al. [2023a] Alan Chan, Herbie Bradley, and Nitarshan Rajkumar.  Reclaiming the Digital Commons: A Public Data Trust for Training Data.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, pages 855–868, New York, NY, USA, August 2023a. Association for Computing Machinery.  ISBN 9798400702310.  doi: 10.1145/3600211.3604658.  URL https://dl.acm.org/doi/10.1145/3600211.3604658.
  * Chan et al. [2023b] Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bhatt, Adrian Weller, David Krueger, and Tegan Maharaj.  Harms from Increasingly Agentic Algorithmic Systems.  In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’23, pages 651–666, New York, NY, USA, June 2023b. Association for Computing Machinery.  ISBN 9798400701924.  doi: 10.1145/3593013.3594033.  URL https://dl.acm.org/doi/10.1145/3593013.3594033.
  * Chase [2022] Harrison Chase.  LangChain 0.0.77 Docs, 2022.  URL https://langchain.readthedocs.io/en/latest/modules/agents/getting_started.html.
  * ChrisHMSFT et al. [2023] ChrisHMSFT, PatrickFarley, mrbullwinkle, eric urban, and aahill.  Data, privacy, and security for Azure OpenAI Service - Azure AI services, June 2023.  URL https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy.
  * Cloud [2023] Google Cloud.  Generative AI, Privacy, and Google Cloud.  Technical report, 2023.  URL https://services.google.com/fh/files/misc/genai_privacy_google_cloud_202308.pdf.
  * Coglianese et al. [2004] Cary Coglianese, Richard Zeckhauser, and Edward Parson.  Seeking Truth for Power: Informational Strategy and Regulatory Policy Making.  _Minnesota Law Review_ , January 2004.  URL https://scholarship.law.upenn.edu/faculty_scholarship/107.
  * Cohen and Havlin [2010] Reuven Cohen and Shlomo Havlin.  _Complex networks: structure, robustness and function_.  Cambridge university press, 2010.
  * Commission [2022a] Securities and Exchange Commission.  17 CFR § 240.17a-4 - Records to be preserved by certain exchange members, brokers and dealers.  _Federal Register_ , April 2022a.
  * Commission [2022b] Securities and Exchange Commission.  17 CFR § 240.17a-3 - Records to be made by certain exchange members, brokers and dealers.  _Federal Register_ , April 2022b.
  * Cooper et al. [2022] A. Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum.  Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning.  In _2022 ACM Conference on Fairness, Accountability, and Transparency_. ACM, June 2022.  doi: 10.1145/3531146.3533150.  URL https://doi.org/10.1145%2F3531146.3533150.
  * Council [2022] PCI Security Standars Council.  Payment Card Industry Data Security Standard, March 2022.
  * Critch and Russell [2023] Andrew Critch and Stuart Russell.  TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI, June 2023.  URL http://arxiv.org/abs/2306.06924.  arXiv:2306.06924 [cs].
  * CTFC and SEC [2010] CTFC and SEC.  Preliminary findings regarding the market events of may 6, 2010.  Technical report, U.S. Commodity Futures Trading Commission and U.S. Securities & Exchange Commission, May 2010.  URL https://www.sec.gov/sec-cftc-prelimreport.pdf.  tex.creationdate: 2023-10-29T16:47:57.
  * Cummings [2004] Mary Cummings.  Automation Bias in Intelligent Time Critical Decision Support Systems.  In _AIAA 1st Intelligent Systems Technical Conference_ , Infotech@Aerospace Conferences. American Institute of Aeronautics and Astronautics, September 2004.  doi: 10.2514/6.2004-6313.  URL https://arc.aiaa.org/doi/10.2514/6.2004-6313.
  * Dal Bó [2006] Ernesto Dal Bó.  Regulatory capture: A review.  _Oxford review of economic policy_ , 22(2):203–225, 2006.  Publisher: Oxford University Press.
  * Davidson et al. [2023] Tom Davidson, Jean-Stanislas Denain, Pablo Villalobos, and Guillem Bas.  AI capabilities can be significantly improved without expensive retraining, December 2023.  URL http://arxiv.org/abs/2312.07413.  arXiv:2312.07413 [cs].
  * De La Garza [2020] Alejandro De La Garza.  States’ Automated Systems Are Trapping Citizens in Bureaucratic Nightmares With Their Lives on the Line.  _TIME_ , May 2020.  URL https://time.com/5840609/algorithm-unemployment/.
  * Degrave et al. [2022] Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de las Casas, Craig Donner, Leslie Fritz, Cristian Galperti, Andrea Huber, James Keeling, Maria Tsimpoukelli, Jackie Kay, Antoine Merle, Jean-Marc Moret, Seb Noury, Federico Pesamosca, David Pfau, Olivier Sauter, Cristian Sommariva, Stefano Coda, Basil Duval, Ambrogio Fasoli, Pushmeet Kohli, Koray Kavukcuoglu, Demis Hassabis, and Martin Riedmiller.  Magnetic control of tokamak plasmas through deep reinforcement learning.  _Nature_ , 602(7897):414–419, February 2022.  ISSN 1476-4687.  doi: 10.1038/s41586-021-04301-9.  URL https://www.nature.com/articles/s41586-021-04301-9.  Number: 7897 Publisher: Nature Publishing Group.
  * Delacroix and Lawrence [2019] Sylvie Delacroix and Neil D Lawrence.  Bottom-up data Trusts: disturbing the ‘one size fits all’ approach to data governance.  _International Data Privacy Law_ , 9(4):236–252, November 2019.  ISSN 2044-3994.  doi: 10.1093/idpl/ipz014.  URL https://doi.org/10.1093/idpl/ipz014.
  * Delacroix et al. [2020] Sylvie Delacroix, Joelle Pineau, and Jessica Montgomery.  Democratising the Digital Revolution: The Role of Data Governance, June 2020.  URL https://papers.ssrn.com/abstract=3720208.
  * DiResta [2019] Renee DiResta.  A New Law Makes Bots Identify Themselves—That’s the Problem.  _Wired_ , July 2019.  ISSN 1059-1028.  URL https://www.wired.com/story/law-makes-bots-identify-themselves/.  Section: tags.
  * Dorner [2021] Florian E. Dorner.  Algorithmic collusion: A critical review, October 2021.  URL http://arxiv.org/abs/2110.04740.  arXiv:2110.04740 [cs].
  * Egan and Heim [2023] Janet Egan and Lennart Heim.  Oversight for Frontier AI through a Know-Your-Customer Scheme for Compute Providers, October 2023.  URL http://arxiv.org/abs/2310.13625.  arXiv:2310.13625 [cs].
  * Ely and Szentes [2023] Jeffrey C. Ely and Balazs Szentes.  Natural selection of artificial intelligence.  2023\.
  * Encrypt [2024] Let’s Encrypt.  Let’s Encrypt Stats - Let’s Encrypt, January 2024.  URL https://letsencrypt.org/stats/.
  * Endsley and Kaber [1999] M. R. Endsley and D. B. Kaber.  Level of automation effects on performance, situation awareness and workload in a dynamic control task.  _Ergonomics_ , 42(3):462–492, March 1999.  ISSN 0014-0139.  doi: 10.1080/001401399185595.
  * Endsley and Kiris [1995] Mica R. Endsley and Esin O. Kiris.  The Out-of-the-Loop Performance Problem and Level of Control in Automation.  _Human Factors_ , 37(2):381–394, June 1995.  ISSN 0018-7208.  doi: 10.1518/001872095779064555.  URL https://doi.org/10.1518/001872095779064555.  Publisher: SAGE Publications Inc.
  * Epoch [2023] Epoch.  Key trends and figures in Machine Learning, 2023.  URL https://epochai.org/trends.
  * Erdil and Besiroglu [2023] Ege Erdil and Tamay Besiroglu.  Algorithmic progress in computer vision, 2023.  _eprint: 2212.05153.
  * Fama and Jensen [1983] Eugene F. Fama and Michael C. Jensen.  Agency Problems and Residual Claims.  _The Journal of Law & Economics_, 26(2):327–349, 1983.  ISSN 0022-2186.  URL https://www.jstor.org/stable/725105.  Publisher: [University of Chicago Press, Booth School of Business, University of Chicago, University of Chicago Law School].
  * Finck and Pallas [2020] Michèle Finck and Frank Pallas.  They who must not be identified—distinguishing personal from non-personal data under the GDPR.  _International Data Privacy Law_ , 10(1):11–36, February 2020.  ISSN 2044-3994.  doi: 10.1093/idpl/ipz026.  URL https://doi.org/10.1093/idpl/ipz026.
  * Frazier [2023] Kevin Frazier.  The Right to Reality, December 2023.  URL https://www.lawfaremedia.org/article/the-right-to-reality.
  * Gan et al. [2024] Chengguang Gan, Qinghao Zhang, and Tatsunori Mori.  Application of LLM Agents in Recruitment: A Novel Framework for Resume Screening, January 2024.  URL http://arxiv.org/abs/2401.08315.  arXiv:2401.08315 [cs].
  * Ganguli et al. [2022] Deep Ganguli, Danny Hernandez, Liane Lovitt, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova Dassarma, Dawn Drain, Nelson Elhage, Sheer El Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Scott Johnston, Andy Jones, Nicholas Joseph, Jackson Kernian, Shauna Kravec, Ben Mann, Neel Nanda, Kamal Ndousse, Catherine Olsson, Daniela Amodei, Tom Brown, Jared Kaplan, Sam McCandlish, Christopher Olah, Dario Amodei, and Jack Clark.  Predictability and Surprise in Large Generative Models.  In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’22, pages 1747–1764, New York, NY, USA, June 2022. Association for Computing Machinery.  ISBN 978-1-4503-9352-2.  doi: 10.1145/3531146.3533229.  URL https://dl.acm.org/doi/10.1145/3531146.3533229.
  * Gebru et al. [2021] Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford.  Datasheets for datasets.  _Communications of the ACM_ , 64(12):86–92, December 2021.  ISSN 0001-0782, 1557-7317.  doi: 10.1145/3458723.  URL https://dl.acm.org/doi/10.1145/3458723.
  * Gilbert et al. [2023] Thomas Krendl Gilbert, Nathan Lambert, Sarah Dean, Tom Zick, Aaron Snoswell, and Soham Mehta.  Reward Reports for Reinforcement Learning.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, pages 84–130, New York, NY, USA, August 2023. Association for Computing Machinery.  ISBN 9798400702310.  doi: 10.1145/3600211.3604698.  URL https://dl.acm.org/doi/10.1145/3600211.3604698.
  * Goodwin [2018] Chloe Goodwin.  Cooperation or resistance?: The role of tech companies in government surveillance.  131:1722–1722, 2018.
  * Google [2023] Google.  Bard Privacy Help Hub - Bard Help, December 2023.  URL https://support.google.com/bard/answer/13594961?sjid=16420951458997305974-EU&visit_id=638406643042311657-3533103185&p=bard_pntos_retention&rd=1#retention&zippy=%2Cwhy-does-google-retain-my-conversations-after-i-turn-off-bard-activity-and-what-does-google-do-with-this-data.
  * Green [2022] Ben Green.  The flaws of policies requiring human oversight of government algorithms.  _Computer Law & Security Review_, 45:105681, July 2022.  ISSN 0267-3649.  doi: 10.1016/j.clsr.2022.105681.  URL https://www.sciencedirect.com/science/article/pii/S0267364922000292.
  * Green [2023] David Green.  Emergence in complex networks of simple agents.  _Journal of Economic Interaction and Coordination_ , 18:1–44, May 2023.  doi: 10.1007/s11403-023-00385-w.
  * Greenblatt et al. [2024] Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, and Fabien Roger.  AI Control: Improving Safety Despite Intentional Subversion, January 2024.  URL http://arxiv.org/abs/2312.06942.  arXiv:2312.06942 [cs].
  * Greenwald and MacAskill [2013] Glenn Greenwald and Ewen MacAskill.  NSA Prism program taps in to user data of Apple, Google and others.  _The Guardian_ , June 2013.  ISSN 0261-3077.  URL https://www.theguardian.com/world/2013/jun/06/us-tech-giants-nsa-data.
  * Guarnieri et al. [2021] Valentina Guarnieri, Maria Moriondo, Mattia Giovannini, Lorenzo Lodi, Silvia Ricci, Laura Pisano, Paola Barbacci, Costanza Bini, Giuseppe Indolfi, Alberto Zanobini, and Chiara Azzari.  Surveillance on Healthcare Workers During the First Wave of SARS-CoV-2 Pandemic in Italy: The Experience of a Tertiary Care Pediatric Hospital.  _Frontiers in Public Health_ , 9, 2021.  ISSN 2296-2565.  URL https://www.frontiersin.org/articles/10.3389/fpubh.2021.644702.
  * Hammond and TBD [2024] Lewis Hammond and TBD.  Multi-Agent Risks from Advanced AI.  Technical report, 2024.
  * Hazell [2023] Julian Hazell.  Large Language Models Can Be Used To Effectively Scale Spear Phishing Campaigns, May 2023.  URL http://arxiv.org/abs/2305.06972.  arXiv:2305.06972 [cs].
  * Hendrycks [2023] Dan Hendrycks.  Natural Selection Favors AIs over Humans, July 2023.  URL http://arxiv.org/abs/2303.16200.  arXiv:2303.16200 [cs].
  * Henzinger et al. [2023] Thomas Henzinger, Mahyar Karimi, Konstantin Kueffner, and Kaushik Mallik.  Runtime Monitoring of Dynamic Fairness Properties.  In _Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’23, pages 604–614, New York, NY, USA, June 2023. Association for Computing Machinery.  ISBN 9798400701924.  doi: 10.1145/3593013.3594028.  URL https://dl.acm.org/doi/10.1145/3593013.3594028.
  * Hobbhahn and Besiroglu [2022] Marius Hobbhahn and Tamay Besiroglu.  Trends in GPU Price-Performance, 2022.  URL https://epochai.org/blog/trends-in-gpu-price-performance.
  * Hobbhahn et al. [2023] Marius Hobbhahn, Lennart Heim, and Gökçe Aydos.  Trends in machine learning hardware, 2023.  URL https://epochai.org/blog/trends-in-machine-learning-hardware.
  * Hoffmann et al. [2022] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre.  Training Compute-Optimal Large Language Models, March 2022.  URL http://arxiv.org/abs/2203.15556.  arXiv:2203.15556 [cs].
  * Holmstrom and Milgrom [1991] Bengt Holmstrom and Paul Milgrom.  Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design.  _Journal of Law, Economics, & Organization_, 7:24–52, 1991.  ISSN 8756-6222.  URL https://www.jstor.org/stable/764957.  Publisher: Oxford University Press.
  * Holmström [1979] Bengt Holmström.  Moral Hazard and Observability.  _The Bell Journal of Economics_ , 10(1):74–91, 1979.  ISSN 0361-915X.  doi: 10.2307/3003320.  URL https://www.jstor.org/stable/3003320.  Publisher: [RAND Corporation, Wiley].
  * Horowitz and Scharre [2021] Michael Horowitz and Paul Scharre.  AI and International Stability: Risks and Confidence-Building Measures.  Technical report, Centrer for a New American Security, January 2021.  URL https://www.cnas.org/publications/reports/ai-and-international-stability-risks-and-confidence-building-measures.
  * House [2023] The White House.  National Cybersecurity Strategy.  Technical report, March 2023.
  * Huang and Siddarth [2023] Saffron Huang and Divya Siddarth.  Generative AI and the Digital Commons, February 2023.  URL https://cip.org/research/generative-ai-digital-commons.
  * ISO [2022] ISO.  ISO/IEC 27001, 2022.  URL https://www.iso.org/standard/27001.
  * Jakesch et al. [2023] Maurice Jakesch, Advait Bhat, Daniel Buschek, Lior Zalmanson, and Mor Naaman.  Co-Writing with Opinionated Language Models Affects Users’ Views.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , CHI ’23, pages 1–15, New York, NY, USA, April 2023. Association for Computing Machinery.  ISBN 978-1-4503-9421-5.  doi: 10.1145/3544548.3581196.  URL https://dl.acm.org/doi/10.1145/3544548.3581196.
  * Javadi et al. [2020] Seyyed Ahmad Javadi, Richard Cloete, Jennifer Cobbe, Michelle Seng Ah Lee, and Jatinder Singh.  Monitoring Misuse for Accountable ’Artificial Intelligence as a Service’.  In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’20, pages 300–306, New York, NY, USA, February 2020. Association for Computing Machinery.  ISBN 978-1-4503-7110-0.  doi: 10.1145/3375627.3375873.  URL https://dl.acm.org/doi/10.1145/3375627.3375873.
  * Javadi et al. [2021] Seyyed Ahmad Javadi, Chris Norval, Richard Cloete, and Jatinder Singh.  Monitoring AI Services for Misuse.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’21, pages 597–607, New York, NY, USA, July 2021. Association for Computing Machinery.  ISBN 978-1-4503-8473-5.  doi: 10.1145/3461702.3462566.  URL https://dl.acm.org/doi/10.1145/3461702.3462566.
  * Jensen and Meckling [1976] Michael C. Jensen and William H. Meckling.  Theory of the firm: Managerial behavior, agency costs and ownership structure.  _Journal of Financial Economics_ , 3(4):305–360, October 1976.  ISSN 0304-405X.  doi: 10.1016/0304-405X(76)90026-X.  URL https://www.sciencedirect.com/science/article/pii/0304405X7690026X.
  * Kaplan et al. [2020] Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.  Scaling Laws for Neural Language Models, January 2020.  URL http://arxiv.org/abs/2001.08361.  arXiv:2001.08361 [cs, stat].
  * Karimi et al. [2021] Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera.  Algorithmic Recourse: From Counterfactual Explanations to Interventions.  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, pages 353–362, New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 978-1-4503-8309-7.  doi: 10.1145/3442188.3445899.  URL https://doi.org/10.1145/3442188.3445899.  event-place: Virtual Event, Canada.
  * Karinshak et al. [2023] Elise Karinshak, Sunny Xun Liu, Joon Sung Park, and Jeffrey T. Hancock.  Working With AI to Persuade: Examining a Large Language Model’s Ability to Generate Pro-Vaccination Messages.  _Proceedings of the ACM on Human-Computer Interaction_ , 7(CSCW1):116:1–116:29, April 2023.  doi: 10.1145/3579592.  URL https://doi.org/10.1145/3579592.
  * Kenton et al. [2023] Zachary Kenton, Ramana Kumar, Sebastian Farquhar, Jonathan Richens, Matt MacDermott, and Tom Everitt.  Discovering agents.  _Artificial Intelligence_ , 322:103963, September 2023.  ISSN 0004-3702.  doi: 10.1016/j.artint.2023.103963.  URL https://www.sciencedirect.com/science/article/pii/S0004370223001091.
  * Kinniment et al. [2023] Megan Kinniment, Lucas Jun Koba Sato, Haoxing Du, Brian Goodrich, Max Hasin, Lawrence Chan, Luke Harold Miles, Tao R. Lin, Hjalmar Wijk, Joel Burget, Aaron Ho, Elizabeth Barnes, and Paul Christiano.  Evaluating Language-Model Agents on Realistic Autonomous Tasks, July 2023.  URL https://evals.alignment.org/Evaluating_LMAs_Realistic_Tasks.pdf.
  * Kittaka et al. [2023] Yuta Kittaka, Susumu Sato, and Yusuke Zennyo.  Self-preferencing by platforms: A literature review.  _Japan and the World Economy_ , 66:101191, 2023.  ISSN 0922-1425.  doi: https://doi.org/10.1016/j.japwor.2023.101191.  URL https://www.sciencedirect.com/science/article/pii/S0922142523000178.
  * Koessler and Schuett [2023] Leonie Koessler and Jonas Schuett.  Risk assessment at AGI companies: A review of popular risk assessment techniques from other safety-critical industries, July 2023.  URL http://arxiv.org/abs/2307.08823.  arXiv:2307.08823 [cs].
  * Korinek and Juelfs [2022] Anton Korinek and Megan Juelfs.  Preparing for the (Non-Existent?) Future of Work, June 2022.  URL https://www.nber.org/papers/w30172.
  * Kott [2018] Alexander Kott.  Intelligent Autonomous Agents are Key to Cyber Defense of the Future Army Networks.  _The Cyber Defense Review_ , 3(3):57–70, 2018.  ISSN 2474-2120.  URL https://www.jstor.org/stable/26554997.  Publisher: Army Cyber Institute.
  * Laffont and Martimort [2002] Jean-Jacques Laffont and David Martimort.  _The Theory of Incentives: The Principal-Agent Model_.  Princeton University Press, 2002.  ISBN 978-0-691-09184-6.  doi: 10.2307/j.ctv7h0rwr.  URL https://www.jstor.org/stable/j.ctv7h0rwr.
  * Liu et al. [2024] Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu.  A Survey of Text Watermarking in the Era of Large Language Models, January 2024.  URL http://arxiv.org/abs/2312.07913.  arXiv:2312.07913 [cs].
  * Liu et al. [2023] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang.  AgentBench: Evaluating LLMs as Agents, October 2023.  URL http://arxiv.org/abs/2308.03688.  arXiv:2308.03688 [cs].
  * Melley [2024] Brian Melley.  Judges in England and Wales Given Cautious Approval to Use AI.  _TIME_ , January 2024.  URL https://time.com/6553030/ai-legal-opinions-england-wales/.
  * Mialon et al. [2023] Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom.  GAIA: a benchmark for General AI Assistants, November 2023.  URL https://arxiv.org/abs/2311.12983v1.
  * Milli et al. [2023] Smitha Milli, Micah Carroll, Yike Wang, Sashrika Pandey, Sebastian Zhao, and Anca D. Dragan.  Engagement, User Satisfaction, and the Amplification of Divisive Content on Social Media, December 2023.  URL http://arxiv.org/abs/2305.16941.  arXiv:2305.16941 [cs].
  * Minkkinen et al. [2022] Matti Minkkinen, Joakim Laine, and Matti Mäntymäki.  Continuous Auditing of Artificial Intelligence: a Conceptualization and Assessment of Tools and Frameworks.  _Digital Society_ , 1(3):21, October 2022.  ISSN 2731-4669.  doi: 10.1007/s44206-022-00022-2.  URL https://doi.org/10.1007/s44206-022-00022-2.
  * Mitchell et al. [2019] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru.  Model Cards for Model Reporting.  In _Proceedings of the Conference on Fairness, Accountability, and Transparency_ , pages 220–229, January 2019.  doi: 10.1145/3287560.3287596.  URL http://arxiv.org/abs/1810.03993.  arXiv:1810.03993 [cs].
  * Morris et al. [2023] Meredith Ringel Morris, Jascha Sohl-dickstein, Noah Fiedel, Tris Warkentin, Allan Dafoe, Aleksandra Faust, Clement Farabet, and Shane Legg.  Levels of AGI: Operationalizing Progress on the Path to AGI, November 2023.  URL http://arxiv.org/abs/2311.02462.  arXiv:2311.02462 [cs].
  * mrbullwinkle and eric urban [2023] mrbullwinkle and eric urban.  Azure OpenAI Service abuse monitoring - Azure OpenAI, July 2023.  URL https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/abuse-monitoring.
  * Mökander et al. [2023] Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi.  Auditing Large Language Models: A Three-Layered Approach, February 2023.  URL https://papers.ssrn.com/abstract=4361607.
  * Naihin et al. [2023] Silen Naihin, David Atkinson, Marc Green, Merwane Hamadi, Craig Swift, Douglas Schonholtz, Adam Tauman Kalai, and David Bau.  Testing Language Model Agents Safely in the Wild, December 2023.  URL http://arxiv.org/abs/2311.10538.  arXiv:2311.10538 [cs].
  * Ngo et al. [2022] Richard Ngo, Lawrence Chan, and Sören Mindermann.  The alignment problem from a deep learning perspective, August 2022.  URL https://arxiv.org/abs/2209.00626v5.
  * Nissenbaum [1996] Helen Nissenbaum.  Accountability in a computerized society.  _Science and Engineering Ethics_ , 2(1):25–42, March 1996.  ISSN 1471-5546.  doi: 10.1007/BF02639315.  URL https://doi.org/10.1007/BF02639315.
  * O’Brien et al. [2023] Joe O’Brien, Shaun Ee, and Zoe Williams.  Deployment Corrections: An incident response framework for frontier AI models, September 2023.  URL http://arxiv.org/abs/2310.00328.  arXiv:2310.00328 [cs].
  * of Human and Health Services [2022] U.S. Department of Human and Health Services.  Guidance Regarding Methods for De-identification of Protected Health Information in Accordance with the Health Insurance Portability and Accountability Act (HIPAA) Privacy Rule, October 2022.  URL https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html.  Last Modified: 2023-02-22T10:17:21-0500.
  * of Trading and Markets [2008] Division of Trading and Markets.  Guide to Broker-Dealer Registration.  Technical report, U.S. Securities and Exchange Commission, April 2008.
  * OpenAI [2023a] OpenAI.  ChatGPT plugins, 2023a.  URL https://openai.com/blog/chatgpt-plugins.
  * OpenAI [2023b] OpenAI.  GPT-4 Technical Report, March 2023b.  URL http://arxiv.org/abs/2303.08774.  arXiv:2303.08774 [cs].
  * OpenAI [2023c] OpenAI.  Introducing GPTs, November 2023c.  URL https://openai.com/blog/introducing-gpts.
  * OpenAI [2024] OpenAI.  Introducing the GPT Store, January 2024.  URL https://openai.com/blog/introducing-the-gpt-store.
  * Ovadya [2023] Aviv Ovadya.  ’Generative CI’ through Collective Response Systems, February 2023.  URL http://arxiv.org/abs/2302.00672.  arXiv:2302.00672 [cs].
  * Pan et al. [2023] Alexander Pan, Chan Jun Shern, Andy Zou, Nathaniel Li, Steven Basart, and others.  Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark, April 2023.  arXiv: 2304.03279 [cs] Issue: arXiv:2304.03279.
  * Park et al. [2023] Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein.  Generative Agents: Interactive Simulacra of Human Behavior.  In _Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology_ , UIST ’23, pages 1–22, New York, NY, USA, October 2023. Association for Computing Machinery.  ISBN 9798400701320.  doi: 10.1145/3586183.3606763.  URL https://dl.acm.org/doi/10.1145/3586183.3606763.
  * Raghavan et al. [2020] Manish Raghavan, Solon Barocas, Jon Kleinberg, and Karen Levy.  Mitigating bias in algorithmic hiring: evaluating claims and practices.  In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_ , FAT* ’20, pages 469–481, New York, NY, USA, January 2020. Association for Computing Machinery.  ISBN 978-1-4503-6936-7.  doi: 10.1145/3351095.3372828.  URL https://doi.org/10.1145/3351095.3372828.
  * Raji et al. [2022a] Inioluwa Deborah Raji, I. Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst.  The Fallacy of AI Functionality.  In _2022 ACM Conference on Fairness, Accountability, and Transparency_ , pages 959–972, Seoul Republic of Korea, June 2022a. ACM.  ISBN 978-1-4503-9352-2.  doi: 10.1145/3531146.3533158.  URL https://dl.acm.org/doi/10.1145/3531146.3533158.
  * Raji et al. [2022b] Inioluwa Deborah Raji, Peggy Xu, Colleen Honigsberg, and Daniel Ho.  Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance.  In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’22, pages 557–571, New York, NY, USA, July 2022b. Association for Computing Machinery.  ISBN 978-1-4503-9247-1.  doi: 10.1145/3514094.3534181.  URL https://dl.acm.org/doi/10.1145/3514094.3534181.
  * Reed et al. [2022] Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-maron, Mai Giménez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas.  A Generalist Agent.  _Transactions on Machine Learning Research_ , 2022.  URL https://openreview.net/forum?id=1ikK0kHjvj.
  * Richards [2023] Toran Bruce Richards.  Auto-GPT: An Autonomous GPT-4 Experiment, April 2023.  URL https://github.com/Significant-Gravitas/Auto-GPT.  original-date: 2023-03-16T09:21:07Z.
  * Richter [2023] Felix Richter.  Infographic: Amazon Maintains Lead in the Cloud Market, August 2023.  URL https://www.statista.com/chart/18819/worldwide-market-share-of-leading-cloud-infrastructure-service-providers.
  * Rismani et al. [2023] Shalaleh Rismani, Renee Shelby, Andrew Smart, Edgar Jatho, Joshua Kroll, AJung Moon, and Negar Rostamzadeh.  From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML.  In _Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems_ , CHI ’23, pages 1–18, New York, NY, USA, April 2023. Association for Computing Machinery.  ISBN 978-1-4503-9421-5.  doi: 10.1145/3544548.3581407.  URL https://dl.acm.org/doi/10.1145/3544548.3581407.
  * Roger and Greenblatt [2023] Fabien Roger and Ryan Greenblatt.  Preventing Language Models From Hiding Their Reasoning, October 2023.  URL http://arxiv.org/abs/2310.18512.  arXiv:2310.18512 [cs].
  * Ruan et al. [2023] Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J. Maddison, and Tatsunori Hashimoto.  Identifying the Risks of LM Agents with an LM-Emulated Sandbox, September 2023.  URL http://arxiv.org/abs/2309.15817.  arXiv:2309.15817 [cs].
  * Russell and Norvig [2021] Stuart J. Russell and Peter Norvig.  _Artificial Intelligence: A Modern Approach_.  4 edition, 2021.
  * Sandbrink et al. [2022] Jonas Sandbrink, Hamish Hobbs, Jacob Swett, Allan Dafoe, and Anders Sandberg.  Differential technology development: An innovation governance consideration for navigating technology risks, September 2022.  URL https://papers.ssrn.com/abstract=4213670.
  * Sandbrink [2023] Jonas B. Sandbrink.  Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools, June 2023.  URL http://arxiv.org/abs/2306.13952.  arXiv:2306.13952 [cs].
  * Scharre [2021] Paul Scharre.  Debunking the AI Arms Race Theory (Summer 2021).  2021\.  URL https://hdl.handle.net/2152/87035.  Publisher: Texas National Security Review.
  * Schelling [1978] Thomas C. Schelling.  _Micromotives and Macrobehavior_.  W. W. Norton & Company, October 1978.  ISBN 0-393-09009-4.
  * Schick et al. [2023] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom.  Toolformer: Language Models Can Teach Themselves to Use Tools, February 2023.  URL http://arxiv.org/abs/2302.04761.  arXiv:2302.04761 [cs].
  * Seger et al. [2023] Elizabeth Seger, Aviv Ovadya, Divya Siddarth, Ben Garfinkel, and Allan Dafoe.  Democratising AI: Multiple Meanings, Goals, and Methods.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, pages 715–722, New York, NY, USA, August 2023. Association for Computing Machinery.  ISBN 9798400702310.  doi: 10.1145/3600211.3604693.  URL https://dl.acm.org/doi/10.1145/3600211.3604693.
  * Sharkey et al. [2023] Lee Sharkey, Clíodhna Ní Ghuidhir, Dan Braun, Jérémy Scheurer, Mikita Balesni, Lucius Bushnaq, Charlotte Stix, and Marius Hobbhahn.  A Causal Framework for AI Regulation and Auditing, November 2023.  URL https://static1.squarespace.com/static/6461e2a5c6399341bcfc84a5/t/654bc268049d687cecac24d8/1699463818729/auditing_framework_web.pdf.
  * Shavit et al. [2023] Yonadav Shavit, Sandhini Agarwal, Miles Brundage, Steven Adler, Cullen O’Keefe, Rosie Campbell, Teddy Lee, Pamela Mishkin, Tyna Eloundou, Alan Hickey, Katarina Slama, Lama Ahmad, Paul McMillan, Alex Beutel, Alexandre Passos, and David G. Robinson.  Practices for Governing Agentic AI Systems, 2023.
  * Sheffield and Francois [2021] Rachel Sheffield and Catherine Francois.  Is Instagram Causing Poorer Mental Health Among Teen Girls? - Is Instagram Causing Poorer Mental Health Among Teen Girls? - United States Joint Economic Committee.  Technical report, United States Congress Joint Economic Committee, December 2021.  URL https://www.jec.senate.gov/public/index.cfm/republicans/2021/12/is-instagram-causing-poorer-mental-health-among-teen-girls.
  * Shelby et al. [2023] Renee Shelby, Shalaleh Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadeh, Paul Nicholas, N’Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, and Gurleen Virk.  Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction.  In _Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society_ , AIES ’23, pages 723–741, New York, NY, USA, August 2023. Association for Computing Machinery.  ISBN 9798400702310.  doi: 10.1145/3600211.3604673.  URL https://dl.acm.org/doi/10.1145/3600211.3604673.
  * Shen et al. [2023] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang.  HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face, December 2023.  URL http://arxiv.org/abs/2303.17580.  arXiv:2303.17580 [cs].
  * Shergadwala et al. [2022] Murtuza N. Shergadwala, Himabindu Lakkaraju, and Krishnaram Kenthapadi.  A Human-Centric Perspective on Model Monitoring.  _Proceedings of the AAAI Conference on Human Computation and Crowdsourcing_ , 10:173–183, October 2022.  ISSN 2769-1349.  doi: 10.1609/hcomp.v10i1.21997.  URL https://ojs.aaai.org/index.php/HCOMP/article/view/21997.
  * Shevlane et al. [2023] Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Allan Dafoe.  Model evaluation for extreme risks, May 2023.  URL https://arxiv.org/abs/2305.15324v2.
  * Siegenfeld and Bar-Yam [2020] Alexander F Siegenfeld and Yaneer Bar-Yam.  An introduction to complex systems science and its applications.  _Complexity_ , 2020:1–16, 2020.  Publisher: Hindawi Limited.
  * Soares et al. [2015] Nate Soares, Benja Fallenstein, Stuart Armstrong, and Eliezer Yudkowsky.  Corrigibility.  In _Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence_ , 2015.
  * Soice et al. [2023] Emily H. Soice, Rafael Rocha, Kimberlee Cordova, Michael Specter, and Kevin M. Esvelt.  Can large language models democratize access to dual-use biotechnology?, June 2023.  URL http://arxiv.org/abs/2306.03809.  arXiv:2306.03809 [cs].
  * Solaiman [2023] Irene Solaiman.  The Gradient of Generative AI Release: Methods and Considerations, February 2023.  URL http://arxiv.org/abs/2302.04844.  arXiv:2302.04844 [cs].
  * Sullivan and Verreault-Julien [2022] Emily Sullivan and Philippe Verreault-Julien.  From Explanation to Recommendation: Ethical Standards for Algorithmic Recourse.  In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_. ACM, July 2022.  doi: 10.1145/3514094.3534185.  URL https://doi.org/10.1145%2F3514094.3534185.
  * Sumers et al. [2023] Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths.  Cognitive Architectures for Language Agents, September 2023.  URL http://arxiv.org/abs/2309.02427.  arXiv:2309.02427 [cs].
  * Sutton and Barto [2018] Richard S. Sutton and Andrew G. Barto.  _Reinforcement learning: An introduction_.  Adaptive computation and machine learning series. The MIT Press, Cambridge, Massachusetts, second edition edition, 2018.  ISBN 978-0-262-03924-6.  tex.lccn: Q325.6 .R45 2018.
  * Team et al. [2023a] Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, and Lei Zhang.  Human-Timescale Adaptation in an Open-Ended Task Space, January 2023a.  URL http://arxiv.org/abs/2301.07608.  arXiv:2301.07608 [cs].
  * Team et al. [2023b] Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, Alexandre Frechette, Charlotte Smith, Laura Culp, Lev Proleev, Yi Luan, Xi Chen, James Lottes, Nathan Schucher, Federico Lebron, Alban Rrustemi, Natalie Clay, Phil Crone, Tomas Kocisky, Jeffrey Zhao, Bartek Perz, Dian Yu, Heidi Howard, Adam Bloniarz, Jack W. Rae, Han Lu, Laurent Sifre, Marcello Maggioni, Fred Alcober, Dan Garrette, Megan Barnes, Shantanu Thakoor, Jacob Austin, Gabriel Barth-Maron, William Wong, Rishabh Joshi, Rahma Chaabouni, Deeni Fatiha, Arun Ahuja, Ruibo Liu, Yunxuan Li, Sarah Cogan, Jeremy Chen, Chao Jia, Chenjie Gu, Qiao Zhang, Jordan Grimstad, Ale Jakse Hartman, Martin Chadwick, Gaurav Singh Tomar, Xavier Garcia, Evan Senter, Emanuel Taropa, Thanumalayan Sankaranarayana Pillai, Jacob Devlin, Michael Laskin, Diego de Las Casas, Dasha Valter, Connie Tao, Lorenzo Blanco, Adrià Puigdomènech Badia, David Reitter, Mianna Chen, Jenny Brennan, Clara Rivera, Sergey Brin, Shariq Iqbal, Gabriela Surita, Jane Labanowski, Abhi Rao, Stephanie Winkler, Emilio Parisotto, Yiming Gu, Kate Olszewska, Yujing Zhang, Ravi Addanki, Antoine Miech, Annie Louis, Laurent El Shafey, Denis Teplyashin, Geoff Brown, Elliot Catt, Nithya Attaluri, Jan Balaguer, Jackie Xiang, Pidong Wang, Zoe Ashwood, Anton Briukhov, Albert Webson, Sanjay Ganapathy, Smit Sanghavi, Ajay Kannan, Ming-Wei Chang, Axel Stjerngren, Josip Djolonga, Yuting Sun, Ankur Bapna, Matthew Aitchison, Pedram Pejman, Henryk Michalewski, Tianhe Yu, Cindy Wang, Juliette Love, Junwhan Ahn, Dawn Bloxwich, Kehang Han, Peter Humphreys, Thibault Sellam, James Bradbury, Varun Godbole, Sina Samangooei, Bogdan Damoc, Alex Kaskasoli, Sébastien M. R. Arnold, Vijay Vasudevan, Shubham Agrawal, Jason Riesa, Dmitry Lepikhin, Richard Tanburn, Srivatsan Srinivasan, Hyeontaek Lim, Sarah Hodkinson, Pranav Shyam, Johan Ferret, Steven Hand, Ankush Garg, Tom Le Paine, Jian Li, Yujia Li, Minh Giang, Alexander Neitz, Zaheer Abbas, Sarah York, Machel Reid, Elizabeth Cole, Aakanksha Chowdhery, Dipanjan Das, Dominika Rogozińska, Vitaly Nikolaev, Pablo Sprechmann, Zachary Nado, Lukas Zilka, Flavien Prost, Luheng He, Marianne Monteiro, Gaurav Mishra, Chris Welty, Josh Newlan, Dawei Jia, Miltiadis Allamanis, Clara Huiyi Hu, Raoul de Liedekerke, Justin Gilmer, Carl Saroufim, Shruti Rijhwani, Shaobo Hou, Disha Shrivastava, Anirudh Baddepudi, Alex Goldin, Adnan Ozturel, Albin Cassirer, Yunhan Xu, Daniel Sohn, Devendra Sachan, Reinald Kim Amplayo, Craig Swanson, Dessie Petrova, Shashi Narayan, Arthur Guez, Siddhartha Brahma, Jessica Landon, Miteyan Patel, Ruizhe Zhao, Kevin Villela, Luyu Wang, Wenhao Jia, Matthew Rahtz, Mai Giménez, Legg Yeung, Hanzhao Lin, James Keeling, Petko Georgiev, Diana Mincu, Boxi Wu, Salem Haykal, Rachel Saputro, Kiran Vodrahalli, James Qin, Zeynep Cankara, Abhanshu Sharma, Nick Fernando, Will Hawkins, Behnam Neyshabur, Solomon Kim, Adrian Hutter, Priyanka Agrawal, Alex Castro-Ros, George van den Driessche, Tao Wang, Fan Yang, Shuo-yiin Chang, Paul Komarek, Ross McIlroy, Mario Lučić, Guodong Zhang, Wael Farhan, Michael Sharman, Paul Natsev, Paul Michel, Yong Cheng, Yamini Bansal, Siyuan Qiao, Kris Cao, Siamak Shakeri, Christina Butterfield, Justin Chung, Paul Kishan Rubenstein, Shivani Agrawal, Arthur Mensch, Kedar Soparkar, Karel Lenc, Timothy Chung, Aedan Pope, Loren Maggiore, Jackie Kay, Priya Jhakra, Shibo Wang, Joshua Maynez, Mary Phuong, Taylor Tobin, Andrea Tacchetti, Maja Trebacz, Kevin Robinson, Yash Katariya, Sebastian Riedel, Paige Bailey, Kefan Xiao, Nimesh Ghelani, Lora Aroyo, Ambrose Slone, Neil Houlsby, Xuehan Xiong, Zhen Yang, Elena Gribovskaya, Jonas Adler, Mateo Wirth, Lisa Lee, Music Li, Thais Kagohara, Jay Pavagadhi, Sophie Bridgers, Anna Bortsova, Sanjay Ghemawat, Zafarali Ahmed, Tianqi Liu, Richard Powell, Vijay Bolina, Mariko Iinuma, Polina Zablotskaia, James Besley, Da-Woon Chung, Timothy Dozat, Ramona Comanescu, Xiance Si, Jeremy Greer, Guolong Su, Martin Polacek, Raphaël Lopez Kaufman, Simon Tokumine, Hexiang Hu, Elena Buchatskaya, Yingjie Miao, Mohamed Elhawaty, Aditya Siddhant, Nenad Tomasev, Jinwei Xing, Christina Greer, Helen Miller, Shereen Ashraf, Aurko Roy, Zizhao Zhang, Ada Ma, Angelos Filos, Milos Besta, Rory Blevins, Ted Klimenko, Chih-Kuan Yeh, Soravit Changpinyo, Jiaqi Mu, Oscar Chang, Mantas Pajarskas, Carrie Muir, Vered Cohen, Charline Le Lan, Krishna Haridasan, Amit Marathe, Steven Hansen, Sholto Douglas, Rajkumar Samuel, Mingqiu Wang, Sophia Austin, Chang Lan, Jiepu Jiang, Justin Chiu, Jaime Alonso Lorenzo, Lars Lowe Sjösund, Sébastien Cevey, Zach Gleicher, Thi Avrahami, Anudhyan Boral, Hansa Srinivasan, Vittorio Selo, Rhys May, Konstantinos Aisopos, Léonard Hussenot, Livio Baldini Soares, Kate Baumli, Michael B. Chang, Adrià Recasens, Ben Caine, Alexander Pritzel, Filip Pavetic, Fabio Pardo, Anita Gergely, Justin Frye, Vinay Ramasesh, Dan Horgan, Kartikeya Badola, Nora Kassner, Subhrajit Roy, Ethan Dyer, Víctor Campos, Alex Tomala, Yunhao Tang, Dalia El Badawy, Elspeth White, Basil Mustafa, Oran Lang, Abhishek Jindal, Sharad Vikram, Zhitao Gong, Sergi Caelles, Ross Hemsley, Gregory Thornton, Fangxiaoyu Feng, Wojciech Stokowiec, Ce Zheng, Phoebe Thacker, Çağlar Ünlü, Zhishuai Zhang, Mohammad Saleh, James Svensson, Max Bileschi, Piyush Patil, Ankesh Anand, Roman Ring, Katerina Tsihlas, Arpi Vezer, Marco Selvi, Toby Shevlane, Mikel Rodriguez, Tom Kwiatkowski, Samira Daruki, Keran Rong, Allan Dafoe, Nicholas FitzGerald, Keren Gu-Lemberg, Mina Khan, Lisa Anne Hendricks, Marie Pellat, Vladimir Feinberg, James Cobon-Kerr, Tara Sainath, Maribeth Rauh, Sayed Hadi Hashemi, Richard Ives, Yana Hasson, YaGuang Li, Eric Noland, Yuan Cao, Nathan Byrd, Le Hou, Qingze Wang, Thibault Sottiaux, Michela Paganini, Jean-Baptiste Lespiau, Alexandre Moufarek, Samer Hassan, Kaushik Shivakumar, Joost van Amersfoort, Amol Mandhane, Pratik Joshi, Anirudh Goyal, Matthew Tung, Andrew Brock, Hannah Sheahan, Vedant Misra, Cheng Li, Nemanja Rakićević, Mostafa Dehghani, Fangyu Liu, Sid Mittal, Junhyuk Oh, Seb Noury, Eren Sezener, Fantine Huot, Matthew Lamm, Nicola De Cao, Charlie Chen, Gamaleldin Elsayed, Ed Chi, Mahdis Mahdieh, Ian Tenney, Nan Hua, Ivan Petrychenko, Patrick Kane, Dylan Scandinaro, Rishub Jain, Jonathan Uesato, Romina Datta, Adam Sadovsky, Oskar Bunyan, Dominik Rabiej, Shimu Wu, John Zhang, Gautam Vasudevan, Edouard Leurent, Mahmoud Alnahlawi, Ionut Georgescu, Nan Wei, Ivy Zheng, Betty Chan, Pam G. Rabinovitch, Piotr Stanczyk, Ye Zhang, David Steiner, Subhajit Naskar, Michael Azzam, Matthew Johnson, Adam Paszke, Chung-Cheng Chiu, Jaume Sanchez Elias, Afroz Mohiuddin, Faizan Muhammad, Jin Miao, Andrew Lee, Nino Vieillard, Sahitya Potluri, Jane Park, Elnaz Davoodi, Jiageng Zhang, Jeff Stanway, Drew Garmon, Abhijit Karmarkar, Zhe Dong, Jong Lee, Aviral Kumar, Luowei Zhou, Jonathan Evens, William Isaac, Zhe Chen, Johnson Jia, Anselm Levskaya, Zhenkai Zhu, Chris Gorgolewski, Peter Grabowski, Yu Mao, Alberto Magni, Kaisheng Yao, Javier Snaider, Norman Casagrande, Paul Suganthan, Evan Palmer, Geoffrey Irving, Edward Loper, Manaal Faruqui, Isha Arkatkar, Nanxin Chen, Izhak Shafran, Michael Fink, Alfonso Castaño, Irene Giannoumis, Wooyeol Kim, Mikołaj Rybiński, Ashwin Sreevatsa, Jennifer Prendki, David Soergel, Adrian Goedeckemeyer, Willi Gierke, Mohsen Jafari, Meenu Gaba, Jeremy Wiesner, Diana Gage Wright, Yawen Wei, Harsha Vashisht, Yana Kulizhskaya, Jay Hoover, Maigo Le, Lu Li, Chimezie Iwuanyanwu, Lu Liu, Kevin Ramirez, Andrey Khorlin, Albert Cui, Tian LIN, Marin Georgiev, Marcus Wu, Ricardo Aguilar, Keith Pallo, Abhishek Chakladar, Alena Repina, Xihui Wu, Tom van der Weide, Priya Ponnapalli, Caroline Kaplan, Jiri Simsa, Shuangfeng Li, Olivier Dousse, Fan Yang, Jeff Piper, Nathan Ie, Minnie Lui, Rama Pasumarthi, Nathan Lintz, Anitha Vijayakumar, Lam Nguyen Thiet, Daniel Andor, Pedro Valenzuela, Cosmin Paduraru, Daiyi Peng, Katherine Lee, Shuyuan Zhang, Somer Greene, Duc Dung Nguyen, Paula Kurylowicz, Sarmishta Velury, Sebastian Krause, Cassidy Hardin, Lucas Dixon, Lili Janzer, Kiam Choo, Ziqiang Feng, Biao Zhang, Achintya Singhal, Tejasi Latkar, Mingyang Zhang, Quoc Le, Elena Allica Abellan, Dayou Du, Dan McKinnon, Natasha Antropova, Tolga Bolukbasi, Orgad Keller, David Reid, Daniel Finchelstein, Maria Abi Raad, Remi Crocker, Peter Hawkins, Robert Dadashi, Colin Gaffney, Sid Lall, Ken Franko, Egor Filonov, Anna Bulanova, Rémi Leblond, Vikas Yadav, Shirley Chung, Harry Askham, Luis C. Cobo, Kelvin Xu, Felix Fischer, Jun Xu, Christina Sorokin, Chris Alberti, Chu-Cheng Lin, Colin Evans, Hao Zhou, Alek Dimitriev, Hannah Forbes, Dylan Banarse, Zora Tung, Jeremiah Liu, Mark Omernick, Colton Bishop, Chintu Kumar, Rachel Sterneck, Ryan Foley, Rohan Jain, Swaroop Mishra, Jiawei Xia, Taylor Bos, Geoffrey Cideron, Ehsan Amid, Francesco Piccinno, Xingyu Wang, Praseem Banzal, Petru Gurita, Hila Noga, Premal Shah, Daniel J. Mankowitz, Alex Polozov, Nate Kushman, Victoria Krakovna, Sasha Brown, MohammadHossein Bateni, Dennis Duan, Vlad Firoiu, Meghana Thotakuri, Tom Natan, Anhad Mohananey, Matthieu Geist, Sidharth Mudgal, Sertan Girgin, Hui Li, Jiayu Ye, Ofir Roval, Reiko Tojo, Michael Kwong, James Lee-Thorp, Christopher Yew, Quan Yuan, Sumit Bagri, Danila Sinopalnikov, Sabela Ramos, John Mellor, Abhishek Sharma, Aliaksei Severyn, Jonathan Lai, Kathy Wu, Heng-Tze Cheng, David Miller, Nicolas Sonnerat, Denis Vnukov, Rory Greig, Jennifer Beattie, Emily Caveness, Libin Bai, Julian Eisenschlos, Alex Korchemniy, Tomy Tsai, Mimi Jasarevic, Weize Kong, Phuong Dao, Zeyu Zheng, Frederick Liu, Fan Yang, Rui Zhu, Mark Geller, Tian Huey Teh, Jason Sanmiya, Evgeny Gladchenko, Nejc Trdin, Andrei Sozanschi, Daniel Toyama, Evan Rosen, Sasan Tavakkol, Linting Xue, Chen Elkind, Oliver Woodman, John Carpenter, George Papamakarios, Rupert Kemp, Sushant Kafle, Tanya Grunina, Rishika Sinha, Alice Talbert, Abhimanyu Goyal, Diane Wu, Denese Owusu-Afriyie, Cosmo Du, Chloe Thornton, Jordi Pont-Tuset, Pradyumna Narayana, Jing Li, Sabaer Fatehi, John Wieting, Omar Ajmeri, Benigno Uria, Tao Zhu, Yeongil Ko, Laura Knight, Amélie Héliou, Ning Niu, Shane Gu, Chenxi Pang, Dustin Tran, Yeqing Li, Nir Levine, Ariel Stolovich, Norbert Kalb, Rebeca Santamaria-Fernandez, Sonam Goenka, Wenny Yustalim, Robin Strudel, Ali Elqursh, Balaji Lakshminarayanan, Charlie Deck, Shyam Upadhyay, Hyo Lee, Mike Dusenberry, Zonglin Li, Xuezhi Wang, Kyle Levin, Raphael Hoffmann, Dan Holtmann-Rice, Olivier Bachem, Summer Yue, Sho Arora, Eric Malmi, Daniil Mirylenka, Qijun Tan, Christy Koh, Soheil Hassas Yeganeh, Siim Põder, Steven Zheng, Francesco Pongetti, Mukarram Tariq, Yanhua Sun, Lucian Ionita, Mojtaba Seyedhosseini, Pouya Tafti, Ragha Kotikalapudi, Zhiyu Liu, Anmol Gulati, Jasmine Liu, Xinyu Ye, Bart Chrzaszcz, Lily Wang, Nikhil Sethi, Tianrun Li, Ben Brown, Shreya Singh, Wei Fan, Aaron Parisi, Joe Stanton, Chenkai Kuang, Vinod Koverkathu, Christopher A. Choquette-Choo, Yunjie Li, T. J. Lu, Abe Ittycheriah, Prakash Shroff, Pei Sun, Mani Varadarajan, Sanaz Bahargam, Rob Willoughby, David Gaddy, Ishita Dasgupta, Guillaume Desjardins, Marco Cornero, Brona Robenek, Bhavishya Mittal, Ben Albrecht, Ashish Shenoy, Fedor Moiseev, Henrik Jacobsson, Alireza Ghaffarkhah, Morgane Rivière, Alanna Walton, Clément Crepy, Alicia Parrish, Yuan Liu, Zongwei Zhou, Clement Farabet, Carey Radebaugh, Praveen Srinivasan, Claudia van der Salm, Andreas Fidjeland, Salvatore Scellato, Eri Latorre-Chimoto, Hanna Klimczak-Plucińska, David Bridson, Dario de Cesare, Tom Hudson, Piermaria Mendolicchio, Lexi Walker, Alex Morris, Ivo Penchev, Matthew Mauger, Alexey Guseynov, Alison Reid, Seth Odoom, Lucia Loher, Victor Cotruta, Madhavi Yenugula, Dominik Grewe, Anastasia Petrushkina, Tom Duerig, Antonio Sanchez, Steve Yadlowsky, Amy Shen, Amir Globerson, Adam Kurzrok, Lynette Webb, Sahil Dua, Dong Li, Preethi Lahoti, Surya Bhupatiraju, Dan Hurt, Haroon Qureshi, Ananth Agarwal, Tomer Shani, Matan Eyal, Anuj Khare, Shreyas Rammohan Belle, Lei Wang, Chetan Tekur, Mihir Sanjay Kale, Jinliang Wei, Ruoxin Sang, Brennan Saeta, Tyler Liechty, Yi Sun, Yao Zhao, Stephan Lee, Pandu Nayak, Doug Fritz, Manish Reddy Vuyyuru, John Aslanides, Nidhi Vyas, Martin Wicke, Xiao Ma, Taylan Bilal, Evgenii Eltyshev, Daniel Balle, Nina Martin, Hardie Cate, James Manyika, Keyvan Amiri, Yelin Kim, Xi Xiong, Kai Kang, Florian Luisier, Nilesh Tripuraneni, David Madras, Mandy Guo, Austin Waters, Oliver Wang, Joshua Ainslie, Jason Baldridge, Han Zhang, Garima Pruthi, Jakob Bauer, Feng Yang, Riham Mansour, Jason Gelman, Yang Xu, George Polovets, Ji Liu, Honglong Cai, Warren Chen, XiangHai Sheng, Emily Xue, Sherjil Ozair, Adams Yu, Christof Angermueller, Xiaowei Li, Weiren Wang, Julia Wiesinger, Emmanouil Koukoumidis, Yuan Tian, Anand Iyer, Madhu Gurumurthy, Mark Goldenson, Parashar Shah, M. K. Blake, Hongkun Yu, Anthony Urbanowicz, Jennimaria Palomaki, Chrisantha Fernando, Kevin Brooks, Ken Durden, Harsh Mehta, Nikola Momchev, Elahe Rahimtoroghi, Maria Georgaki, Amit Raul, Sebastian Ruder, Morgan Redshaw, Jinhyuk Lee, Komal Jalan, Dinghua Li, Ginger Perng, Blake Hechtman, Parker Schuh, Milad Nasr, Mia Chen, Kieran Milan, Vladimir Mikulik, Trevor Strohman, Juliana Franco, Tim Green, Demis Hassabis, Koray Kavukcuoglu, Jeffrey Dean, and Oriol Vinyals.  Gemini: A Family of Highly Capable Multimodal Models, December 2023b.  URL http://arxiv.org/abs/2312.11805.  arXiv:2312.11805 [cs].
  * Urbina et al. [2022] Fabio Urbina, Filippa Lentzos, Cédric Invernizzi, and Sean Ekins.  Dual use of artificial-intelligence-powered drug discovery.  _Nature Machine Intelligence_ , 4(3):189–191, March 2022.  ISSN 2522-5839.  doi: 10.1038/s42256-022-00465-9.  URL https://www.nature.com/articles/s42256-022-00465-9.
  * Valmeekam et al. [2023a] Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati.  Can Large Language Models Really Improve by Self-critiquing Their Own Plans?, October 2023a.  URL http://arxiv.org/abs/2310.08118.  arXiv:2310.08118 [cs].
  * Valmeekam et al. [2023b] Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati.  Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change), April 2023b.  URL http://arxiv.org/abs/2206.10498.  arXiv:2206.10498 [cs].
  * Van Loo [2019] Rory Van Loo.  Regulatory Monitors: Policing Firms in the Compliance Era.  _Columbia Law Review_ , 119(2):369, January 2019.  URL https://scholarship.law.bu.edu/faculty_scholarship/265.
  * Venkatasubramanian and Alfano [2020] Suresh Venkatasubramanian and Mark Alfano.  The philosophical basis of algorithmic recourse.  In _Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency_. ACM, January 2020.  doi: 10.1145/3351095.3372876.  URL https://doi.org/10.1145%2F3351095.3372876.
  * Verma [2023a] Pranshu Verma.  The rise of AI fake news is creating a ‘misinformation superspreader’.  _Washington Post_ , December 2023a.  ISSN 0190-8286.  URL https://www.washingtonpost.com/technology/2023/12/17/ai-fake-news-misinformation/.
  * Verma [2023b] Pranshu Verma.  They thought loved ones were calling for help. It was an AI scam.  _Washington Post_ , March 2023b.  ISSN 0190-8286.  URL https://www.washingtonpost.com/technology/2023/03/05/ai-voice-scam/.
  * Wang et al. [2023] Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen.  A Survey on Large Language Model based Autonomous Agents, September 2023.  arXiv: 2308.11432 [cs] Issue: arXiv:2308.11432.
  * Wang et al. [2021] Zihan Wang, Olivia Byrnes, Hu Wang, Ruoxi Sun, Congbo Ma, Huaming Chen, Qi Wu, and Minhui Xue.  Data Hiding with Deep Learning: A Survey Unifying Digital Watermarking and Steganography, July 2021.  URL https://arxiv.org/abs/2107.09287v3.
  * Warren [2024] Tom Warren.  Microsoft’s new Copilot Pro brings AI-powered Office features to the rest of us.  _The Verge_ , January 2024.  URL https://www.theverge.com/2024/1/15/24038711/microsoft-copilot-pro-office-ai-apps.
  * Wei et al. [2023] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou.  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models, January 2023.  URL http://arxiv.org/abs/2201.11903.  arXiv:2201.11903 [cs].
  * Weidinger et al. [2023] Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Mateos-Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, Iason Gabriel, Verena Rieser, and William Isaac.  Sociotechnical Safety Evaluation of Generative AI Systems, October 2023.  URL http://arxiv.org/abs/2310.11986.  arXiv:2310.11986 [cs].
  * Widder et al. [2023] David Gray Widder, Meredith Whittaker, and Sarah Myers West.  Open (for Business): Big Tech, Concentrated Power, and the Political Economy of Open AI, August 2023.  URL https://ssrn.com/abstract=4543807.
  * Willison [2023] Simon Willison.  Prompt injection: What’s the worst that can happen?, April 2023.  URL https://simonwillison.net/2023/Apr/14/worst-that-can-happen/.
  * Wu et al. [2023] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W. White, Doug Burger, and Chi Wang.  AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation Framework.  2023\.  _eprint: 2308.08155.
  * Xu et al. [2023] Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu.  Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game, October 2023.  URL https://arxiv.org/abs/2310.18940v2.
  * Zhang et al. [2023] Hanlin Zhang, Benjamin L. Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak.  Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models, November 2023.  URL http://arxiv.org/abs/2311.04378.  arXiv:2311.04378 [cs].
  * Zilka et al. [2022] Miri Zilka, Holli Sargeant, and Adrian Weller.  Transparency, Governance and Regulation of Algorithmic Tools Deployed in the Criminal Justice System: a UK Case Study.  In _Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society_. ACM, July 2022.  doi: 10.1145/3514094.3534200.  URL https://doi.org/10.1145%2F3514094.3534200.
  * Zou et al. [2023] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson.  Universal and Transferable Adversarial Attacks on Aligned Language Models, July 2023.  URL https://arxiv.org/abs/2307.15043v2.
  * Zwetsloot and Dafoe [2019] Remco Zwetsloot and Allan Dafoe.  Thinking about risks from AI: Accidents, misuse and structure.  _Lawfare. February_ , 11:2019, 2019.

##  Appendix A Potential Information to Include on an Agent Card

###  A.1. The Underlying System

  * •

Evaluations of the system’s degree of agency [Mialon et al., 2023, Ruan et
al., 2023, Liu et al., 2023];

  * •

Evaluations of the system’s generality: its ability to accomplish a broad
array of tasks to some specified performance threshold [Morris et al., 2023];

  * •

Red flags, such as previous incidents or results of previous dangerous
capabilities and alignment evaluations [Shevlane et al., 2023];

  * •

Dependencies of the agent, such as with an ecosystem graph [Bommasani et al.,
2023] (e.g., whether the agent is a fine-tuned variant of another model).

###  A.2. The Specific Instance of the Agent

  * •

How the agent instance was created (e.g., by its user directly or by another
agent?);

  * •

The agent’s goal, including both what the user specifies and what the system
appears to be achieving [Gilbert et al., 2023];

  * •

Any tools or services that the agent can access (e.g., spinning up another
agent through an API call, physical manipulation of robotics);

  * •

The agent’s permissions (e.g., whether it has sudo access in a terminal);

  * •

Details about the persistence of the agent, such as whether the agent has a
set lifetime;

  * •

The sector of the intended deployment environment (e.g., finance);

  * •

The number of people the system can directly impact and the severity of such
impact;

  * •

The degree and ease of human oversight over the agent.

###  A.3. The Actors Involved in the Creation and Operation of the Agent

  * •

The user;

  * •

The compute provider;

  * •

Developers of the underlying system, including developers of any scaffolding
or component foundation models;

  * •

Any humans with whom the agent has interacted;

  * •

Tools and services providers that the agent can use.