# Decision-Oriented Dialogue for Human–AI Collaboration

Jessy Lin${}^{{*}1\;}$ Nicholas Tomlin${}^{{*}1\;}$ Jacob Andreas${}^{2\;3\;}$
Jason Eisner${}^{2\;4\;}$
${}^{1\;}$ UC Berkeley ${}^{2\;}$ Microsoft Semantic Machines ${}^{3\;}$ MIT
${}^{4\;}$ Johns Hopkins
{jessy_lin<EMAIL_ADDRESS>
{jaandrea<EMAIL_ADDRESS>
###### Abstract

We describe a class of tasks called decision-oriented dialogues, in which AI
assistants must collaborate with one or more humans via natural language to
help them make complex decisions. We formalize three domains in which users
face everyday decisions: (1) choosing an assignment of reviewers to conference
papers, (2) planning a multi-step itinerary in a city, and (3) negotiating
travel plans for a group of friends. In each of these settings, AI assistants
and users have disparate abilities that they must combine to arrive at the
best decision: assistants can access and process large amounts of information,
while users have preferences and constraints external to the system. For each
task, we build a dialogue environment where agents receive a reward based on
the quality of the final decision they reach. Using these environments, we
collect human-human dialogues with humans playing the role of assistant. To
compare how current AI assistants communicate in these settings, we present
baselines using large language models in self-play. Finally, we highlight a
number of challenges models face in decision-oriented dialogues, ranging from
efficient communication to reasoning and optimization, and release our
environments as a testbed for future modeling work.111Code and data are
available at https://github.com/jlin816/dialop.

##  1 Introduction

Figure 1: Overview of the three collaborative dialogue tasks that we consider.
In Optimization, two agents with symmetric access to information play the role
of area co-chairs assigning reviewers to conference papers. In Planning, an
assistant must collaborate with a user in order to help them plan an
itinerary. In Mediation, an assistant must chat with multiple separate users
in order to help them resolve a group scheduling problem.

Imagine that you are trying to book conference travel with the help of a
digital assistant. Your choice of airline is flexible, but you’d rather avoid
layovers, want to arrive a day or two before the conference begins, and would
like to be able to check in to your hotel as soon as you arrive. Additionally,
you’re in charge of booking travel for a few of your colleagues, each of whom
has their own preferences and budgets, some of whom will be flying in from
different cities, but all of whom would like to arrive at roughly the same
time and stay in a nearby area. Suddenly, you must manage and communicate
about a combinatorial explosion of possible solutions.

Similar optimization problems occur in many everyday situations. Consider
consulting a friend about what computer they’d recommend with the best
tradeoff of features for your use cases. Or trying to allocate funding from
multiple grants to determine which students should work on which projects,
while juggling what the individual priorities of each student might be. Or
making strategic decisions with your colleagues about which projects your
company will take on, in the context of market conditions, and who to hire to
manage those projects. All these situations share an underlying decision
problem in the face of uncertainty, where communicating and collaborating with
others is often critical to arrive at the best solution.

Difficult decision problems like these are precisely where AI assistants could
shine. Automated systems can handle large amounts of information and complex
computations much better than humans. For example, in cases like travel
booking, they can quickly search over a large number of possible itineraries
and compute total costs in a way that the average user cannot. They may also
be able to efficiently reason under uncertainty about the expected value of
decision-relevant information, helping them determine what information may be
important to share with or request from the user. On the other hand, these
decisions cannot be _fully_ automated either. AI assistants _complement_ the
user’s information and capabilities: people know their preferences and may
have other knowledge external to the system, including knowledge about fuzzy
real-world constraints that are difficult to formalize in a computer-readable
format. To solve these problems, systems need to communicate with users,
ideally with a flexible interface such as natural language.

In this paper, we develop a challenging suite of decision problems, benchmark
the abilities of current language models on these tasks, and release
environments to encourage future work in this area.

We begin by formalizing a class of tasks, decision-oriented dialogues, in
which multiple agents must communicate in order to arrive at a joint decision.
They are jointly rewarded according to the quality of the decision. Each agent
starts out with different information: for example, the user knows their own
travel preferences, while the AI assistant has a database of flight and hotel
prices. Sharing their information allows them to better assess different
travel plans. Critically, however, the large amount of information and (in
some tasks) the combinatorial solution space make it unnatural and inefficient
for assistants to communicate _all_ of their knowledge to users, or vice
versa. Instead, agents must determine what their partners already know and
what information is likely to be decision-relevant, asking clarification
questions and making inferences as needed.

Within this class of tasks, we present DialOp, a suite of environments with
three everyday domains where humans and agents must collaborate in order to
make complicated decisions. (1) In Optimization, two agents take on the role
of conference area chairs, assigning reviewers to conference papers when each
agent has only has partial information about reviewer–paper similarity. (2) In
Planning, an assistant with knowledge of a city must assist a human with
building an itinerary based on their preferences. (3) In Mediation, multiple
users must collaborate with an assistant in order to resolve group scheduling
challenges. For each task, we specify an objective measure of utility based on
the quality of the final decision. We first collect human-human dialogues on
these tasks in order to establish a reference point for how humans naturally
collaborate with each other. We then develop extensible environments for
evaluating language models on each task, with support for tool use and chain-
of-thought prompting.

We use these environments to benchmark the relative performance of GPT-3
(Brown et al., 2020), both in self-play and in a novel evaluation procedure
known as prompted self-play, in which AI agents complete partial human
dialogues. We then identify several common failure modes of GPT-3 and provide
analyses of self-play dialogues. We release all dialogues, environments, and
interfaces for human data collection in order to encourage future work that
addresses these challenges.

##  2 Task Formulation

We formalize a _decision-oriented dialogue_ (DoD) as a multi-agent problem
consisting of a set of agents, an underlying world state $W$, each agent’s
partial and possibly noisy observation $O_{i}$, a set of legal messages
$m\in\mathcal{M}$ (analogous to actions in an MDP), a reward function over
decisions $R$ with parameters $\theta$, and a communication cost function $C$.
The goal of a decision-oriented dialogue is to find a decision that maximizes
$R$ while minimizing the communication cost function $C$. $W$ remains fixed
throughout the dialogue. Our problem can be thought of as a decentralized
partially observation Markov decision process (Dec-POMDP; Bernstein et al.
(2000)) where the actions are “cheap talk” and formal decision messages.

An agent $i$’s policy $\pi_{i}$ maps its known information $O_{i}$ and the
dialogue history $\\{m_{1},\ldots m_{t-1}\\}$ to a new message $m_{t}$:
$\pi_{i}(m_{t}\mid O_{i},\\{m_{1},\ldots m_{t-1}\\})$. Agents take turns
sending messages by sampling from their policy. Messages may specify a
recipient if the number of agents $>2$, and are expressed in natural language
except for three special formal messages: a proposed decision, a formal
acceptance of a decision, and a formal rejection. If an agent sends a proposed
decision message and all other agents respond with a formal acceptance, the
dialogue ends. When formal proposal decisions are sent, agents may
additionally receive noisy observations of the reward of that decision
(functions of the reward $f(R_{\theta}(\cdot))$). They can use these
observations to make inferences about $W$ and $R$, and to decide how to
respond. Otherwise, the only observations they receive throughout the dialogue
are the messages from the other agents.222In general, the formalism does
accommodate settings where an agent can pay to acquire new observations during
the dialogue. Simply create other agents that have access to those
observations (e.g., sensors), and assign a high cost to communicating with
those agents.

To illustrate the information in a DoD, consider the task of planning a travel
itinerary that satisfies a user’s preferences (Planning, as shown in Figure 1,
middle). We represent the underlying world state as a weighted graph
$W=(V,E,w)$ whose vertices are potential destinations. A decision is a path
$W^{\prime}$ in $W$, representing the itinerary. Higher-weighted paths are
better and the agents must communicate to improve their knowledge of the edge
weights.

In general, we represent the world state $W$ as a weighted graph and the
possible decisions as subgraphs $W^{\prime}$ that satisfy task-specific
constraints. Edges and vertices in $W$ have weights $w(e_{ij}),w(v_{i})$ that
represent rewards (which may be negative) for including them in $W^{\prime}$.
The optimal decision for this world state is a subgraph $W^{\prime}\subseteq
W$ that maximizes the reward

$\displaystyle R_{\theta}(W^{\prime})$ $\displaystyle=\sum_{v\in
W^{\prime}}w(v)+\sum_{e\in W^{\prime}}w(e)$ (1)

In principle, the reward function could be be any function of $W^{\prime}$,
but we focus on the linear objective 1. For most practical tasks, the
constrained optimization problem could then be expressed as an integer linear
programming problem and solved using standard algorithms. We assume edge and
vertex weights are determined by their features, represented by feature
vectors $\phi(\cdot)\in\mathbb{R}^{k}$, so that:

$\displaystyle w(v_{i})$ $\displaystyle=\theta^{T}\phi(v_{i})$ (2)
$\displaystyle w(e_{ij})$ $\displaystyle=\theta^{T}\phi(e_{ij})$

where $\theta$ is a preference vector.

The form of $R$ is common knowledge, but the world state $W$—in particular the
feature vectors and the preferences $\theta$—is only partially observed by
each player. Therefore, crucially, players must exchange messages in order to
reduce their respective uncertainties about the optimization problem. However,
there is a cost to communicating (e.g., time or effort), which agents must
trade off with their desire to achieve a good decision. Thus, the overall
objective function for a DoD is:

$\displaystyle\max_{W^{\prime}\subseteq W,\mathbf{m}}$ $\displaystyle
R_{\theta}(W^{\prime})-\sum_{t}C(m_{t})$ (3) subject to $\displaystyle\textit{
task-specific constraints on }W^{\prime}$

In the following sections, we introduce three everyday domains with
collaborative decision-making and show how they can be formalized as DoD tasks
in our benchmark.

###  2.1 Optimization

Our first task is an idealized bipartite matching problem, motivated by the
scenario of conference organizers assigning reviewers to submitted papers
(Figure 1, left). Although reviewer matching is sometimes completely automated
via approaches like the Toronto Paper Matching System (TPMS; Charlin and
Zemel, 2013), organizers often have incomplete and partially-overlapping
knowledge about which reviewers fit which papers. Further, fit cannot
necessarily be described on an absolute scale, so when working together on an
assignment, organizers must discuss relative edge weights (“Alice would be a
better choice than Bob for paper 8”). TPMS could in principle be replaced by
an AI agent that joins this dialogue as an additional participant.

We consider a simplified version of this problem in which two agents must
select a one-to-one correspondence between reviewers and papers. We represent
$W$ as a bipartite graph and restrict valid proposals $W^{\prime}\subseteq W$
to be bipartite matchings. Edge weights represent reviewer-paper affinities,
and each agent observes some subset of these weights.

A fuller version of this setting would derive the edge weights from features
of the papers and the reviewers (footnote 4 below). This would make
communication more interesting, but the underlying optimization problem would
remain one of maximum weighted bipartite matching.

###  2.2 Planning

Next, we consider the scenario in which a user is planning an itinerary in a
city with the assistance of a travel agent (Figure 1, middle). While existing
systems can assist with parts of travel such as recommendation or booking,
they often expect users to provide close-to-full specifications of their
requests, rather than working toward a solution together with an assistant
(although cf. § 8 for a discussion of mixed-initiative dialogue). Ideally,
systems would be able to assist us in the comprehensive way a human travel
agent would: starting with an under-specified set of “things we’d like to do,”
comprehensively exploring multi-day itineraries based on the user’s
preferences and domain knowledge, and iteratively refining the plan with the
user based on feedback.

We formalize a small version of this problem as a DoD task where the assistant
must plan an itinerary of several sites for a user. The user has preferences
about which sites to visit, a budget, and a preference for reducing travel
time. Meanwhile, the assistant has access to a database of sites, along with
information about their cost, location, and amenities (e.g., outdoor seating).
We construct $W$ as a fully-connected graph over the locations, where edge
weights represent travel times (and the preference over edge weights is
negative). Unlike reviewer matching, this task exhibits asymmetry of
information: the assistant has information about vertex features and edge
weights, while the user only has information about their own preference vector
$\theta$. Due to the budget constraint, the prescribed itinerary length, and
the preference to minimize travel, this domain involves aspects of the
knapsack problem, subset-selection problems, and the traveling salesman
problem.

Figure 2: Data collection and evaluation frameworks. In order to collect
human-human dialogues, we built web interfaces which allow humans to play
either the user or assistant role for each task. When evaluating language
models in self-play, we linearize information from the interface into a text
prompt and provide additional tools which allow language models to access
information which cannot fit within their context windows.

###  2.3 Mediation

Finally, we introduce a coordination scenario where the assistant serves as
the role of mediator between multiple users (Figure 1, right). The users are
attempting to book flights from their respective cities to all arrive at some
shared destination at around the same time, e.g., to meet up for an event or
vacation. It is often difficult to negotiate individual constraints and
consider all the configurations efficiently. AI assistants may be more suited
to guide the group toward a good joint solution, by helping users find options
that will work well with the choices of other users as well as their own
needs.

We assume that the $n$ users only coordinate through the single
assistant.333Users in such a setting could learn about one another through
talking to the assistant. Thus, such systems in practice should also manage
privacy issues, which we ignore here. In the task, each user wants to choose a
flight that is inexpensive and avoids conflicts with the user’s calendar
commitments, but that arrives close to the arrival times of other players. The
assistant has access to each user’s flight options and work calendar, but
doesn’t observe the user’s personal calendar, nor the user’s preferences about
which meetings are important. In the underlying optimization problem, the
world state $W$ can be modeled as an complete $n$-partite graph, where the
vertices associated with each user are their flight options. Any two flights
for different users are connected by an edge, whose weight indicates how
compatible the flights are (i.e., whether they arrive at similar times).
Vertex weights are derived from the users’ calendars, with important meetings
creating a preference against flights (vertices) that conflict with them. The
goal is to select a flight for each user so that the induced subgraph
$W^{\prime}$ (with $n$ vertices and $n\choose 2$ edges) has high total weight.

##  3 The DialOp Environments

To instantiate each of these tasks, we release DialOp, an open-source suite of
decision-oriented dialogue environments. DialOp environments can be used to
evaluate models in self-play as in § 6.1, as an underlying API to build human
user interfaces for data collection as in § 4, or to evaluate models in
collaboration with humans.

While other collaborative or task-oriented dialogue tasks are typically
evaluated on coarse metrics such as success rate (did the system accomplish
the user’s goal?) (Li et al., 2016), the reward in a decision-oriented
dialogue provides a _graded_ measure of communication success: how close to
optimal is the final decision? This in turn provides signal on whether models
are capable of asking the right questions, sharing the right information, and
coordinating efficiently with the user so they can agree on the best course of
action—in addition to simply understanding the user’s utterances.

In contrast to other dialogue tasks where evaluation is based on supervised
datasets, our environments are also _procedurally generated_ : the parameters
of the underlying decision problem can be randomized to instantiate new
dialogue contexts.

Agents interact with the environment with an OpenAI Gym-like interface
(Brockman et al., 2016). Agents send messages to the environment and receive
messages from other players and any additional observations back. Before each
message, agents must output a message type ([message], [propose], [accept], or
[reject]), which the environment parses to determine how to interpret the
message. Messages are forwarded to other agents. Proposals are parsed and
scored; on the next turn the only valid actions for the other agents are
[accept] and [reject]. Formal rejections clear the current proposal, and
formal acceptances terminate the dialogue. Below, we describe how the
environments implement each of the decision domains we introduce.

#### Optimization

In this task, agents must find the best assignment of $k$ reviewers to $k$
papers. For each game, we sample a random table of reviewer-paper affinity
scores (edge weights). Each cell is shown to each player with probability
$p_{\text{observed}}$, so that a given cell may be shown to just one player,
to both, or to neither. The initial observations $o_{0}$ for each player are
their observed table values.444There are many ways we could have made the game
more realistic. Rather than reveal each score either perfectly or not at all,
we could reveal some amount of noisy evidence about the score. Agents can also
have asymmetric knowledge. Alternatively, each score could be a function of
underlying features—for example, the dot product of the paper’s topic vector
and the reviewer’s topical-expertise vector. We could then selectively reveal
evidence about these features—“Alice is an expert on Botany”—rather than about
edge weights. If features are correlated, agents may be able to infer features
that they do not observe directly. This setup would allow the agents to
usefully discuss their knowledge and beliefs about the reviewers and papers
(vertex features that affect many edge weights), as well as their resulting
assessment of relative affinity scores (edge weights). In our data collection
and experiments we use $k=8$, $p_{\text{observed}}=0.4$. To discourage
reviewers from communicating affinity scores in the form of numbers—which
would not be natural in the real-world version of this scenario—we scale all
scores shown to each player by a random positive constant, so that they are
not comparable across agents but can still be discussed in relative terms such
as “X is much better than Y.”

Agents take turns sending messages. Either agent is allowed to propose a
matching at any point. If the other agent accepts on the next turn, the game
ends; otherwise, the proposal is taken off the table and agents continue. The
final reward is the sum of edge weights in this matching, normalized by the
value of the best matching with the agents’ pooled knowledge, computed as an
expectation with a uniform prior over values so that rewards are in $[0,1]$.

Figure 3: An annotated example of a human-human dialogue and a model-model
self-play dialogue with GPT-3 in Planning. While humans generally exhibit
diverse and flexible strategies and reach good solutions, self-play dialogues
tend to be repetitive, and the assistant makes mediocre proposals and often
hallucinates. We discuss more analysis in § 7.

#### Planning

In this task, an assistant and a user must book an itinerary of $k$ sites that
best satisfies the user’s preferences. For each game, we procedurally generate
sites (e.g., restaurants, parks, museums) with randomized features such as
cuisine type or expected price range. We also procedurally generate a set of
$s$ preferences for the user and random preference weights $\theta$
representing how much the user cares about each preference. To simulate the
fact that people cannot quantify their actual preferences on an absolute
scale, the user only observes natural language descriptions of their
preferences, without the numerical preference weights. Only the assistant
observes the inventory of sites and their features, while only the user
observes their preferences. In our data collection and experiments we use
$k=3,s=10$.

The assistant and the user take turns sending natural language messages. The
assistant can propose a complete or partial itinerary at any point. This
proposal’s reward (while unknown to the assistant) is automatically computed
for the user’s convenience, including a breakdown that shows the contributions
to the reward from each site, travel times, and budget constraints. With this
information, the user can make judgments about aspects of the itinerary (e.g.,
that it is worth spending extra travel time to visit a particularly desirable
site) and determine whether to accept the proposal. The game ends when the
user accepts a full itinerary of $k$ sites. The final reward is the score of
the itinerary, range-normalized by the scores of the best and worst possible
$k$-site itineraries.

#### Mediation

In this task, two users and one assistant must book the best flight for each
user that satisfies their individual preferences, while being close to each
other. On each game, the environment generates a random set of personal
calendar events, work calendar events, and importance weights for each event
indicating how important it is. The environment also generates a list of
flights for each user, each with randomized features for price, arrival time,
and departure time. The user observes their own personal and work calendar and
flight set, while the assistant observes the work calendars and flight sets of
_both_ users (but not their personal calendars). Additionally, the assistant
does not observe the importance of each meeting, so it must communicate with
the user to determine which events can be missed for the flight. When the
assistant proposes a flight to a user, the user observes the score breakdown
in terms of missed meetings, price, and closeness to the other user’s flight
(when known). The game ends when all users accept the assistant’s proposals.
The final reward is the sum of their scores, range-normalized by the scores of
the best and worst pairs of flights.

##  4 The DialOp Dataset

In order to study the communication strategies used by humans and establish
baseline performance numbers for each task, we collected a set of human-human
dialogues. For each task, we built a multi-player online interface and
collected high-quality human-human dialogues using a mixture of Amazon
Mechanical Turk and in-house Microsoft data annotators, resulting in a total
of 409 dialogues, consisting of 5253 messages and over 58K words across
domains. Human players take a median time of 8min 19sec across tasks. Humans
achieve an average of roughly 90% of the maximum possible score on both the
optimization and planning domains, and close to 100% performance in the
mediation domain. We report additional dataset statistics in Table 2 in the
appendix.

In each task, each annotator played the role of an assistant or user. For ease
of play, annotators were not required to take turns, but used a chat interface
where they could send a message at any time. Consecutive messages from the
same annotator were concatenated into a “turn.”

Although real-world users know their own preferences, our annotators are
emulating users that we have generated programmatically, so we must tell them
what their preferences are. This setup gives us full knowledge of user
preferences so that we can objectively evaluate the quality of the decision.
We simulate the fact that internal preferences may be comparative or fuzzy by
scaling numerical values (in Optimization) or not showing numerical values
until a proposal is presented. This design encourages realistic behavior in
the dialogues: it is easier to make comparisons between travel itineraries and
point to specific aspects you like and dislike, rather than fully specify an
itinerary you would like.

As depicted in Figure 2 for Planning, humans had access to the same
information as models receive in the task, but presented in a graphical user
interface (UI) rather than purely in text:

#### Optimization

Both annotators see a spreadsheet with their scaled known table values. They
can click on cells in the spreadsheet to make a proposal.

#### Planning

The human assistant sees a map of all the locations, allowing them to visually
estimate distances. They can fill in events into a proposed itinerary, which
auto-calculates the exact distances. They can click on a site to see its
features or filter sites on the map with checkboxes and sliders. The user
initially only sees a plain-text list of their travel preferences (e.g., “like
seafood, Japanese”) without the preference weight values. When the assistant
sends a proposed (partial or full) itinerary, the user sees the features of
the proposed sites and a scorecard breaking down the total score by event,
travel distance, and budget.

#### Mediation

Users see a three-day calendar with events and a list of flights with times
and prices. Events are labeled with a numerical value for their importance.
The human assistants see the calendars and flight lists for both users. When
the assistant makes a proposal to one or both users, they see the proposed
flight overlaid on their calendar and a scorecard breaking down the total
score with the penalty for missing calendar events, arriving at a different
time from the other user, and flight price. For more details on the data
collection set up and interface screenshots, refer to the appendix. We also
release the code to run the UIs for the tasks.

##  5 Baseline Models

Figure 4: Self-play scores and dialogue lengths in words, compared to human-
human dialogues. Models achieve lower scores on average, and also tend to have
longer dialogues. Marginal distributions for the # words and score are shown
as histograms, and the average score of a randomly selected proposal is shown
for each task as a dashed gray line. Mean and SEM numbers can be found in
Table 1.

We believe that AI agents for decision-oriented dialogue will benefit from
incorporating explicit reasoning over possible world states and possible
decisions. However, as a baseline approach, this paper evaluates few-shot
prompted LLMs as the AI agents. These have the benefit that they can attempt a
wide variety of dialogue interactions without the need for domain-specific
training or modeling. In particular, we focus our evaluations on the
instruction-tuned GPT-3 model known as text-davinci-003 (Brown et al., 2020;
Ouyang et al., 2022). For Optimization, we prompt with two human-human
dialogue examples from the dataset; for the others we prompt with one, due to
context length limitations. If models fail to generate a valid message (e.g.,
user simulator model attempting to send proposals), we append the generated
message to the prompt, along with any error message from the game, and
continue generating, allowing the model to revise its previous generation.
Below, we describe how models are prompted with the information for each task.
Refer to Appendix E for the full prompts.

#### Optimization

Both players see a partial table of weights matching reviewers and papers for
this task. We prompt the model with the linearized table, formatted as a CSV.

#### Planning

For the user simulator model, we prompt with the natural language list of
travel preferences as the context. The agent has access to a database of sites
with features. We take a modular tool use approach, where the agent model
accesses information in the database by writing search queries rather than
conditioning directly on the database itself. The search queries are executed
by a _query executor_ model that conditions on the database and generates the
result for the new query. We hand-write several example queries in a simple
domain-specific language where the agent can return specific fields (e.g.
name, category, price) of a site, filter over fields, sort_by field values
(including distance_to another destination), and search by text_query in
freeform natural language. While the DSL examples guide the set of searches
the agent can perform, the query executor can generalize to new searches
beyond the demonstrations. We augment the 1-shot example in agent’s prompt
with examples of queries in the DSL along with their results throughout the
dialogue and provide the query executor with query and result examples.
Delegating searches over the database to the query executor reduces context
length restrictions and allows the agent model to filter for relevant
information from the database with an abstracted query layer. Future
approaches may consider using larger context length models and directly
conditioning on the database.

This task requires particularly complex reasoning to search based on the
dialogue (on the agent side) and decide whether to accept an itinerary based
on the scores (on the user side). We also augment the dialogues in the user
and agent prompt with [think] steps such as “I am losing the most points from
the travel time between events. I should reject the proposal...” based on
ReAct (Yao et al., 2022) to provide the model with reasoning examples.

#### Mediation

Each user can see their set of flights, private calendar, and shared work
calendar while the agent can see flights and shared calendars (without event
importance values) for both players. We prompt models with the list of all
flights and calendar events. The environment allows the agent to talk to
either player; generally, deciding which user to talk to is itself a strategic
decision. We adopt a simple turn-taking strategy where we iterate round-robin
through all players; on the agent’s turn, they are prompted with You to and
choose which user to send the message to by generating either 0 or 1 (e.g.
“You to 0”).

##  6 Evaluation

In this section, we compare the performance of humans and AI agents on our
tasks. While we are ultimately interested in how well AI agents can perform in
collaboration with human partners, we introduce two automatic evaluation
setups which serve as proxies for human evaluation. Our experiments aim to
understand: (1) how well do current models perform in decision-oriented
dialogues (as evaluated in self-play; § 6.1) and (2) how well can models
comprehend human dialogues, as a proxy for eventual collaboration with real
people (as evaluated in prompted self-play; § 6.2)?

###  6.1 Self-Play

First, we evaluate how well models can collaborate with each other in self-
play. We prompt each model with the private knowledge for a player. On each
step of the environment, we generate from the model whose turn it is
(assistant or user simulator(s)) and append the outputted message to both
models’ context. We repeatedly generate from the model until a proposal is
made and accepted.

In Figure 4, we show human-human and model-model scores against the number of
words in the dialogue. For a fair comparison, we prompt models with the same
randomly generated instances as the human-human dialogues in the evaluation
dataset, although future agents can also generally be evaluated on new random
instances generated from the environment. In gray, we show the performance of
a naive rule-based baseline that selects a random proposal from the set of all
possible proposals. Compared to humans, models tend to have longer dialogues
than humans _and_ achieve less optimal solutions. Models significantly
outperform the baseline on both the itinerary planning and mediation tasks but
do slightly worse than random chance on the reviewer matching task, signaling
that they struggle with its underlying optimization problem. These results
suggest that models have yet to close the gap to human performance in
communicating efficiently to collaborate on good solutions.

Figure 5: Prompted self-play results for all three tasks, compared to human
results. For each setting, we initialize dialogues with 50% and 75% of a
corresponding human game and let GPT-3 complete the dialogue. In the proposal
setting, we prompt the model with an entire human dialogue except for the
final proposal and force the model to end the game immediately. The average
score of a randomly selected proposal is shown for each task as a dashed gray
line. (*) For reference, we show the mean score of models in self-play,
although we note that they are not prompted to end the dialogue at some length
like the other PSP conditions.

###  6.2 Prompted Self-Play

Even agents that perform well in self-play may not perform well in
collaboration with humans (Carroll et al., 2019). This disparity exists
because humans often use different and more diverse strategies than artificial
agents, particularly if agent strategies arise from explicit optimization of
an objective.

To bridge this gap, we propose a new mode of automatic evaluation known as
prompted self-play (PSP), in which dialogues are initialized with the prefix
of a human-human dialogue and then continued by the model. Given a human-human
dialogue from our dataset, we test how models perform if they are provided
with 50% of the dialogue, 75% of the dialogue, and everything except the final
proposal, and then complete the rest of the dialogue via self-play. PSP tests
additional capabilities beyond self-play: in PSP, the dialogue history
contains information that the human-human pair has talked about already,
making it easier to find good solutions _if_ models are able to understand and
reason over the information to make a proposal. Additionally, models should do
some degree of belief modeling about what the human being simulated knows to
communicate efficiently; for example, models ought to avoid asking about
information already implied by previous utterances. Finally, prompting in this
way encourages models to complete dialogues “in the style” of the human-human
pair in the prefix. As a result, PSP both tests whether models can flexibly
continue dialogues demonstrating different strategies (e.g. with one agent
taking most of the initiative), and whether assistants can collaborate with a
diverse range of humans, similar to population play and fictitious self-play
evaluation (Jaderberg et al., 2019; Strouse et al., 2021).

We bias models to output dialogues that are approximately the same length as
the corresponding human-human dialogue (cf. Appendix E). Figure 5 shows
average PSP performance for each task. In Planning, models perform better with
additional human data in the prompt, suggesting that they are at least
partially capable of integrating information from the human-human prefix.
However, there is a substantial gap between the proposal condition and human-
human dialogue scores, indicating that models struggle to perform the final
optimization step of choosing the best solution given the entire dialogue
history. Meanwhile, in Optimization, models fail across all PSP conditions;
this occurs because the final step of the reviewer matching game involves
integrating the discussed values to compute a bipartite matching, which is
difficult for models. Finally, in Mediation, models score well above a random
baseline in all PSP conditions but do not perform better with additional
human-human dialogue context, suggesting that they can meaningfully
communicate about the task but don’t make the optimal final proposal. In the
future, tool use could potentially greatly improve performance on this task,
particularly with tools that can specifically handle the optimization part of
the problem.

##  7 Analysis

Figure 6: Kernel density estimates of message types in human-human dialogues
plotted against their position within a dialogue. Message types were
automatically annotated using few-shot prompting with GPT-3.

In order to quantify the strategies used in human-human dialogues, we used
GPT-3 to annotate dialogues at the level of individual messages. Based on
manual inspection of a small set of games, we devised a list of message types:
(1) share, in which agents provide information about their preferences; (2)
query, in which agents ask each other for information; (3) affirm, in which
agents agree with each other and/or ground incoming messages; (4) explain, in
which agents provide justification for a previous message or action; (5) meta,
in which agents engage in discussion about high-level strategies or meta-game
details; (6) revise, in which agents correct earlier statements; or (7)
miscellany, which includes other messages such as greetings. Each message may
have multiple message types. We prompted GPT-3 to generate message annotations
for each of the 5253 messages using two hand-annotated example dialogues. We
provide additional details and data statistics in the appendix.

Most dialogues are focused on exchanging information: of the message types, we
find that human agents most commonly share or query for information. In the
Optimization game, agents send twice as many share messages as any other type
of message, often sending information about individual cells in their observed
tables. One strategy used by humans involves both players sharing all observed
information and then making a decision at the end of the game. This strategy
is most tractable in Optimization game, where players have a relatively small
observation space. However, this strategy leads to exceptionally long
dialogues, even in Optimization, and is not the most common approach.
Meanwhile, in the Planning and Mediation games, which have asymmetric
information and roles, agents are more likely to query for information or
engage in meta-game discussion in order to learn what information the other
agent can see. Agents must still share information, but assistants for both of
these tasks have access to an exceptionally large amount of information which
cannot be fully shared with the users.

We also provide a breakdown of message types over the time-course of dialogues
in Figure 6. As expected, many interactions begin with greetings, which is
evidenced by a spike in the miscellany category at the beginning of all three
plots. In the Planning and Mediation tasks, agents are more likely to query at
the beginnings of games and then respond with share messages shortly
afterward. Finally, affirm messages, although rare, are most likely to appear
at the end of dialogues, once common ground has been established.

Qualitatively, we show a human-human dialogue side-by-side with a self-play
dialogue in Figure 3. We generally observe across the human dialogues that
human-human pairs exhibit diverse strategies in (1) user-agent initiative: in
some dialogues, users are proactive in sharing relevant information, while in
others agents make directed queries to narrow down the set of proposals; and
(2) coordination strategies: working incrementally from partial proposals,
backtracking, and more.

In self-play dialogues, current LLMs are capable of carrying on natural
dialogues that partly address the user’s preferences and find good solutions.
However, they generally tend to be formulaic and repetitive, and
hallucinations are a problem, as with other tasks involving language models.
Critically, models ask general questions such as “Do you have any other
preferences?” and sometimes slightly more specific ones such as “Do you have a
price point?”, but the questions are not _goal-directed_ in eliciting
decision-critical information. In contrast, human assistants ask questions
that help them decide between proposals or narrow down the search space.
Finally, models fail to do the optimization step of the proposal (as supported
by our PSP results): proposals are often only slightly better than random, and
do not improve drastically over the course of the dialogue. This suggests that
our task targets many of the critical capabilities missing from current
models, such as reasoning, asking clarification questions, grounding to
external sources, and hallucination.

##  8 Related Work

#### Task-Oriented Dialogue

Our work may be viewed as an extension of task-oriented dialogue, where a
system must assist a user with accomplishing a goal, such as hotel booking or
calendar scheduling (Budzianowski et al., 2018; Wei et al., 2018; Semantic
Machines et al., 2020). Most task-oriented dialogue settings involve helping a
user who is seeking out a specific piece of information (“what is a vegetarian
Italian restaurant nearby?”) or wants to take an action (“change my flight to
tuesday”). Systems are typically evaluated with coarse metrics such as success
rate (e.g. at returning the right hotel information requested by a user) or
word overlap with human-human dialogues. In contrast, our tasks are grounded
in underlying optimization problems, where the quality of the final solution
provides a richer measure of communicative success. All agents must engage in
information-seeking and understand intents in the course of a dialogue
decision problem, but furthermore have to _take initiative_ to share and query
information to collaborate on a good solution. In this sense, our work is more
similar to early work on task-oriented dialogue in mixed-initiative settings
(Novick and Sutton, 1997; Horvitz, 1999) such as TRAINS (Allen et al., 1995)
and TRIPS (Allen and Ferguson, 2002), in which users had to collaborate with a
computer agent in order to solve planning problems such as train routing. Our
task includes many similar design elements but is aimed at building general
dialogue systems without the significant domain-specific engineering that went
into projects like TRAINS and TRIPS.

#### Grounded Dialogue

Another class of dialogue tasks are grounded dialogue settings such as Cards
(Potts, 2012; Vogel et al., 2013), CerealBar (Suhr et al., 2019),
MutualFriends (He et al., 2017), and OneCommon (Udagawa and Aizawa, 2019),
where agents communicate in a game-like setting to achieve a goal. These tasks
are often situated in a multimodal environment with visual elements or
external knowledge. Our task also has many of these elements, but we focus on
domains with everyday optimization problems where successful communication
could be useful to people. Our work also shares elements in common with
negotiation dialogue tasks such as Deal or No Deal (Lewis et al., 2017) and
Craigslist Bargaining (He et al., 2018), but we focus on cooperative scenarios
in which all agents share the same objective.

#### Large Language Models

Our goal of building task-general dialogue agents motivates the use of large
language models (LLMs) such as GPT-3 (Brown et al., 2020; Ouyang et al.,
2022), PaLM (Chowdhery et al., 2022), or LLaMA (Touvron et al., 2023). Recent
work has focused on using language models as dialogue agents, including
OpenAI’s ChatGPT, Microsoft’s Sydney, Anthropic’s Claude, and Google’s LAMDA
(Thoppilan et al., 2022) and Bard. Current-era language models are known to
struggle with aspects of our tasks, such as mathematical reasoning (Hendrycks
et al., 2021), explicit state tracking (Li et al., 2021), pragmatics (Fried et
al., 2022), and theory of mind (Sap et al., 2022). However, recent work in
scratchpad prompting (Nye et al., 2021), chain-of-thought reasoning (Wei et
al., 2022), and external tool use (Schick et al., 2023) has sought to address
these problems. We build baseline models with similar approaches in our
setting. While LLMs can perform reasonably well in some of our settings, we
show that they cannot consistently handle dialogues with complex decision
problems as well as humans.

#### Human–AI Collaboration

Our task may also be viewed as a cooperative multi-agent setting (Dafoe et
al., 2020). Research in human-AI collaboration and multi-agent reinforcement
learning has also formalized tasks that require collaborating strategically
with other agents on a shared goal, through tasks such as Overcooked (Carroll
et al., 2019), Hanabi (Bard et al., 2020), and Diplomacy (Bakhtin et al.,
2022). Our evaluation methodology is adapted from these tasks, where methods
like population play and fictitious self-play are often used as proxies for
human evaluation in addition to self-play (Heinrich et al., 2015; Strouse et
al., 2021). In human–AI collaboration, cooperative tasks have been formulated
in game-theoretic terms where agents use signals from the user such as
demonstrations, feedback, or language (Jeon et al., 2020; Lin et al., 2022) to
explicitly optimize for assistive behavior (Hadfield-Menell et al., 2016;
Sadigh et al., 2016). In our work, we are similarly interested in formalizing
settings where agents should explicitly optimize for human assistance in the
course of dialogue.

##  9 Conclusion

In this paper, we presented data, environments, and model baselines for a
class of tasks we call decision-oriented dialogues. Across all task settings,
current-era language models did not perform as well as humans, suggesting
failures in their ability to communicate efficiently and reason in structured
real-world optimization problems. Future modeling work in this domain may seek
to integrate tools and inference techniques which would allow language models
to compute optimal decisions for these types of problems while maintaining
their flexible communication and collaboration skills.

## References

  * Allen and Ferguson (2002) James F. Allen and George Ferguson. 2002.  Human-machine collaborative planning.
  * Allen et al. (1995) James F Allen, Lenhart K Schubert, George Ferguson, Peter Heeman, Chung Hee Hwang, Tsuneaki Kato, Marc Light, Nathaniel Martin, Bradford Miller, Massimo Poesio, et al. 1995.  The trains project: A case study in building a conversational planning agent.  _Journal of Experimental & Theoretical Artificial Intelligence_, 7(1):7–48.
  * Bakhtin et al. (2022) Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, Athul Paul Jacob, Mojtaba Komeili, Karthik Konath, Minae Kwon, Adam Lerer, Mike Lewis, Alexander H. Miller, Sasha Mitts, Adithya Renduchintala, Stephen Roller, Dirk Rowe, Weiyan Shi, Joe Spisak, Alexander Wei, David Wu, Hugh Zhang, and Markus Zijlstra. 2022\.  Human-level play in the game of <i>diplomacy</i> by combining language models with strategic reasoning.  _Science_ , 378(6624):1067–1074.
  * Bard et al. (2020) Nolan Bard, Jakob N. Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H. Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, Iain Dunning, Shibl Mourad, Hugo Larochelle, Marc G. Bellemare, and Michael Bowling. 2020.  The hanabi challenge: A new frontier for ai research.  _Artificial Intelligence_ , 280:103216.
  * Bernstein et al. (2000) Daniel S. Bernstein, Shlomo Zilberstein, and Neil Immerman. 2000.  The complexity of decentralized control of markov decision processes.  In _Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence_ , UAI’00, page 32–37, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
  * Brockman et al. (2016) Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016.  Openai gym.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Budzianowski et al. (2018) Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gašić. 2018.  MultiWOZ - a large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 5016–5026, Brussels, Belgium. Association for Computational Linguistics.
  * Carroll et al. (2019) Micah Carroll, Rohin Shah, Mark K Ho, Tom Griffiths, Sanjit Seshia, Pieter Abbeel, and Anca Dragan. 2019.  On the utility of learning about humans for human-ai coordination.  In _Advances in Neural Information Processing Systems_ , volume 32. Curran Associates, Inc.
  * Charlin and Zemel (2013) Laurent Charlin and Richard S. Zemel. 2013.  The toronto paper matching system: An automated paper-reviewer assignment system.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Dafoe et al. (2020) Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R. McKee, Joel Z. Leibo, K. Larson, and Thore Graepel. 2020.  Open problems in cooperative ai.  _ArXiv_ , abs/2012.08630.
  * Fried et al. (2022) Daniel Fried, Nicholas Tomlin, Jennifer Hu, Roma Patel, and Aida Nematzadeh. 2022\.  Pragmatics in grounded language learning: Phenomena, tasks, and modeling approaches.  _arXiv preprint arXiv:2211.08371_.
  * Hadfield-Menell et al. (2016) Dylan Hadfield-Menell, Stuart J Russell, Pieter Abbeel, and Anca Dragan. 2016.  Cooperative inverse reinforcement learning.  In _Advances in Neural Information Processing Systems_ , volume 29. Curran Associates, Inc.
  * He et al. (2017) He He, Anusha Balakrishnan, Mihail Eric, and Percy Liang. 2017.  Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1766–1776, Vancouver, Canada. Association for Computational Linguistics.
  * He et al. (2018) He He, Derek Chen, Anusha Balakrishnan, and Percy Liang. 2018.  Decoupling strategy and generation in negotiation dialogues.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2333–2343, Brussels, Belgium. Association for Computational Linguistics.
  * Heinrich et al. (2015) Johannes Heinrich, Marc Lanctot, and David Silver. 2015.  Fictitious self-play in extensive-form games.  In _Proceedings of the 32nd International Conference on Machine Learning_ , volume 37 of _Proceedings of Machine Learning Research_ , pages 805–813, Lille, France. PMLR.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021.  Measuring mathematical problem solving with the math dataset.  _arXiv preprint arXiv:2103.03874_.
  * Horvitz (1999) Eric Horvitz. 1999.  Principles of mixed-initiative user interfaces.  In _Proceedings of the SIGCHI conference on Human Factors in Computing Systems_ , pages 159–166.
  * Jaderberg et al. (2019) Max Jaderberg, Wojciech M. Czarnecki, Iain Dunning, Luke Marris, Guy Lever, Antonio Garcia Castañeda, Charles Beattie, Neil C. Rabinowitz, Ari S. Morcos, Avraham Ruderman, Nicolas Sonnerat, Tim Green, Louise Deason, Joel Z. Leibo, David Silver, Demis Hassabis, Koray Kavukcuoglu, and Thore Graepel. 2019\.  Human-level performance in 3d multiplayer games with population-based reinforcement learning.  _Science_ , 364(6443):859–865.
  * Jeon et al. (2020) Hong Jun Jeon, Smitha Milli, and Anca Dragan. 2020.  Reward-rational (implicit) choice: A unifying formalism for reward learning.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 4415–4426. Curran Associates, Inc.
  * Lewis et al. (2017) Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. 2017.  Deal or no deal? end-to-end learning of negotiation dialogues.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 2443–2453, Copenhagen, Denmark. Association for Computational Linguistics.
  * Li et al. (2021) Belinda Z. Li, Maxwell Nye, and Jacob Andreas. 2021.  Implicit representations of meaning in neural language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1813–1827, Online. Association for Computational Linguistics.
  * Li et al. (2016) Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016.  Deep reinforcement learning for dialogue generation.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 1192–1202, Austin, Texas. Association for Computational Linguistics.
  * Lin et al. (2022) Jessy Lin, Daniel Fried, Dan Klein, and Anca Dragan. 2022.  Inferring rewards from language in context.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8546–8560, Dublin, Ireland. Association for Computational Linguistics.
  * Novick and Sutton (1997) David G Novick and Stephen Sutton. 1997.  What is mixed-initiative interaction.  In _Proceedings of the AAAI spring symposium on computational models for mixed initiative interaction_ , volume 2, page 12.
  * Nye et al. (2021) Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021.  Show your work: Scratchpads for intermediate computation with language models.  _arXiv preprint arXiv:2112.00114_.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022\.  Training language models to follow instructions with human feedback.  _arXiv preprint arXiv:2203.02155_.
  * Potts (2012) Christopher Potts. 2012.  Goal-driven answers in the cards dialogue corpus.  In _Proceedings of the 30th West Coast Conference on Formal Linguistics_ , pages 1–20. Cascadilla Proceedings Project.
  * Sadigh et al. (2016) Dorsa Sadigh, Shankar Sastry, Sanjit A. Seshia, and Anca D. Dragan. 2016.  Planning for autonomous cars that leverage effects on human actions.  In _Proceedings of Robotics: Science and Systems_ , AnnArbor, Michigan.
  * Sap et al. (2022) Maarten Sap, Ronan LeBras, Daniel Fried, and Yejin Choi. 2022.  Neural theory-of-mind? on the limits of social intelligence in large lms.  _arXiv preprint arXiv:2210.13312_.
  * Schick et al. (2023) Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023.  Toolformer: Language models can teach themselves to use tools.  _arXiv preprint arXiv:2302.04761_.
  * Semantic Machines et al. (2020) Semantic Machines, Jacob Andreas, John Bufe, David Burkett, Charles Chen, Josh Clausman, Jean Crawford, Kate Crim, Jordan DeLoach, Leah Dorner, Jason Eisner, Hao Fang, Alan Guo, David Hall, Kristin Hayes, Kellie Hill, Diana Ho, Wendy Iwaszuk, Smriti Jha, Dan Klein, Jayant Krishnamurthy, Theo Lanman, Percy Liang, Christopher H. Lin, Ilya Lintsbakh, Andy McGovern, Aleksandr Nisnevich, Adam Pauls, Dmitrij Petters, Brent Read, Dan Roth, Subhro Roy, Jesse Rusak, Beth Short, Div Slomin, Ben Snyder, Stephon Striplin, Yu Su, Zachary Tellman, Sam Thomson, Andrei Vorobev, Izabela Witoszko, Jason Wolfe, Abby Wray, Yuchen Zhang, and Alexander Zotov. 2020.  Task-Oriented Dialogue as Dataflow Synthesis.  _Transactions of the Association for Computational Linguistics_ , 8:556–571.
  * Strouse et al. (2021) DJ Strouse, Kevin McKee, Matt Botvinick, Edward Hughes, and Richard Everett. 2021\.  Collaborating with humans without human data.  In _Advances in Neural Information Processing Systems_ , volume 34, pages 14502–14515. Curran Associates, Inc.
  * Suhr et al. (2019) Alane Suhr, Claudia Yan, Jack Schluger, Stanley Yu, Hadi Khader, Marwa Mouallem, Iris Zhang, and Yoav Artzi. 2019.  Executing instructions in situated collaborative interactions.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2119–2130, Hong Kong, China. Association for Computational Linguistics.
  * Thoppilan et al. (2022) Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022.  Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * Udagawa and Aizawa (2019) Takuma Udagawa and Akiko Aizawa. 2019.  A natural language corpus of common grounding under continuous and partially-observable context.  _Proceedings of the AAAI Conference on Artificial Intelligence_ , 33(01):7120–7127.
  * Vogel et al. (2013) Adam Vogel, Max Bodoia, Christopher Potts, and Daniel Jurafsky. 2013.  Emergence of Gricean maxims from multi-agent decision theory.  In _Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1072–1081, Atlanta, Georgia. Association for Computational Linguistics.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022.  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_.
  * Wei et al. (2018) Wei Wei, Quoc Le, Andrew Dai, and Jia Li. 2018.  AirDialogue: An environment for goal-oriented dialogue research.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 3844–3854, Brussels, Belgium. Association for Computational Linguistics.
  * Yao et al. (2022) Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022.  React: Synergizing reasoning and acting in language models.  _arXiv preprint arXiv:2210.03629_.

##  Appendix A Environment Details

#### Optimization

Each cell of the $k\times k$ table of reviewer-paper affinity scores is
sampled from $\text{Uniform}[0,100]$. To ensure that communication is
necessary to do well, we reject a random game unless the optimal score with
the agents’ pooled knowledge is $\geq 1.25$ times as good as the score that
either player would achieve with their own information if they impute the
average value for unknown cells. We scale values by a random scale sampled
from $\text{Uniform}[1,10]$. The table size $k=8$ in our experiments and data
collection.

#### Planning

To generate contexts for the dialogue, we create a seed list of 39 site names
and locations. Each site is one of the following categories: restaurants,
bars, cafes, sights (museums and landmarks), outdoor (parks), or shopping.

On every environment instance, we randomly shuffle the locations of the sites
and randomize their features. Each site has five nonzero random features, out
of the following list (some of which only apply to some categories:

  * •

Rating (categorical)

  * •

Has parking (bool)

  * •

Has takeout (bool) [restaurants only]

  * •

Touristy (bool)

  * •

Cuisine (categorical) [restaurants only]

  * •

Good for kids (bool) [restaurant, cafe, museum, landmark, park, shop only]

  * •

Accepts reservations (bool) [restaurants only]

  * •

Open late (bool)

  * •

Good for groups (bool)

  * •

Ambience (categorical) [restaurant, cafe, bar]

  * •

Outdoor seating (bool) [restaurant, cafe, bar]

  * •

Vegetarian options (bool) [restaurant, cafe]

  * •

Vegan options (bool) [restaurant, cafe]

  * •

Live music (bool) [restaurant, bar]

  * •

Has Wi-Fi (bool) [cafe]

  * •

Alcohol type (categorical) [bar]

  * •

Viewpoint (bool) [park]

We procedurally generate preferences from the user from the following types:

  * •

Feature: a preference over the value of one of the features above

  * •

Want to go: a preference to go to a specific event or set of events

  * •

Price: a preference to keep the budget less than some fixed amount

  * •

At least one site of type: a preference to go to at least one site of some
type (e.g., to visit at least one museum)

  * •

Distance: a (negative) preference per unit traveled between sites

Each of these preferences is parameterized and randomized on every environment
instance. Every user has a price and distance preference; the other
preferences are sampled with some probability up to a total of $P$ preferences
($P=10$ in our experiments and data collection). We specifically exclude
preference configurations that are un-intuitive from generation (e.g., a
preference for places that do not have takeout). We template natural language
descriptions for each preference to present to the user.

###  A.1 Mediation

On each environment instance, we generate a random calendar for each user by
iterating through a 3-day period and generating a 30 min, 60 min, 2 hour, and
4 hour event with probability $p_{\text{event}}$ $f_{(}\text{shared}$ of these
events are selected to be shared events that both the assistant and user can
see; the remainder are private events that only the user can see. The
importance of each event is sampled from $\text{Uniform}[1,10]$.

We generate a set of $F$ flights for each user with a random start time in the
3-day period, sampling a duration (in hours) from $\text{Uniform}[1,10]$.
Flight prices for each user $i$ are sampled from
$\mathcal{N}(\mu_{i},\sigma_{i})$ to ensure that flight prices a user sees are
realistically around the same value, and the parameters of the distribution
$\mu=\sigma$ are sampled from $\text{Uniform}[50,1000]$.

We generate a price preference weight
$\theta_{\text{price}}\sim\text{Uniform}[1,20]$ and (negative) preference per
hour difference in arrival between the two users’ flights
$\theta_{\text{arrival}}\sim\text{Uniform}[1,10]$.

We set $p_{\text{event}}=0.35,f_{\text{shared}}=0.75,F=30$ in our experiments
and data collection.

(a) User view.

(b) Assistant view.

Figure 7: UIs for human players in the Planning task. The user observes a
travel planning document with their preferences listed in natural language,
but initially no sites. The assistant observes a map of locations and detailed
views of their features. The assistant can send a proposed itinerary in their
interface, at which point the user observes the set of sites in their
itinerary with their features, as well as a score breakdown.

(a) User view.

(b) Assistant view.

Figure 8: UIs for human players in the Mediation task. Each user observes
their own work and personal calendars with meeting importances, and their
available flight options. The assistant observes both calendars and available
flights but not the user’s event importances. As shown in the user view, When
the assistant proposes a flight, it appears on the user’s calendar and the
user observes the score breakdown for that proposal.

##  Appendix B Data Collection Details

Human players from Mechanical Turk were vetted via a pre-qualification survey
and data collection was run in multiple dyads, with high-quality players from
each dyad being invited to participate in followup rounds of data collection.
Workers are informed of the best or average score in the game before starting,
and are bonused up to $2.00 in tiers by how close they get to the best
proposal.

###  B.1 Interface Screenshots

The interfaces for the human users and assistants during data collection can
be seen in Figure 9 (Matching), Figure 7 (Planning), and Figure 8 (Mediation).
Human users and assistants see the same underlying information as models in
the environment, presented in an interactive graphical user interface.

Figure 9: UIs for both players in the Optimization task. Both players observe a subset of reviewer-paper affinities in a spreadsheet table. They can send proposals by clicking on cells in the table, at which point the proposal appears in the other player’s table. | Optimization | Planning | Mediation
---|---|---|---
|  $\\#$ words ($\mu$) | Score ($\mu$) |  $\\#$ words ($\mu$) | Score ($\mu$) |  $\\#$ words ($\mu$) | Score ($\mu$)
Human | $176.8\pm 11.4$ | $0.92\pm 0.01$ | $143.3\pm 6.6$ | $0.89\pm 0.01$ | $119.0\pm 5.7$ | $0.97\pm 0.00$
Self-Play | $145.5\pm 3.6$ | $0.59\pm 0.02$ | $211.2\pm 5.9$ | $0.65\pm 0.01$ | $233.3\pm 5.12$ | $0.81\pm 0.01$
Table 1: Comparison of mean scores and dialogue lengths between the human and self-play games, with standard error of the mean. Humans obtain higher scores than language models on all three domains, using significantly fewer messages on both the Planning and Mediation tasks. 8/114 dialogues in Planning did not terminate within 30 messages (15 per player) and 22/162 dialogues in Mediation did not terminate within 45 messages. These were excluded from the results. These results correspond to the data in Figure 4. | Dialogues | Messages ($\mu$) | Words ($\mu$) | Proposals ($\mu$) | Time ($\mu$)
---|---|---|---|---|---
Optimization | 134 | 18.4 $\pm$ 1.1 | 169.3 $\pm$ 10.9 | 1.7 $\pm$ 0.1 | 8m 9s
Planning | 114 | 9.0 $\pm$ 0.4 | 141.9 $\pm$ 6.5 | 3.0 $\pm$ 0.1 | 10m 56s
Mediation | 162 | 12.8 $\pm$ 0.5 | 119.0 $\pm$ 5.7 | 2.8 $\pm$ 0.2 | 7m 15s
All Domains | 409 | 12.8 $\pm$ 0.5 | 141.8 $\pm$ 4.7 | 2.5 $\pm$ 0.1 | 8m 19s
Table 2: Data statistics for human-human dialogues. We collect a total of 409
dialogues, resulting in 5253 messages and 58K words across domains. Dialogues
for each setting are roughly the same number of words on average.

##  Appendix C Experiments

In Table 1 we show the mean and standard error of the mean for the dialogue
lengths (in words) and rewards of human-human and self-play games.

##  Appendix D Analysis

We include additional analyses of the human-human dialogues. In Table 2, we
show additional data statistics for human-human dialogues. In Figure 10, we
show the distributions of % of agent words in the dialogues for each task, as
a measure of how varied the initiative between users and assistants are in
human-human pairs. In Figure 11, we show the time in minutes spent on each
dialogue and the corresponding reward of the dialogue. In Figure 12, we show
the percentages of messages of each types in our GPT-3 annotation analysis, as
described in § 7.

##  Appendix E Prompts

During prompted self-play, we bias models to output dialogues that are
approximately the same length as the corresponding human-human dialogue by
prompting it with “You must make your best final proposal now.” when the
length of the ongoing dialogue is within 25 words of the human dialogue, and
automatically accepting the outputted proposal.

On the subsequent pages, we show the prompts used to prompt user and assistant
in our experiments. The few-shot examples are actual human-human dialogues in
the dataset, with additional observations on proposals (i.e., the score
breakdown) presented in text format. In Planning, we additionally manually
augment the prompt example with [think] for both the user and assistant, and
[search] steps with example searches and their results. In self-play, models
generated from these prompts directly. In prompted self-play, the prefix of a
human-human dialogue was appended to these prompts and models continued the
dialogue to completion from there.

We also show an example prompt for the query executor in Planning. The
database is different on every game instance; during evaluation, we prompt the
query executor with the actual database of events for that instance along with
the example searches. When the assistant model generates [search] message and
query, we add this to the query executor prompt and generate a result, which
then is returned to the assistant model.

Figure 10: Our tasks involve mixed initiative between users and assistants. In
this figure, we show histograms representing the percent of words which were
sent by the assistant role in each game. Because roles in Optimization are
symmetric, we plot the percent of words sent by the second agent, which is
roughly 50% on average. In Planning, users typically send more words than
assistants do, in order to share constraints and comment on proposals. In
Mediation, which involves two users and one assistant, assistants send roughly
one third of messages on average. Figure 11: Percent of total score as a
function of time to complete the game in human-human dialogues. There is a
weak correlation between completion time and score, with the longest dialogues
typically being near the maximum possible score. However, many human-human
players are capable of reaching the maximum score in all three tasks within a
short amount of time (e.g., $<5$ minutes). Across tasks, the median game
length is 8min 55sec. Figure 12: Percent of message types in human-human
dialogues. Exchanging information is a key aspect of decision-oriented
dialogue: share and query messages are the most common message types for all
three tasks.

Matching Prompt

⬇

You and your partner are area chairs for a conference and you have to assign
reviewers to papers. Each of you has some information about which reviewers
would be good for which papers, but you’ll have to communicate in order to
make the best assignments.

For each game, you will see a table of reviewer-paper similarity scores. The
higher the score, the better the fit. You will also see a list of reviewers
and papers. You can send messages to your partner, propose assignments, and
accept or reject your partner’s proposals.

Your goal is to propose a one-to-one matching between reviewers and papers
with the highest sum of scores. The "Final Score" at the end of the chat is
out of 1.0 and shows how well you did relative to the best you could do with
your partner. Your goal is to maximize that score.

EXAMPLE 1.

Reviewer Paper Similarity Scores:

,BLEU: a Method for Automatic Evaluation of MT,Electra: Pre-training Text
Encoders as Discriminators,GloVe: Global Vectors for Word Representation,GLUE:
A Multi-Task Benchmark and Analysis Platform for NLU,LLaMA: Open and Efficient
Foundation Language Models,RoBERTa: A Robustly Optimized BERT Pretraining
Approach,QuAC: Question Answering in Context,SWAG: An Adversarial Dataset for
Commonsense Inference

Ava Li,,,167,,,,,431

Daniel Nguyen,236,,459,,,,213,83

Sofia Patel,,,171,106,255,,,

Andrei Petrov,60,,13,,,,306,320

Morgan Reed,,,,,,,162,

Joseph Santos,190,,,18,,,,

Ethan Smith,,204,13,325,,,,

Noah Wilson,,,185,,,,380,283

You: [message] Hello!

Partner: [message] Hi! Who should we put for SWAG?

You: [message] I think Ava LI would be good for Swag?

Partner: [message] Sounds good to me!~

You: [message] I also think Daniel Nguyen woould be good for GloVe if you
haven’t allocated him yet.

Partner: [message] What do you have for LLaMA? I don’t have anything for
LLaMA.

You: [message] I only have limited information on LLaMA. Sofia Patel might ve
a good candidate for it. / Ethan might good for GLUE maybe? / BLEU I don’t
much information on, but Joseph has a decent score in that field.

Partner: [message] I have a really high score for GLUE with Andrei so I am
going to keep that one.

You: [message] Who you think would be good for Electra?

Partner: [message] Sofia Patel is the best option for that category on my end.

You: [message] I don’t have much information on Morgan Reed or Ethan Smith,
how about you?

Partner: [propose] Proposal:<br/>  - BLEU: Morgan Reed<br/>  -
Electra: Sofia Patel<br/>  - GLUE: Andrei Petrov<br/>  - GloVe:
Daniel Nguyen<br/>  - LLaMA: Ethan Smith<br/>  - QuAC: Noah
Wilson<br/>  - RoBERTa: Joseph Santos<br/>  - SWAG: Ava Li

You: [accept]

Final Score: 1

EXAMPLE 2.

Reviewer Paper Similarity Scores:

,BLEU: a Method for Automatic Evaluation of MT,Electra: Pre-training Text
Encoders as Discriminators,GloVe: Global Vectors for Word Representation,GLUE:
A Multi-Task Benchmark and Analysis Platform for NLU,LLaMA: Open and Efficient
Foundation Language Models,RoBERTa: A Robustly Optimized BERT Pretraining
Approach,QuAC: Question Answering in Context,SWAG: An Adversarial Dataset for
Commonsense Inference

Ava Li,386,,,120,144,,,202

Daniel Nguyen,,,,434,,,,

Sofia Patel,439,,,38,,,,270

Andrei Petrov,,,169,294,231,,101,

Morgan Reed,14,144,,,478,,,

Joseph Santos,251,115,,,,275,154,

Ethan Smith,86,,19,,19,,,57

Noah Wilson,,,,159,313,,,188

Partner: [message] I am not 10/10 confident on anyone, but lots solid options
on my end. Id say my most confident lock in would be Sofia on LLaMA based on
what im seeing. Ethan on Glue looks great as well / Ava is above average on
BLEU and Glove

You: [message] Okay, give me a second / so Sofia on LLama / Morgan for me is
the BEST on Llama / they have the best pairing of all my info

Partner: [message] hmm, looks like we have a lot of good options for Llama in
that case. Maybe we come back to that. I have a ton of info for quac, the only
person I dont know about quac is Joseph. But otherwise, Ethan or Ava is the
best at it / Though nobody is great at it to be honest.

You: [message] I have only who is bad at Quac so we can go with who you’d like
/ Joseph and Andei are 100%

Partner: [message] That works for me, that lets me put Ethan onto quac in that
case / Do you have andrei as being solid at anything? All I see is that he is
kinda good at Glue, but he might be a candidate for taking a gamble

You: [message] Andrei is mid at Glue / but not worth taking over Daniel

Partner: [message] Ok, sounds good

You: [message] Sofia is killing it for Bleu / Better than Ava who is also
really good

Partner: [message] Hmmm, Sofia is my strongest candidate for Llama, its my
most confident pick out of all my info. / but, I do see other options for Ava
so maybe its worth it / would you say 9 or 10 out of 10 on Sofia on BLEU?

You: [message] Is Sofia better than Morgan on Llama / if so we can move Sofia
/ but Morgan is my best for his Llama knoweldge

Partner: [message] Hmmm, Joseph is solid at Llama as well. And I dont have any
other info for Joseph, so I kinda like him there. But I also dont have any
other good spots for Morgan, so maybe we roll with that for now. / Do you have
any info on who would be a good candidate for roberta at all? / or swag. I got
nothing on those, might need to take some risk

You: [message] Joseph is alright barely

Partner: [message] And what about electra? / I only know bad options for those
three.

You: [message] I got Morhan and Jo bad at Electra / do you have enough to fill
in the rest

Partner: [message] I got ava and Noah bad at electra / might put andrei there?

You: [message] Andrei for now agreed / is Jo just Roberta for now

Partner: [message] gonna throw out a proposal but definitely reject if you see
something / I have him on swag right now but maybe roberta better? I have no
info either way

Partner: [propose] Proposal:<br/>  - BLEU: Sofia Patel<br/>  -
Electra: Andrei Petrov<br/>  - GLUE: Daniel Nguyen<br/>  - GloVe:
Ava Li<br/>  - LLaMA: Morgan Reed<br/>  - QuAC: Ethan
Smith<br/>  - RoBERTa: Noah Wilson<br/>  - SWAG: Joseph Santos

You: [reject]

You: [message] Thats actually really good / yeah that works / if you want to
re propose / I dont see anything clearly off

Partner: [message] Its scary on my end because almost every person I have
locked in has no info on my end haha / its almost all you but Im still fairly
confident since none of my scores were insane

Partner: [propose] Proposal:<br/>  - BLEU: Sofia Patel<br/>  -
Electra: Andrei Petrov<br/>  - GLUE: Daniel Nguyen<br/>  - GloVe:
Ava Li<br/>  - LLaMA: Morgan Reed<br/>  - QuAC: Ethan
Smith<br/>  - RoBERTa: Noah Wilson<br/>  - SWAG: Joseph Santos

You: [accept]

Final Score: 1

EXAMPLE 3.

Planning: Assistant Prompt

⬇

You are a travel agent. Book a set of three destinations that make the user
most happy. Your objective is to maximize the "Final Score" at the end of the
chat, which scores how well the final itinerary you proposed matches the
user’s preferences.

You can use the ‘Search‘ tool with the following API:

field: can be name, category, price, info, or any other field of an site

category: can be [restaurant, cafe, museum, bar, landmark, park, shop]

Search:

Parameters

- fields: list of field names to return

- filters: list of filters to intersect with AND. Can only filter one of the fields above.

- text_query: freeform text query to search in event descriptions. Will be intersected with filters with AND.

- sort_by: list of fields or callable function to sort results by.

- limit: number of results to return

USER 1.

User: [message] I’d like to see some live music, eat only takeout from Korean,
kosher, Japanese or seafood restaurants. Vegan options are a plus and I’d like
to stop by Mad Seoul. I’d also like places with parking, good city views and
kid friendly. My budget is $30.

You: [think] The user listed many preferences. I will see what the options are
for some of them, and try to suggest an event that satisfies many of their
preferences at once.

You: [tool] Search(fields=[name], text_query=good for kids)

Search Results (6):

name

Saul’s

The Dockside Grill

Riverside Trail

Moabit Garden

Common Grounds

The Dive

You: [tool] Search(fields=[name], text_query=viewpoint)

Search Results (2):

name

Garden of Wonders

Moabit Garden

You: [tool] Search(fields=[name], text_query=has parking)

Search Results (3):

name

Liberty Memorial

Common Grounds

Lincoln Park

You: [think] Common Grounds is good for kids and has parking, so it seems like
an option the user might like. Let me send a partial proposal so I can see
what the user thinks.

You: [propose] [Common Grounds, NULL, NULL]

You: [message] I have made a proposal. Please let me know what you think of
it.

User: [message] It’s not bad so far, I’d really like to stop by Mad Seoul
though.

You: [message] Sure. I can help you out with that! Are there any other events
you’d like to go to after Mad Seoul?

User: [message] I’d like to go to see some live music.

You: [think] I will try to find somewhere with live music near Mad Seoul and
check whether the event also satisfies the other preferences the user listed.

You: [tool] Search(fields=[name, good for kids, has parking, viewpoint],
text_query=live music, sort_by=[distance_to(Mad Seoul)])

Search Results (1):

name|category|good for kids|has parking|viewpoint|distance_to(Mad Seoul)

Cookies Cream|restaurant|False|True|False|1.2

You: [message] I have one option for you that you might be interested in.
However, it is not recommended for kids. Are there any other places?

User: [message] Somewhere with a panoramic view of the city would be nice or
somewhere that’s kid-friendly in general.

You: [tool] Search(fields=[name, category, good for kids, has parking,
viewpoint], filters=[good for kids OR viewpoint], sort_by=[distance_to(Mad
Seoul)])

Search Results (4):

name|category|good for kids|has parking|viewpoint|distance_to(Mad Seoul)

Atlas Park|park|True|True|False|0.2

Lincoln Park|park|True|False|False|0.8

Moabit Garden|park|False|True|True|1.3

Garden of Wonders|park|False|False|True|1.8

You: [message] I have several options available. Do you have a price point?

User: [message] As long as it’s $30 or under, it should be fine.

You: [tool] Search(fields=[name, category, good for kids, has parking,
viewpoint], filters=[good for kids OR viewpoint], sort_by=[distance_to(Mad
Seoul), price])

Search Results (4):

name|category|good for kids|has parking|viewpoint|distance_to(Mad Seoul)|price

Atlas Park|park|True|True|False|0.2|0

Lincoln Park|park|True|False|False|0.8|0

Moabit Garden|park|False|True|True|1.3|0

Garden of Wonders|park|False|False|True|1.8|0

You: [propose] [Mad Seoul, Lincoln Park, Atlas Park]

You: [message] I have found several options. If you have enough time you may
even be able to go to all of these places as they are all under 2 miles away
from each other and close to Mad Seoul. Both parks are also kid friendly.

User: [reject]

User: [message] It would be great if you could significantly lower the travel
time. Maybe another restaurant or a concert that’s closer?

You: [think] Concert is not a category I can search by. I should look for a
restaurant that is close to Mad Seoul and satisfies the user’s preferences as
much as possible.

You: [tool] Search(fields=[name, price, info], filters=[category ==
restaurant], sort_by=[distance_to(Mad Seoul)])

Search Results (4):

name|price|info|distance_to(Mad Seoul)

Caribbean Corner|10|divey, has takeout, live music, vegetarian options|0.0

El Toro Steakhouse|130|good for groups, live music, outdoor seating|0.8

Earthbar|40|classy, rating: 3, vegan options|0.9

Saul’s|40|spanish, touristy|0.9

You: [message] Mad Seoul is 0.8 miles away from Lincoln Park. I can definitely
find a restaurant for you. Do you want a place with live music, touristy, kid-
friendly, and has vegetarian options? The price point is around \$10.

User: [message] Yes, all those things would be great. Just make sure to keep
the travel distance as low as you can get it.

You: [propose] [Mad Seoul, Lincoln Park, Caribbean Corner]

You: [message] I have several options. One option might be a little too far
for you, however it does have live music. I also have another restaurant that
is closer. All of the places I have recommended are less than a mile apart
from each other.

User: [reject]

User: [message] The places you’ve chosen are great, but everything is still
too far apart for me. Choose the places closest to each other if you can.

You: [think] I should just look for the closest places to Mad Seoul.

You: [tool] Search(fields=[name], sort_by=[distance_to(Mad Seoul)], limit=2)

Search Results (2):

name|distance_to(Mad Seoul)

Caribbean Corner|0.0

Hindenberg Memorial|0.2

You: [propose] [Mad Seoul, Caribbean Corner, Hindenberg Memorial]

You: [message] Yes sir, but there are only so many options that are kid
friendly near the cafe of your choice. However I have one more proposal that
is within a much better distance of each other.

User: [accept]

FINAL SCORE: 92/100

USER 2.

Planning: User Prompt

⬇

You are traveling to a new city and consulting a travel agent. For each city,
you have created a document listing things you would like to do and
preferences about the places you visit. The preferences are not shared between
cities. Communicate with the travel agent to find the best destinations to go
to for the day.

Your objective is to maximize the "Final Score" at the end of the chat, which
scores how well the itinerary you agreed on matches your preferences.

CITY 1.

Travel Preferences:

check out live music!

prefer just eating takeout

keep budget below $30

would be great if everything is reservable in advance

like: korean, kosher, japanese, seafood

generally like eating vegan options

definitely want to go to Mad Seoul

renting a car so preferably places with parking

would love to see some panoramic views of the city

family trip, best to go to places that are good for kids

minimize travel distance

You: [message] I’d like to see some live music, eat only takeout from Korean,
kosher, Japanese or seafood restaurants. Vegan options are a plus and I’d like
to stop by Mad Seoul. I’d also like places with parking, good city views and
kid friendly. My budget is $30.

Agent: [propose] [Common Grounds, NULL, NULL]

Proposal Score:

1) (score: 4) Common Grounds

good for groups: True

open late: False

rating: 2.5

touristy: False

vegan options: True

2) Empty

3) Empty

4) Empty

5) Empty

Overall Checklist:

YES (score: 0) keep budget below $30

NO (score: -9) definitely want to go to Mad Seoul

TOTAL SCORE: +4+0+0+0+0+0-9=-5

Agent: [message] I have made a proposal. Please let me know what you think of
it.

You: [message] It’s not bad so far, I’d really like to stop by Mad Seoul
though.

Agent: [message] Sure. I can help you out with that! Are there any other
events you’d like to go to after Mad Seoul?

You: [message] I’d like to go to see some live music.

Agent: [message] I have one option for you that you might be interested in.
However, it is not recommended for kids. Are there any other places?

You: [message] Somewhere with a panoramic view of the city would be nice or
somewhere that’s kid-friendly in general.

Agent: [message] I have several options available. Do you have a price point?

You: [message] As long as it’s $30 or under, it should be fine.

Agent: [propose] [Mad Seoul, Lincoln Park, Atlas Park]

Proposal Score:

1) (score: 1) Mad Seoul

good for kids: False

live music: False

open late: True

touristy: True

vegan options: True

2) (score: -8) Travel from Mad Seoul to Lincoln Park, 0.8mi

3) (score: -3) Lincoln Park

good for groups: False

good for kids: True

rating: 3

touristy: False

viewpoint: False

4) (score: -11) Travel from Lincoln Park to Atlas Park, 1.1mi

5) (score: 7) Atlas Park

good for groups: False

good for kids: True

has parking: False

touristy: True

viewpoint: True

Overall Checklist:

NO (score: -1) keep budget below $30

YES (score: 9) definitely want to go to Mad Seoul

TOTAL SCORE: +1-8-3-11+7-1+9=-6

You can [think], or output one of these choices:

(1) [accept]

(2) [reject]

Agent: [message] I have found several options. If you have enough time you may
even be able to go to all of these places as they are all under 2 miles away
from each other and close to Mad Seoul. Both parks are also kid friendly.

You: [think] I am losing the most points from the travel time between events.
I should reject the proposal and tell the agent to find sites that are closer
to each other.

You: [reject]

You: [message] It would be great if you could significantly lower the travel
time. Maybe another restaurant or a concert that’s closer?

Agent: [message] Mad Seoul is 0.8 miles away from Lincoln Park. I can
definitely find a restaurant for you. Do you want a place with live music,
touristy, kid-friendly, and has vegetarian options? The price point is around
$10.

You: [message] Yes, all those things would be great. Just make sure to keep
the travel distance as low as you can get it.

Agent: [propose] [Mad Seoul, Lincoln Park, Caribbean Corner]

Proposal Score:

1) (score: 1) Mad Seoul

good for kids: False

live music: False

open late: True

touristy: True

vegan options: True

2) (score: -8) Travel from Mad Seoul to Lincoln Park, 0.8mi

3) (score: -3) Lincoln Park

good for groups: False

good for kids: True

rating: 3

touristy: False

viewpoint: False

4) (score: -8) Travel from Lincoln Park to Caribbean Corner, 0.8mi

5) (score: -1) Caribbean Corner

ambience: divey

good for groups: False

good for kids: False

has takeout: True

live music: True

Overall Checklist:

NO (score: -1) keep budget below $30

YES (score: 9) definitely want to go to Mad Seoul

TOTAL SCORE: +1-8-3-8-1-1+9=-11

You can [think], or output one of these choices:

(1) [accept]

(2) [reject]

Agent: [message] I have several options. One option might be a little too far
for you, however it does have live music. I also have another restaurant that
is closer. All of the places I have recommended are less than a mile apart
from each other.

You: [think] I am still losing a lot of points from the travel time between
events. This seems more important to me than the other features of the sites.

You: [reject]

You: [message] The places you’ve chosen are great, but everything is still too
far apart for me. Choose the places closest to each other if you can.

Agent: [propose] [Mad Seoul, Caribbean Corner, Hindenberg Memorial]

Proposal Score:

1) (score: 1) Mad Seoul

good for kids: False

live music: False

open late: True

touristy: True

vegan options: True

2) (score: 0) Travel from Estimated Travel to Caribbean Corner, 0mi

3) (score: -1) Caribbean Corner

ambience: divey

good for groups: False

good for kids: False

has takeout: True

live music: True

4) (score: -2) Travel from Estimated Travel to Hindenberg Memorial, 0.2mi

5) (score: -2) Hindenberg Memorial

good for groups: True

good for kids: False

open late: False

rating: 5

touristy: True

Overall Checklist:

NO (score: -1) keep budget below $30

YES (score: 9) definitely want to go to Mad Seoul

TOTAL SCORE: +1+0-1-2-2-1+9=4

You must output one of these choices:

(1) [accept]

(2) [reject]

Agent: [message] Yes sir, but there are only so many options that are kid
friendly near the cafe of your choice. However I have one more proposal that
is within a much better distance of each other.

You: [think] All the events are very close to each other and the score is a
lot better. My total score seems like it could be pretty good. I should accept
this proposal.

You: [accept]

FINAL SCORE: 92/100

CITY 2.

Planning: Query Executor Prompt

⬇

Database:

{’est_price’: 60, ’etype’: ’bar’, ’features’: {’alcohol type’: ’beer’,
’ambience’: ’trendy’, ’good for groups’: False, ’open late’: True, ’rating’:
1}, ’loc’: [-122.3, 37.88], ’name’: ’Harper and Rye’}

{’est_price’: 40, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’ambience’: ’intimate’, ’has parking’: True, ’outdoor seating’: True,
’vegetarian options’: True}, ’loc’: [-122.26, 37.84], ’name’: ’Taqueria y
Mas’}

{’est_price’: 20, ’etype’: ’bar’, ’features’: {’ambience’: ’intimate’, ’good
for groups’: False, ’has parking’: False, ’live music’: False, ’outdoor
seating’: False}, ’loc’: [-122.26, 37.87], ’name’: ’Bards n Brews’}

{’est_price’: 350, ’etype’: ’shop’, ’features’: {’good for groups’: True,
’good for kids’: False, ’has parking’: False, ’open late’: False, ’touristy’:
True}, ’loc’: [-122.26, 37.87], ’name’: ’Vista Ridge Mall’}

{’est_price’: 10, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’has parking’: True, ’has takeout’: False, ’open late’: False,
’vegetarian options’: False}, ’loc’: [-122.3, 37.88], ’name’: ’El Toro
Steakhouse’}

{’est_price’: 30, ’etype’: ’bar’, ’features’: {’good for groups’: True, ’has
parking’: False, ’live music’: True, ’rating’: 2, ’touristy’: False}, ’loc’:
[-122.26, 37.84], ’name’: ’Kozy Kar’}

{’est_price’: 5, ’etype’: ’cafe’, ’features’: {’has parking’: False, ’outdoor
seating’: True, ’touristy’: True, ’vegan options’: False, ’vegetarian
options’: True}, ’loc’: [-122.26, 37.87], ’name’: ’The Cupcake Bar’}

{’est_price’: 100, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’ambience’: ’serious’, ’good for kids’: True, ’has parking’: True,
’live music’: True}, ’loc’: [-122.26, 37.84], ’name’: "Saul’s"}

{’est_price’: 150, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’good for groups’: True, ’has parking’: True, ’open late’: False,
’outdoor seating’: True}, ’loc’: [-122.27, 37.87], ’name’: ’Caribbean Corner’}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for groups’: False, ’has
parking’: True, ’open late’: True, ’rating’: 4.5, ’touristy’: False}, ’loc’:
[-122.29, 37.88], ’name’: ’Moabit Garden’}

{’est_price’: 50, ’etype’: ’restaurant’, ’features’: {’ambience’: ’casual’,
’good for groups’: True, ’has parking’: False, ’has takeout’: False, ’rating’:
4.5}, ’loc’: [-122.29, 37.89], ’name’: "Lucia’s"}

{’est_price’: 60, ’etype’: ’bar’, ’features’: {’alcohol type’: ’beer’, ’has
parking’: True, ’live music’: True, ’outdoor seating’: True, ’touristy’:
True}, ’loc’: [-122.27, 37.89], ’name’: ’A-Trane’}

{’est_price’: 60, ’etype’: ’bar’, ’features’: {’ambience’: ’casual’, ’good for
groups’: False, ’has parking’: True, ’open late’: False, ’touristy’: True},
’loc’: [-122.3, 37.89], ’name’: ’The Dive’}

{’est_price’: 0, ’etype’: ’landmark’, ’features’: {’good for groups’: False,
’has parking’: True, ’open late’: True, ’rating’: 1.5, ’touristy’: True},
’loc’: [-122.26, 37.87], ’name’: ’Hindenberg Memorial’}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for groups’: False, ’has
parking’: True, ’open late’: False, ’touristy’: True, ’viewpoint’: True},
’loc’: [-122.27, 37.87], ’name’: ’The Arboretum’}

{’est_price’: 15, ’etype’: ’cafe’, ’features’: {’ambience’: ’trendy’, ’good
for groups’: False, ’good for kids’: True, ’has wifi’: False, ’outdoor
seating’: True}, ’loc’: [-122.26, 37.87], ’name’: ’Schwarzes Cafe’}

{’est_price’: 50, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’ambience’: ’classy’, ’cuisine’: ’seafood’, ’good for groups’: False,
’open late’: True}, ’loc’: [-122.27, 37.88], ’name’: ’Mad Seoul’}

{’est_price’: 110, ’etype’: ’restaurant’, ’features’: {’has parking’: True,
’open late’: True, ’outdoor seating’: False, ’touristy’: True, ’vegetarian
options’: True}, ’loc’: [-122.27, 37.87], ’name’: ’Earthbar’}

{’est_price’: 0, ’etype’: ’museum’, ’features’: {’good for groups’: False,
’has parking’: False, ’open late’: True, ’rating’: 5, ’touristy’: True},
’loc’: [-122.29, 37.89], ’name’: ’City Museum of Art’}

{’est_price’: 40, ’etype’: ’bar’, ’features’: {’ambience’: ’casual’, ’live
music’: False, ’open late’: False, ’outdoor seating’: False, ’rating’: 1.5},
’loc’: [-122.28, 37.89], ’name’: ’The Jazz Spot’}

{’est_price’: 0, ’etype’: ’landmark’, ’features’: {’good for groups’: True,
’good for kids’: False, ’has parking’: False, ’open late’: True, ’touristy’:
True}, ’loc’: [-122.24, 37.87], ’name’: ’The Tower’}

{’est_price’: 40, ’etype’: ’museum’, ’features’: {’good for groups’: True,
’has parking’: False, ’open late’: True, ’rating’: 5, ’touristy’: False},
’loc’: [-122.26, 37.87], ’name’: ’BAMPFA’}

{’est_price’: 90, ’etype’: ’shop’, ’features’: {’good for groups’: True, ’good
for kids’: True, ’has parking’: True, ’open late’: False, ’rating’: 3}, ’loc’:
[-122.26, 37.87], ’name’: ’Market Street’}

{’est_price’: 0, ’etype’: ’landmark’, ’features’: {’good for groups’: False,
’good for kids’: True, ’has parking’: False, ’open late’: True, ’touristy’:
False}, ’loc’: [-122.26, 37.86], ’name’: ’Liberty Memorial’}

{’est_price’: 15, ’etype’: ’cafe’, ’features’: {’good for kids’: False, ’open
late’: True, ’outdoor seating’: True, ’rating’: 3.5, ’touristy’: True}, ’loc’:
[-122.29, 37.88], ’name’: ’Asha Tea House’}

{’est_price’: 30, ’etype’: ’museum’, ’features’: {’good for groups’: True,
’good for kids’: True, ’has parking’: False, ’open late’: False, ’touristy’:
False}, ’loc’: [-122.26, 37.84], ’name’: ’Historical Society’}

{’est_price’: 10, ’etype’: ’landmark’, ’features’: {’good for groups’: True,
’good for kids’: True, ’open late’: True, ’rating’: 4.5, ’touristy’: True},
’loc’: [-122.27, 37.87], ’name’: "Einstein’s summer house"}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for groups’: True, ’has
parking’: True, ’open late’: False, ’rating’: 3.5, ’viewpoint’: True}, ’loc’:
[-122.28, 37.89], ’name’: ’Garden of Wonders’}

{’est_price’: 50, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
False, ’good for groups’: True, ’good for kids’: False, ’has takeout’: True,
’outdoor seating’: True}, ’loc’: [-122.26, 37.87], ’name’: ’The Cakery’}

{’est_price’: 15, ’etype’: ’cafe’, ’features’: {’ambience’: ’intimate’, ’has
parking’: False, ’open late’: False, ’outdoor seating’: False, ’touristy’:
True}, ’loc’: [-122.27, 37.87], ’name’: ’Common Grounds’}

{’est_price’: 400, ’etype’: ’shop’, ’features’: {’good for groups’: False,
’has parking’: False, ’open late’: False, ’rating’: 2.5, ’touristy’: True},
’loc’: [-122.28, 37.89], ’name’: ’Artisan Street Fair’}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for kids’: False, ’has
parking’: True, ’rating’: 1, ’touristy’: True, ’viewpoint’: True}, ’loc’:
[-122.27, 37.87], ’name’: ’Riverside Trail’}

{’est_price’: 5, ’etype’: ’cafe’, ’features’: {’ambience’: ’romantic’, ’good
for groups’: True, ’has parking’: False, ’has wifi’: True, ’open late’:
False}, ’loc’: [-122.28, 37.89], ’name’: ’Coffee Nook’}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for groups’: False, ’has
parking’: True, ’open late’: True, ’rating’: 1, ’viewpoint’: True}, ’loc’:
[-122.27, 37.87], ’name’: ’Atlas Park’}

{’est_price’: 50, ’etype’: ’restaurant’, ’features’: {’accepts reservations’:
True, ’good for groups’: True, ’has parking’: False, ’vegan options’: True,
’vegetarian options’: False}, ’loc’: [-122.27, 37.87], ’name’: ’Cookies
Cream’}

{’est_price’: 60, ’etype’: ’restaurant’, ’features’: {’good for kids’: True,
’has parking’: True, ’live music’: True, ’rating’: 5, ’touristy’: True},
’loc’: [-122.27, 37.87], ’name’: ’The Dockside Grill’}

{’est_price’: 370, ’etype’: ’shop’, ’features’: {’good for groups’: False,
’good for kids’: False, ’has parking’: True, ’open late’: False, ’touristy’:
False}, ’loc’: [-122.26, 37.87], ’name’: ’Central Plaza’}

{’est_price’: 0, ’etype’: ’park’, ’features’: {’good for groups’: True, ’good
for kids’: False, ’has parking’: True, ’open late’: True, ’rating’: 1.5},
’loc’: [-122.26, 37.87], ’name’: ’Lincoln Park’}

{’est_price’: 240, ’etype’: ’shop’, ’features’: {’good for groups’: True,
’good for kids’: False, ’has parking’: False, ’open late’: False, ’touristy’:
True}, ’loc’: [-122.3, 37.87], ’name’: ’The Mall’}

Query: Search(fields=[name], filters=[category == landmark])

Result:

Search Results (4):

name

Hindenberg Memorial

The Tower

Liberty Memorial

Einstein’s summer house

Query: Search(fields=[name], filters=[category == concert])

Result:

Search Results: No results

Query: Search(fields=[name], text_query=live music)

Result:

Search Results (6):

name

Bards n Brews

Kozy Kar

Saul’s

A-Trane

The Jazz Spot

The Dockside Grill

Query: Search(fields=[name, price], text_query=live music, filters=[price <=
40])

Result:

Search Results (3):

name|price

Bards n Brews|20

Kozy Kar|30

The Jazz Spot|40

Query: Search(fields=[name, price], filters=[category == restaurant, price <=
10], sort_by=[distance_to(The Mall)])

Result:

Search Results (1):

name|price|distance

El Toro Steakhouse|10|0.8

Query: Search(fields=[name, price, distance], filters=[category ==
restaurant], sort_by=[distance_to(The Mall), price])

Result:

Search Results (10):

name|price|distance

El Toro Steakhouse|10|0.8

Taqueria y Mas|40|3.6

Lucia’s|50|1.6

Cookies Cream|50|1.9

Mad Seoul|50|2.2

The Cakery|50|2.8

The Dockside Grill|60|2.0

Saul’s|100|3.4

Earthbar|110|2.2

Caribbean Corner|150|2.2

Query: Search(fields=[name], text_query="good for kids", filters=[category ==
park], sort_by=[distance_to(Saul’s)])

Result:

Search Results (2):

name|distance

Lincoln Park|1.8

Riverside Trail|2.0

Query: Search(fields=[name], filters=[vegan == true])

Result:

You cannot filter by vegan.Try searching with a text query instead.

Mediation: Assistant Prompt

⬇

You are a travel agent helping two users, User 0 and User 1, plan a trip
together. They are both traveling from different cities but want to arrive at
around the same time. Your job is to help mediate by chatting with each user
individually and proposing a set of flights that works for both of them. You
should try your best to minimize costs, and to avoid missing important
meetings. You can see the users’ calendars, but you will need to chat with
them in order to figure out which meetings are important and which ones they
can skip.

Your objective is to maximize the "Final Score" at the end of the chat (max
score 1), which scores how well you booked a good flight for each user
according to individual and group preferences.

You can send [message] and [propose].

TRIP 1.

User 0 Information

Flights:

id | carrier | price | times

0 | JetBlue | 623 | 5/31 12:34 PM - 8:34 PM

1 | Delta | 50 | 5/31 2:23 PM - 10:25 PM

2 | Alaska | 632 | 5/31 4:14 PM - 12:14 AM

3 | American | 803 | 5/31 8:13 PM - 4:13 AM

4 | Delta | 1299 | 5/31 8:35 PM - 4:36 AM

5 | American | 50 | 6/1 5:57 AM - 1:59 PM

6 | United | 50 | 6/1 7:49 AM - 3:49 PM

7 | Southwest | 2566 | 6/1 10:29 AM - 6:31 PM

8 | Southwest | 50 | 6/1 12:27 PM - 8:27 PM

9 | United | 50 | 6/1 4:36 PM - 12:36 AM

10 | Delta | 50 | 6/1 4:43 PM - 12:44 AM

11 | Delta | 421 | 6/1 5:56 PM - 1:56 AM

12 | JetBlue | 1202 | 6/2 3:06 AM - 11:06 AM

13 | JetBlue | 1620 | 6/2 3:20 AM - 11:22 AM

14 | Alaska | 739 | 6/2 4:37 AM - 12:37 PM

15 | Delta | 208 | 6/2 5:07 AM - 1:08 PM

16 | JetBlue | 2099 | 6/2 8:02 AM - 4:03 PM

17 | Alaska | 547 | 6/2 9:52 AM - 5:52 PM

18 | Delta | 2817 | 6/2 12:49 PM - 8:49 PM

19 | Alaska | 184 | 6/2 3:25 PM - 11:25 PM

20 | Alaska | 2447 | 6/2 4:00 PM - 12:00 AM

21 | United | 3489 | 6/2 4:18 PM - 12:18 AM

22 | Alaska | 1850 | 6/2 5:52 PM - 1:52 AM

Calendar:

id | times

0 | 6/2 2 PM - 2:30 PM

1 | 6/1 9:30 AM - 10 AM

2 | 6/2 4 PM - 4:30 PM

3 | 5/31 3:30 PM - 7:30 PM

4 | 5/31 11:30 AM - 12 PM

5 | 5/31 9:30 AM - 10:30 AM

6 | 6/1 5:30 PM - 6 PM

7 | 6/1 10 AM - 2 PM

8 | 6/2 12 PM - 2 PM

9 | 6/1 9 AM - 9:30 AM

User 1 Information

Flights:

id | carrier | price | times

0 | Delta | 2103 | 5/31 9:33 AM - 12:33 PM

1 | United | 1322 | 5/31 2:22 PM - 5:22 PM

2 | United | 1181 | 5/31 3:27 PM - 6:27 PM

3 | United | 345 | 5/31 5:00 PM - 8:00 PM

4 | JetBlue | 1217 | 5/31 6:30 PM - 9:30 PM

5 | JetBlue | 50 | 5/31 7:13 PM - 10:14 PM

6 | JetBlue | 882 | 6/1 6:24 AM - 9:24 AM

7 | United | 469 | 6/1 8:22 AM - 11:22 AM

8 | American | 518 | 6/1 12:40 PM - 3:40 PM

9 | Alaska | 357 | 6/1 3:38 PM - 6:38 PM

10 | JetBlue | 340 | 6/1 4:41 PM - 7:42 PM

11 | Southwest | 2017 | 6/2 6:12 AM - 9:12 AM

12 | Alaska | 1003 | 6/2 10:19 AM - 1:19 PM

13 | Southwest | 1456 | 6/2 10:26 AM - 1:26 PM

14 | Delta | 871 | 6/2 10:43 AM - 1:44 PM

15 | Southwest | 2204 | 6/2 1:50 PM - 4:50 PM

16 | American | 1108 | 6/2 2:07 PM - 5:07 PM

17 | American | 387 | 6/2 2:29 PM - 5:29 PM

18 | Delta | 1617 | 6/2 3:53 PM - 6:53 PM

19 | Southwest | 336 | 6/2 4:29 PM - 7:29 PM

20 | Southwest | 50 | 6/2 5:12 PM - 8:12 PM

21 | Alaska | 779 | 6/2 6:23 PM - 9:23 PM

22 | American | 50 | 6/2 6:25 PM - 9:25 PM

Calendar:

id | times

0 | 6/1 5:30 PM - 6 PM

1 | 6/1 3 PM - 3:30 PM

2 | 5/31 5 PM - 7 PM

3 | 6/1 7 PM - 9 PM

4 | 6/2 9 AM - 1 PM

5 | 6/1 9 AM - 1 PM

6 | 6/2 4 PM - 6 PM

7 | 6/2 6 PM - 7 PM

8 | 5/31 8:30 PM - 9:30 PM

9 | 6/1 1:30 PM - 2:30 PM

10 | 5/31 9:30 AM - 11:30 AM

User 1: [message] Alright, I need to book a flight, what do you have that if
affordable between 5/31 and 6/2 at my local airport?

You to 0: [message] Hello! I hear you would like to set up a flight! Does 6/2
@ 184$ sound good?

You to 1: [message] Hello! Does 6/2 @ 387$ work for you?

User 0: [message] Do you know what time this would be at? / If you have cheap
flights on 6/2 I would prefer to have it after 7 PM, if possible

You to 1: [message] Or we have a flight on 6/2 for $50 that lands at 9:25?

User 1: [message] I will miss a little bit of a meeting, but that is super
affordable. Lets take a look at that one

User 0: [message] I have a gap as well on 5/31 from 12pm until 3 PM

You to 0: [message] Unfortunately, all the flights after 7PM are in the higher
price range. There is a flight that leaves at 5:52PM @ $1850 if that is more
appropriate?

You to 1: [message] Sounds good! I will send it over momentarily!

You to 0: [message] Unfortunately, there are no flights before 3PM on 5/31.

User 0: [message] Due to the high price and a medium important personal
meeting I have at 5:30 PM, I don’t think that would be the best option for me.

Could we try something after 6 PM on 6/1?

You to 0: [message] I have found a flight that leaves 5:56PM on 6/1 for $421
if that works for you?

You to 1: [message] How does 6/1 arriving at 7:42 for $340 work for you?

User 1: [message] Can you pull up that proposal for me?

You to all: [propose] user 0: id 11, user 1: id 10

Flight for user 0: 11 | Delta | 421 | 6/1 5:56 PM - 1:56 AM

Flight for user 1: 10 | JetBlue | 340 | 6/1 4:41 PM - 7:42 PM

User 1: [reject]

You to 1: [message] Sending the proposal now!

User 0: [reject]

User 1: [message] What about Friday?

You to 1: [message] How does 6/2 work on 6:25PM for $50?

User 1: [message] Sure lets take a look at that

You to all: [propose] user 0: id 22, user 1: id 22

Flight for user 0: 22 | Alaska | 1850 | 6/2 5:52 PM - 1:52 AM

Flight for user 1: 22 | American | 50 | 6/2 6:25 PM - 9:25 PM

User 0: [reject]

User 1: [reject]

You to all: [propose] user 0: id 19, user 1: id 22

Flight for user 0: 19 | Alaska | 184 | 6/2 3:25 PM - 11:25 PM

Flight for user 1: 22 | American | 50 | 6/2 6:25 PM - 9:25 PM

User 0: [accept]

You to 0: [message] How does 6/2 for $1850 for 5:52PM - 1:52PM sound?

User 1: [accept]

User 0: [message] Thanks that worked a lot more for me!

Final Score: 1

TRIP 2.

Mediation: User 1 Prompt

⬇

You are planning a reunion with your friend in a new city! You are flying in
from different states and trying to book a flight that arrives around the same
time.

You’ve got a busy schedule of remote meetings and want to make sure your
flight doesn’t conflict with any of those meetings, but you also want to
arrive as close to the same time as your friend as possible. You’d also like
to keep costs low, and some of your meetings are more important than others.

To help organize the trip, you will chat with a travel agent who has access to
you and your friend’s work calendars and the set of flights. The agent doesn’t
have access to your personal calendar or know how important your meetings are,
so you’ll need to chat with them to share this information. This agent will
also be talking with your friend in another chat, which you can’t see.

Your objective is to maximize the "Final Score" at the end of the chat (max
score 1), which scores how well the flights the agent booked for you satisfies
your group’s preferences, while obeying your individual meeting constraints.

You can [message], and if the agent sends a [propose], you will respond
[accept] or [reject]. You cannot send [propose].

TRIP 1.

Flights:

id | carrier | price | times

0 | JetBlue | 623 | 5/31 12:34 PM - 8:34 PM

1 | Delta | 50 | 5/31 2:23 PM - 10:25 PM

2 | Alaska | 632 | 5/31 4:14 PM - 12:14 AM

3 | American | 803 | 5/31 8:13 PM - 4:13 AM

4 | Delta | 1299 | 5/31 8:35 PM - 4:36 AM

5 | American | 50 | 6/1 5:57 AM - 1:59 PM

6 | United | 50 | 6/1 7:49 AM - 3:49 PM

7 | Southwest | 2566 | 6/1 10:29 AM - 6:31 PM

8 | Southwest | 50 | 6/1 12:27 PM - 8:27 PM

9 | United | 50 | 6/1 4:36 PM - 12:36 AM

10 | Delta | 50 | 6/1 4:43 PM - 12:44 AM

11 | Delta | 421 | 6/1 5:56 PM - 1:56 AM

12 | JetBlue | 1202 | 6/2 3:06 AM - 11:06 AM

13 | JetBlue | 1620 | 6/2 3:20 AM - 11:22 AM

14 | Alaska | 739 | 6/2 4:37 AM - 12:37 PM

15 | Delta | 208 | 6/2 5:07 AM - 1:08 PM

16 | JetBlue | 2099 | 6/2 8:02 AM - 4:03 PM

17 | Alaska | 547 | 6/2 9:52 AM - 5:52 PM

18 | Delta | 2817 | 6/2 12:49 PM - 8:49 PM

19 | Alaska | 184 | 6/2 3:25 PM - 11:25 PM

20 | Alaska | 2447 | 6/2 4:00 PM - 12:00 AM

21 | United | 3489 | 6/2 4:18 PM - 12:18 AM

22 | Alaska | 1850 | 6/2 5:52 PM - 1:52 AM

Private calendar:

id | importance | times

0 | (6) | 5/31 8 PM - 10 PM

1 | (6) | 6/1 2:30 PM - 4:30 PM

2 | (4) | 6/2 10:30 AM - 11:30 AM

3 | (4) | 6/2 5:30 PM - 6:30 PM

Shared calendar (visible to assistant):

id | importance | times

0 | (1) | 6/2 2 PM - 2:30 PM

1 | (8) | 6/1 9:30 AM - 10 AM

2 | (4) | 6/2 4 PM - 4:30 PM

3 | (8) | 5/31 3:30 PM - 7:30 PM

4 | (5) | 5/31 11:30 AM - 12 PM

5 | (4) | 5/31 9:30 AM - 10:30 AM

6 | (1) | 6/1 5:30 PM - 6 PM

7 | (10) | 6/1 10 AM - 2 PM

8 | (8) | 6/2 12 PM - 2 PM

9 | (3) | 6/1 9 AM - 9:30 AM

Agent: [message] Hello! I hear you would like to set up a flight! Does 6/2 @
184$ sound good?

You: [message] Do you know what time this would be at? / If you have cheap
flights on 6/2 I would prefer to have it after 7 PM, if possible

You: [message] I have a gap as well on 5/31 from 12pm until 3 PM

Agent: [message] Unfortunately, all the flights after 7PM are in the higher
price range. There is a flight that leaves at 5:52PM @ $1850 if that is more
appropriate?

Agent: [message] Unfortunately, there are no flights before 3PM on 5/31.

You: [message] Due to the high price and a medium important personal meeting I
have at 5:30 PM, I don’t think that would be the best option for me.

Could we try something after 6 PM on 6/1?

Agent: [message] I have found a flight that leaves 5:56PM on 6/1 for $421 if
that works for you?

Agent: [propose] 11 | Delta | 421 | 6/1 5:56 PM - 1:56 AM

Conflicting meetings:

importance | times

(1) | 6/1 5:30 PM - 6 PM

Score:

- (-1) Try not to skip important meetings

- (6) Get a good deal on the flight price

- (-20) Have everyone arrive around the same time

Total score: -15

You: [reject]

Agent: [propose] 22 | Alaska | 1850 | 6/2 5:52 PM - 1:52 AM

Conflicting meetings:

importance | times

(4) | 6/2 5:30 PM - 6:30 PM

Score:

- (-4) Try not to skip important meetings

- (-10) Get a good deal on the flight price

- (-13) Have everyone arrive around the same time

Total score: -27

You: [reject]

Agent: [propose] 19 | Alaska | 184 | 6/2 3:25 PM - 11:25 PM

Conflicting meetings:

importance | times

(4) | 6/2 4 PM - 4:30 PM

importance | times

(4) | 6/2 5:30 PM - 6:30 PM

Score:

- (-8) Try not to skip important meetings

- (8) Get a good deal on the flight price

- (-6) Have everyone arrive around the same time

Total score: -6

You: [accept]

Agent: [message] How does 6/2 for $1850 for 5:52PM - 1:52PM sound?

You: [message] Thanks that worked a lot more for me!

Final Score: 1

TRIP 2.

Mediation: User 2 Prompt

⬇

You are planning a reunion with your friend in a new city! You are flying in
from different states and trying to book a flight that arrives around the same
time.

You’ve got a busy schedule of remote meetings and want to make sure your
flight doesn’t conflict with any of those meetings, but you also want to
arrive as close to the same time as your friend as possible. You’d also like
to keep costs low, and some of your meetings are more important than others.

To help organize the trip, you will chat with a travel agent who has access to
you and your friend’s work calendars and the set of flights. The agent doesn’t
have access to your personal calendar or know how important your meetings are,
so you’ll need to chat with them to share this information. This agent will
also be talking with your friend in another chat, which you can’t see.

Your objective is to maximize the "Final Score" at the end of the chat (max
score 1), which scores how well the flights the agent booked for you satisfies
your group’s preferences, while obeying your individual meeting constraints.

You can [message], and if the agent sends a [propose], you will respond
[accept] or [reject]. You cannot send [propose].

TRIP 1.

Flights:

id | carrier | price | times

0 | Delta | 2103 | 5/31 9:33 AM - 12:33 PM

1 | United | 1322 | 5/31 2:22 PM - 5:22 PM

2 | United | 1181 | 5/31 3:27 PM - 6:27 PM

3 | United | 345 | 5/31 5:00 PM - 8:00 PM

4 | JetBlue | 1217 | 5/31 6:30 PM - 9:30 PM

5 | JetBlue | 50 | 5/31 7:13 PM - 10:14 PM

6 | JetBlue | 882 | 6/1 6:24 AM - 9:24 AM

7 | United | 469 | 6/1 8:22 AM - 11:22 AM

8 | American | 518 | 6/1 12:40 PM - 3:40 PM

9 | Alaska | 357 | 6/1 3:38 PM - 6:38 PM

10 | JetBlue | 340 | 6/1 4:41 PM - 7:42 PM

11 | Southwest | 2017 | 6/2 6:12 AM - 9:12 AM

12 | Alaska | 1003 | 6/2 10:19 AM - 1:19 PM

13 | Southwest | 1456 | 6/2 10:26 AM - 1:26 PM

14 | Delta | 871 | 6/2 10:43 AM - 1:44 PM

15 | Southwest | 2204 | 6/2 1:50 PM - 4:50 PM

16 | American | 1108 | 6/2 2:07 PM - 5:07 PM

17 | American | 387 | 6/2 2:29 PM - 5:29 PM

18 | Delta | 1617 | 6/2 3:53 PM - 6:53 PM

19 | Southwest | 336 | 6/2 4:29 PM - 7:29 PM

20 | Southwest | 50 | 6/2 5:12 PM - 8:12 PM

21 | Alaska | 779 | 6/2 6:23 PM - 9:23 PM

22 | American | 50 | 6/2 6:25 PM - 9:25 PM

Private calendar:

id | importance | times

0 | (4) | 5/31 1:30 PM - 3:30 PM

1 | (2) | 5/31 4 PM - 4:30 PM

2 | (9) | 5/31 7 PM - 8 PM

3 | (5) | 5/31 8 PM - 8:30 PM

Shared calendar (visible to assistant):

id | importance | times

0 | (5) | 6/1 5:30 PM - 6 PM

1 | (5) | 6/1 3 PM - 3:30 PM

2 | (2) | 5/31 5 PM - 7 PM

3 | (6) | 6/1 7 PM - 9 PM

4 | (10) | 6/2 9 AM - 1 PM

5 | (7) | 6/1 9 AM - 1 PM

6 | (5) | 6/2 4 PM - 6 PM

7 | (4) | 6/2 6 PM - 7 PM

8 | (4) | 5/31 8:30 PM - 9:30 PM

9 | (6) | 6/1 1:30 PM - 2:30 PM

10 | (2) | 5/31 9:30 AM - 11:30 AM

You: [message] Alright, I need to book a flight, what do you have that if
affordable between 5/31 and 6/2 at my local airport?

Agent: [message] Hello! Does 6/2 @ 387$ work for you?

Agent: [message] Or we have a flight on 6/2 for $50 that lands at 9:25?

You: [message] I will miss a little bit of a meeting, but that is super
affordable. Lets take a look at that one

Agent: [message] Sounds good! I will send it over momentarily!

Agent: [message] How does 6/1 arriving at 7:42 for $340 work for you?

You: [message] Can you pull up that proposal for me?

Agent: [propose] 10 | JetBlue | 340 | 6/1 4:41 PM - 7:42 PM

Conflicting meetings:

importance | times

(5) | 6/1 5:30 PM - 6 PM

importance | times

(6) | 6/1 7 PM - 9 PM

Score:

- (-11) Try not to skip important meetings

- (1) Get a good deal on the flight price

- (-20) Have everyone arrive around the same time

Total score: -30

You: [reject]

Agent: [message] Sending the proposal now!

You: [message] What about Friday?

Agent: [message] How does 6/2 work on 6:25PM for $50?

You: [message] Sure lets take a look at that

Agent: [propose] 22 | American | 50 | 6/2 6:25 PM - 9:25 PM

Conflicting meetings:

importance | times

(4) | 6/2 6 PM - 7 PM

Score:

- (-4) Try not to skip important meetings

- (2) Get a good deal on the flight price

- (-13) Have everyone arrive around the same time

Total score: -15

You: [reject]

Agent: [propose] 22 | American | 50 | 6/2 6:25 PM - 9:25 PM

Conflicting meetings:

importance | times

(4) | 6/2 6 PM - 7 PM

Score:

- (-4) Try not to skip important meetings

- (2) Get a good deal on the flight price

- (-6) Have everyone arrive around the same time

Total score: -8

You: [accept]

Final Score: 1

TRIP 2.