# Machine Learning for Utility Prediction in
Argument-Based Computational Persuasion

Ivan Donadello1 Anthony Hunter2 Stefano Teso4 Mauro Dragoni3

###### Abstract

Automated persuasion systems (APS) aim to persuade a user to believe something
by entering into a dialogue in which arguments and counterarguments are
exchanged. To maximize the probability that an APS is successful in persuading
a user, it can identify a global policy that will allow it to select the best
arguments it presents at each stage of the dialogue whatever arguments the
user presents. However, in real applications, such as for healthcare, it is
unlikely the utility of the outcome of the dialogue will be the same, or the
exact opposite, for the APS and user. In order to deal with this situation,
games in extended form have been harnessed for argumentation in Bi-party
Decision Theory. This opens new problems that we address in this paper: (1)
How can we use Machine Learning (ML) methods to predict utility functions for
different subpopulations of users? and (2) How can we identify for a new user
the best utility function from amongst those that we have learned? To this
extent, we develop two ML methods, EAI and EDS, that leverage information
coming from the users to predict their utilities. EAI is restricted to a fixed
amount of information, whereas EDS can choose the information that best
detects the subpopulations of a user. We evaluate EAI and EDS in a simulation
setting and in a realistic case study concerning healthy eating habits.
Results are promising in both cases, but EDS is more effective at predicting
useful utility functions.

##  1 Introduction

Persuasion is an activity that involves one party trying to induce another
party to believe something. It is an important and multifaceted human
facility. Often, it can involve a dialogue in which arguments and
counterarguments are exchanged between the persuader and the persuadee. For
example, a doctor can use arguments to persuade a patient to eat a more
healthy diet, and the patient can give counterarguments based on their
understanding and preferences. Also, the doctor may provide counterarguments
to attempt to overturn misconceptions held by the patient.

An automated persuasion system (APS) is a software system that takes on the
role of a persuader, and a user is the persuadee (Hunter et al. 2019). It aims
to use convincing arguments in order to persuade the user. Whether an argument
is convincing depends on its own nature, on the context of the argumentation,
and on the user’s characteristics. An APS maintains a model of the user to
choose the best moves in the dialogue (Hadoux and Hunter 2019). During the
turn of the APS, it presents an argument to the user and then user can select
their counterarguments from a menu. An APS can also use a natural language
interface to allow users input their argument in free text (Chalaguine and
Hunter 2020).

In order for an APS to choose the moves it makes in the dialogue (i.e., the
arguments it presents), it needs to use a strategy. Within the literature on
computational models of arguments, key approaches to strategies are: Mechanism
design where it is a one-step process rather for multi-step dialogues (Rahwan
and Larson 2008; Rahwan, Larson, and Tohmé 2009; Fan and Toni 2012); Planning
systems to optimize choice of arguments based on belief in premises (Black,
Coles, and Bernardini 2014; Black, Coles, and Hampson 2017) or minimize the
number of moves made (Atkinson, Bench-Capon, and Bench-Capon 2012); Machine
learning of strategies such as Reinforcement Learning (Monteserin and Amandi
2013; Rosenfeld and Kraus 2016b; Rach, Minker, and Ultes 2018; Katsumi et al.
2018; Riveret et al. 2019; Alahmari 2020) and transfer learning (Rosenfeld and
Kraus 2016a); Probabilistic methods to select a move based on what an agent
believes the other is aware of (Rienstra, Thimm, and Oren 2013), to
approximately predict the argument an opponent might put forward based on data
about the moves made by the opponent in previous dialogues (Hadjinikolis et
al. 2013), using a decision-theoretic lottery (Hunter and Thimm 2016), using
POMDPs when there is uncertainty about the internal state of the opponent
(Hadoux et al. 2015); and Local strategies based for example on the concerns
(i.e., issues raised or addressed by an argument) of the persuadee (Hadoux and
Hunter 2019; Chalaguine and Hunter 2020).

Apart from local strategies based on concerns, the above proposals do not take
the user’s needs into account. Yet, if an APS is to persuade someone to accept
some arguments, then how those arguments relate to their needs is fundamental
in deciding how to argue with them. This can be captured by considering the
utility of each argument from the point of view of both the APS and the user
(i.e., we have a utility function for each agent). For this we can harness
games in extensive form (Osborne and Rubinstein 1994). For argumentation, this
has led to Bi-party Decision Theory (BDT) (Hadoux, Hunter, and Polberg 2018)
that generates a policy that is a good compromise in maximizing the APS and
user utilities. However, BDT requires a utility function to be constructed for
groups of users. We address this by using Machine Learning (ML) methods that
learn utility functions from data about subpopulations of users. Then, an
appropriate utility function for a new user (i.e., which subpopulation the
user belongs to) has to be determined. This can be performed by asking
questions. However, a user might be reluctant to answer too many questions
(e.g., because they become bored, or it takes too much effort). So the
challenge is to ask fewer questions and still get a useful utility function.
We address these novel problems by studying the following research questions:
RQ1 How do we use ML-methods to identify a utility function for each
subpopulation of a set of users? RQ2 How do we optimize the number of
questions we ask a new user to identify an appropriate utility function?

Differently from Recommender Systems (RSs) (Ricci, Rokach, and Shapira 2015),
our proposal is based on BDT (Section 2) and, therefore, it does not provide
“one-shot” recommendations based only on users’ utilities but a dialogue that
considers both users’ and system (i.e., biparty) utilities. BDT explicitly
models the persuader’s and user’s needs. This allows a better customization of
the APS according to the specific domain and subpopulation. Conversational RSs
(CRSs) (Jannach et al. 2021) go beyond standard “one-shot” RSs by creating a
dialogue with users. However, as far as we know, no method tackles the problem
of learning a utility function in a BDT setting for CRSs. Reinforcement
Learning (RL) (Rosenfeld and Kraus 2016b) is orthogonal to our proposal but it
is data inefficient as it involves large amounts of interaction. Our method is
less data hungry as it only requires some utility elicitation from users with
questionnaires. We addresses the limitations of BDT by developing two ML
methods (Section 3) that compute the utility functions from a dataset of user
needs. These ML methods allow the optimization of the number of questions to
ask. The evaluation using synthetic data (Section 4) and a case study
(Sections 5) reveals promising results and answers to both RQ1 and RQ2.

##  2 Bi-party Decision Theory

Decision trees (DT) are an important approach to making optimal decisions
(Peterson 2009). In computational persuasion, a DT is used for representing
all possible dialogues between two agents: each path from the root to a leaf
alternates _decision nodes_ with _chance nodes_. The former are associated
with the persuader (a.k.a. the proponent) and represent the argument to be
posed by the APS. The latter are associated with the persuadee (a.k.a. the
opponent) and represent the argument posed by the user. Each arc $(n,n_{i})$
is labelled with the argument posited by the corresponding agent that has been
selected at node $n$. Figure 1 contains an example of DT for the persuasion
goal of reducing the red meat consumption.

$n_{1}$$n_{2}$$n_{3}$$n_{4}$$n_{5}$$n_{6}$$n_{7}$$n_{8}$ [$a_{1}$] Low red
meat consumption is necessary for a healthy diet. [$a_{2}$] It is really
difficult to change diet. [$a_{3}$] I really like the taste of meat. [$a_{4}$]
Think about the benefits of reducing red meat. [$a_{5}$] Try to reduce red
meat slowly. [$a_{6}$] White meat can be an alternative. [$a_{7}$] Fish is a
tasty alternative to meat. Figure 1: A decision tree for a persuasive dialogue
for the red meat persuasion goal. Proponent (decision) nodes are solid boxes
and opponent (chance) nodes are dashed boxes.

A _policy_ associates a decision node with the best argument to pose by the
proponent by considering the points of view of both the proponent and the
opponent. In Bi-party Decision Theory, these viewpoints are modelled using two
_utility functions_ $U^{p}$ and $U^{o}$. These provide each leaf with a
utility value that represents the benefit, for the corresponding agent, of
accepting the APS arguments in the dialogue from the root to that leaf

A bimaximax policy maximizes $U^{p}$ at a decision node and $U^{o}$ at a
chance node. Some notation is necessary before a formal definition of the
policy. Let $T$ be a DT, $L$ be a labeling function that associates an
argument to each arc $(n,n_{i})$ between two nodes of $T$ and ${\sf
Children}(T,n)$ be the set of children of $n$. The ${\sf AMax}(T,U,n)$
function returns the children of $n$ with highest utility $U$: ${\sf
AMax}(T,U,n)=\\{n^{\prime}\in{{\sf Children}}(T,n)\mid\forall
n^{\prime\prime}\in{{\sf Children}}(T,n),U(n^{\prime})\geq
U(n^{\prime\prime})\\}$ Let $\delta\in\mathbb{R}$ be a discount factor that
decreases the utility of longer branches. Indeed, shorter dialogues can be
more persuasive than longer ones that require more attention from the user.
Definition 1 defines the bimaximax policy.

######  Definition 1.

A bimaximax policy for $(T,L,U^{p},U^{o},\delta)$ is $\Pi:{\sf
Nodes}(T)\rightarrow{\sf Nodes}(T)$ defined as follows using the calculation
of the $Q^{p}:{\sf Nodes}(T)\rightarrow\mathbb{R}$ and $Q^{o}:{\sf
Nodes}(T)\rightarrow\mathbb{R}$ functions.

  * •

If $n$ is a leaf node, then $Q^{p}(n)=U^{p}(n)$ and $Q^{o}(n)=U^{o}(n)$.

  * •

If $n$ is a chance node, and $n_{i}\in{\sf AMax}(T,Q^{o},n)$, then
$Q^{o}(n)=\delta\times Q^{o}(n_{i})$ and $Q^{p}(n)=\delta\times Q^{p}(n_{i})$.

  * •

If $n$ is a decision node, and $n_{i}\in{\sf AMax}(T,Q^{p},n)$, then
$Q^{o}(n)=\delta\times Q^{o}(n_{i})$ and $Q^{p}(n)=\delta\times Q^{p}(n_{i})$
and $\Pi(n)=\langle n_{i},L(n,n_{i})\rangle$.

For the example in Figure 1, the APS aims to persuade the user to adopt
healthy alternatives to red meat, the leaf utility values are then
$U^{p}(n_{5})=2$, $U^{p}(n_{6})=3$, $U^{p}(n_{7})=4$, $U^{p}(n_{8})=6$. The
user is interested in consuming white meat and to slowly reduce his/her
consumption of red meat: $U^{o}(n_{5})=2$, $U^{o}(n_{6})=6$, $U^{o}(n_{7})=7$,
$U^{o}(n_{8})=4$. The utility values are then propagated to the higher levels
of the tree as follows, where $Q^{p/o}(n)$ denotes $(Q^{p}(n),Q^{o}(n))$:
$Q^{p/o}(n_{1})=(3,6)$, $Q^{p/o}(n_{2})=(3,6)$, $Q^{p/o}(n_{3})=(3,6)$,
$Q^{p/o}(n_{4})=(6,4)$, with $\delta=1$, $\Pi(n_{1})=(n_{2},L(n_{2}))$,
$\Pi(n_{3})=(n_{6},L(n_{6}))$ and $\Pi(n_{4})=(n_{8},L(n_{8}))$.

Definition 1 does not provide a criterion for the choice of $n_{i}$ when
$|{\sf AMax}(T,Q,n)|>1$. We therefore adopt a random choice that improves the
variability of the APS messages.

###  2.1 Simulated Dialogues

The bimaximax policy defines a rule for choosing the label/argument to pose
given a decision node $n$. Given a chance node $n_{i}$, the next decision node
$n_{j}$ is selected by the user via, for example, a menu. However, when real
users are not available the next decision node has to be computed according to
some _simulated opponent policy_ $\Pi^{o}$. This policy can be computed by
statistical methods that rely on datasets of conversations with users, by
harvesting arguments from the web or with the use of questionnaires. Here, we
simply define the opponent policy as $\Pi^{o}(n_{i})=\langle
n_{j},L(n_{i},n_{j})\rangle$ with $n_{j}\in{\sf AMax}(T,Q^{o},n_{i})$. In our
example $\Pi^{o}(n_{2})=(n_{3},L(n_{3}))$. We implement Definition 1 in a
procedure, called $\mathrm{Bimaximax}$, that propagates up the leaf utilities
at the parents, with a post-order tree traversal, and outputs $\Pi$,
$\Pi^{o}$. A _simulated dialogue procedure_ , Algorithm 1, consists in the
computation of the policies $\Pi,\Pi^{o}$ (line 2) and in alternating between
the proponent and opponent arguments (given by $\Pi,\Pi^{o}$) from the root to
a leaf (lines from 4 to 9). The output path $\bm{p}$ contains the
proponent/opponent nodes from the root to a leaf. In our example,
$\bm{p}=\langle n_{1},n_{2},n_{3},n_{6}\rangle$.

Algorithm 1 $\mathrm{SimDialogue}(T,L,\bm{u}^{p},\bm{u}^{o},\delta)$

Input: $T,L,\bm{u}^{p},\bm{u}^{o},\delta$
Output: The path
$\bm{p}=\langle\mathrm{root}(T),n_{1},\ldots,n_{0pt(T)}\rangle$

1: $\bm{p}:=\langle\rangle$

2: $\Pi,\Pi^{o}:=\mathrm{Bimaximax}(T,L,\bm{u}^{p},\bm{u}^{o},\delta)$

3: $n:=\mathrm{root}(T)$

4: while $\mathrm{isNotLeaf}(n)$ do

5: if $\mathrm{isDecision}(n)$ then

6: $\langle n,l_{n}\rangle:=\Pi(n)$

7: else

8: $\langle n,l_{n}\rangle:=\Pi^{o}(n)$

9: $\bm{p}.\mathrm{append}(n)$

10: return $\bm{p}.\mathrm{append}(n)$

Let $\bm{u}^{p}=\langle U^{p}(n_{1}),U^{p}(n_{2}),\ldots\rangle$ and
$\bm{u}^{o}=\langle U^{o}(n_{1})$, $U^{o}(n_{2}),\ldots\rangle$ be the
proponent and opponent utility vectors respectively, with $n_{i}\in{\sf
leaves}(T)$. The values of $\bm{u}^{p}$ can be authored by domain experts from
the substantial evidence in healthcare literature on the health benefits of
specific behaviour changes, e.g., the value of changing from high red meat
consumption to white meat consumption (Abete et al. 2014). Instead, the
setting of $\bm{u}^{o}$ is an open challenge. In the next section, we define
two ML methods able to predict an accurate estimation for $\bm{u}^{o}$ from a
dataset of users’ utilities. These methods can be used also in a dialogue
procedure with real users where the policy $\Pi^{o}$ will be replaced, for
example, by a menu in a smartphone app that implements the APS.

##  3 Machine Learning for Utility Prediction

Let $T$ be a DT, a corresponding utility dataset $\mathcal{U}^{o}$ contains
the utility values for a population sample. We encode $\mathcal{U}^{o}$ as a
matrix where each row $j$ contains the utility vector $\bm{u}^{o}_{j}$ of the
arguments in ${\sf leaves}(T)$ for a given user. Such a dataset can be
gathered by using, for example, utility elicitation techniques based on
questionnaires (Lenert et al. 1997; Lenert, Sherbourne, and Reyna 2001). Here,
we assume that $\mathcal{U}^{o}$ contains samples of subpopulations (as
commonly done in RSs), that is, clusters of users who have similar utilities.
For example, according to the leaf arguments in Figure 1, we can have three
subpopulations/clusters in $\mathcal{U}^{o}$: $C_{1}$, $C_{2}$ and $C_{3}$.
The users in $C_{1}$ like to eat food containing animal proteins, the users in
$C_{2}$ have difficulties changing their diet habits whereas the users in
$C_{3}$ are more open minded. Such a dataset can be gathered with cluster
sampling techniques (Turk and Borkowski 2005).

We develop two ML methods, trained on $\mathcal{U}^{o}$, for predicting an
estimation of the opponent utility vector $\hat{\bm{u}}^{o}$, for a given
user, as close as possible to the true vector $\bm{u}^{o}$. This implies
choosing some features of the users as input to our ML methods to predict
$\hat{\bm{u}}^{o}$. These can be demographic information or some arguments in
${\sf leaves}(T)$. We consider these input utilities as the _evidence_ to be
obtained in order to predict the rest of the utility vector. Therefore, the
predicted utility vector is composed from the evidence $\bm{u}_{1:e}^{o}$,
with $e\in[1,Le-1]$ and $Le=|{\sf leaves}(T)|$, and the predicted utilities:
$\hat{\bm{u}}_{e}^{o}=\langle\bm{u}_{1:e}^{o};\hat{\bm{u}}_{e+1:Le}^{o}\rangle$.
In our example, let us suppose that a user in cluster $C_{1}$ has true
utilities $\bm{u}^{o}=\langle 5,4,8,9\rangle$. A ML method, trained on
$\mathcal{U}^{o}$, can ask as evidence the utilities for $n_{5},n_{6}$ ($e=2$
out of 4 possible questions) to predict the utilities of $n_{7},n_{8}$: 9 and
7, for example. Therefore, $\hat{\bm{u}}_{2}^{o}=\langle 5,4,9,7\rangle$ and
this is a good prediction as the true path returned by $\mathrm{SimDialogue}$
is $\bm{p}=\langle n_{1},n_{2},n_{4},n_{8}\rangle$ whereas the path returned
by using $\hat{\bm{u}}_{2}^{o}$ is $\hat{\bm{p}}=\langle
n_{1},n_{2},n_{4},n_{7}\rangle$ that is quite similar to the true one. Indeed,
both $n_{7}$ and $n_{8}$ contain arguments related to the animal proteins
cluster.

Our proposal consists of providing $\mathrm{SimDialogue}$ with the opponent
utility vectors $\hat{\bm{u}}_{e}^{o}$ predicted with ML methods as input. The
evidence $e$ obtained from the user has to be enough to have a good dialogue
path $\hat{\bm{p}}$ but without asking too many questions of the user
(otherwise they may disengage). For example, asking 50 utilities of a DT with
60 leaves will bring reliable paths but would be too many questions. More
formally, let $\mathrm{ML}$ be a machine learning method trained on
$\mathcal{U}^{o}$ to compute the opponent utility vector
$\hat{\bm{u}}_{e}^{o}$ from $\bm{u}^{o}_{1:e}$, that is,
$\mathrm{ML}(\bm{u}^{o}_{1:e})=\hat{\bm{u}}_{e}^{o}$, and let $\mathcal{R}$ be
a regret function that measures the difference from the path returned by
$\mathrm{SimDialogue}$ with the one returned by $\mathrm{SimDialogue(ML)}$,
here the notation $\mathrm{SimDialogue(ML)}$ is a shortcut for
$\mathrm{SimDialogue}(T,L,\bm{u}^{p},\mathrm{ML}(\bm{u}^{o}_{j,1:e}),\delta))$,
more details in Section 4.2. Given a test set $\mathcal{U}_{TE}^{o}$, the
optimal number $e^{*}$ of evidence to ask about is:

$e^{*}=\operatorname*{argmin}_{e\in[1,Le-1]}\sum_{j=1}^{|\mathcal{U}_{TE}^{o}|}\mathcal{R}(\mathrm{SimDialogue}(T,L,\bm{u}^{p},\bm{u}_{j}^{o},\delta),\\\
\mathrm{SimDialogue}(T,L,\bm{u}^{p},\mathrm{ML}(\bm{u}^{o}_{j,1:e}),\delta))+\mathcal{E}(e)$
(1)

with $\mathcal{E}$ a monotonic-increasing function that models the user effort
in answering the asked evidence whose parameters can be estimated, for
example, by measuring the user disengagement rate. This is to avoid having an
APS that asks all the possible questions as evidence. The following
subsections present our ML methods for utility prediction.

###  3.1 Evidence as Amount of Information

An _evidence as amount of information_ (EAI) method uses a portion of the
arguments in $\mathcal{U}^{o}$ (limited by the evidence $e$) to train a
supervised regressor to predict the utility values of the other arguments.
This is the standard _regression_ task performed with _supervised learning_
techniques where a training set $\mathcal{X}\times\mathcal{Y}$ is used to
compute an estimator function $f$ for $\mathcal{Y}$. The set $\mathcal{X}$
contains feature vectors $\bm{x}$ whereas $\mathcal{Y}$ contains numeric
values $y$ to be estimated from $\bm{x}$. The training is performed such that
the output $\hat{y}$ of $f(\bm{x})$ has to be as close as possible to the true
value $y$. In our case, given an evidence index $e$, the set $\mathcal{X}$ is
given by the first $e$ arguments in $\mathcal{U}^{o}$ and $\mathcal{Y}$ is one
of the remaining arguments. More formally, the training set is
$\mathcal{U}^{o}_{*,1:e}\times\mathcal{U}^{o}_{*,e+l}$ with $l\in[1,Le-e]$.
The notation $\bm{A}_{*,i:j}$ indicates the columns from $i$ to $j$ (included)
of matrix $\bm{A}$. Therefore, for each evidence index $e\in[1,Le-1]$ multiple
regressions have to be performed from $\mathcal{U}^{o}_{*,1:e}$: one for
predicting the utility values for each remaining argument with index in
$[e+1,Le]$. As the ML method for computing $f$, we use a Support Vector
Regression $\mathrm{SVR}$ that linearly penalizes the predictions $\hat{y}$
that are at least $\epsilon$ away from the true value $y$. The advantage of an
EAI method is that many off-the-shelf ML methods for computing $f$ can be
used. However, such methods are restricted to the provided portion
$\mathcal{U}^{o}_{*,1:e}$ of the dataset to learn the other utility values,
without any possibility of automatically choosing a better set of arguments to
ask about. The next method addresses this issue. We opted for $\mathrm{SVR}$
(and $\mathrm{RF}$ in the next section) because it is a known-working
technique that does not require large training set to reach reasonable
performance. Our focus is not on the choice of the ML method but on the
usefulness of adopting ML for effective, data driven, argumentation.

###  3.2 Evidence as Depth of Searching

An _evidence as depth of searching_ (EDS) method has a limited amount of
evidence $e$ to be obtained from users, but, differently from the EAI-based
approach, the EDS method is given access to the whole dataset
$\mathcal{U}^{o}$. Therefore, the EDS method can search the most appropriate
evidence for the regression of the utility function. This search is limited by
the quantity $e$. We call the method Cluster and RAndom forest MEan Regressor
($\mathrm{CRAMER}$) and, in brief, it performs clustering on $\mathcal{U}^{o}$
to find subpopulations. Then a Random Forest (RF) is trained to learn the most
important utilities of arguments to ask in order to classify a new user into a
subpopulation. The utility vector $\hat{\bm{u}}_{e}^{o}$ is given by the asked
utilities and by the centroids of the subpopulation. We detail the method with
the running example.

Firstly, $\mathrm{CRAMER}$ performs clustering on $\mathcal{U}^{o}$ and
discovers the underlying clusters and computes their centroids. These are the
mean of the utilities of each user in the cluster. Numeric examples for the
centroids for $C_{1},C_{2}$ and $C_{3}$ are: $\bm{c}(C_{1})=\langle
5.0,4.0,9.2,9.1\rangle$, $\bm{c}(C_{2})=\langle 3.1,8.5,7.2,1.9\rangle$ and
$\bm{c}(C_{3})=\langle 8.1,8.6,2.7,7.3\rangle$. The second step of
$\mathrm{CRAMER}$ is to train a RF classifier on $\mathcal{U}^{o}$ to learn
from user utilities what are the main arguments that characterize a cluster.
RF classifiers require a max depth parameter for setting the maximum depth of
their classification trees, i.e., decision trees trained for classification
tasks. We set this with $e$, that is the number of arguments to be asked. A
high value for max depth could generate classification trees that overfit
$\mathcal{U}^{o}$ (with consequently low performance) and could be too
demanding for the users. In our example, if we set $e=2$, a random forest is
able to classify a new user by simply asking the utilities only for $n_{8}$
(fish) and $n_{7}$ (white meat). Indeed, high utilities for both these
arguments mean that the user is in $C_{1}$, whereas a low utility for the fish
argument classifies the user in $C_{2}$. A high utility for the fish and a low
one for the white meat argument bring the classification to $C_{3}$. Lastly,
the method joins the asked (true) utilities with the centroid (inferred)
utilities in the predicted cluster to obtain the utility vector
$\hat{\bm{u}}_{e}^{o}$ of the user. For example, the utility vector of a user
answering with utility value of 8 for the white meat argument and with utility
7 for the fish argument (cluster $C_{1}$) is $\hat{\bm{u}}_{2}^{o}=\langle
5,4,8,7\rangle$.

##  4 Empirical Evaluation

The aim of the evaluation is to test the ability of $\mathrm{SimDialogue(ML)}$
(prediction of $\hat{\bm{u}}^{o}$ with $\mathrm{ML}$ and consequent use of
$\hat{\bm{u}}^{o}$ in Algorithm 1) of returning persuasive arguments as good
as the ones returned by $\mathrm{SimDialogue}$. However, as big datasets of
APS dialogues with real users are not available, we test
$\mathrm{SimDialogue(ML)}$ on synthetic datasets with the use of simulations.
We use the output of the $\mathrm{SimDialogue}$ procedure as a _gold standard_
and evaluate how much the output of $\mathrm{SimDialogue(ML)}$ differs from
the gold standard. The source code and the supplementary material are online
at https://github.com/ivanDonadello/ML-Argument-Based-Computational-
Persuasion.

###  4.1 Simulation Design

We compare $\mathrm{SimDialogue}$ and $\mathrm{SimDialogue(ML)}$ on different
abstract DTs and population samples with their own utilities. Our aim is to
perform a fair comparison between the gold standard and
$\mathrm{SimDialogue(ML)}$ trying to avoid possible bias. Good simulation
results reveal possible good results in real cases too. The simulation steps
are as follows:

Trees Generation: given a tree height and a list of branching factors as
input, a random abstract DT is generated with a breadth-first algorithm. This
recursively generates random children of a node according to a branching
factor taken from the input list. This is repeated until the input height has
been reached. We repeat this process to obtain a set
$\mathbb{T}=\\{T_{1},T_{2},\ldots\\}$ of abstract DTs. As they have no
particular meaning, the labelling $L$ is not necessary. Each DT represents the
structure of a possible persuasion dialogue between an APS and a user.

Datasets Generation: for each $T\in\mathbb{T}$, we synthesize a set
$\mathbb{U}_{T}^{o}=\\{\mathcal{U}_{T,1}^{o},\mathcal{U}_{T,2}^{o},\ldots\\}$
of datasets of opponent utilities. Each dataset
$\mathcal{U}_{T,i}^{o}=\\{\bm{u}^{o}_{T,i}\\}_{j}$ represents samples of
subpopulations containing the utility vector $\bm{u}^{o}_{T,i,j}$ for a user
indexed by $j$ belonging to a particular subpopulation. This allows the EAI
and EDS methods to learn utility functions for subpopulations. Each dataset is
used for: i) training by procedure $\mathrm{SimDialogue(ML)}$, ii) testing by
both $\mathrm{SimDialogue(ML)}$ and $\mathrm{SimDialogue}$. Several datasets
of sobpopulations allow us to check the results of $\mathrm{SimDialogue(ML)}$
in a robust way. If we tested our method on only one dataset, the results
could be affected by the quality of that particular dataset.

We synthesize a dataset $\mathcal{U}_{T,i}^{o}$ by: i) creating clusters of
users; ii) assigning each user a utility vector $\bm{u}_{T,i,j}^{o}$ such that
users in the same cluster have similar utilities. Let $T$ be a DT,
$C\in\mathbb{N}$ be an input number of clusters and $CW\in\mathbb{R}$ be an
input center width, i) we sample $C$ center points (vectors with ${\sf
leaves}(T)$ dimensions) from a multivariate uniform distribution with values
in $[-CW,CW]$. These vectors are the centers of the clusters. ii) For each
center vector $\bm{c}$, we sample $m$ vectors (with ${\sf leaves}(T)$
dimensions) from a multivariate normal distribution with mean
$\bm{\mu}=\bm{c}$ and covariance matrix $\bm{\Sigma}=\sigma_{C}^{2}\bm{I}$,
with $\bm{I}$ the identity matrix and $\sigma_{C}^{2}$ the cluster variance
given as input. We sample the $m$ vectors such as every cluster has roughly
the same number of vectors. All these sampled vectors compose the dataset
$\mathcal{U}_{T,i}^{o}$.

Simulation Run: for each pair $\langle T,\mathcal{U}_{T,i}^{o}\rangle$ and for
a given evidence index $e\in[1,Le-1]$, we split $\mathcal{U}_{T,i}^{o}$ into a
training set, for training a given $\mathrm{ML}$ method, and a test set. Let
$j$ be the index of a sample in the test set, we run
$\mathrm{SimDialogue}(T,L,\bm{u}^{p},\mathrm{ML}(\bm{u}_{T,i,j,1:e}^{o}),\delta)$
and $\mathrm{SimDialogue}(T,L,\bm{u}^{p},\bm{u}_{T,i,j}^{o},\delta)$. Remember
that an EAI method takes a portion of the training set given by $e$, whereas
an EDS method takes the whole training set but can ask only $e$ questions. For
a statistical significance of the results, we use the $k$-fold cross
validation technique. The dataset $\mathcal{U}_{T,i}^{o}$ is split into $k$
parts, $k-1$ parts are used as training set for $\mathrm{SimDialogue(ML)}$ and
the remaining part is left as test set for both $\mathrm{SimDialogue(ML)}$ and
$\mathrm{SimDialogue}$. In this way, $k$ splits/folds of the original dataset
$\mathcal{U}_{T,i}^{o}$ are obtained and for each split we run both
$\mathrm{SimDialogue(ML)}$ and $\mathrm{SimDialogue}$.

Comparison: the paths $\hat{\bm{p}}_{T,i,j,e,k}$ and $\bm{p}_{T,i,j,k}$
returned by the $\mathrm{SimDialogue(ML)}$ and $\mathrm{SimDialogue}$
procedures, respectively, are compared according to the following metrics.

###  4.2 Metrics for the Empirical Evaluation

Given a user in $\mathcal{U}^{o}$, let $\hat{\bm{u}}_{e}^{o}$ be their
estimated utility vector and $\bm{u}^{o}$ be the true one. Given
$\hat{\bm{p}}=\mathrm{SimDialogue}(T,L,\bm{u}^{p},\hat{\bm{u}}_{e}^{o},\delta)$
and $\bm{p}=\mathrm{SimDialogue}(T,L,\bm{u}^{p},\bm{u}^{o},\delta)$ (hereafter
we remove the subscripts $T,i,e,k$ if not necessary), we consider the last
(leaf) nodes of these paths: $\hat{n}=\hat{\bm{p}}_{h}$, $n=\bm{p}_{h}$ with
$h$ the height of $T$, we call $n$ the _true node_ and $\hat{n}$ the
_predicted node_ and we compute the evaluation metrics by only comparing $n$
with $\hat{n}$. This strategy is motivated by the random choice of $n_{i}$
when $|{\sf AMax}(T,Q,n)|>1$: this randomness can easily select a different
branch in $T$ for $\mathrm{SimDialogue(ML)}$ w.r.t. $\mathrm{SimDialogue}$ for
the same utility vector. This side effect would highly penalize the
performance if we compared the whole paths resulting in a non-fair comparison.

The aim of the evaluation is to test the ability of $\mathrm{SimDialogue(ML)}$
to return similar nodes to $\mathrm{SimDialogue}$. This similarity can be
easily tested with the standard accuracy, i.e., whether $n_{j}=\hat{n}_{j}$
for a given user indexed with $j$ in $\mathcal{U}^{o}$. However, as random
choices can easily fail this testing, the similarity between $n_{j}$ and
$\hat{n}_{j}$ is relaxed by measuring an _argument distance_ between the
arguments in $n_{j}$ and in $\hat{n}_{j}$, respectively. This is the
$\mathcal{R}$ term in Equation (1). The nodes $n_{j}$ and $\hat{n}_{j}$ have a
small argument distance if the opponent and proponent utilities of the
argument in $n_{j}$ are close to the corresponding ones of the argument in
$\hat{n}_{j}$. Good performance would indicate that an APS based on
$\mathrm{SimDialogue(ML)}$ is able to provide arguments as similar as possible
(w.r.t. the utilities) to the gold standard arguments returned by
$\mathrm{SimDialogue}$. More formally, the mean argument distance ($Mad$) is
the mean, over all the dataset samples, of the Manhattan distance between the
proponent and opponent utilities of the true node and the predicted one:
$Mad=\frac{1}{|\mathcal{U}^{o}|}\sum_{j=1}^{|\mathcal{U}^{o}|}|U^{p}(n_{j})-U^{p}(\hat{n}_{j})|+|U^{o}(n_{j})-U^{o}(\hat{n}_{j})|.$
Notice that, if we decompose the mean argument distance according to the
proponent
($\frac{1}{|\mathcal{U}^{o}|}\sum_{j=1}^{|\mathcal{U}^{o}|}|U^{o}(n_{j})-U^{o}(\hat{n}_{j})|$)
and the opponent
($\frac{1}{|\mathcal{U}^{o}|}\sum_{j=1}^{|\mathcal{U}^{o}|}|U^{p}(n_{j})-U^{p}(\hat{n}_{j})|$)
dimensions we obtain the _mean absolute errors_ (Mae) which are standard
metrics for regression tasks. These metrics depend on $T,i,e,k$, therefore the
output of the simulations is a set of argument distances. These values need to
be aggregated to have a global estimation of the performance.

$\mathbf{k}$-fold aggregation The first aggregation is performed on the number
of folds $k$ of the $k$-fold cross validation. For each triple $\langle
T,i,e\rangle$ we compute the mean and the standard deviation of the metrics
over all the $k$ values obtaining the values $\mu(Mad)_{T,i,e}$ and
$\sigma(Mad)_{T,i,e}$.

Evidence aggregation The second aggregation regards the evidence $e$ requested
as input to users in order to compute the utility values of the remaining
arguments. We want to study the trend of the performance metrics with a
increasing amount of evidence. This amount is measured as the percentage of
asked utility arguments over all possible arguments/leaves of a DT. This
allows us to check whether there exists a percentage of evidence from which
the performance is acceptable. We therefore aggregate the mentioned means by
averaging over $\mathbb{T}$ and $\mathbb{U}_{T}^{o}$ according to the evidence
percentage. Let $Ep\in\\{0.1,0.2,\ldots 1.0\\}$ be the evidence percentage and
$E=\left\lceil{Ep\cdot(Le-1)})\right\rceil$ be the amount of utility of
arguments to ask for $T$ and for a given $Ep$, the mean argument distance
aggregation is the average of the set $\\{\mu(Mad)\\}_{T,i,e}$, with
$e\in\\{1,2,\ldots E\\}.$ The standard deviation of this aggregations is
computed from this set by estimating the pooled variance.

###  4.3 Simulation Setting

Some values of the parameters in the simulation are uniformly (randomly)
selected from a set of alternatives. This avoids the bias of having only one
value resulting in more general results. These are: the tree height in
$\\{4,5,6\\}$, the proponent utility in $\\{1,2,\ldots 11\\}$, the number of
clusters $C\in\\{4,6,8,10\\}$ and the center width
$CW\in\\{0.5,1.0,2.5,3.0\\}$. This setting for $CW$, allow us to have
different levels of clustering performance avoiding the bias of population
samples that are perfectly separable into clusters. The branching factor for
generating the children of a node is 2 or 3 for 90% of the time, 4 for 10% of
the time. This allows us to have realistic DTs with a reasonable number of
leaves. Other parameters have a single value: the number of synthetic trees
($|\mathbb{T}|=10$) and datasets ($|\mathbb{U}^{o}_{T}|=10$), the size of
$\mathcal{U}^{o}_{T,i}$ is 2000, the cluster variance $\sigma^{2}_{C}$ is 1.0,
the discount factor $\delta$ in $\mathrm{Bimaximax}$ is 1 as it is not
relevant for the simulations and $k=5$ for the $k$-fold cross validation. The
hyperparameters for $\mathrm{SVR}$ are $C=1$, $\epsilon=0.1$ and the radial
basis function as a kernel. For the clustering in $\mathrm{CRAMER}$ we used
KMeans with the number of clusters found through grid search in
$\\{4,6,8,10\\}$. The random forest in $\mathrm{CRAMER}$ has 100 estimators
with the minimum number of samples required: i) to split a node is 2, ii) to
be a leaf is 1.

###  4.4 Results

Figure 2: Performance of $\mathrm{SimDialogue(ML)}$. Results are aggregated
according to the trees, the datasets and the evidence. Vertical bars represent
the sampled standard deviation. Best viewed in colors.

We measure the performance of $\mathrm{SimDialogue(ML)}$ (with the average
mean argument distance) according to different levels of evidence percentage
$E_{p}$. This allows us to check: i) whether ML methods are able to learn
utility functions from data in comparison to non-ML baselines (RQ1) and ii)
whether there exists a percentage of evidence from which the performance are
acceptable (RQ2). We compare some baselines with the gold standard in the same
way done for $\mathrm{SimDialogue(ML)}$:

  * •

Random Regressor ($\mathrm{RandR}$) randomly selects integer utility values
between the maximum and the minimum utility for each argument in the columns
of of $\mathcal{U}^{o}_{*,e+1:Le}$.

  * •

Mean Regressor ($\mathrm{MeanR}$) computes the mathematical mean (over all the
users) of the utility values of each argument in the columns of
$\mathcal{U}^{o}_{*,e+1:Le}$.

  * •

CLUster and MEan Regressor ($\mathrm{CLUMER}$) performs clustering to find the
subpopulations in $\mathcal{U}^{o}_{*,1:e}$. Once a new user is assigned to a
cluster, the centroids of the clusters are used as utility values for the
remaining arguments in the columns of $\mathcal{U}^{o}_{*,e+1:Le}$.

Aggregated performance are reported in Figure 2 whereas Appendix A of the
supplementary material provides examples of performance for some random DTs
and datasets. As expected, the EAI method and the baselines increase their
performance as the evidence percentage $E_{p}$ increases. Indeed, more asked
questions allow more accurate performance. In addition, the increase of the
performance is also due to a robustness property of the bimaximax policy:
i.e., the system ability to capture regions of high user’s utility regardless
the other regions. If few leaves have a random utility most probably their
utility will not be the highest one. Indeed, during the bottom-up utility
propagation phase in $\mathrm{Bimaximax}$, the utility of these few leaves
will be discarded in favour of higher utilities from other branches as the
bimaximax policy selects the highest utility at each node. These few leaves do
not have the power to change the prediction of the true node. As $E_{p}$
increases, the number of random utilities decreases. This explains also the
unexpected performance improvement of $\mathrm{RandR}$ as $E_{p}$ increases.

Discussion for RQ1 The results for the argument distances are promising as
with 10% of evidence $\mathrm{SVR}$ obtains a $Mad$ of 2.8. This means that
the predicted node has proponent and opponent utilities that on average have
distance 1.4 from the true node. As the utility values range in a Likert scale
from 1 to 11, a 1.4 distance is an acceptable error. Therefore, the ML methods
are able to compute the utility values for the population sample by predicting
nodes that contain arguments that are very close (from the utilities point of
view) to the arguments contained in the true nodes.

Discussion for RQ2 An acceptable number of questions as evidence is not
trivial to identify. For example, 10% of evidence could present low
performance for small trees (e.g., with 20 leaves only 2 questions are asked)
therefore choosing 30% could represent a better choice. However, this could
result in a high amount of requested information (e.g., 30 questions for a
tree with 100). To this extent, $\mathrm{CRAMER}$ gives us a different view.
For this method, 10% of evidence is required to identify the subpopulation and
the utilities are the centroids of each subpopulation. Therefore, for a tree
with 20 leaves, a depth of 2 for the random forest is sufficient to identify 4
subpopulations. In general, a depth of $D$ identifies $2^{D}$ subpopulations.
It is important to notice that the level of depth of search is a _maximum_
level for the random forest, that is, the random forest can use less
information to identify subpopulations. This can be seen in the picture as the
performance of $\mathrm{CRAMER}$ are stable as the depth of search increases.
The EAI methods need 50% of evidence to have the same performance of
$\mathrm{CRAMER}$.

The advantage of an EDS method relies in its ability of automatically
selecting the most appropriate arguments to ask from $\mathcal{U}^{o}$
obtaining good performance with a low $E_{p}$. Differently, an EAI method has
only a partial view of the whole information but it increases the performance
with higher $E_{p}$. The drawback of $\mathrm{CRAMER}$ is its rough
computation of $\hat{\bm{u}_{e}^{o}}$ given a subpopulation. $\mathrm{SVR}$
instead has a more accurate computation based on the error optimization of the
predicted utility. This can be seen by its better performance with respect to
the EAI method $\mathrm{CLUMER}$. In addition, $\mathrm{CLUMER}$ and
$\mathrm{SVR}$ have similar performance as the former explicitly leverages the
cluster structure of the users, the latter achieves the same through the
underlying Gaussian kernel.

##  5 A Realistic Case Study

Figure 3: Performance of $\mathrm{SimDialogue(ML)}$ for the red meat case
study. Results are aggregated according to the $k$ folds. Vertical bars
represent the standard deviation. Best viewed in colors.

We evaluate $\mathrm{SimDialogue(ML)}$ on a realistic case study where the DT
contains real arguments (taken from the literature) regarding the reduction of
red meat consumption. Both the DT and the synthetic dataset are provided. This
is the first step for implementing an APS with a smartphone app for following
healthy lifestyles. This was commissioned by the Local Healthcare Department
of Trento (Italy), for the deployment in the Trentino area.

Regarding the DT, we gathered the arguments and their relations from diet and
behavior change literature (Abidi et al. 2014; Stankevitz et al. 2017). We
obtained a DT with 23 leaves and height of 4. Concerning $\bm{u}^{p}$, we
labeled each leaf with a topic describing the content of the arguments in the
corresponding path from the root. In our example, the path
$n_{1},n_{2},n_{4},n_{8}$ is labeled with “Fish as alternative”. Each topic is
assigned with a proponent utility according to the topic importance from a
diet point of view. Each leaf $n$ is assigned with the proponent utility of
the corresponding topic. These topics are: vegetarianism ($U^{p}(n)=8$), fish
as alternative ($U^{p}(n)=6$), white meat as alternative ($U^{p}(n)=4$),
thinking of alternatives ($U^{p}(n)=3$) and reducing red meat consumption
($U^{p}(n)=2$).

Regarding the dataset, we created 6 user profiles with their demographics
(sex, age, school degree, level of meat consumption and level of physical
activity) and utility values according to the profile, e.g, young student or
woman in career. Each profile is then transformed into a numeric vector. This
dataset of 6 examples is expanded to 2000 examples by repeating and shuffling
the 6 example vectors. Then a matrix of gaussian random noise ($\mu=0$,
$\sigma^{2}=1$) is added to the expanded dataset in order to have variability.
DT topics and dataset profiles are in Appendix C.

The evaluation has been performed as in the previous section. As there is only
one DT and dataset, we present the non-averaged metrics in Figure 3. This
realistic case study shows similar findings to the simulations. However,
differently from the simulations, here demographic information is available as
evidence for all the methods. The $Mad$ is good as with only the demographic
information we have a $Mad$ of around 1.7 for $\mathrm{CRAMER}$ and around 2.1
for both $\mathrm{SVR}$ and $\mathrm{CLUMER}$. Here, the simple clustering and
centroids approach of $\mathrm{CLUMER}$ is sufficient to achieve the same
results of $\mathrm{SVR}$. $\mathrm{CRAMER}$ presents stable performance. As
the number of questions increases both $\mathrm{CLUMER}$ and $\mathrm{SVR}$
require 5 questions (out of 22 possible questions, i.e., 23% more of
questions) to get a $Mad$ close to the one of $\mathrm{CRAMER}$. The mean
absolute error of the proponent utility is much lower compared to the opponent
one. This is due to a bias in the dataset as most of the final true nodes have
8 as proponent utility value. This bias does not regard the opponent utility,
see the statistic in Appendix D. A qualitative comparison of the arguments
returned by $\mathrm{CLUMER}$ and $\mathrm{SVR}$ for each profile is in
Appendix E.

##  6 Conclusion

In computational persuasion, Bi-party Decision Theory is a promising approach
for an APS to choose the best arguments to persuade a user. These strategies
are based on utility functions. However, no methods deal with the construction
of such functions for a new user in an efficient way. In this paper, we
addressed this problem by developing two ML models (EAI and EDS) to learn
these utility functions from datasets. The evaluation with simulations and
with a realistic case study show that both EAI and EDS are able to learn
utility functions of subpopulation of users with comparable performance.
However, EDS is more efficient as it requires less user information.

As future work, we want to test these methods with real users in a living lab.
In addition, a combination of EAI with EDS could improve the performance
requiring a lower number of questions to ask about. The use of neural networks
for learning the utilities will be studied also in an interactive APS that it
adapts the utilities as it interacts with the user (Dragone, Teso, and
Passerini 2017).

## Acknowledgements

The research of ST was partially supported by TAILOR, a project funded by EU
Horizon 2020 research and innovation programme under GA No 952215.

## References

  * Abete et al. (2014) Abete, I.; Romaguera, D.; Vieira, A. R.; de Munain, A. L.; and Norat, T. 2014.  Association between total, processed, red and white meat consumption and all-cause, CVD and IHD mortality: a meta-analysis of cohort studies.  _British Journal of Nutrition_ , 112(5): 762–775.
  * Abidi et al. (2014) Abidi, S.; Vallis, M.; Abidi, S. S. R.; Piccinini-Vallis, H.; and Imran, S. A. 2014\.  D-WISE: Diabetes Web-Centric Information and Support Environment: conceptual specification and proposed evaluation.  _Canadian Journal of Diabetes_ , 38(3): 205–211.
  * Alahmari (2020) Alahmari, S. 2020.  _Reinforcement Learning for Argumentation_.  Ph.D. thesis, University of York, York, UK.
  * Atkinson, Bench-Capon, and Bench-Capon (2012) Atkinson, K.; Bench-Capon, P.; and Bench-Capon, T. J. M. 2012.  Efficiency in Persuasion Dialogues.  In _ICAART (2)_ , 23–32. SciTePress.
  * Black, Coles, and Bernardini (2014) Black, E.; Coles, A. J.; and Bernardini, S. 2014.  Automated Planning of Simple Persuasion Dialogues.  In _CLIMA_ , volume 8624 of _Lecture Notes in Computer Science_ , 87–104. Springer.
  * Black, Coles, and Hampson (2017) Black, E.; Coles, A. J.; and Hampson, C. 2017.  Planning for Persuasion.  In _AAMAS_ , 933–942. ACM.
  * Chalaguine and Hunter (2020) Chalaguine, L.; and Hunter, A. 2020.  A Persuasive Chatbot Using a Crowd-Sourced Argument Graph and Concerns.  In _COMMA_ , volume 326 of _FAIA_ , 9–20. IOS Press.
  * Dragone, Teso, and Passerini (2017) Dragone, P.; Teso, S.; and Passerini, A. 2017.  Constructive Preference Elicitation.  _Frontiers Robotics AI_ , 4: 71.
  * Fan and Toni (2012) Fan, X.; and Toni, F. 2012.  Mechanism Design for Argumentation-based Persuasion.  In _COMMA_ , volume 245 of _Frontiers in Artificial Intelligence and Applications_ , 322–333. IOS Press.
  * Hadjinikolis et al. (2013) Hadjinikolis, C.; Siantos, Y.; Modgil, S.; Black, E.; and McBurney, P. 2013.  Opponent Modelling in Persuasion Dialogues.  In _IJCAI_ , 164–170. IJCAI/AAAI.
  * Hadoux et al. (2015) Hadoux, E.; Beynier, A.; Maudet, N.; Weng, P.; and Hunter, A. 2015.  Optimization of Probabilistic Argumentation with Markov Decision Models.  In _IJCAI_ , 2004–2010. AAAI Press.
  * Hadoux and Hunter (2019) Hadoux, E.; and Hunter, A. 2019.  Comfort or Safety? Gathering and Using the Concerns of a Participant for Better Persuasion.  _Argument & Computation_, 10: 113–147.
  * Hadoux, Hunter, and Polberg (2018) Hadoux, E.; Hunter, A.; and Polberg, S. 2018.  Biparty Decision Theory for Dialogical Argumentation.  In _COMMA_ , volume 305 of _Frontiers in Artificial Intelligence and Applications_ , 233–240. IOS Press.
  * Hunter et al. (2019) Hunter, A.; Chalaguine, L.; Czernuszenko, T.; Hadoux, E.; and Polberg, S. 2019.  Towards Computational Persuasion via Natural Language Argumentation Dialogues.  In _KI_ , volume 11793 of _LNCS_ , 18–33. Springer.
  * Hunter and Thimm (2016) Hunter, A.; and Thimm, M. 2016.  Optimization of Dialectical Outcomes in Dialogical Argumentation.  _International Journal of Approximate Reasoning_ , 78: 73–102.
  * Jannach et al. (2021) Jannach, D.; Manzoor, A.; Cai, W.; and Chen, L. 2021.  A Survey on Conversational Recommender Systems.  _ACM Comput. Surv._ , 54(5): 105:1–105:36.
  * Katsumi et al. (2018) Katsumi, H.; Hiraoka, T.; Yoshino, K.; Yamamoto, K.; Motoura, S.; Sadamasa, K.; and Nakamura, S. 2018.  Optimization of Information-Seeking Dialogue Strategy for Argumentation-Based Dialogue System.  In _Proceedings of DEEP-DIAL@AAAI’19_ , volume abs/1811.10728. ArXiv.
  * Lenert et al. (1997) Lenert, L.; Morss, S.; Goldstein, M. K.; Bergen, M.; Faustman, W.; and Garber, A. M. 1997.  Measurement of the validity of utility elicitations performed by computerized interview.  _Medical Care_ , 35(9): 915–920.
  * Lenert, Sherbourne, and Reyna (2001) Lenert, L. A.; Sherbourne, C. D.; and Reyna, V. 2001.  Utility elicitation using single-item questions compared with a computerized interview.  _Medical Decision Making_ , 21(2): 97–104.
  * Monteserin and Amandi (2013) Monteserin, A.; and Amandi, A. 2013.  A reinforcement learning approach to improve the argument selection effectiveness in argumentation-based negotiation.  _Expert Systems with Applications_ , 40: 2182–2188.
  * Osborne and Rubinstein (1994) Osborne, M.; and Rubinstein, A. 1994.  _A Course in Game Theory_.  MIT Press.
  * Peterson (2009) Peterson, M. 2009.  _An Introduction to Decision Theory_.  Cambridge University Press.
  * Rach, Minker, and Ultes (2018) Rach, N.; Minker, W.; and Ultes, S. 2018.  Markov Games for Persuasive Dialogue.  In _COMMA_ , volume 305 of _Frontiers in Artificial Intelligence and Applications_ , 213–220. IOS Press.
  * Rahwan and Larson (2008) Rahwan, I.; and Larson, K. 2008.  Pareto Optimality in Abstract Argumentation.  In _AAAI_ , 150–155. AAAI Press.
  * Rahwan, Larson, and Tohmé (2009) Rahwan, I.; Larson, K.; and Tohmé, F. A. 2009.  A Characterisation of Strategy-Proofness for Grounded Argumentation Semantics.  In _IJCAI_ , 251–256.
  * Ricci, Rokach, and Shapira (2015) Ricci, F.; Rokach, L.; and Shapira, B., eds. 2015.  _Recommender Systems Handbook_.  Springer.  ISBN 978-1-4899-7636-9.
  * Rienstra, Thimm, and Oren (2013) Rienstra, T.; Thimm, M.; and Oren, N. 2013.  Opponent Models with Uncertainty for Strategic Argumentation.  In _IJCAI_ , 332–338. IJCAI/AAAI.
  * Riveret et al. (2019) Riveret, R.; Gao, Y.; Governatori, G.; Rotolo, A.; Pitt, J.; and Sartor, G. 2019\.  A probabilistic argumentation framework for reinforcement learning agents - Towards a mentalistic approach to agent profiles.  _Autonomous Agents and Multi-Agent Systems_ , 33(1-2): 216–274.
  * Rosenfeld and Kraus (2016a) Rosenfeld, A.; and Kraus, S. 2016a.  Providing Arguments in Discussions on the Basis of the Prediction of Human Argumentative Behavior.  _ACM Transactions on Interactive Intelligent Systems_ , 6(4): 30:1–30:33.
  * Rosenfeld and Kraus (2016b) Rosenfeld, A.; and Kraus, S. 2016b.  Strategical Argumentative Agent for Human Persuasion.  In _ECAI_ , volume 285 of _Frontiers in Artificial Intelligence and Applications_ , 320–328. IOS Press.
  * Stankevitz et al. (2017) Stankevitz, K.; Dement, J.; Schoenfisch, A.; Joyner, J.; Clancy, S. M.; Stroo, M.; and Østbye, T. 2017.  Perceived barriers to healthy eating and physical activity among participants in a workplace obesity intervention.  _Journal of Occupational and Environmental Medicine_ , 59(8): 746–751.
  * Turk and Borkowski (2005) Turk, P.; and Borkowski, J. J. 2005.  A review of adaptive cluster sampling: 1990–2003.  _Environmental and Ecological Statistics_ , 12(1): 55–94.


[2010]Primary 35D40;
Secondary 60H30

CHAPTER: INTRODUCTION

CHAPTER: UNIQUENESS OF VISCOSITY
SOLUTIONS OF PARABOLIC PDES

CHAPTER: EXISTENCE OF VISCOSITY SOLUTIONS

CHAPTER: REGULARITY ESTIMATES

CHAPTER: APPLICATION

§ STOCHASTIC BURGERS EQUATIONS

§ STOCHASTIC 2-D NAVIER-STOKES EQUATIONS

§.§.§ Acknowledgements

This work has been partially supported by the Deutsche Forschungsgemeinschaft (DFG) via RTG 2131
"High-dimensional phenomena in probability - Fluctuations and discontinuity".

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

journaltitleElectronic Journal of Differential Equations
titleViscosity solutions of the Cauchy problem for second-order nonlinear partial differential equations in Hilbert spaces

Berlin Heidelberg

seriesLecture Notes in Mathematics
titleSecond Order PDE's in Finite and Infinite Dimension

journaltitlearXiv e-prints
titleLocal Lipschitz continuity in the initial value and strong completeness for nonlinear stochastic differential equations
Mathematics - Probability,60H10,60H15,34F05

abstractWe show that if a sequence of piecewise affine linear processes converges in the strong sense with a positive rate to a stochastic process that is strongly Hölder continuous in time, then this sequence converges in the strong sense even with respect to much stronger Hölder norms and the convergence rate is essentially reduced by the Hölder exponent. Our first application hereof establishes pathwise convergence rates for spectral Galerkin approximations of stochastic partial differential equations. Our second application derives strong convergence rates of multilevel Monte Carlo approximations of expectations of Banach-space-valued stochastic processes.
journaltitleIMA Journal of Numerical Analysis
titleConvergence in Hölder norms with applications to Monte Carlo methods in infinite dimensions

journaltitleBulletin of the American Mathematical Society
titleUser's guide to viscosity solutions of second order partial differential equations

Elsevier, Paris

journaltitleC. R. Acad. Sci., Paris, Sér. I
titleCondition d'unicité pour les solutions généralisées des équations de Hamilton-Jacobi du premier ordre.

Cambridge University Press

seriesEncyclopedia of Mathematics and its Applications
titleStochastic Equations in Infinite Dimensions

Basel [u.a.]

titleKolmogorov equations for stochastic PDEs
Stochastische partielle Differentialgleichung / Kolmogorovsche Differentialgleichungen

New York

Dover Publications

seriesDover books on advanced mathematics
titleAdvanced Calculus of Several Variables

journaltitleJournal of Mathematical Analysis and Applications
titleA numerical approach to Kolmogorov equation in high dimension based on Gaussian analysis
Kolmogorov equation,Numerical solution,Iteration schema,Gaussian process

Society for Industrial
Applied Mathematics

abstractThe paper is concerned with fully nonlinear second order Hamilton–Jacobi–Bellman–Isaacs equations of elliptic type in separable Hilbert spaces which have unbounded first and second order terms. The viscosity solution approach is adapted to the equations under consideration and the existence and uniqueness of viscosity solutions are proved. A stochastic optimal control problem driven by a parabolic stochastic PDE with control of Dirichlet type on the boundary is considered. It is proved that the value function of this problem is the unique viscosity solution of the associated Hamilton–Jacobi–Bellman equation.
journaltitleSIAM Journal on Control and Optimization
titleSecond Order Hamilton–Jacobi Equations in Hilbert Spaces and Stochastic Boundary Control
49l25 ; 93e20 ; 35r15 ; Stochastic Boundary Control ; Hamilton–Jacobi Equations ; Viscosity Solutions

journaltitleCommunications on Pure and Applied Mathematics
titleBellman equations associated to the optimal feedback control of stochastic navier-stokes equations

journaltitleProbability Theory and Related Fields
titleThe infinitesimal generator of the stochastic Burgers equation

journaltitleThe Annals of Probability
titleLoss of regularity for Kolmogorov equations

Berlin-New York

seriesLecture Notes in Mathematics
titleGeometric Theory of Semilinear Parabolic Equations

journaltitleCommunications in Partial Differential Equations
titleViscosity solutions of nonlinear second-order partial differential equations in Hilbert spaces

ETH Zürich

howpublishedLecture Notes
titleStochastic Partial Differential Equations: Analysis and Numerical Approximations
.ethz.ch/ grsam/NASPDE_MATH_FS15/script/NASPDE_130.pdf

ETH Zürich

howpublishedLecture Notes
titleStochastic Partial Differential Equations: Analysis and Numerical Approximations

titleSpatial Sobolev regularity for stochastic Burgers equations with additive trace class noise

titlePerturbation Theory for Linear Operators

journaltitleMathematische Annalen
titleÜber die analytischen Methoden in der Wahrscheinlichkeitsrechnung

journaltitleJournal of Functional Analysis
titleViscosity Solutions of Fully Nonlinear Second-Order Equations and Optimal Stochastic Control in Infinite Dimensions. III. Uniqueness of Viscosity Solutions for General Second-Order Equations

journaltitleJournal of Functional Analysis
titleSPDE in Hilbert space with locally monotone coefficients

Scuola Normale Superiore

titleInterpolation Theory

Springer-Verlag Berlin Heidelberg

seriesLecture Notes in Mathematics
titleMulti-Layer Potentials and Boundary Problems

Springer-Verlag Berlin Heidelberg

seriesSpringer Monographs in Mathematics
titleDirect Methods in the Theory of Elliptic Equations

booktitleStochastic partial differential equations and their applications
titleBackward stochastic differential equations and quasilinear parabolic partial differential equations

titleNonlinear Expectations and Stochastic Calculus under Uncertainty

journaltitleComptes Rendus Mathematique - C R MATH
titleA new approach to Kolmogorov equations in infinite dimensions and applications to the stochastic 2D Navier-Stokes equation

Institute of Mathematical Statistics

journaltitleThe Annals of Probability
titleKolmogorov equations in infinite dimensions: Well-posedness and regularity of solutions, with applications to stochastic generalized Burgers equations

New York

de Gruyter

seriesde Gruyter Series in Nonlinear Analysis and Applications; 3
titleSobolev Spaces of Fractional Order, Nemytskij Operators, and Nonlinear Partial Differential Equations

journaltitleJournal of the London Mathematical Society
titleOn Restrictions And Extensions Of The Besov And Triebel–Lizorkin Spaces With Respect To Lipschitz Domains

journaltitleMath. Ann.
titleOptimization of Functions on Certain Subsets of Banach Spaces

Amsterdam, New York, Oxford

North-Holland Publishing Company

titleInterpolation Theory, Function Spaces, Differential Operators

journaltitleJournal of Functional Analysis
titleStochastic evolution equations in UMD Banach spaces

The University of Chicago Press

titleEssential Results of Functional Analysis