# When Is It Acceptable to Break the Rules? Knowledge Representation of Moral
Judgement Based on Empirical Data

Edmond Awad<EMAIL_ADDRESS>
University of Exeter, UK
Sydney Levine<EMAIL_ADDRESS>
Massachusetts Institute of Technology and Harvard University, USA
Andrea Loreggia<EMAIL_ADDRESS>
European University Institute, Italy
Nicholas Mattei<EMAIL_ADDRESS>
Tulane University, USA
Iyad Rahwan<EMAIL_ADDRESS>
Center for Humans & Machines, Max Planck Institute for Human Development,
Germany
Francesca Rossi<EMAIL_ADDRESS>
IBM Research, USA
Kartik Talamadupula<EMAIL_ADDRESS>
IBM Research, USA
Joshua Tenenbaum<EMAIL_ADDRESS>
Massachusetts Institute of Technology, USA
Max Kleiman-Weiner<EMAIL_ADDRESS>
Massachusetts Institute of Technology, USA

###### Abstract

One of the most remarkable things about the human moral mind is its
flexibility. We can make moral judgments about cases we have never seen
before. We can decide that pre-established rules should be broken. We can
invent novel rules on the fly. Capturing this flexibility is one of the
central challenges in developing AI systems that can interpret and produce
human-like moral judgment. This paper details the results of a study of real-
world decision makers who judge whether it is acceptable to break a well-
established norm: “no cutting in line.” We gather data on how human
participants judge the acceptability of line-cutting in a range of scenarios.
Then, in order to effectively embed these reasoning capabilities into a
machine, we propose a method for modeling them using a preference-based
structure, which captures a novel modification to standard “dual process”
theories of moral judgment.

##  1 Introduction

One of the most remarkable things about the human moral mind is its
flexibility: we can make moral judgments about cases we have never seen
before. Yet, on the face of things, morality often seems highly rigid. Legal
codes, employee handbooks, and IRB training manuals all seem to suggest that
morality is a system of clearly defined rules. Indeed, the past few decades of
research in moral psychology has revealed that human moral judgment is (at
least partially) driven by rules (?, ?, ?, ?, ?, ?). But sometimes, people
agree that it is morally appropriate to break the rules. And sometimes, new
rules need to be created when none exist to govern the case at hand. The field
of moral psychology is just now beginning to explore and understand this kind
of flexibility – and there are scant attempts to describe this capacity of the
moral mind in computational terms (for a notable exception see (?)).

Meanwhile, the flexibility of the human moral mind poses a challenge for AI
engineers. AI systems that dynamically navigate the human world will sometimes
need to predict and produce human-like moral judgments. Yet current tools for
building AI systems fall short of capturing moral flexibility. In order to
build artificial agents that can work effectively with humans, we must equip
them with knowledge about how humans make moral judgements (?). Hence, AI
researchers must engage with the study of human decision makers to understand
how they think, and provide a potential formal reasoning methodology to
validate and test modeling the results (?).

The goal of this paper is two-fold. First, we aim to computationally
characterize the way that humans both follow and break moral rules in everyday
settings, adding to the literature on the computational modeling of moral
flexibility. Second, we aim to capture this pattern of judgments through a
novel generalization of a common formalism – CP-nets (?) – suggesting a way
forward for the development of morally competent machines. Note that we
consider this work to be a contribution to descriptive ethics, i.e., what
people think is morally right or wrong, as opposed to normative ethics, i.e.,
what is _actually_ right or wrong.

Our guide in this project will be dual process theories of moral judgment,
which propose that human moral judgment can be understood as a combination of
_fast_ , heuristic-based thinking and _slow_ , deliberative thinking (?, ?).
Our proposal is that the combination of these two modes of thinking enables
human moral flexibility, specifically the ability to figure out when pre-
established rules should be broken.

When we follow pre-established moral rules (i.e.: heuristics), we are often
using fast thinking. In contrast, when we are careful and deliberate in our
moral decision-making, we are using slow thinking (?, ?). Up until now, slow
thinking has been invoked almost exclusively to explain how people incorporate
utility-maximization into their moral judgments. We propose that slow thinking
can also enable moral flexibility; _humans use slow thinking to figure out
when the rules should be overridden or when new rules need to be created_.
Moreover, we argue that there are two kinds of slow thinking: outcome-based
thinking and agreement-based thinking, which we detail below. Capturing this
dual process model of moral flexibility in CP-nets is the engineering
challenge that this paper attempts to tackle.

##  2 The Importance of Embedding Ethics Principles in AI Systems

The idea of teaching machines right from wrong has become an important
research topic in both AI (?) and related fields (?). Our goal is to formalize
and understand how to build AI agents that can act in ways that are morally
acceptable to humans (?, ?, ?). This challenge has been pursued using a range
of computer science approaches, including taking sequences of actions in a
reactive environment (?) and teaching agents how to respond in specific
environments (?). Many of these projects address what is called the _value-
alignment_ problem (?), that is, the problem of building machines that behave
according to values aligned with human ones (?, ?, ?, ?, ?).

Concerns about the ways in which autonomous decision making systems behave
when deployed in the real world are growing in society. Stakeholders worry
about AI systems achieving goals in ways that are morally unacceptable to the
community impacted by the decisions, also called “specification gaming”
behaviors (?). Thus, there is a growing need to understand how to constrain
the actions of an AI system by providing boundaries within which the system
must operate. To tackle this problem, it is useful to take inspiration from
humans, who often constrain the decisions and actions they take according to a
number of exogenous priorities, be they moral, ethical, religious, or business
values (?, ?, ?), and we may want the systems we build to be restricted in
their actions by similar principles (?). The overriding concern is that the
agents we construct may not obey these values while maximizing some objective
function111https://www.wired.com/story/when-bots-teach-themselves-to-cheat/ -
May, 31st 2021 (?). But humans are also experts at figuring out when the
constraints can and should be broken. This is a hallmark of human moral
flexibility – humans can figure out when simple rules should be overridden.
Can we enable AI to do the same?

There are two popular techniques to encoding constraints into AI systems:
bottom-up and top-down (?). A bottom-up approach involves teaching a machine
what is right and wrong by providing examples of “correct” decisions, e.g.,
providing a reinforcement learning system with demonstrations from humans in a
setting such as self driving cars or exploring mazes (?, ?, ?). In top-down
approaches, behavioral guidelines are specified by explicit rules or
constraints on the decisions space, e.g., by making a rule that a self-driving
car must never strike a human (?). However, both of these kinds of models will
struggle to determine when constraints should be overridden. Top-down
approaches will err when the rules or constraints on the system are too
general to deal with a particular edge case or unusual circumstance. Bottom-up
approaches will err when a case that should be an exception to a rule differs
dramatically from anything in the training set.

In this paper, we describe cases where humans think it is acceptable to
violate a rule. We then propose a formalism that will help the AI community
move forward in encoding this kind of moral flexibility into AI systems.

##  3 Fast and Slow Thinking Guide Moral Judgment

###  3.1 Brief Overview of Previous Work

Many human decision-making processes can be characterized by the System
1/System 2 model of cognition, often known as the “thinking fast and slow”
approach, (?). This model describes two broad reasoning approaches: (1)
thinking fast, or System 1, which relies on predefined heuristics or rules,
and (2) thinking slow, or System 2, that is more deliberate. Each of these
systems has their merits. System 1 thinking is efficient, requiring only
limited computational resources and contextual information. When System 1
heuristics are deployed in the environments they were intended for, they often
produce good-enough decisions; though they sometimes can lead to sub-optimal
choices when deployed in edge cases. System 2 decision-processes, on the other
hand, are cognitively intensive and allow the decision-maker to flexibly
integrate information from disparate sources, leading to decisions that can be
carefully tailored to the case at hand. These decision-processes can be used
in combination in a way that optimizes payoffs given the limitations of the
computational resources available (?, ?, ?).

The System 1/System 2 model of decision-making has a well-known analogue in
the moral psychology literature, known the “dual process” model of moral
judgment (?, ?, ?, ?). Dual process models argue that we make moral judgments
using a combination of heuristic-like rules (System 1-style reasoning) and
more deliberative System 2 methods. The dual process models draw their
inspiration from two of the major branches of moral philosophy. The rule-based
System 1 is inspired by _deontological_ theories of moral philosophy, which
focus on constraints on actions. The deliberative System 2 is typically
associated with _consequentialist_ theories, which focus on maximizing the
utility of outcomes. System 1 applies a hard and fast rule and does not
consider the subtleties or complexities of the current scenario. In contrast,
System 2 acts more like a utilitarian reasoner, it attempts to quantify
utility losses and gains, which enable it to make a judgment. For example,
when Bob refrains from taking something he cannot afford from a store even
though he really wants it, he may be following an intuitive moral rule: no
stealing. When Susan decides to donate $5 to buy mosquito nets rather than
chocolate bars for children in crisis, she may be using System 2 reasoning and
comparing the values of the outcomes of the candidate actions.

###  3.2 A New Kind of Slow Thinking: Contractualism

We mentioned above that the central ideas of two of the main branches of moral
philosophy (rules and outcomes) are represented in theories of moral
psychology. Curiously, the central idea of the third major family of
philosophical views – _contractualism_ – has been virtually absent from
theories of moral psychology. Contractualist views ask us to consider what
agreements could be adopted that would lead to mutual benefit (?, ?, ?, ?).
Recent work has begun to show that, despite being neglected for so long,
contractualist mechanisms play a critical role in the moral mind (?, ?, ?).

The specific agreement-based process we focus on in this paper is
_universalization_ , which is a way of making a moral decision by asking “what
if everyone felt at liberty to do that?” (?). When people universalize, they
imagine a hypothetical world where everyone is allowed to act in a certain
way. If things go well in that hypothetical world, the action in question
should be allowed. If things go badly, the action should be prohibited. This
counts as agreement-based because it is a way of asking if any person is
taking a special privilege for themselves that they couldn’t grant to
everyone. If everyone could feel free to do the action, then presumably mutual
agreement could be reached that the action is permitted. Universalization is
likely to be a System 2 process because running the universalization
computation is resource-intensive and requires a lot of information about the
particular decision-making context (?). However, more empirical work needs to
be done to be certain that this is the case.

###  3.3 Fast and Slow Thinking Explain Moral Flexibility

We propose that System 1 processes enables rule-following behavior, as is
widely agreed upon in the literature. In contrast, System 2 processes can
enable moral flexibility. Specifically, humans use System 2-thinking to figure
out when the rules should be overridden. Moreover, we argue that there are two
kinds of System 2 thinking: outcome-based thinking and agreement-based
thinking.

Sometimes people decide that it is morally acceptable to break previously
established rules because doing so would bring about a better outcome than
following the rule (?, ?). In general, there is a rule against non-consensual
harmful contact, i.e., no hitting. But if pushing someone out of the way of a
speeding train could save their life, then it is morally permissible to do so.
Yet in other cases, people may decide that it is morally acceptable to break a
rule because the person whom the rule protects would agree to have the rule
violated (?). There is rule against taking things that don’t belong to you
(or: no stealing), but it may be OK to take coffee grounds from your co-
worker’s desk if you know they would consent to it, given that they are likely
to be caught in a similar situation in the future and want to take some coffee
from you.

In sum, we propose that both contractualist (agreement-based) and
consequentialist (outcome-based) System 2 reasoning help humans figure out
when it is acceptable to override the rules. We aim to be able to capture this
computationally, as described in Section 5 below.

###  3.4 Towards a Psychological Triple Theory

There have been various attempts by philosophers to unite the three major
threads of moral philosophy (outcomes, rules, and agreement) into a single
unified view (?, ?). Parfit called his unified view a “Triple Theory”. To
date, no psychological theory has attempted to explain how rules, outcomes,
and agreement are all integrated in the moral mind. Our work is one step on
the way to creating a Psychological Triple Theory.

##  4 Our Paradigm: Waiting in Line

Our paradigm involves asking our subjects when it is morally acceptable to
break simple and generally agreed upon rules. We depart from the typical work
in this area which has often focused on high-stakes judgments in somewhat
uncommon scenarios, e.g., a runaway trolley headed towards innocents (?), and
we rather probe people’s moral intuitions in a very common everyday scenarios:
standing in line to receive a service. Intuitively, it seems like an easy feat
to figure out how to wait in line. In fact, one simple widely agreed rule
governs the process of waiting in line: each person in line is helped in the
order that they arrive (first-in-first-out). If true, navigating a situation
that requires waiting in line would simply involve getting in the back of the
line and waiting your turn.

However, a few moments’ reflection reveals that we can intuitively evaluate
all kinds of exceptions to this seemingly simple rule about waiting in line.
For example, say that you are in a deli and have waited in line and ordered a
bowl of soup, but just as you are about to begin eating the soup, your spoon
falls on the floor. It is probably acceptable for you to get a new spoon
without waiting in line. Or say that you just want a glass of tap-water. Here,
too, usually you are allowed to cut to the front of the line without waiting.
Or if you are assisting someone who has just fallen off a bike outside, you
can probably purchase a bottle of water without waiting in line. On the other
hand, it is probably not acceptable to cut to the front of the line to order a
soda, even though that might delay the line just as much as the person
requesting tap water. How do humans figure out when it is acceptable to cut in
line and when it’s not? And what kind of reasoning do people use in making
their judgments? We propose that System 2 reasoning, which typically involves
utility calculations and/or agreement-based reasoning, helps subjects figure
out when it is OK to break the rules.

We use three different contexts for waiting in line: a line at a deli, at an
airport, and at a bathroom. In each context, we present participants with a
series of reasons why someone may want to cut the line. We ask subjects to
judge whether it is acceptable to cut in line in the cases described. We also
ask subjects to evaluate each scenario on a series of other measures, which
tap into the underlying cognitive mechanisms that are driving their moral
judgments. For instance, we ask subjects to estimate how long the cutter would
delay the line, the benefit to the cutter, the detriment to the line, and so
forth. Using these metrics, we can generate an expected utility calculation
for the action. We also ask subjects what would happen if this type of line-
cutting were always allowed, a proxy for whether everyone would agree to allow
this person to cut (?). We also code each of these cases for obvious instances
of a rule violation, where the rule about cutting in line is construed in
simple terms, e.g., you must wait in line at a deli if you intend to buy
something. Full experimental details are provided in Section 6.

Each scenario is modelled in our computational framework through a set of
_scenario variables_ , such as the location or the reason to ask to cut the
line, and gives rise to a number of descriptive _evaluation variables_
dependent on the scenario variables. We then model this data in a preference-
based framework that links scenarios, evaluation variables, and moral
judgement, claiming it can serve as a central modeling and reasoning framework
for human moral reasoning and decision making.

##  5 Our Computational Approach

###  5.1 A Computational Approach to Moral Judgment is Critical for AI
Development

The vast majority of work characterizing human moral decision-making, both in
psychology (?, ?) and experimental philosophy (?, ?) has focused on
identifying factors that are relevant to moral judgment (e.g., affect, rules,
utility calculations). Our proposal contributes to an emerging body of work
that goes beyond simply identifying these factors, but also seeks to
characterize _in computational terms_ the mechanisms underlying moral
judgments (?, ?, ?, ?, ?, ?). Describing the mind in computational terms is a
critical step in building AI that can produce and interpret human moral
judgment (?, ?, ?). Moreover, our work goes even further and describes the way
that current methods of implementing morality in AI systems would need to be
modified to capture the computational mechanisms we describe.

###  5.2 CP-Nets: A Preference-Based Approach

Decisions and actions are linked to the concept of preferences: we choose one
among all possible decisions because we prefer the resulting state-of-affairs
over the ones generated by the other decisions. The issue of modelling and
reasoning with preferences in an artificial agent has been the subject of a
very active research area for many years. Several frameworks have been
defined, and their properties studied, for example related to expressivity,
computational complexity, and easiness of preference elicitation (?). The
centrality of preferences is true also in the case of moral judgement: we
consider a decision more morally acceptable than another one if its impact on
others is preferred according to our moral values (?, ?). Finding a way to
model and reason with such values, and corresponding preferences, is central
to build artificial agents that behave in a way that is aligned to humans
values (?).

In this work, we specifically investigate the conditions under which humans
decide to break previously established rules in multi-agent scenarios. We then
use a preference-based framework to to model and embed the underlying
processes they use into a machine. We exploit an existing preference modelling
and reasoning framework and we generalize it to include also scenario and
evaluation variables. Conditional Preference networks (CP-nets) are a
graphical model for compactly representing conditional and qualitative
preferences (?). CP-nets are comprised of sets of ceteris paribus preference
statements (cp-statements). For instance, the cp-statement, “I prefer red wine
to white wine if meat is served,” asserts that, given two meals that differ
only in the kind of wine served and both containing meat, the meal with red
wine is preferable to the meal with white wine. CP-nets have been extensively
used in the preference reasoning, preference learning, and social choice
literature as a formalism for working with qualitative preferences (?, ?, ?).
CP-nets have even been used to compose web services (?) and other decision aid
systems (?). While there are many formalisms to choose from when modeling
preferences, we focus on CP-nets as they are graphical and intuitive.
Transforming the results of our experiments into other formalisms is an
interesting and important direction for future work.

Label |  Scenario Description |  Main Func. |  Already Waited
---|---|---|---
Deli Scenarios |  |  |
Spoon |  A customer who is eating soup at the deli dropped his spoon on the floor and needs another one. |  False |  True
Water |  A customer who is eating lunch at the deli wants more a refill on tap water. |  False |  True
Soda |  A customer who is eating lunch at the deli wants to buy another soda. |  True |  True
Catering Order |  A customer wants to ask a series of questions about a catering order that he will pick up next week. |  False |  False
Fasted |  A customer walks in who has just finished fasting for 24 hours for a colonoscopy and is extremely hungry. |  True |  False
Diabetic |  A customer walks in who is diabetic and urgently needs sugar. |  True |  False
Oven Repair |  The oven-repair technician shows up and needs to ask the cashier a series of questions about the oven so he can fix it. |  False |  False
Soap |  A customer uses the last of the hand soap in the bathroom. |  False |  False
Toilet Paper |  A customer notices that the bathroom is out of toilet paper. |  False |  False
Spouse |  A customer walks in who is married to a customer who is currently placing an order with the cashier. |  True |  False
Father |  The father of the family is currently placing an order with the cashier. |  True |  False
Sandwich |  A customer walks in who wants to order a sandwich. |  True |  False
Bathroom Scenarios |  |  |
Wash Hands |  Someone at the back of the line just needs to wash their hands. |  False |  False
Cleaner |  Someone arrives who needs to clean the bathroom. |  False |  False
Vomit |  Someone at the back of the line needs to throw up immediately. |  True |  False
Get Jacket |  Someone at the back of the line thinks they forgot their jacket in the bathroom. |  False |  True
Friend |  Someone at the back of the line is a friend of someone at the front of the line. |  True |  False
Aid |  Someone at the back of the line is an aid to an elderly person at the front of the line. |  True |  False
Use Bathroom |  Someone at the back of the line needs to use the bathroom. |  True |  False
Airport Scenarios |  |  |
Departure in 20min |  The flight leaves in 20 minutes. |  True |  False
Crying Baby |  Standing with a baby who is crying very loudly. |  True |  False
Forgot Jacket |  Forgot their jacket at the check-in counter. |  False |  True
Cafe Worker |  Works at a cafe inside the airport. |  False |  False
Go to Bathroom |  Has to leave the line to go to the bathroom. |  True |  True
Departure in 3h |  The flight leaves in 3 hours. |  True |  False
Table 1: Description of all 25 scenarios used in our experiments. Variable |  Prompt
---|---
Global Welfare |  Think about the well-being of all the people in line combined. How are they affected by the person cutting in line?
First Person Welfare |  How much worse off/better off is the first person in line?
Middle Person Welfare |  How much worse off/better off is a person standing in the middle of the line?
Last Person Welfare |  How much worse off/better off is the last person in line?
Line Cutter Welfare |  How much worse off/better off is the person that cut in line?
Universalization |  Think about the person who cut in line. How much worse off/better off would it be for people who come to the deli if everyone who was in this situation cut in line?
Likelihood |  On any given day, how likely is it that this scenario going to happen?
Judgement |  Is it acceptable to cut the line? (yes or no).
Table 2: Questions that each respondent was asked in order to evaluate each
possible scenario.

##  6 Experimental Details

To gain insight into how humans make moral judgements we ran an experimental
study on Amazon MTurk. Informed consent was given by all participants. This
study was approved by the Massachusetts Institute of Technology IRB. 407
subjects participated in the study. Following attention checks, the data from
301 subjects was retained for analysis. Subjects were randomly assigned to one
of three story contexts:

Deli.

waiting in line at a deli (12 scenarios);

Bathroom.

waiting in line for the bathroom (7 scenarios);

Airport.

waiting in line at the airport security screening (6 scenarios).

Participants were asked to read a set of short vignettes about people who
wanted to cut in line for a range of different reasons. For example: “Imagine
that there are five people who are waiting in line at a deli to order
sandwiches for lunch. There is only one person (the cashier) working at the
deli. A customer who is eating soup at the deli dropped his spoon on the floor
and needs another one. Is it OK for that person to ask the cashier for a new
spoon without waiting in line?” Subjects were allowed to answer “yes” or “no”.
Complete descriptions of the scenarios are given in Table 1.

Subjects were also asked to evaluate each scenario on a range of different
metrics which might correspond to the cognitive mechanisms underlying their
moral judgments. The full list of evaluation questions is given in Table 2.
Subjects responded on a scale of -50 (a lot worse off), 0 (not affected), +50
(a lot better off); likelihood was judged between 0 (not likely) and 100 (very
likely).

Half the subjects were shown the evaluation questions for all the scenarios
followed by the permissibility questions for all the scenarios; the other half
of the subjects received the blocks of questions in the reverse order. This
was designed to test, and eliminate if necessary, the effects of evaluation
then judgement versus judgement then evaluation.

Variable |  Scenario Description
---|---
Reason |  The particular reason for cutting the line, see Table 1.
Location |  The particular location of the scenario: _Deli_ , _Airport_ , or __.
Main Service |  Whether or not the individual cutting the line had the goal of accessing the main service.
Table 3: Variables that went in to constructing each scenario.

Finally, we coded each vignette for the presence of a possible confounding
feature: whether or not the person attempting to cut had the goal of accessing
the main service the line was providing. The main service for the deli line
was the sale of an item, for the airport scenario it was security screening,
and for the bathroom it was the use of the toilet. Note that defining what
counts as the “main service” being provided to the line is open to
interpretation. We think that variation in this interpretation likely impacts
subjects’ judgments about the acceptability of cutting in line. For instance,
one might view the main service of the deli line as receiving anything from
the cashier (rather than purchasing something). Our characterizations of the
main function of the line are rough approximations meant to describe one view
that seems commonly held. These coding can be found in Table 1.

At the end of the survey, participants were given an attention check as
follows: “Finally, we are interested in learning some facts about you to see
how our survey respondents answer questions differently from each other. This
is an attention check. If you are reading this, please do not answer this
question (do not check any of the boxes). Instead, in the box below labeled
‘Other’ please write ‘I am paying attention’. Thanks very much!” This was
followed with a list of levels of education and a free-response box labeled
“Other.” Participants who checked any box or failed to write ”I am paying
attention” in the appropriate place were screened out of the study.

Figure 1:  Moral judgements about cutting in line in each of the scenarios.
Color indicates if the person cutting in line is requesting the main service
or not (blue for Yes, red for No). Error bars are $95\%$ confidence intervals.
As we can see, a simple rule, such as “it is ok to cut if you are not
requesting the main service” is not sufficient to explain variation.

##  7 Data Analysis

###  7.1 Moral Judgments

Figure 2: Box-plots depicting the distribution of responses to the evaluation
questions, polled across scenarios.

Figure 1 shows overall moral permissibility data (i.e., ok or not-ok judgment)
for each of the scenarios. The most notable feature of this data is that moral
permissibility is graded; cutting the line is endorsed probabilistically by
our subjects rather than in an all-or-none fashion. This is the first hint
that our subjects are not using a simple rule to figure out when it is
permissible to cut in line. A rule like “don’t cut” for instance, would
produce unanimously low permissibility for all cases.

A slightly more sophisticated version of this rule-based approach would be
that subjects use the rule “don’t cut” but realize that the rule is not
operative in certain scenarios. This would yield a slightly more complex rule
such as “cutting is allowed only when you are not requesting the main
service.” (See Figure 1.) This would yield a binary all-or-none pattern of
results, with some instances of cutting being permissible and some not.
Instead, it seems that a more sophisticated understanding of the computations
behind subjects’ moral judgments is required. The aim of our model is to be
able to predict our participants’ graded permissibility judgments.

###  7.2 Checking for Order Effects

First, we checked if order (whether subjects were asked the evaluation
questions first and then asked to make a moral judgement, or the opposite) had
any effect on moral judgments. To test this we ran a Wilcoxon signed-rank test
against the null hypothesis that _changing the order of evaluation and
judgement does not influence the judgement value_. We ran this for all 25
scenarios described in Table 1. The test did not reject the null hypothesis
for any of the 25 scenarios. Hence, under the conditions of this study, asking
individuals to think closely about the evaluation questions did not cause them
to change their judgements. We conjecture that this happens because people
already went through the internal evaluation process when they made a
decision. Thus asking for it before or after does not influence the decision
they made. We therefore pooled subjects in both order conditions for the
remaining analyses.

Figure 3: Cross-correlation matrix for all scenarios. The moral judgement is
labeled as PREF and is negatively correlated with all the evaluating metrics,
indicating that as any measure gets worse, the moral permissibility goes down.

###  7.3 Evaluation Questions

Figure 2 depicts the distribution of responses by all participants for the
evaluation questions.We conjectured that the different evaluation variables
might draw out importantly different elements of the scenarios, allowing for
the emergence of a better predictor by using multiple variables. For instance,
the well-being of the first person could matter most because she has the most
to lose if someone cuts in line, i.e., her change in wait time is
proportionally greatest. On the other hand, it might emerge that the last
person is most important because waiting additional time might actually make
it no longer worthwhile to wait at all. Furthermore, _Global Welfare_ or
_Middle Person Welfare_ might be the best because they provide an aggregate
estimate or average estimate. As it turns out, people do not judge these
metrics differently (See Fig.2). Additionally, there is a fair amount of
negative skew for the question _Line Cutter Welfare_ , indicating that some
participants felt that even if cutting the line was allowed, they were not
receiving much benefit.

###  7.4 Relationship Between Evaluation Questions and Moral Judgment

Finally, we wanted to see if there were any strong correlations between the
various evaluation questions and the moral judgement. Figure 3 shows the
cross-correlation between all responses from the subjects. As one might
expect, the questions about the individuals in line are highly correlated,
further indicating that most subjects respond to these questions with similar
evaluations. In addition, as predicted, all evaluation variables are
negatively correlated with the moral judgement variable, indicating that as
there are more negative impacts of the action, the less likely it is judged
permissible to cut in line. Universality has the strongest negative
correlation with the moral judgement variable as well, indicating that
participants seem to consider the question “what if everyone did this” (the
System 2 contractualist method of reasoning) when deciding if it was OK to
cut.

##  8 Modelling and Reasoning with Preferences

According to Sen, moral judgements are a form of preferences, driven by moral
reasoning (?). The issue of modelling and reasoning with preferences in an AI
system has been the subject of a very active research area for many years, and
has produced many frameworks to deal with preferences and embed them into an
AI decision making system. Different frameworks differ on properties related,
for example, to expressivity, computational complexity, and easiness of
preference elicitation (?). In this paper we propose a novel extsnison to CP-
nets, a popular formalism for reasoning with preferences (?, ?). We first
provide background on CP-nets and how one may use them to model the questions
in our experiments.

Formally, a CP-net has a set of features (or variables)
$F=\\{X_{1},\ldots,X_{n}\\}$ with finite domains $D(X_{1}),\ldots,D(X_{n})$.
For each feature $X_{i}$, we are given a set of parent features $Pa(X_{i})$
that can affect the preferences over the values of $X_{i}$. This defines a
dependency graph in which each node $X_{i}$ has $Pa(X_{i})$ as its immediate
predecessors. An acyclic CP-net is one in which the dependency graph is
acyclic. Given this structural dependency information among a CP-net’s
variables, one needs to specify the preference over the values of each
variable $X_{i}$ for each complete assignment to the parent variables,
$Pa(X_{i})$. This preference is assumed to take the form of a total or partial
order over $D(X_{i})$. A cp-statement for some feature $X_{i}$ that has
parents $Pa(X_{i})=\\{x_{1},\ldots,x_{n}\\}$ and domain
$D(X_{i})=\\{a_{1},\ldots,a_{m}\\}$ is a total ordering over $D(X_{i})$. The
set of cp-statements regarding a certain variable $X_{i}$ is called the cp-
table for $X_{i}$. For example, given a CP-net with features $A$, $B$, $C$,
and $D$. Each variable has binary domain containing $f$ and $\overline{f}$ if
$F$ is the name of the feature. Here, statement $a\succ\overline{a}$
represents the unconditional preference for $A=a$ over $A=\overline{a}$, while
statement $c:d\succ\overline{d}$ states that $D=d$ is preferred to
$D=\overline{d}$, given that $C=c$.

Among the several generalizations of CP-nets, in this work we focus on an
expansion of the traditional CP-nets model to model cases where individuals
have indifference over the values of some features, that is, some preference
information is missing (?). A cp-statement $z:a_{i}\approx a_{j}$ means that
the person is indifferent between $a_{i}$ and $a_{j}$ given the assignment to
the parents variables $z$, i.e. $a_{i}\succeq a_{j}$ and $a_{j}\succeq a_{i}$.
A lack of information over one of the values of a variable is modeled with
empty cp-statements for that value.

###### Example 1

John is in line at the airport. When he is the first of the line, a person
approaches him and kindly ask to cut the line because her flight is going to
leave in 5 minutes. Due to security constraints, he always prefers not to let
anyone cut the line at the airport. He has however different preferences when
he is not at the airport: usually he lets a person to cut the line if he is on
time while he prefers not to let someone to cut the line if he is in late with
his daily schedule.

$S$$T$$P$ $a,\bar{a}$
---
$o>\bar{o}$
---
$a,o:$ | $\bar{c}>c$
---|---
$a,\bar{o}:$ | $\bar{c}>c$
$\bar{a},o:$ | $c>\bar{c}$
$\bar{a},\bar{o}:$ | $\bar{c}>c$

Figure 4: The CP-net representing John’s preferences described in Example 1.
Variable $S$ denotes the scenario: $a$ for “at the airport”, and
$\overline{a}$ for “not at the airport”; variable $T$ represents time: $o$ for
“on time”, and $\overline{o}$ for “not on time”; and a preference variable $P$
represent the judgement over cutting the line: i.e., $c$ is for “ok to cut the
line”, and $\overline{c}$ is for “not ok to cut the line”.

The CP-net in Figure 4, with features $S$, $T$, and $P$, represents John’s
preferences as described in Example 1. Specifically, variable $S$ denotes the
scenario and takes the values $a$ for ”at the airport”, and $\overline{a}$ for
”not at the airport”. while variable $T$ represents time, and takes value $o$
for ”on time” and $\overline{o}$ for ”not on time”. Finally we have our
preference variable $P$ which has values for cut ($c$) and not cut
($\overline{c}$). Observe that we do not report any preference between the
values of variable $S$ and hence there is no cp-table for that variable. For
variable $P$, we have a complete preference order over his domain for each
possible combination of values of the parent variables $P$ and $T$.

$ao\overline{c}$
---
$aoc$
---
$a\overline{oc}$
---
$a\overline{o}c$
---
$\overline{a}oc$
---
$\overline{a}o\overline{c}$
---
$\overline{a}o\overline{c}$
---
$\overline{aoc}$
---

Figure 5: The preorder induced by the CP-net in Figure 4. The two components
are due to the indifference on the domain of the variable $A$.

The semantics of CP-nets depends on the notion of a worsening flip: a change
in the value of a variable to a less preferred value according to the cp-
statement for that variable. One outcome $\alpha$ is preferred to or
_dominates_ another outcome $\beta$ (written $\alpha\succ\beta$) if and only
if there is a chain of worsening flips from $\alpha$ to $\beta$. This
definition induces a preorder (i.e. a binary relation which is reflexive and
transitive) over the outcomes. Indifference induces loop between pairs of
outcomes while the lack of information induces incomparability in the preorder
(i.e. given two outcomes $o,p$, if neither $o\preceq p$ nor $p\preceq o$ are
valid, then we say that $o$ and $p$ are incomparable, denoted with $o\bowtie
p$.). This incomparability makes the preference graph disconnected. In
particular, in the induced preference graph there is a connected component for
each combination of values of the variables with missing cp-statements.

For instance, Figure 5 gives the full induced preference order for the CP-net
shown in Figure 4. The component on the left side describes preferences over
the airport scenario ($S=a$), while the component on the right side describes
John’s preferences hen not at the airport ($S=\overline{a}$). Clearly,
outcomes in different components cannot be compared because they describe
preferences over different scenarios. In this, incomparability is a useful too
that allows us to model different scenarios. While CP-nets are usually a
compact way to express preferences, their induced order can be exponentially
larger. This is why it is important to reason on CP-nets and not on their
semantics. Two useful questions that one may ask are related to optimality and
dominance (?, ?). Finding the optimal outcome of a CP-net is NP-hard (?) in
general but can be found in polynomial time for acyclic CP-nets, by assigning
the most preferred value (according to the CP–tables) for each variable in the
order given by the dependencies. On the other hand, checking the dominance
between two outcomes is a computationally difficult problem.

##  9 Moral Judgements, Fast and Slow Thinking, and Preferences

In a standard CP-net, there is only one kind of variable: those needed to
express preferences. There is no ability to describe the context in which a
preference-based decision making process takes place, nor to model other
auxiliary evaluation variables that may be needed, or useful, to declare our
own preferences. In some sense, a CP-net is a useful tool only when it is
clear what the context is, and if no reasoning on the context is needed in
order to state the preferences, or when such reasoning takes place outside the
CP-net formalism.

Given this limitation, there have been calls to extend this model into
reasoning and preference systems in computer science in a principled and exact
way (?). We therefore modified the CP-net formalism to be able to capture the
psychological mechanisms at play in our subjects’ moral judgments. As
described in detail in Section 3, we hypothesized that subjects were using a
combination of System 1 (rule-based) and System 2 (consequentialist and
contractualist) thinking. We attempt to model this two-modality reasoning
process in the CP-net formalism so that we can embed it within a machine, to
allow the machine to reason both fast and slow about ethical principles (?,
?). Our motivation is to extend the semantics of CP-nets in order to model
both the snap judgements that do not take into account the particularities of
the scenario, System 1 thinking, as well as provide the ability to reason
about these details, using System 2 thinking, if necessary (?).

Scenario VariablesEvaluation VariablesPreference Variables Reason Location
First Person Welfare Middle Person Welfare Last Person Welfare Global Welfare
Line Cutter Welfare Universalization Likelihood Judgement Figure 6: Our model
that blends the notion of preferences with that of Scenario and Evaluation
Variables. While individuals cannot set or have preferences over the Scenario
Variables, they will possess their own subjective evaluations over the
Evaluation Variables given a setting to the Scenario Variables. Given both the
Scenario Variables and the Evaluation Variables, the agent can then decide on
a preference over the single Preference Variable.

##  10 SEP-nets: Scenarios, Evaluation, and Preferences

Analyzing the scenarios of our vignettes and our conjecture of how the
subjects reason about the scenarios and then respond to the single preference
question (whether somebody should be allowed to cut the line or not), we
propose a generalization of the CP-net formalism to handle variables
associated with the context. We propose extending the formalism consisting of:

  * •

a set of _scenario variables (SVs)_ to define a decision making context over
which there is no preference to be stated,

  * •

a set of _evaluation variables (EVs)_ to model the evaluation process that
takes place in the subjects’ minds while reasoning over the given context,

  * •

to decide their preference over the _preference variables (PVs)_ that are
already modelled in CP-nets.

Figure 6 describes this generalized preference framework visually using all
the variables from our experiments, that include the following kinds of
variables:

  1. 1.

Scenario Variables. A set of variables that describe the context, such as
location, whether or not the agent had already waited in line, whether or not
the agent was using the main function of the line, and the size of the line.
In addition, we need a variable to specify the main reason or motivation for
cutting the line. We observe that the agent does not have the ability to set
values for these variables, nor does the agent have preferences over their
values, as these values are set by the environment or context within which the
decision is taking place. These variables do not depend on any other variable
(that is, there is no incoming dependency arrow), meaning that this is part of
the input to a decision making AI system.

  2. 2.

Evaluation Variables. A set of variables that a person (or an AI system)
considers (and estimates the value of) to reason about the given scenario.
These are, for example, the well-being of the first in line, the well-being of
the cutter, and others as discussed in the experimental details section. In
our experiments we have 8 of such variables. These variables have a real
valued range as a domain and the user selects one point in the range, which
represents her estimate for that variable’s value. However, no preferences for
the values of these variables is required. All the evaluation variables depend
on the scenario variables. This follows our conjecture that people need to
examine the specific scenario in order to start an evaluation phase in which
they identify the evaluation variables and estimate a value for them.

  3. 3.

Preference Variables. These are already included in a standard CP-net. In the
setting under study, the agent expresses preference over a single value, that
models whether or not, given both the values of the SVs and the values for the
EVs, it is acceptable to cut the line, i.e., the moral judgement. The single
preference variable depends on the evaluation variables. This again follow
from the conjecture that a person needs to first perform a level of
consequentialist or contractualist estimation in order to decide whether the
rule, that states that a line cannot be cut, can be violated.

As noted above, CP-nets and their variants, e.g., probabilistic CP-nets (?),
allow only for preference variables, and there is no option for creating a
dependency between the preference variables to scenario and context variables.
We rather envision a three-layer generalization where, as shown in Figure 6,
the single preference variable depends on the evaluation variables, which in
turn depend on the scenario variables. However, a finer grained analysis may
show that there are evaluation variables that do not depend on the scenario
variables. For example, the evaluation variable that has to do with the
likelihood of the event happening does not have any relationship with whether
or not the cutter is concerned with the main function of the line.

###  10.1 SEP-nets: Formal Syntax and Semantics

In this section, we formally define a generalization of CP-nets, called SEP-
nets. This will allow us to have a compact framework to express and reason
about knowledge on moral preferences. We call this generalization SEP-nets
since it includes three kinds of variables, representing Scenarios,
Evaluations, and Preferences. When SEP-nets include only preference features,
they collapse to being standard CP-nets.

###### Definition 1

An SEP-net consists of

  * •

A set of features (or variables) $V=S\cup E\cup P$. Given a variable $x$, the
domain of $x$ is a finite set if $x\in S$ or $x\in P$, while it is an ordered
set if $x\in E$. For simplicity, we assume this ordered set is a numerical
range and we define it by its minimum and maximum element (min and max). Each
variable can be only in one of three sets, that is, $S\cap E=\emptyset$,
$S\cap P=\emptyset$, and $P\cap E=\emptyset$.

  * •

As in a standard CP-net, each variable $x$ has a set of parent variables
$Pa(x)$, on which it depends on. However, $Pa(x)=\emptyset$ if $x\in S$;
$Pa(x)\subseteq S\cup E$ if $x\in E$, and $Pa(x)\subseteq S\cup E\cup P$ if
$x\in P$. This models a three-level acyclic structure, where scenario
variables are independent, evaluation variables can depend only on scenario
variables or other evaluation variables, and preference variables can depend
on any variable.

  * •

If $x\in E$, an evaluation function $ef(x)$ identifies a single value in the
domain of $x$, between min and max. Thus $ef:(min,max)\rightarrow[min,max]$.
This function gives an estimate of the value for that variable. If $x\in P$, a
standard CP-table states the preferences over the domain of $x$: for each
combination of values in $Pa(x)$, the table provides a total order of
$Dom(x)$. No CP-table or evaluation function is associated with scenario
variables.

Preference variables may depend directly on either all or a subset of the
scenario variables. Moreover, they may depend on certain values of a scenario
variable, but not others. For example, as we will see in our data analysis in
Section 10.2, the single preference variable is dependent on the location
variable when the location is an airport or a bathroom. This direct dependency
between preferences and scenarios models a sort of System 1 approach, where
people make a moral judgement (or any other preference decision) by just
looking at the situation at hand and without performing any sophisticated
reasoning. On the other hand, when preference variables depend on evaluation
variables, which in turn depend on scenario variables, we model a sort of
System 2 approach, where people consider a scenario and a preference question,
and make an estimate of the consequences of the various options before
finalizing their decision or judgement.

When there are no scenario nor evaluation variables, but just preference
variables, this is a standard CP-net, so it semantics is defined as usual for
CP-nets (?). When instead we have a full SEP-net, its semantics is an order
over all the SEP-outcomes, each being an assignment of values to all the
variables in their respective domains. Given two outcomes
$o=[s_{1},\ldots,s_{n},e_{1},\ldots,e_{m},p_{1},\ldots,pk]$ and
$o^{\prime}=[s^{\prime}_{1},\ldots,s^{\prime}_{n},e^{\prime}_{1},\ldots,e^{\prime}_{m},p^{\prime}_{1},\ldots,p^{\prime}k]$,
we have $o<o^{\prime}$ if

  * •

$[s_{1}\ldots,s_{n}]=[s^{\prime}_{1}\ldots,s^{\prime}_{n}]$. This means that
we are considering the same scenario.

  * •

$[e_{1}\ldots,e_{m}]=[e^{\prime}_{1}\ldots,e^{\prime}_{m}]=[ef(e_{1}),\ldots,ef(e_{m})]$.
That is, the evaluation variables are set to their estimates as given by their
estimation functions.

  * •

$[p_{1}\ldots,p_{k}]<_{p}[p^{\prime}_{1}\ldots,p^{\prime}_{k}]$ in the order
$<_{p}$ induced by the CP-net which is obtained by just considering the
preference variables (in $P$) and the dependencies among them.

So, outcomes that differ on the scenario are not connected in the order
induced by a SEP-net. Moreover, any outcome with the evaluation variables set
to values that are different from the value given by their estimate function
are not connected to any other outcome. If the dependencies among preference
variables define an acyclic graph, the result is a set of partial orders, one
for each scenario. Each of such partial orders has the same shape as the
induced order of the CP-net obtained by the given SEP-net by setting the
scenario variables to any value in their domain and the evaluation variables
to their estimate value.

Given an SEP-net and its induced set of partial orders, the optimal outcomes
are the top elements in the induced partial orders, thus one for each
scenario, and can be efficiently computed by 1) choosing any scenario, 2)
setting the evaluation variables to their estimate value, given the chosen
scenario, and 3) taking the most preferred values for the preference
variables.

###  10.2 From Experimental Data to a SEP-net

Starting from our study, we will now build a corresponding SEP-net. To do
that, we must identify the variables and the dependencies among them as
detailed in our vignettes from Section 6. The variables of this SEP-net can be
seen in Figure 7. The scenario variables (SVs) describe all features of a
scenario, and therefore refer to Reason and Location. Reason has a domain that
includes all 25 reasons for cutting the line as listed in Table 1 and Location
has domain $\\{Airport,Bathroom,Deli\\}$. The 7 evaluation variables (EVs) in
Figure 7 correspond to the evaluation questions asked to the subjects. All
have the domain $[-50,50]$, except Likelihood that has the domain $[0,100]$.
There is only one preference variable (PV), corresponding to the moral
judgement question. Its domain is simply yes/no.

In order to build the SEP-net corresponding to the data collected in our
survey, we need to understand which SVs influence the way individuals respond
to EVs. If we can find a relationship between these variables, then we say
that EVs depend on SVs and validate our model. We also check whether SVs
influence the PV. To test for dependency, we run Wilcoxon signed-rank tests
(?), a non-parametric t-test for comparing paired data samples from the
evaluations of individuals. Using this we test if the following four null
hypotheses can be rejected:

  1. 1.

NH1: Location does not affect the EVs;

  2. 2.

NH2: Reason does not affect the EVs;

  3. 3.

NH3: Location does not affect the PV;

  4. 4.

NH4: Evaluation variables do not affect the PV.

#### NH1: Location Does Not Affect the Evaluation Variables.

In order to determine this, we ran a Wilcoxon signed-rank test for each pair
of locations and evaluation variable. The results of this are depicted in
Table 4.

Scenario | Deli-Bath | Deli-Airpt | Airpt-Bath
---|---|---|---
Global Welfare | 0.2782 | 0.8954 | 0.3028
First Person Welfare | 0.1779 | 0.0932 | 0.9696
Middle Person Welfare | 0.1012 | 0.1390 | 0.3478
Last Person Welfare | 0.0064 | 0.1444 | 0.2763
Line Cutter Welfare | 0.0069 | 0.0123 | 0.3467
Universalization | 0.4848 | 0.4356 | 0.1567
Likelihood | 0.0008 | 0.0138 | 0.2430
Table 4: p-values for the Wilcoxon signed-rank test against the null-
hypothesis NH1: the location does not affect evaluation variables. The null-
hypothesis is rejected if $p<0.05$, these cases are reported in bold.

NH1 is only rejected in a few cases, specifically between Deli and Bathroom
when evaluating the effect on the last person, cutter, and likelihood as well
as between the Deli and Airport when evaluating the effect on the cutter and
likelihood. Hence we can say that NH1 is only partially rejected, and that
some Evaluation Variables are affected by the Location, specifically: last
person, cutter, and likelihood. We have depicted these results in Figure 7
where we build an SEP net from our experimental data. It is interesting to
notice that there are a set of variables that seem to be independent of
location. This implies that no matter what location, the value of the EV does
not change. Hence, it may be that some rules, no matter what is happening, are
never ok to violate.

#### NH2: Reason Does Not Affect the Evaluation Variables

For each pair of scenarios, we checked the Wilcoxon signed-rank test for each
of the evaluation variables; we omit the full $7\times{25\choose 2}$ pairs for
readability. For NH2 we can only reject the null hypothesis in some cases,
however, some evaluation variables are significantly affected for each
assignment to the reason variable. Hence, we can say that individuals evaluate
the scenario differently based on what is happening, and that the reason can
and does influence all evaluation variables, with the exception of the Line
Cutter variable, as shown in Figure 7.

#### NH3: Location Does Not Affect the PV

We investigated whether or not the location had an effect on the preference
variable. In order to test this, we selected four reasons for each location,
since there were different numbers per location, aggregated these, and
compared the mean responses to the to the Preference Variable or moral
judgement. The complete results are depicted in Table 5.

Scenario | p-value | Rejected
---|---|---
Deli-Bath | 8.759E-01 | False
Deli-Air | 1.548E-08 | True
Air-Bath | 2.662E-10 | True
Table 5: p-values for the Wilcoxon signed-rank test against the null-
hypothesis NH3: location does not affect the preference variable. The null-
hypothesis is rejected if $p<0.05$.

From this we can reject NH3 for all pairs except Deli and Bathroom. This
indicates that in some cases location may be sufficient to evaluate the
vignette and make a decision.. This is represented by the arc between location
and preference variable in Figure 7. It is interesting to note that
participants seemed to evaluate the airport as being a significantly different
location than the deli or the bathroom.

#### NH4: Evaluation Variables Do Not Affect the Preference Variable

In order to assess the influence of evaluation variables over the preference
variable we need to place them all in a comparable range. To do this we
compute quartiles of each evaluation variable to bucket them and then compare
the effect of this groups on the decision using the Wilcoxon signed-rank test.
For each combination of Scenario and Evaluation Variable, we pair groups in
order to perform the test. The full results are available in the appendix.

The results suggest that the evaluation variables have some influence on the
preference variable, except for the cutter variable. Indicating that in
general participants were not thinking of the benefit to the cutter, but only
the cost to others. This is depicted in Figure 7 as we have moved the cutter
variable out of the box of the rest of the evaluation variables, since is
perceived differently.

Based on the findings we can construct a partial graph of the dependencies
between variables, the resulting SEP-net corresponding to the data of our
study is shown in Figure 7.

Scenario VariablesEvaluation VariablesPreference VariablesScenario
VariablesEVs for ReasonEVs for LocationPreference Variables Reason Location
First Person Welfare Middle Person Welfare Global Welfare Universalization
Last Person Welfare Likelihood Line Cutter Welfare Judgement Figure 7: The
SEP-net corresponding to the data collected in our study. SVs influence the
way individuals evaluate each scenario and make a decision. For the sake of
readability, we group evaluation variables based whether they depend on a
particular SV in order to reduce the number of arrows. Given the SVs and the
EVs, the agent can then decide on a preference over the single PV. Note that
_Line Cutter Welfare_ does not have any effect on the PVs, while the _Already
Waited_ variable is completely missing

##  11 Conclusion and Future Work

We have taken a first step to model and understand the question of when humans
think it is morally acceptable to break rules. We constructed and studied a
suite of hypothetical scenarios relating to this question, and collected human
judgments about these scenarios. We showed that existing structures in the
preference reasoning literature are insufficient for modeling this data, and
we defined a generalization of CP-nets, called SEP-nets, which allow the
linkage of preferences with scenarios and context evaluation.

Using this novel technique, we showed that humans seem to employ a complex set
of preferences when determining if it is morally acceptable to break a
previously established rule. Subjects seem to take into account the particular
elements of a given scenario, including location and reasons to break the
rule. Moreover, we found evidence that System 2 processes were operative in
rule-breaking decisions: moral judgments were influenced by calculations of
the impact of rule-violations on the well-being of others (consequentialist
reasoning) as well as notions of what everyone would agree to (contractualist
reasoning). Other times, System 1 reasoning seemed to be operative; sometimes,
outcomes and agreement had no relationship to moral permissibility and rather
rules were considered inviolable. Together, this pattern of data begins to
suggest that moral rigidity and moral flexibility are driven by fast and slow
thinking.

There are several ways in which we plan to extend this work. We mention here
two concrete directions:

#### Comparing Preferences and Measuring Value Deviation.

We defined generalized CP-nets (called SEP-nets) in a way that is consistent
with classical CP-nets and probabilistic CP-nets. The aim is to understand how
to use a (generalized) preference structure to effectively learn and reason
with morality-driven preferences, and to embed them into an AI system. Another
fruitful direction for future work is to attempt to adapt our results to other
preference reasoning formalisms, e.g., soft-constraints or weighted logical
representations (?). Reasoning about preferences and decision making requires
also being able to compare preferences. For instance, in a multi-agent system,
it is important to understand whether an agent is deviating from a societal
priority or norm. To do that, recent studies proposed metric spaces over
preferences (?) which can be used to implement value alignment procedures.
Such metric spaces may be extended to be applicable to SEP-nets. Such metrics
would be useful to allow a system designer to intervene in cases where
individuals (or artificial agents) behave differently than expected, or are
operating outside the norms or rules of a society.

#### Prescriptive Plans Based on Moral Preferences.

The AI research community has not only been active in understanding how to
make single decisions based on preferences, but also on creating plans,
consisting of sequences of actions, that would respect or follow certain
preferences (?). This work can be exploited to extend the use of the moral
preferences discussed in this paper into more prescriptive AI techniques such
as automated planning (?). Although prior efforts from the planning
perspective all investigate the generation of plans that take into account
pre-specified utilitarian preferences, the question of where those utilities
and preferences manifest from has not been addressed very adequately so far.
We are currently actively investigating methods that seek to use the data
collected in this work to automatically generate preferences in the notation
used by planning formalisms (?). The generation of such preferences will in
turn enable us to generate prescriptive plans for agents or systems that
conform to the moral standards of that agent or system. Specifically, we will
transform the problem from a classification-based setting into a generative
model, and then present plan (action) alternatives that agents can choose
from. To situate this in the context of the current question of study: this
extension would enable us to move from determining whether it was acceptable
to break a rule, to generating ways to do so that are most in accordance with
some preference and cost function that takes moral obligations into account.

## References

  * Alexander Alexander, J. (2012).  Experimental philosophy: An introduction.  Polity.
  * Alkoby, Rath, & Stone Alkoby, S., Rath, A., & Stone, P. (2019).  Teaching social behavior through human reinforcement for ad hoc teamwork-the STAR framework.  In Proc. of the 18th AAMAS.
  * Allen, Smit, & Wallach Allen, C., Smit, I., & Wallach, W. (2005).  Artificial morality: Top-down, bottom-up, and hybrid approaches.  Ethics and Information Technology, 7(3), 149–155.
  * Allen Allen, T. E. (2013).  CP-nets with indifference.  In 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1488–1495. IEEE.
  * Arnold, Thomas, Kasenberg, & Scheutzs Arnold, T., Thomas, Kasenberg, D., & Scheutzs, M. (2017).  Value alignment or misalignment - what will keep systems accountable?.  In AI, Ethics, and Society, Papers from the 2017 AAAI Workshop.
  * Awad, Dsouza, Kim, Schulz, Henrich, Shariff, Bonnefon, & Rahwan Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.-F., & Rahwan, I. (2018).  The moral machine experiment.  Nature, 563(7729), 59.
  * Balakrishnan, Bouneffouf, Mattei, & Rossi Balakrishnan, A., Bouneffouf, D., Mattei, N., & Rossi, F. (2019).  Incorporating behavioral constraints in online AI systems.  In Proc. of the 33rd AAAI.
  * Baumard, André, & Sperber Baumard, N., André, J.-B., & Sperber, D. (2013).  A mutualistic approach to morality: The evolution of fairness by partner choice.  Behavioral and Brain Sciences, 36(1), 59–78.
  * Benton, Coles, & Coles Benton, J., Coles, A., & Coles, A. (2012).  Temporal planning with preferences and time-dependent continuous costs.  In Proc. 22nd ICAPS.
  * Bonnefon, Shariff, & Rahwan Bonnefon, J.-F., Shariff, A., & Rahwan, I. (2020).  The moral psychology of AI and the ethical opt-out problem.  Oxford University Press, Oxford, UK.
  * Booch, Fabiano, Horesh, Kate, Lenchner, Linck, Loreggia, Murgesan, Mattei, Rossi, et al. Booch, G., Fabiano, F., Horesh, L., Kate, K., Lenchner, J., Linck, N., Loreggia, A., Murgesan, K., Mattei, N., Rossi, F., et al. (2021).  Thinking fast and slow in AI.  In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, pp. 15042–15046.
  * Boutilier, Brafman, Domshlak, Hoos, & Poole Boutilier, C., Brafman, R., Domshlak, C., Hoos, H., & Poole, D. (2004).  CP-nets: A tool for representing and reasoning with conditional ceteris paribus preference statements.  Journal of Artificial Intelligence Research, 21, 135–191.
  * Brafman & Chernyavsky Brafman, R. I., & Chernyavsky, Y. (2005).  Planning with goal preferences and constraints..  In ICAPS, pp. 182–191.
  * Brandt, Conitzer, Endriss, Lang, & Procaccia Brandt, F., Conitzer, V., Endriss, U., Lang, J., & Procaccia, A. D. (Eds.). (2016).  Handbook of Computational Social Choice.  Cambridge University Press.
  * Cornelio, Goldsmith, Mattei, Rossi, & Venable Cornelio, C., Goldsmith, J., Mattei, N., Rossi, F., & Venable, K. (2013).  Updates and uncertainty in CP-nets.  In Proc. of the 26th AUSAI.
  * Cushman Cushman, F. (2013).  Action, outcome, and value: A dual-system framework for morality.  Personality and social psychology review, 17(3), 273–292.
  * Domshlak, Hüllermeier, Kaci, & Prade Domshlak, C., Hüllermeier, E., Kaci, S., & Prade, H. (2011).  Preferences in AI: An overview.  Artificial Intelligence, 175(7), 1037–1052.
  * Doris, Group, et al. Doris, J. M., Group, M. P. R., et al. (2010).  The moral psychology handbook.  OUP Oxford.
  * Engelmann & Waldmann Engelmann, N., & Waldmann, M. R. (2022).  How to weigh lives. a computational model of moral judgment in multiple-outcome structures.  Cognition, 218, 104910.
  * Gauthier Gauthier, D. (1986).  Morals by agreement.  Oxford University Press on Demand.
  * Gerevini & Long Gerevini, A., & Long, D. (2005).  Plan constraints and preferences in pddl3..
  * Glazier, Loreggia, Mattei, Rahgooy, Rossi, & Venable Glazier, A., Loreggia, A., Mattei, N., Rahgooy, T., Rossi, F., & Venable, K. B. (2021).  Making human-like trade-offs in constrained environments by learning from demonstrations.  arXiv preprint arXiv:2109.11018.
  * Goldsmith, Lang, Truszczyński, & Wilson Goldsmith, J., Lang, J., Truszczyński, M., & Wilson, N. (2008).  The computational complexity of dominance and consistency in CP-nets.  Journal of Artificial Intelligence Research, 33(1), 403–432.
  * Greene Greene, J. D. (2014).  Moral tribes: Emotion, reason, and the gap between us and them.  Penguin.
  * Habermas Habermas, J. (1990).  Moral consciousness and communicative action.  MIT press.
  * Haidt Haidt, J. (2007).  The new synthesis in moral psychology.  science, 316(5827), 998–1002.
  * Hare Hare, R. M. (1981).  Moral thinking: Its levels, method, and point.  Oxford: Clarendon Press; New York: Oxford University Press.
  * Harsanyi Harsanyi, J. C. (1977).  Morality and the theory of rational behavior.  Social Research, 44(4), 623.
  * Holyoak & Powell Holyoak, K. J., & Powell, D. (2016).  Deontological coherence: A framework for commonsense moral reasoning..  Psychological Bulletin, 142(11), 1179.
  * Jiang, Hwang, Bhagavatula, Bras, Forbes, Borchardt, Liang, Etzioni, Sap, & Choi Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. L., Forbes, M., Borchardt, J., Liang, J., Etzioni, O., Sap, M., & Choi, Y. (2021).  Delphi: Towards machine ethics and norms.  arXiv preprint arXiv:2110.07574.
  * Kahneman Kahneman, D. (2011).  Thinking, Fast and Slow.  Macmillan.
  * Kim, Kleiman-Weiner, Abeliuk, Awad, Dsouza, Tenenbaum, & Rahwan Kim, R., Kleiman-Weiner, M., Abeliuk, A., Awad, E., Dsouza, S., Tenenbaum, J. B., & Rahwan, I. (2018).  A computational model of commonsense moral decision making.  In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 197–203.
  * Kleiman-Weiner, Gerstenberg, Levine, & Tenenbaum Kleiman-Weiner, M., Gerstenberg, T., Levine, S., & Tenenbaum, J. B. (2015).  Inference of intention and permissibility in moral decision making..  In CogSci. Citeseer.
  * Kleiman-Weiner, Saxe, & Tenenbaum Kleiman-Weiner, M., Saxe, R., & Tenenbaum, J. B. (2017).  Learning a commonsense moral theory.  Cognition, 167, 107–123.
  * Knobe Knobe, J. (2007).  Experimental philosophy.  Philosophy Compass, 2(1), 81–92.
  * Levine, Kleiman-Weiner, Chater, Cushman, & Tenenbaum Levine, S., Kleiman-Weiner, M., Chater, N., Cushman, F., & Tenenbaum, J. B. (2018).  The cognitive mechanisms of contractualist moral decision-making..  In CogSci. Citeseer.
  * Levine, Kleiman-Weiner, Schulz, Tenenbaum, & Cushman Levine, S., Kleiman-Weiner, M., Schulz, L., Tenenbaum, J., & Cushman, F. (2020).  The logic of universalization guides moral judgment.  Proceedings of the National Academy of Sciences, 117(42), 26158–26169.
  * Levine & Leslie Levine, S., & Leslie, A. (2021).  Preschoolers use the means principle to make moral judgments.  psyarxiv.com/np9a5.
  * Lieder & Griffiths Lieder, F., & Griffiths, T. L. (2020).  Resource-rational analysis: understanding human cognition as the optimal use of limited computational resources.  Behavioral and Brain Sciences, 43.
  * Loreggia, Mattei, Rossi, & Venable Loreggia, A., Mattei, N., Rossi, F., & Venable, K. B. (2018a).  On the distance between CP-nets.  In Proc. of the 17th AAMAS, pp. 955–963. International Foundation for Autonomous Agents and Multiagent Systems.
  * Loreggia, Mattei, Rossi, & Venable Loreggia, A., Mattei, N., Rossi, F., & Venable, K. B. (2018b).  Preferences and ethical principles in decision making.  In Proc. 1st AIES.
  * Loreggia, Mattei, Rossi, & Venable Loreggia, A., Mattei, N., Rossi, F., & Venable, K. B. (2018c).  Value alignment via tractable preference distance.  In Yampolskiy, R. V. (Ed.), Artificial Intelligence Safety and Security, chap. 16. CRC Press.
  * Loreggia, Mattei, Rossi, & Venable Loreggia, A., Mattei, N., Rossi, F., & Venable, K. B. (2020a).  CPMetric: Deep siamese networks for metric learning on structured preferences.  In El Fallah Seghrouchni, A., & Sarne, D. (Eds.), Artificial Intelligence. IJCAI 2019 International Workshops, pp. 217–234, Cham. Springer International Publishing.
  * Loreggia, Mattei, Rossi, & Venable Loreggia, A., Mattei, N., Rossi, F., & Venable, K. B. (2020b).  Modeling and reasoning with preferences and ethical priorities in AI systems.  Ethics of Artificial Intelligence, 127.
  * Mann, Whitney, et al. Mann, H., Whitney, D., et al. (1947).  On a test of whether one of two random variables is stochastically larger than the other.  Annals of Mathematical Statistics, 18(1), 50–60.
  * Mikhail Mikhail, J. (2011).  Elements of moral cognition: Rawls’ linguistic analogy and the cognitive science of moral and legal judgment.  Cambridge University Press.
  * Nichols & Mallon Nichols, S., & Mallon, R. (2006).  Moral dilemmas and moral rules.  Cognition, 100(3), 530–542.
  * Noothigattu, Bouneffouf, Mattei, Chandra, Madan, Varshney, Campbell, Singh, & Rossi Noothigattu, R., Bouneffouf, D., Mattei, N., Chandra, R., Madan, P., Varshney, K., Campbell, M., Singh, M., & Rossi, F. (2019).  Teaching AI agents ethical values using reinforcement learning and policy orchestration.  In Proc. of the 28th IJCAI.
  * Parfit Parfit, D. (2011).  On what matters: volume one, Vol. 1.  Oxford University Press.
  * Pu, Faltings, Chen, Zhang, & Viappiani Pu, P., Faltings, B., Chen, L., Zhang, J., & Viappiani, P. (2011).  Usability guidelines for product recommenders based on example critiquing research.  In Ricci, F., Rokach, L., Shapira, B., & Kantor, P. B. (Eds.), Recommender Systems Handbook, pp. 511–545. Springer.
  * Rawls Rawls, J. (1971).  A theory of justice.  Harvard university press.
  * Rossi, Venable, & Walsh Rossi, F., Venable, K., & Walsh, T. (2011).  A Short Introduction to Preferences: Between Artificial Intelligence and Social Choice.  Morgan and Claypool.
  * Rossi & Loreggia Rossi, F., & Loreggia, A. (2019).  Preferences and ethical priorities: Thinking fast and slow in AI.  In Proc. of the 18th AAMAS, pp. 3–4.
  * Rossi & Mattei Rossi, F., & Mattei, N. (2019).  Building ethically bounded AI.  In Proc. of the 33rd AAAI(Blue Sky Track).
  * Russell Russell, S. (2019).  Human compatible: Artificial intelligence and the problem of control.  Penguin.
  * Russell, Dewey, & Tegmark Russell, S., Dewey, D., & Tegmark, M. (2015).  Research priorities for robust and beneficial artificial intelligence.  AI Magazine, 36(4), 105–114.
  * Scanlon et al. Scanlon, T., et al. (1998).  What we owe to each other.  Harvard University Press.
  * Sen Sen, A. (1974).  Choice, ordering and morality.  In Körner, S. (Ed.), Practical Reason. Blackwell, Oxford.
  * Simon Simon, H. A. (1955).  A behavioral model of rational choice.  The quarterly journal of economics, 69(1), 99–118.
  * Simon Simon, H. A. (1956).  Rational choice and the structure of the environment..  Psychological review, 63(2), 129.
  * Svegliato, Nashed, & Zilberstein Svegliato, J., Nashed, S. B., & Zilberstein, S. (2021).  Ethically compliant sequential decision making.  In Proceedings of the 35th AAAI International Conference on Artificial Intelligence (AAAI).
  * van Baar, Chang, & Sanfey van Baar, J. M., Chang, L. J., & Sanfey, A. G. (2019).  The computational and neural substrates of moral strategies in social decision-making.  Nature communications, 10(1), 1–14.
  * Wallach & Allen Wallach, W., & Allen, C. (2008).  Moral machines: Teaching robots right from wrong.  Oxford University Press.
  * Wang, Shao, Zhou, Wan, & Bouguettaya Wang, H., Shao, S., Zhou, X., Wan, C., & Bouguettaya, A. (2009).  Web service selection with incomplete or inconsistent user preferences.  In Proc. 7th International Conference on Service-Oriented Computing, pp. 83–98. Springer.
  * Yu, Shen, Miao, Leung, Lesser, & Yang Yu, H., Shen, Z., Miao, C., Leung, C., Lesser, V. R., & Yang, Q. (2018).  Building ethics into artificial intelligence..  In Proc. 27th IJCAI, pp. 5527–5533.

##  A Complete Statistical Tables

Scenario | PREFthenEVAL | PREFthenEVAL | p-value | Rejected
---|---|---|---|---
DFO_SP | 0.1212 (0.3264) | 0.1692 (0.3750) | 0.5059 | False
DFO_WA | 0.4697 (0.4991) | 0.4154 (0.4928) | 0.5860 | False
DFO_SO | 0.6970 (0.4596) | 0.6923 (0.4615) | 0.8535 | False
DFO_CA | 0.7576 (0.4285) | 0.6154 (0.4865) | 0.0743 | False
DFO_CO | 0.7424 (0.4373) | 0.6308 (0.4826) | 0.1979 | False
DFO_SU | 0.0909 (0.2875) | 0.1385 (0.3454) | 0.4281 | False
DFO_BR | 0.0758 (0.2646) | 0.1692 (0.3750) | 0.1414 | False
DFO_HS | 0.1061 (0.3079) | 0.1385 (0.3454) | 0.6379 | False
DFO_TP | 0.2121 (0.4088) | 0.1692 (0.3750) | 0.4579 | False
DFO_MA | 0.2879 (0.4528) | 0.3231 (0.4677) | 0.5860 | False
DFO_FA | 0.2424 (0.4285) | 0.3231 (0.4677) | 0.3596 | False
DFO_SW | 0.8182 (0.3857) | 0.7538 (0.4308) | 0.4236 | False
BFO_HA | 0.5507 (0.4974) | 0.5217 (0.4995) | 0.7317 | False
BFO_CL | 0.1594 (0.3661) | 0.2899 (0.4537) | 0.0971 | False
BFO_TU | 0.1014 (0.3019) | 0.1884 (0.3910) | 0.1414 | False
BFO_JA | 0.1594 (0.3661) | 0.2319 (0.4220) | 0.3248 | False
BFO_FR | 0.7101 (0.4537) | 0.7536 (0.4309) | 0.5730 | False
BFO_EL | 0.1014 (0.3019) | 0.2029 (0.4022) | 0.0948 | False
BFO_BR | 0.7391 (0.4391) | 0.7681 (0.4220) | 0.7238 | False
AFO_MN | 0.4030 (0.4905) | 0.4219 (0.4939) | 0.6096 | False
AFO_BA | 0.4328 (0.4955) | 0.4531 (0.4978) | 0.7526 | False
AFO_JA | 0.4478 (0.4973) | 0.5625 (0.4961) | 0.1088 | False
AFO_CA | 0.5224 (0.4995) | 0.5781 (0.4939) | 0.3471 | False
AFO_BR | 0.4478 (0.4973) | 0.4062 (0.4911) | 0.4285 | False
AFO_HR | 0.8209 (0.3834) | 0.7969 (0.4023) | 0.5607 | False
Table 6: p-values for the Wilcoxon signed-rank test against the null-hypothesis: changing the order of evaluation or preference does not influence the preference value. The null-hypothesis is rejected if $p<0.05$. Table 7: p-values for the Wilcoxon signed-rank test against the null-hypothesis NH4: evaluation variable does not affect the preference. The null-hypothesis is rejected if $p<0.05$, these cases are reported in bold. Scenario | EV | Group 0-1 | Group 0-2 | Group 0-3 | Group 1-2 | Group 1-3 | Group 2-3
---|---|---|---|---|---|---|---
DFO_SP | everyone | 0.1489 | 0.4237 | 0.4840 | 1.0000 | 0.7656 | 0.7656
DFO_WA | everyone | 0.3506 | 0.3458 | 0.0004 | 1.0000 | 0.0050 | 0.3458
DFO_SO | everyone | 1.0000 | 0.3458 | 0.0001 | 0.7728 | 0.0001 | 0.0369
DFO_CA | everyone | 0.1817 | 0.0726 | 0.0000 | 0.1817 | 0.0001 | 0.1096
DFO_CO | everyone | 0.3506 | 0.0719 | 0.0000 | 0.0719 | 0.0000 | nan
DFO_SU | everyone | 0.2986 | 0.7728 | 0.0369 | 1.0000 | 0.4237 | 1.0000
DFO_BR | everyone | 0.2330 | 1.0000 | 1.0000 | 1.0000 | 0.7897 | 0.7656
DFO_HS | everyone | 0.1489 | 1.0000 | 0.7728 | 0.1489 | 0.7656 | 0.0719
DFO_TP | everyone | 0.1169 | 0.0411 | 0.0077 | 0.7728 | 0.2986 | 0.7728
DFO_MA | everyone | 0.0026 | 0.7768 | 0.0219 | 0.1489 | 0.5653 | 0.4840
DFO_FA | everyone | 0.0048 | 0.3458 | 0.0019 | 0.4237 | 0.7768 | 0.0719
DFO_SW | everyone | 1.0000 | 0.0060 | 0.0001 | 0.0060 | 0.0006 | 0.7656
BFO_HA | everyone | 0.0134 | 0.0004 | 0.0000 | 0.0572 | 0.0030 | 0.7897
BFO_CL | everyone | 0.4835 | 0.0355 | 0.0003 | 0.3506 | 0.0107 | 0.2330
BFO_TU | everyone | 0.0658 | 0.7728 | 0.8016 | 1.0000 | 0.1817 | 1.0000
BFO_JA | everyone | 0.7728 | 0.7656 | 0.0719 | 0.5653 | 0.7768 | 0.1096
BFO_FR | everyone | 0.0197 | 0.0369 | 0.0000 | 0.0719 | 0.0000 | 0.0197
BFO_EL | everyone | 0.0369 | 0.3458 | 0.5653 | 1.0000 | 0.1450 | 1.0000
BFO_BR | everyone | 0.2330 | 0.4840 | 0.0000 | 0.4840 | 0.0002 | 0.0726
AFO_MN | everyone | 0.8293 | 1.0000 | 0.0628 | 0.4840 | 0.0140 | 0.4237
AFO_BA | everyone | 0.4579 | 0.1779 | 0.0005 | 0.5653 | 0.0019 | 0.0411
AFO_JA | everyone | 0.1779 | 0.1169 | 0.0030 | 0.5653 | 0.0572 | 0.4281
AFO_CA | everyone | 0.0135 | 0.4840 | 0.0002 | 0.7728 | 0.0235 | 0.4840
AFO_BR | everyone | 0.0572 | 0.0488 | 0.0411 | 0.8016 | 0.5941 | 0.7768
AFO_HR | everyone | 0.1489 | 1.0000 | 0.0000 | 1.0000 | 0.0000 | 0.0719
DFO_SP | first_person | 0.7656 | 1.0000 | 1.0000 | 0.0915 | 0.2273 | 0.7656
DFO_WA | first_person | 0.5297 | 0.4281 | 0.0026 | 0.1647 | 0.0006 | 0.0572
DFO_SO | first_person | 0.3506 | 0.4840 | 0.0001 | 1.0000 | 0.0004 | 0.0234
DFO_CA | first_person | 0.1489 | 0.0006 | 0.0001 | 0.0235 | 0.0026 | 0.7768
DFO_CO | first_person | 1.0000 | 0.1450 | 0.0000 | 0.1817 | 0.0000 | 0.0658
DFO_SU | first_person | 0.3506 | 0.0411 | 0.0411 | 0.4237 | 0.4840 | 0.7728
DFO_BR | first_person | 0.7728 | 0.7656 | 0.1294 | 0.3458 | 0.1096 | 0.2986
DFO_HS | first_person | 0.5297 | 0.1294 | 0.4840 | 0.4237 | 1.0000 | 0.2330
DFO_TP | first_person | 0.0050 | 0.0004 | 0.0006 | 0.7728 | 0.5297 | 1.0000
DFO_MA | first_person | 0.0412 | 0.0083 | 0.0034 | 0.2669 | 0.6179 | 0.7897
DFO_FA | first_person | 0.4281 | 0.0006 | 0.0234 | 0.1096 | 0.1817 | 0.7656
DFO_SW | first_person | nan | 0.0107 | 0.0000 | 0.0107 | 0.0000 | 0.1450
BFO_HA | first_person | 0.0234 | 0.0000 | 0.0000 | 0.0002 | 0.0019 | 1.0000
BFO_CL | first_person | 0.5941 | 0.0077 | 0.0002 | 0.1294 | 0.2986 | 1.0000
BFO_TU | first_person | 0.0394 | 0.1294 | 0.8016 | 0.7728 | 0.1817 | 0.1294
BFO_JA | first_person | 0.7728 | 0.3458 | 0.7728 | 0.8016 | 0.3929 | 0.5941
BFO_FR | first_person | 0.7728 | 0.0394 | 0.0000 | 0.0019 | 0.0015 | 0.0051
BFO_EL | first_person | 0.0197 | 0.0411 | 0.5297 | 0.7728 | 0.2986 | 0.0411
BFO_BR | first_person | 0.2986 | 0.1489 | 0.0000 | 0.7728 | 0.0002 | 0.0411
AFO_MN | first_person | 0.2081 | 0.7768 | 0.0311 | 1.0000 | 0.0009 | 0.0234
AFO_BA | first_person | 0.8121 | 0.0235 | 0.0001 | 0.0084 | 0.0002 | 0.0019
AFO_JA | first_person | 0.3506 | 0.0915 | 0.0219 | 0.0759 | 0.0003 | 0.0197
AFO_CA | first_person | 0.1779 | 0.0658 | 0.0005 | 1.0000 | 0.0124 | 0.0411
AFO_BR | first_person | 0.6179 | 0.0134 | 0.6179 | 0.0048 | 0.1169 | 0.0488
AFO_HR | first_person | nan | 0.0719 | 0.0000 | 0.2330 | 0.0001 | 0.0018
DFO_SP | middle_person | 0.4237 | 0.7656 | 0.4237 | 0.0411 | 0.1096 | 0.4237
DFO_WA | middle_person | 0.7768 | 0.0235 | 0.0011 | 0.2357 | 0.0011 | 0.1779
DFO_SO | middle_person | 1.0000 | 0.0234 | 0.0000 | 0.0411 | 0.0000 | 0.1450
DFO_CA | middle_person | 0.2330 | 0.0060 | 0.0000 | 0.0394 | 0.0002 | 0.7656
DFO_CO | middle_person | 1.0000 | 0.3458 | 0.0000 | 0.7728 | 0.0000 | 0.0719
DFO_SU | middle_person | 0.2330 | 0.2330 | 0.2330 | 0.7728 | 0.0726 | 0.7728
DFO_BR | middle_person | 0.5297 | 0.7728 | 0.7897 | 1.0000 | 0.5297 | 0.2986
DFO_HS | middle_person | 1.0000 | 0.0369 | 0.0369 | 0.0369 | 0.2330 | 0.2330
DFO_TP | middle_person | 0.0915 | 0.0219 | 0.0140 | 0.3506 | 0.0726 | 1.0000
DFO_MA | middle_person | 0.0026 | 0.0030 | 0.0268 | 0.5297 | 0.8213 | 0.5653
DFO_FA | middle_person | 0.0083 | 0.0107 | 0.0135 | 0.1294 | 0.7768 | 0.7728
DFO_SW | middle_person | 0.7728 | 0.1489 | 0.0001 | 0.1489 | 0.0000 | 0.0369
BFO_HA | middle_person | 0.5297 | 0.0002 | 0.0004 | 0.0001 | 0.0000 | 0.8016
BFO_CL | middle_person | 0.5941 | 0.0004 | 0.0009 | 0.1096 | 0.0726 | 1.0000
BFO_TU | middle_person | 0.0411 | 0.1817 | 0.1779 | 0.7656 | 0.4840 | 1.0000
BFO_JA | middle_person | 0.2330 | 0.7728 | 1.0000 | 0.2081 | 0.2669 | 0.4840
BFO_FR | middle_person | 0.4237 | 0.0134 | 0.0000 | 0.0394 | 0.0000 | 0.0915
BFO_EL | middle_person | 0.4237 | 0.0369 | 1.0000 | 0.7728 | 0.1450 | 0.1294
BFO_BR | middle_person | 0.7656 | 0.0369 | 0.0000 | 0.2330 | 0.0001 | 0.0658
AFO_MN | middle_person | 0.4281 | 0.8121 | 0.0009 | 1.0000 | 0.0140 | 0.0235
AFO_BA | middle_person | 0.2357 | 0.0107 | 0.0005 | 0.2357 | 0.0051 | 0.0084
AFO_JA | middle_person | 0.0572 | 0.0219 | 0.0001 | 0.8016 | 0.0011 | 0.0077
AFO_CA | middle_person | 0.0488 | 0.0355 | 0.0002 | 0.8213 | 0.0140 | 0.0077
AFO_BR | middle_person | 0.4579 | 0.0235 | 0.0235 | 0.0005 | 0.0916 | 0.2273
AFO_HR | middle_person | 1.0000 | 0.1489 | 0.0000 | 0.1489 | 0.0000 | 0.0034
DFO_SP | last_person | 0.2330 | 0.1489 | 0.1489 | 0.7656 | 1.0000 | 1.0000
DFO_WA | last_person | 0.2669 | 0.0719 | 0.0050 | 0.7728 | 0.0077 | 1.0000
DFO_SO | last_person | 0.7897 | 0.1489 | 0.0000 | 0.3458 | 0.0001 | 1.0000
DFO_CA | last_person | 0.1096 | 0.0369 | 0.0000 | 0.4237 | 0.0003 | 0.3458
DFO_CO | last_person | 1.0000 | 0.1489 | 0.0000 | 0.1489 | 0.0000 | 1.0000
DFO_SU | last_person | 0.0134 | 1.0000 | 0.0134 | 1.0000 | 0.2330 | nan
DFO_BR | last_person | 0.7897 | 0.7728 | 1.0000 | 0.7728 | 0.7897 | 1.0000
DFO_HS | last_person | 0.0107 | 0.0411 | 0.1096 | 1.0000 | 0.4840 | 0.7728
DFO_TP | last_person | 0.1450 | 0.0060 | 0.0394 | 0.1096 | 0.3506 | 0.7656
DFO_MA | last_person | 0.0124 | 0.2330 | 0.0051 | 0.4237 | 0.5297 | 0.2330
DFO_FA | last_person | 0.0011 | 0.0726 | 0.0030 | 1.0000 | 0.7768 | 1.0000
DFO_SW | last_person | 0.3458 | 0.0369 | 0.0000 | 0.0369 | 0.0002 | 0.7728
BFO_HA | last_person | 0.0411 | 0.0077 | 0.0001 | 0.2669 | 0.0124 | 0.7656
BFO_CL | last_person | 0.0572 | 0.0045 | 0.0083 | 0.2986 | 0.3506 | 0.7656
BFO_TU | last_person | 0.8016 | 0.4840 | 0.5653 | 1.0000 | 0.8016 | 1.0000
BFO_JA | last_person | 0.4237 | 1.0000 | 0.4237 | 0.1096 | 0.3506 | 0.4237
BFO_FR | last_person | 0.0197 | 0.0107 | 0.0000 | 0.0197 | 0.0004 | 0.1294
BFO_EL | last_person | 0.1817 | 0.5297 | 0.2986 | 0.1817 | 0.2986 | 0.3506
BFO_BR | last_person | 1.0000 | 0.3458 | 0.0000 | 1.0000 | 0.0001 | 0.2330
AFO_MN | last_person | 0.5941 | 0.2330 | 0.0084 | 0.4840 | 0.0015 | 0.1294
AFO_BA | last_person | 0.1414 | 0.3929 | 0.0005 | 0.7656 | 0.0124 | 0.0369
AFO_JA | last_person | 0.2081 | 0.1294 | 0.0019 | 0.4237 | 0.0519 | 0.2330
AFO_CA | last_person | 0.0289 | 0.1489 | 0.0219 | 1.0000 | 0.0948 | 0.4237
AFO_BR | last_person | 0.8121 | 0.3506 | 0.6179 | 1.0000 | 1.0000 | 0.7897
AFO_HR | last_person | 0.3458 | 0.0719 | 0.0000 | 0.0719 | 0.0000 | 0.0077
DFO_SP | cutter | 0.7656 | 0.7897 | 0.7897 | 0.5297 | 0.7897 | 1.0000
DFO_WA | cutter | 0.0759 | 0.3014 | 0.8121 | 0.6179 | 0.2273 | 0.3318
DFO_SO | cutter | 1.0000 | 0.0915 | 0.7768 | 0.0355 | 0.8016 | 0.1294
DFO_CA | cutter | 0.1096 | 0.8016 | 0.0134 | 0.8121 | 0.2669 | 0.1779
DFO_CO | cutter | 0.1450 | 0.3506 | 0.7768 | 0.4835 | 0.0197 | 0.1450
DFO_SU | cutter | 0.7728 | 0.4237 | 0.1489 | 0.4840 | 0.2330 | 0.7728
DFO_BR | cutter | 0.7897 | 0.2986 | 0.4237 | 0.7768 | 0.2330 | 0.0726
DFO_HS | cutter | 0.7728 | 0.2330 | 1.0000 | 0.7656 | 1.0000 | 1.0000
DFO_TP | cutter | 0.5653 | 0.1294 | 0.5941 | 1.0000 | 1.0000 | 1.0000
DFO_MA | cutter | 0.2669 | 0.4281 | 0.5653 | 0.7897 | 0.6179 | 0.6179
DFO_FA | cutter | 0.8213 | 0.2273 | 0.7897 | 1.0000 | 0.2273 | 0.3014
DFO_SW | cutter | 0.7656 | 0.7897 | 1.0000 | 0.8121 | 0.8121 | 0.8016
BFO_HA | cutter | 0.0759 | 0.1779 | 0.3826 | 0.6179 | 0.5941 | 0.5941
BFO_CL | cutter | 0.4281 | 0.2330 | 0.5941 | 0.2986 | 0.5297 | 0.1817
BFO_TU | cutter | 0.1294 | 0.5297 | 0.4840 | 0.4840 | 1.0000 | 0.2330
BFO_JA | cutter | 0.8213 | 0.1096 | 0.1817 | 0.0915 | 0.2273 | 0.0719
BFO_FR | cutter | 0.6179 | 0.3014 | 0.3929 | 0.2357 | 0.5941 | 0.0948
BFO_EL | cutter | 0.4281 | 0.4840 | 0.2669 | 0.4237 | 1.0000 | 1.0000
BFO_BR | cutter | 0.0034 | 0.2330 | 0.4237 | 0.0394 | 0.0011 | 0.0134
AFO_MN | cutter | 0.2669 | 0.0084 | 0.0394 | 0.1169 | 0.8121 | 0.1096
AFO_BA | cutter | 0.5653 | 0.5297 | 1.0000 | 0.8016 | 0.4835 | 0.3318
AFO_JA | cutter | 0.3248 | 0.0658 | 0.3014 | 1.0000 | 0.1169 | 0.0628
AFO_CA | cutter | 0.0235 | 0.0311 | 0.4281 | 0.4835 | 1.0000 | 0.1779
AFO_BR | cutter | 0.0412 | 0.3318 | 0.0134 | 0.5653 | 0.3506 | 0.4281
AFO_HR | cutter | 0.0034 | 0.0011 | 0.3458 | 0.0572 | 0.0394 | 0.0045
DFO_SP | univers. | 0.1489 | 0.4237 | 0.1489 | 0.5941 | 0.2273 | 0.2986
DFO_WA | univers. | 0.0726 | 0.0015 | 0.0018 | 0.0346 | 0.0045 | 0.3929
DFO_SO | univers. | 0.2986 | 0.0369 | 0.0000 | 0.7768 | 0.0000 | 0.0077
DFO_CA | univers. | 0.0411 | 0.0060 | 0.0000 | 0.0060 | 0.0018 | 0.2986
DFO_CO | univers. | 0.4237 | 0.0019 | 0.0000 | 0.0107 | 0.0000 | 0.5297
DFO_SU | univers. | 0.7728 | 0.3458 | 0.7728 | 0.1817 | 0.1817 | 1.0000
DFO_BR | univers. | 0.7768 | 0.4840 | 0.7897 | 0.7656 | 0.7768 | 0.7768
DFO_HS | univers. | 0.2330 | nan | 1.0000 | 0.2330 | 0.3506 | 0.7728
DFO_TP | univers. | 0.0411 | 0.0726 | 0.0234 | 0.1817 | 0.1450 | 1.0000
DFO_MA | univers. | 0.0572 | 0.0140 | 0.0015 | 0.1169 | 0.0658 | 0.7656
DFO_FA | univers. | 0.1134 | 0.0034 | 0.0124 | 0.0411 | 0.2669 | 0.7656
DFO_SW | univers. | 0.7728 | 0.0060 | 0.0002 | 0.0060 | 0.0011 | 0.7768
BFO_HA | univers. | 0.8213 | 0.0078 | 0.0004 | 0.0030 | 0.0003 | 0.7656
BFO_CL | univers. | 0.4281 | 0.0018 | 0.0005 | 0.1096 | 0.0411 | 1.0000
BFO_TU | univers. | 1.0000 | 0.0726 | 0.5297 | 0.0411 | 0.7768 | 0.1294
BFO_JA | univers. | 0.3506 | 0.2330 | 0.7656 | 0.1779 | 1.0000 | 1.0000
BFO_FR | univers. | 0.1489 | 0.0006 | 0.0000 | 0.0140 | 0.0001 | 0.0311
BFO_EL | univers. | 0.7728 | 0.7768 | 0.4237 | 1.0000 | 0.8016 | 0.7768
BFO_BR | univers. | 0.3458 | 0.0197 | 0.0000 | 0.1294 | 0.0003 | 0.0658
AFO_MN | univers. | 1.0000 | 0.2986 | 0.0355 | 1.0000 | 0.0140 | 0.1817
AFO_BA | univers. | 0.4281 | 0.0084 | 0.0050 | 0.0355 | 0.0048 | 0.7897
AFO_JA | univers. | 0.5941 | 0.0628 | 0.0011 | 0.1414 | 0.0015 | 0.1450
AFO_CA | univers. | 0.0948 | 0.0011 | 0.0019 | 0.0197 | 0.1779 | 0.3929
AFO_BR | univers. | 0.0394 | 0.0001 | 0.0140 | 0.0011 | 0.0948 | 0.1450
AFO_HR | univers. | 0.3458 | 0.1489 | 0.0000 | 0.4237 | 0.0005 | 0.0135
DFO_SP | likelihood | 0.0726 | 0.0197 | 0.1817 | 0.7656 | 0.4840 | 0.4840
DFO_WA | likelihood | 1.0000 | 0.7897 | 0.0726 | 0.8213 | 0.0572 | 0.1662
DFO_SO | likelihood | 0.7728 | 0.4840 | 0.0034 | 0.5941 | 0.2081 | 0.1096
DFO_CA | likelihood | 0.7897 | 0.0135 | 0.0015 | 0.0355 | 0.0050 | 0.1450
DFO_CO | likelihood | 0.7728 | 0.0009 | 0.0001 | 0.0011 | 0.0001 | 0.0107
DFO_SU | likelihood | 0.3506 | 0.7728 | 1.0000 | 0.4237 | 0.7656 | 1.0000
DFO_BR | likelihood | 1.0000 | 0.7728 | 1.0000 | 1.0000 | 1.0000 | 0.7656
DFO_HS | likelihood | 0.4840 | 1.0000 | 1.0000 | 0.5297 | 0.7768 | 0.7768
DFO_TP | likelihood | 0.8213 | 0.0197 | 0.0726 | 0.0915 | 0.2669 | 0.7768
DFO_MA | likelihood | 0.0015 | 0.0006 | 0.0004 | 0.7768 | 0.5653 | 0.8016
DFO_FA | likelihood | 0.0107 | 0.0045 | 0.0009 | 0.1450 | 0.2273 | 0.8016
DFO_SW | likelihood | 0.0719 | 0.0034 | 0.0001 | 0.1096 | 0.0045 | 0.2273
BFO_HA | likelihood | 0.2081 | 0.0771 | 0.0124 | 0.5059 | 0.2357 | 0.8213
BFO_CL | likelihood | 0.3929 | 0.0083 | 0.0083 | 0.3506 | 0.1294 | 0.7768
BFO_TU | likelihood | 0.5297 | 0.3929 | 1.0000 | 0.7768 | 0.4840 | 0.4840
BFO_JA | likelihood | 0.2669 | 0.3929 | 0.1450 | 0.1817 | 1.0000 | 0.2273
BFO_FR | likelihood | 0.7656 | 0.1294 | 0.0077 | 0.2669 | 0.0630 | 0.2610
BFO_EL | likelihood | 0.3506 | 0.4840 | 0.1096 | 0.2986 | 0.5297 | 0.5297
BFO_BR | likelihood | 0.7656 | 0.0005 | 0.0077 | 0.0009 | 0.0077 | 1.0000
AFO_MN | likelihood | 0.1817 | 0.0045 | 0.2081 | 0.0948 | 0.6379 | 0.2669
AFO_BA | likelihood | 0.2273 | 0.0002 | 0.0030 | 0.0048 | 0.0488 | 0.8121
AFO_JA | likelihood | 0.2669 | 0.0031 | 0.0006 | 0.0135 | 0.0015 | 0.2081
AFO_CA | likelihood | 0.3014 | 0.0311 | 0.0007 | 0.2357 | 0.0051 | 0.2669
AFO_BR | likelihood | 0.0140 | 0.0219 | 0.0234 | 0.6379 | 0.8016 | 1.0000
AFO_HR | likelihood | 0.1489 | 0.0034 | 0.0004 | 0.0411 | 0.0019 | 0.0572
Table 8: p-values for the Wilcoxon signed-rank test against the null-hypothesis NH2: reason does not affect evaluation variables. The null-hypothesis is rejected if $p<0.05$, these cases are reported in bold. p-values of EVs for any pair of scenarios. Scenario1 | Scenario2 | everyone | first_person | middle_person | last_person | cutter | univers. | likelihood |
---|---|---|---|---|---|---|---|---|---
DFO_SP | DFO_WA | 0.2576 | 0.2927 | 0.0810 | 0.1037 | 0.5957 | 0.0393 | 0.9389 |
DFO_SP | DFO_SO | 0.0098 | 0.0238 | 0.0050 | 0.0029 | 0.1911 | 0.0007 | 0.4266 |
DFO_SP | DFO_CA | 0.0001 | 0.0005 | 0.0000 | 0.0000 | 0.1741 | 0.0000 | 0.0000 |
DFO_SP | DFO_CO | 0.0026 | 0.0117 | 0.0023 | 0.0020 | 0.0083 | 0.0033 | 0.0000 |
DFO_SP | DFO_SU | 0.1426 | 0.1960 | 0.1692 | 0.0200 | 0.0000 | 0.1937 | 0.0000 |
DFO_SP | DFO_BR | 0.1524 | 0.0510 | 0.0742 | 0.0533 | 0.6433 | 0.6679 | 0.0000 |
DFO_SP | DFO_HS | 0.6004 | 0.9483 | 0.9699 | 0.6789 | 0.0006 | 0.6405 | 0.0120 |
DFO_SP | DFO_TP | 0.1848 | 0.3437 | 0.2102 | 0.0895 | 0.8971 | 0.6394 | 0.0072 |
DFO_SP | DFO_MA | 0.0372 | 0.2239 | 0.0545 | 0.0270 | 0.3939 | 0.0150 | 0.8042 |
DFO_SP | DFO_FA | 0.0041 | 0.0051 | 0.0002 | 0.0004 | 0.7502 | 0.0003 | 0.6971 |
DFO_SP | DFO_SW | 0.0001 | 0.0001 | 0.0006 | 0.0001 | 0.0800 | 0.0000 | 0.0000 |
DFO_SP | BFO_HA | 0.1429 | 0.0113 | 0.0872 | 0.1495 | 0.8587 | 0.0080 | 0.0003 |
DFO_SP | BFO_CL | 0.1021 | 0.0826 | 0.0134 | 0.0279 | 0.0016 | 0.4187 | 0.0015 |
DFO_SP | BFO_TU | 0.4174 | 0.0014 | 0.0265 | 0.7002 | 0.0020 | 0.4160 | 0.0001 |
DFO_SP | BFO_JA | 0.9744 | 0.4098 | 0.4025 | 0.6791 | 0.8361 | 0.0838 | 0.0019 |
DFO_SP | BFO_FR | 0.0002 | 0.0299 | 0.0002 | 0.0372 | 0.1481 | 0.0000 | 0.0160 |
DFO_SP | BFO_EL | 0.7901 | 0.1970 | 0.0631 | 0.5477 | 0.2458 | 0.4133 | 0.0010 |
DFO_SP | BFO_BR | 0.0003 | 0.0001 | 0.0003 | 0.0103 | 0.0412 | 0.0000 | 0.0000 |
DFO_SP | AFO_MN | 0.0005 | 0.0162 | 0.0055 | 0.0239 | 0.0143 | 0.0024 | 0.0526 |
DFO_SP | AFO_BA | 0.4057 | 0.6631 | 0.1543 | 0.4234 | 0.0817 | 0.1124 | 0.0007 |
DFO_SP | AFO_JA | 0.0225 | 0.1121 | 0.0223 | 0.0276 | 0.7590 | 0.0074 | 0.0004 |
DFO_SP | AFO_CA | 0.0109 | 0.2094 | 0.0005 | 0.0007 | 0.3199 | 0.0009 | 0.0024 |
DFO_SP | AFO_BR | 0.1522 | 0.3195 | 0.1226 | 0.2088 | 0.2656 | 0.0529 | 0.0002 |
DFO_SP | AFO_HR | 0.0006 | 0.0201 | 0.0004 | 0.0005 | 0.1467 | 0.0000 | 0.0000 |
DFO_WA | DFO_SO | 0.0809 | 0.2738 | 0.2395 | 0.2102 | 0.4666 | 0.3133 | 0.3244 |
DFO_WA | DFO_CA | 0.0012 | 0.0057 | 0.0004 | 0.0052 | 0.3893 | 0.0387 | 0.0000 |
DFO_WA | DFO_CO | 0.0193 | 0.0547 | 0.0377 | 0.0678 | 0.0635 | 0.3542 | 0.0000 |
DFO_WA | DFO_SU | 0.4977 | 0.9118 | 0.8108 | 0.6923 | 0.0000 | 0.1857 | 0.0000 |
DFO_WA | DFO_BR | 0.8102 | 0.4233 | 0.3115 | 0.4445 | 0.8805 | 0.2035 | 0.0000 |
DFO_WA | DFO_HS | 0.2931 | 0.3330 | 0.3982 | 0.1198 | 0.0000 | 0.0024 | 0.0016 |
DFO_WA | DFO_TP | 0.6326 | 0.9119 | 0.7483 | 0.7279 | 0.5474 | 0.1665 | 0.0008 |
DFO_WA | DFO_MA | 0.2616 | 0.8089 | 0.3557 | 0.3847 | 0.7136 | 0.5187 | 0.9242 |
DFO_WA | DFO_FA | 0.0234 | 0.0219 | 0.0109 | 0.0399 | 0.6142 | 0.0931 | 0.5696 |
DFO_WA | DFO_SW | 0.0010 | 0.0094 | 0.0030 | 0.0117 | 0.3936 | 0.0061 | 0.0000 |
DFO_WA | BFO_HA | 0.4243 | 0.2953 | 0.6165 | 0.9080 | 0.7770 | 0.4610 | 0.0001 |
DFO_WA | BFO_CL | 0.2198 | 0.3432 | 0.0710 | 0.3384 | 0.0006 | 0.3357 | 0.0020 |
DFO_WA | BFO_TU | 0.8157 | 0.0393 | 0.0792 | 0.2403 | 0.0009 | 0.5588 | 0.0000 |
DFO_WA | BFO_JA | 0.4414 | 0.2188 | 0.5775 | 0.0281 | 0.1790 | 0.5868 | 0.0011 |
DFO_WA | BFO_FR | 0.0015 | 0.0790 | 0.0040 | 0.2848 | 0.3623 | 0.0033 | 0.0103 |
DFO_WA | BFO_EL | 0.5926 | 0.0305 | 0.0017 | 0.1431 | 0.0320 | 0.2020 | 0.0000 |
DFO_WA | BFO_BR | 0.0010 | 0.0004 | 0.0028 | 0.3873 | 0.0366 | 0.0029 | 0.0000 |
DFO_WA | AFO_MN | 0.0247 | 0.2322 | 0.1289 | 0.3033 | 0.0340 | 0.1198 | 0.0495 |
DFO_WA | AFO_BA | 0.9641 | 0.7808 | 0.5697 | 0.5959 | 0.2882 | 0.6050 | 0.0003 |
DFO_WA | AFO_JA | 0.0627 | 0.9041 | 0.3082 | 0.5442 | 0.2743 | 0.4018 | 0.0000 |
DFO_WA | AFO_CA | 0.0603 | 0.7259 | 0.1034 | 0.0662 | 0.8009 | 0.3581 | 0.0011 |
DFO_WA | AFO_BR | 0.5243 | 0.5340 | 0.6524 | 0.6965 | 0.0769 | 1.0000 | 0.0000 |
DFO_WA | AFO_HR | 0.0061 | 0.1679 | 0.0275 | 0.0744 | 0.0703 | 0.0278 | 0.0000 |
DFO_SO | DFO_CA | 0.0535 | 0.0364 | 0.0324 | 0.0303 | 0.9677 | 0.1498 | 0.0001 |
DFO_SO | DFO_CO | 0.4275 | 0.4653 | 0.4120 | 0.4079 | 0.1008 | 0.9270 | 0.0000 |
DFO_SO | DFO_SU | 0.3132 | 0.2521 | 0.6091 | 0.2707 | 0.0000 | 0.0336 | 0.0000 |
DFO_SO | DFO_BR | 0.3188 | 0.9848 | 0.6210 | 0.4240 | 0.6132 | 0.0149 | 0.0000 |
DFO_SO | DFO_HS | 0.0053 | 0.0445 | 0.0096 | 0.0157 | 0.0000 | 0.0002 | 0.0449 |
DFO_SO | DFO_TP | 0.0763 | 0.4105 | 0.2800 | 0.3020 | 0.2503 | 0.0098 | 0.0453 |
DFO_SO | DFO_MA | 0.7235 | 0.6324 | 0.6505 | 0.7717 | 0.5832 | 0.7200 | 0.2890 |
DFO_SO | DFO_FA | 0.2642 | 0.4386 | 0.0935 | 0.2806 | 0.5107 | 0.4452 | 0.4994 |
DFO_SO | DFO_SW | 0.0150 | 0.0214 | 0.0306 | 0.1591 | 0.6057 | 0.0338 | 0.0000 |
DFO_SO | BFO_HA | 0.4831 | 0.9282 | 0.2899 | 0.1298 | 0.4661 | 0.9400 | 0.0049 |
DFO_SO | BFO_CL | 0.8919 | 0.8587 | 0.6538 | 0.9365 | 0.0002 | 0.1602 | 0.0233 |
DFO_SO | BFO_TU | 0.1491 | 0.1689 | 0.8089 | 0.0874 | 0.0212 | 0.2677 | 0.0004 |
DFO_SO | BFO_JA | 0.0147 | 0.0516 | 0.0739 | 0.0031 | 0.0988 | 0.1454 | 0.0109 |
DFO_SO | BFO_FR | 0.0765 | 0.9372 | 0.0817 | 0.8678 | 0.6543 | 0.0787 | 0.0471 |
DFO_SO | BFO_EL | 0.0363 | 0.0106 | 0.0000 | 0.0295 | 0.0156 | 0.0327 | 0.0095 |
DFO_SO | BFO_BR | 0.0189 | 0.0370 | 0.3002 | 0.8900 | 0.1013 | 0.0219 | 0.0003 |
DFO_SO | AFO_MN | 0.5799 | 0.9279 | 0.7795 | 0.7396 | 0.0504 | 0.8105 | 0.2208 |
DFO_SO | AFO_BA | 0.1235 | 0.2341 | 0.2139 | 0.1560 | 0.5754 | 0.1070 | 0.0094 |
DFO_SO | AFO_JA | 0.9093 | 0.3036 | 0.7394 | 0.7887 | 0.1056 | 0.9334 | 0.0123 |
DFO_SO | AFO_CA | 0.7481 | 0.4742 | 0.7371 | 0.5164 | 0.9396 | 0.8671 | 0.0259 |
DFO_SO | AFO_BR | 0.1693 | 0.2174 | 0.3793 | 0.1020 | 0.0317 | 0.3568 | 0.0045 |
DFO_SO | AFO_HR | 0.1154 | 0.3854 | 0.4894 | 0.5011 | 0.0312 | 0.1873 | 0.0000 |
DFO_CA | DFO_CO | 0.2840 | 0.1993 | 0.2225 | 0.1837 | 0.2728 | 0.1201 | 0.0004 |
DFO_CA | DFO_SU | 0.0087 | 0.0057 | 0.0011 | 0.0132 | 0.0001 | 0.0008 | 0.0055 |
DFO_CA | DFO_BR | 0.0075 | 0.1540 | 0.0114 | 0.0221 | 0.3660 | 0.0007 | 0.5218 |
DFO_CA | DFO_HS | 0.0000 | 0.0012 | 0.0001 | 0.0001 | 0.0000 | 0.0000 | 0.0112 |
DFO_CA | DFO_TP | 0.0020 | 0.0116 | 0.0020 | 0.0051 | 0.3191 | 0.0003 | 0.0292 |
DFO_CA | DFO_MA | 0.0304 | 0.0109 | 0.0044 | 0.0143 | 0.6385 | 0.0315 | 0.0000 |
DFO_CA | DFO_FA | 0.4449 | 0.6688 | 0.2591 | 0.2504 | 0.5822 | 0.3860 | 0.0000 |
DFO_CA | DFO_SW | 0.9393 | 0.6834 | 0.4835 | 0.7528 | 0.5466 | 0.8922 | 0.4865 |
DFO_CA | BFO_HA | 0.0349 | 0.0365 | 0.0018 | 0.0005 | 0.4741 | 0.1456 | 0.1777 |
DFO_CA | BFO_CL | 0.0599 | 0.1245 | 0.0347 | 0.0371 | 0.0002 | 0.0087 | 0.0445 |
DFO_CA | BFO_TU | 0.0049 | 0.3064 | 0.0219 | 0.0002 | 0.0480 | 0.0163 | 0.4576 |
DFO_CA | BFO_JA | 0.0001 | 0.0012 | 0.0002 | 0.0000 | 0.2330 | 0.0117 | 0.0306 |
DFO_CA | BFO_FR | 0.6946 | 0.1611 | 0.2320 | 0.1063 | 0.7065 | 0.8919 | 0.0019 |
DFO_CA | BFO_EL | 0.0003 | 0.0002 | 0.0000 | 0.0001 | 0.0186 | 0.0003 | 0.1268 |
DFO_CA | BFO_BR | 0.7582 | 0.8718 | 0.4569 | 0.0154 | 0.1262 | 0.5079 | 0.9991 |
DFO_CA | AFO_MN | 0.1564 | 0.0912 | 0.0299 | 0.0142 | 0.1866 | 0.2167 | 0.0011 |
DFO_CA | AFO_BA | 0.0042 | 0.0093 | 0.0029 | 0.0006 | 0.5638 | 0.0048 | 0.0322 |
DFO_CA | AFO_JA | 0.1055 | 0.0162 | 0.0079 | 0.0152 | 0.1015 | 0.0906 | 0.0965 |
DFO_CA | AFO_CA | 0.1097 | 0.0169 | 0.0658 | 0.1182 | 0.9431 | 0.2823 | 0.0523 |
DFO_CA | AFO_BR | 0.0023 | 0.0034 | 0.0031 | 0.0004 | 0.0359 | 0.0141 | 0.0571 |
DFO_CA | AFO_HR | 0.7626 | 0.1247 | 0.0797 | 0.1343 | 0.0067 | 0.7590 | 0.2729 |
DFO_CO | DFO_SU | 0.0949 | 0.0956 | 0.0365 | 0.0808 | 0.0072 | 0.0424 | 0.2502 |
DFO_CO | DFO_BR | 0.0365 | 0.7662 | 0.1419 | 0.2085 | 0.0409 | 0.0145 | 0.0139 |
DFO_CO | DFO_HS | 0.0013 | 0.0144 | 0.0057 | 0.0052 | 0.0000 | 0.0030 | 0.0000 |
DFO_CO | DFO_TP | 0.0662 | 0.1088 | 0.1125 | 0.0683 | 0.0317 | 0.0083 | 0.0000 |
DFO_CO | DFO_MA | 0.1873 | 0.2092 | 0.0853 | 0.2264 | 0.0942 | 0.4718 | 0.0000 |
DFO_CO | DFO_FA | 0.8298 | 0.8334 | 0.8426 | 0.6798 | 0.0238 | 0.6041 | 0.0000 |
DFO_CO | DFO_SW | 0.3395 | 0.2520 | 0.4734 | 0.6084 | 0.4423 | 0.0516 | 0.0041 |
DFO_CO | BFO_HA | 0.0916 | 0.2928 | 0.0608 | 0.0560 | 0.0293 | 0.8990 | 0.0000 |
DFO_CO | BFO_CL | 0.2265 | 0.7764 | 0.8040 | 0.4438 | 0.0000 | 0.1502 | 0.0000 |
DFO_CO | BFO_TU | 0.0467 | 0.8564 | 0.1806 | 0.0117 | 0.1374 | 0.3126 | 0.0000 |
DFO_CO | BFO_JA | 0.0021 | 0.0147 | 0.0138 | 0.0006 | 0.0092 | 0.1365 | 0.0000 |
DFO_CO | BFO_FR | 0.4887 | 0.2877 | 0.8508 | 0.3308 | 0.1592 | 0.0728 | 0.0000 |
DFO_CO | BFO_EL | 0.0186 | 0.0003 | 0.0000 | 0.0013 | 0.0011 | 0.0371 | 0.0000 |
DFO_CO | BFO_BR | 0.4100 | 0.1199 | 0.4167 | 0.6674 | 0.9771 | 0.0749 | 0.0000 |
DFO_CO | AFO_MN | 0.8771 | 0.3757 | 0.4481 | 0.2101 | 0.8702 | 0.9670 | 0.0000 |
DFO_CO | AFO_BA | 0.0321 | 0.0363 | 0.0703 | 0.0413 | 0.3614 | 0.2996 | 0.0000 |
DFO_CO | AFO_JA | 0.4268 | 0.1705 | 0.1705 | 0.1330 | 0.0016 | 0.8324 | 0.0000 |
DFO_CO | AFO_CA | 0.5125 | 0.2991 | 0.6100 | 0.8659 | 0.1423 | 0.8928 | 0.0000 |
DFO_CO | AFO_BR | 0.0667 | 0.0213 | 0.1008 | 0.0098 | 0.0007 | 0.3860 | 0.0000 |
DFO_CO | AFO_HR | 0.7014 | 0.8485 | 0.5030 | 0.7670 | 0.0005 | 0.2446 | 0.0172 |
DFO_SU | DFO_BR | 0.7235 | 0.3874 | 0.5418 | 0.8906 | 0.0000 | 0.8361 | 0.1686 |
DFO_SU | DFO_HS | 0.0541 | 0.3312 | 0.1801 | 0.1098 | 0.0000 | 0.1147 | 0.0000 |
DFO_SU | DFO_TP | 0.8582 | 0.8737 | 0.8659 | 0.7367 | 0.0000 | 0.6684 | 0.0000 |
DFO_SU | DFO_MA | 0.4901 | 0.9728 | 1.0000 | 0.9531 | 0.0000 | 0.0558 | 0.0000 |
DFO_SU | DFO_FA | 0.0722 | 0.0900 | 0.0115 | 0.0667 | 0.0000 | 0.0031 | 0.0000 |
DFO_SU | DFO_SW | 0.0053 | 0.0108 | 0.0131 | 0.0149 | 0.0005 | 0.0002 | 0.0759 |
DFO_SU | BFO_HA | 0.6559 | 0.3705 | 0.9962 | 0.5622 | 0.0000 | 0.0646 | 0.0000 |
DFO_SU | BFO_CL | 0.3593 | 0.3463 | 0.1599 | 0.6404 | 0.0000 | 0.7510 | 0.0000 |
DFO_SU | BFO_TU | 0.8081 | 0.0701 | 0.2850 | 0.2083 | 0.0453 | 0.6706 | 0.0004 |
DFO_SU | BFO_JA | 0.0978 | 0.0892 | 0.2171 | 0.0838 | 0.0000 | 0.6200 | 0.0001 |
DFO_SU | BFO_FR | 0.0303 | 0.1785 | 0.0084 | 0.3835 | 0.0001 | 0.0002 | 0.0000 |
DFO_SU | BFO_EL | 0.4325 | 0.0874 | 0.0006 | 0.0594 | 0.0000 | 0.7412 | 0.0001 |
DFO_SU | BFO_BR | 0.0087 | 0.0026 | 0.0070 | 0.4262 | 0.0037 | 0.0001 | 0.0004 |
DFO_SU | AFO_MN | 0.1097 | 0.2752 | 0.3327 | 0.7949 | 0.0109 | 0.0073 | 0.0000 |
DFO_SU | AFO_BA | 0.6066 | 0.5587 | 0.8254 | 0.6463 | 0.0005 | 0.6756 | 0.0000 |
DFO_SU | AFO_JA | 0.3734 | 0.8786 | 0.3586 | 0.5100 | 0.0000 | 0.0265 | 0.0000 |
DFO_SU | AFO_CA | 0.1553 | 0.5866 | 0.2438 | 0.1065 | 0.0000 | 0.0394 | 0.0001 |
DFO_SU | AFO_BR | 0.8828 | 0.9243 | 0.9981 | 0.3715 | 0.0000 | 0.3940 | 0.0000 |
DFO_SU | AFO_HR | 0.0204 | 0.1071 | 0.0956 | 0.0558 | 0.0000 | 0.0024 | 0.1297 |
DFO_BR | DFO_HS | 0.2090 | 0.0881 | 0.1724 | 0.0688 | 0.0000 | 0.4675 | 0.0015 |
DFO_BR | DFO_TP | 0.8169 | 0.1972 | 0.5300 | 0.5001 | 0.7440 | 0.8298 | 0.0014 |
DFO_BR | DFO_MA | 0.5071 | 0.5528 | 0.7177 | 0.9787 | 0.7058 | 0.1112 | 0.0000 |
DFO_BR | DFO_FA | 0.0200 | 0.3037 | 0.0459 | 0.1492 | 0.9137 | 0.0041 | 0.0000 |
DFO_BR | DFO_SW | 0.0026 | 0.2297 | 0.0998 | 0.0422 | 0.2159 | 0.0001 | 0.9020 |
DFO_BR | BFO_HA | 0.4656 | 0.7679 | 0.7479 | 0.5418 | 0.8558 | 0.0289 | 0.0508 |
DFO_BR | BFO_CL | 0.3459 | 0.9916 | 0.3215 | 0.7511 | 0.0052 | 0.7204 | 0.0197 |
DFO_BR | BFO_TU | 0.5545 | 0.8919 | 0.5198 | 0.1829 | 0.0044 | 0.3475 | 0.1252 |
DFO_BR | BFO_JA | 0.4828 | 0.0476 | 0.1381 | 0.0513 | 0.4381 | 0.3644 | 0.0053 |
DFO_BR | BFO_FR | 0.0084 | 0.7158 | 0.0513 | 0.6100 | 0.3346 | 0.0000 | 0.0002 |
DFO_BR | BFO_EL | 0.6043 | 0.0065 | 0.0002 | 0.1432 | 0.2052 | 0.5916 | 0.0328 |
DFO_BR | BFO_BR | 0.0044 | 0.0259 | 0.0490 | 0.3277 | 0.0139 | 0.0000 | 0.3269 |
DFO_BR | AFO_MN | 0.0388 | 0.9962 | 0.7226 | 0.9611 | 0.0382 | 0.0109 | 0.0002 |
DFO_BR | AFO_BA | 0.8623 | 0.1492 | 0.6974 | 0.2403 | 0.2257 | 0.1937 | 0.0079 |
DFO_BR | AFO_JA | 0.2056 | 0.4909 | 0.6834 | 0.9972 | 0.2472 | 0.0295 | 0.0155 |
DFO_BR | AFO_CA | 0.1581 | 0.5009 | 0.2471 | 0.1940 | 0.4402 | 0.0191 | 0.0192 |
DFO_BR | AFO_BR | 0.6800 | 0.2301 | 0.5683 | 0.2621 | 0.1601 | 0.1150 | 0.0125 |
DFO_BR | AFO_HR | 0.0060 | 0.5641 | 0.4492 | 0.3752 | 0.1139 | 0.0014 | 0.7200 |
DFO_HS | DFO_TP | 0.1078 | 0.6612 | 0.2457 | 0.1772 | 0.0013 | 0.3081 | 0.6320 |
DFO_HS | DFO_MA | 0.0130 | 0.1519 | 0.0975 | 0.0234 | 0.0000 | 0.0023 | 0.0195 |
DFO_HS | DFO_FA | 0.0012 | 0.0035 | 0.0007 | 0.0011 | 0.0003 | 0.0002 | 0.0305 |
DFO_HS | DFO_SW | 0.0000 | 0.0018 | 0.0007 | 0.0003 | 0.0000 | 0.0000 | 0.0021 |
DFO_HS | BFO_HA | 0.0154 | 0.0241 | 0.2125 | 0.2499 | 0.0003 | 0.0003 | 0.2199 |
DFO_HS | BFO_CL | 0.0364 | 0.0669 | 0.0171 | 0.0415 | 0.4888 | 0.1183 | 0.5339 |
DFO_HS | BFO_TU | 0.2220 | 0.0088 | 0.0149 | 0.9008 | 0.0000 | 0.1014 | 0.0638 |
DFO_HS | BFO_JA | 0.7273 | 0.6789 | 0.3359 | 0.5109 | 0.0023 | 0.0214 | 0.5457 |
DFO_HS | BFO_FR | 0.0002 | 0.1084 | 0.0002 | 0.0140 | 0.0000 | 0.0000 | 0.7312 |
DFO_HS | BFO_EL | 0.3642 | 0.1080 | 0.0230 | 0.9587 | 0.0040 | 0.1803 | 0.3194 |
DFO_HS | BFO_BR | 0.0001 | 0.0001 | 0.0004 | 0.0284 | 0.0000 | 0.0000 | 0.0242 |
DFO_HS | AFO_MN | 0.0009 | 0.0415 | 0.0081 | 0.0454 | 0.0000 | 0.0006 | 0.6825 |
DFO_HS | AFO_BA | 0.2733 | 0.8105 | 0.3032 | 0.2474 | 0.0000 | 0.0302 | 0.5620 |
DFO_HS | AFO_JA | 0.0077 | 0.3384 | 0.0373 | 0.0377 | 0.0016 | 0.0014 | 0.1957 |
DFO_HS | AFO_CA | 0.0037 | 0.1900 | 0.0045 | 0.0009 | 0.0000 | 0.0010 | 0.5545 |
DFO_HS | AFO_BR | 0.0301 | 0.4695 | 0.1611 | 0.2695 | 0.0100 | 0.0173 | 0.3772 |
DFO_HS | AFO_HR | 0.0000 | 0.0133 | 0.0031 | 0.0007 | 0.0061 | 0.0000 | 0.0005 |
DFO_TP | DFO_MA | 0.5282 | 0.8267 | 0.8482 | 0.6594 | 0.6172 | 0.1244 | 0.0109 |
DFO_TP | DFO_FA | 0.0413 | 0.0342 | 0.0179 | 0.0225 | 0.4673 | 0.0035 | 0.0372 |
DFO_TP | DFO_SW | 0.0044 | 0.0043 | 0.0061 | 0.0165 | 0.0508 | 0.0003 | 0.0027 |
DFO_TP | BFO_HA | 0.6377 | 0.5470 | 0.9914 | 0.9318 | 0.8010 | 0.0215 | 0.2455 |
DFO_TP | BFO_CL | 0.4996 | 0.3082 | 0.2014 | 0.5846 | 0.0062 | 0.9801 | 0.4365 |
DFO_TP | BFO_TU | 0.8771 | 0.1602 | 0.4017 | 0.3396 | 0.0016 | 0.8503 | 0.0524 |
DFO_TP | BFO_JA | 0.1010 | 0.4174 | 0.5924 | 0.0436 | 0.8185 | 0.4994 | 0.7421 |
DFO_TP | BFO_FR | 0.0198 | 0.1772 | 0.0129 | 0.3053 | 0.2125 | 0.0001 | 1.0000 |
DFO_TP | BFO_EL | 0.3495 | 0.1301 | 0.0016 | 0.2353 | 0.2303 | 0.9606 | 0.2568 |
DFO_TP | BFO_BR | 0.0066 | 0.0026 | 0.0353 | 0.3586 | 0.0266 | 0.0001 | 0.0308 |
DFO_TP | AFO_MN | 0.0274 | 0.3467 | 0.2238 | 0.6182 | 0.0087 | 0.0100 | 0.5482 |
DFO_TP | AFO_BA | 0.6789 | 0.5314 | 0.6993 | 0.5771 | 0.0732 | 0.3648 | 0.4339 |
DFO_TP | AFO_JA | 0.1849 | 0.8712 | 0.3698 | 0.2678 | 0.5901 | 0.0289 | 0.2762 |
DFO_TP | AFO_CA | 0.1952 | 0.5547 | 0.1233 | 0.0322 | 0.2899 | 0.0181 | 0.5488 |
DFO_TP | AFO_BR | 0.9109 | 0.8539 | 0.7699 | 0.7729 | 0.3994 | 0.1853 | 0.3049 |
DFO_TP | AFO_HR | 0.0261 | 0.1265 | 0.1232 | 0.0556 | 0.1927 | 0.0006 | 0.0005 |
DFO_MA | DFO_FA | 0.1921 | 0.1360 | 0.0616 | 0.0488 | 0.7517 | 0.2468 | 0.5774 |
DFO_MA | DFO_SW | 0.0236 | 0.0245 | 0.0111 | 0.0619 | 0.6798 | 0.0139 | 0.0000 |
DFO_MA | BFO_HA | 0.8999 | 0.6959 | 0.8416 | 0.2930 | 0.9485 | 0.7714 | 0.0008 |
DFO_MA | BFO_CL | 0.8740 | 0.2484 | 0.2792 | 0.9805 | 0.0003 | 0.1910 | 0.0074 |
DFO_MA | BFO_TU | 0.5488 | 0.1190 | 0.4285 | 0.0765 | 0.0204 | 0.4930 | 0.0000 |
DFO_MA | BFO_JA | 0.0266 | 0.2357 | 0.1350 | 0.0068 | 0.1786 | 0.3497 | 0.0012 |
DFO_MA | BFO_FR | 0.0859 | 0.3307 | 0.0518 | 0.7903 | 0.6589 | 0.0289 | 0.0181 |
DFO_MA | BFO_EL | 0.0529 | 0.0638 | 0.0003 | 0.0752 | 0.0463 | 0.1110 | 0.0009 |
DFO_MA | BFO_BR | 0.0459 | 0.0032 | 0.0159 | 0.9157 | 0.0823 | 0.0143 | 0.0000 |
DFO_MA | AFO_MN | 0.2381 | 0.4675 | 0.5766 | 0.9363 | 0.0521 | 0.3679 | 0.0069 |
DFO_MA | AFO_BA | 0.3155 | 0.3827 | 0.6660 | 0.3115 | 0.2556 | 0.3231 | 0.0022 |
DFO_MA | AFO_JA | 0.5457 | 0.7916 | 0.6513 | 0.9688 | 0.2585 | 0.6927 | 0.0003 |
DFO_MA | AFO_CA | 0.3954 | 0.8679 | 0.2591 | 0.1620 | 0.7440 | 0.5770 | 0.0013 |
DFO_MA | AFO_BR | 0.4481 | 0.5917 | 0.7516 | 0.2225 | 0.0570 | 0.7426 | 0.0002 |
DFO_MA | AFO_HR | 0.0367 | 0.2188 | 0.1363 | 0.2037 | 0.0181 | 0.0762 | 0.0000 |
DFO_FA | DFO_SW | 0.2637 | 0.6313 | 0.7257 | 0.6190 | 0.2953 | 0.0938 | 0.0000 |
DFO_FA | BFO_HA | 0.1318 | 0.1639 | 0.0179 | 0.0187 | 0.8462 | 0.2728 | 0.0013 |
DFO_FA | BFO_CL | 0.2482 | 0.5497 | 0.4262 | 0.1196 | 0.0047 | 0.0197 | 0.0180 |
DFO_FA | BFO_TU | 0.0815 | 0.6297 | 0.2328 | 0.0097 | 0.0062 | 0.0658 | 0.0002 |
DFO_FA | BFO_JA | 0.0014 | 0.0050 | 0.0018 | 0.0003 | 0.4487 | 0.0229 | 0.0053 |
DFO_FA | BFO_FR | 0.6141 | 0.3764 | 0.8169 | 0.4101 | 0.2486 | 0.1992 | 0.0989 |
DFO_FA | BFO_EL | 0.0057 | 0.0004 | 0.0000 | 0.0016 | 0.1987 | 0.0051 | 0.0015 |
DFO_FA | BFO_BR | 0.4763 | 0.3574 | 0.8937 | 0.3082 | 0.0826 | 0.0324 | 0.0000 |
DFO_FA | AFO_MN | 0.6886 | 0.2519 | 0.2081 | 0.0883 | 0.0281 | 0.6791 | 0.0717 |
DFO_FA | AFO_BA | 0.0669 | 0.0449 | 0.0310 | 0.0258 | 0.2153 | 0.0418 | 0.0085 |
DFO_FA | AFO_JA | 0.4490 | 0.0782 | 0.0994 | 0.1043 | 0.3014 | 0.5205 | 0.0002 |
DFO_FA | AFO_CA | 0.6758 | 0.1053 | 0.3230 | 0.6402 | 0.5172 | 0.5270 | 0.0007 |
DFO_FA | AFO_BR | 0.0500 | 0.0220 | 0.0126 | 0.0065 | 0.0917 | 0.1028 | 0.0022 |
DFO_FA | AFO_HR | 0.6789 | 0.7501 | 0.3642 | 0.7147 | 0.1332 | 0.6252 | 0.0000 |
DFO_SW | BFO_HA | 0.0123 | 0.0785 | 0.0174 | 0.0026 | 0.1515 | 0.0475 | 0.0593 |
DFO_SW | BFO_CL | 0.0121 | 0.2360 | 0.3707 | 0.1099 | 0.0001 | 0.0004 | 0.0083 |
DFO_SW | BFO_TU | 0.0011 | 0.4050 | 0.2033 | 0.0033 | 0.0310 | 0.0007 | 0.0976 |
DFO_SW | BFO_JA | 0.0003 | 0.0015 | 0.0031 | 0.0000 | 0.0496 | 0.0004 | 0.0171 |
DFO_SW | BFO_FR | 0.5051 | 0.1165 | 0.8427 | 0.1131 | 0.8340 | 0.8487 | 0.0001 |
DFO_SW | BFO_EL | 0.0006 | 0.0000 | 0.0000 | 0.0005 | 0.0224 | 0.0001 | 0.0456 |
DFO_SW | BFO_BR | 0.7572 | 0.4820 | 0.9425 | 0.3036 | 0.4167 | 0.3530 | 0.2251 |
DFO_SW | AFO_MN | 0.1477 | 0.0585 | 0.1678 | 0.0654 | 0.3487 | 0.1192 | 0.0003 |
DFO_SW | AFO_BA | 0.0019 | 0.0120 | 0.0093 | 0.0132 | 0.8132 | 0.0021 | 0.0075 |
DFO_SW | AFO_JA | 0.0226 | 0.0224 | 0.0490 | 0.0481 | 0.0276 | 0.0766 | 0.0195 |
DFO_SW | AFO_CA | 0.1274 | 0.0378 | 0.0894 | 0.3498 | 0.4823 | 0.0595 | 0.0070 |
DFO_SW | AFO_BR | 0.0021 | 0.0033 | 0.0049 | 0.0018 | 0.0089 | 0.0054 | 0.0209 |
DFO_SW | AFO_HR | 0.3880 | 0.2626 | 0.2537 | 0.4828 | 0.0083 | 0.5035 | 0.4915 |
BFO_HA | BFO_CL | 0.9513 | 0.9731 | 0.2271 | 0.3662 | 0.0006 | 0.0777 | 0.3749 |
BFO_HA | BFO_TU | 0.5064 | 0.3666 | 0.5149 | 0.4142 | 0.0010 | 0.1618 | 0.6592 |
BFO_HA | BFO_JA | 0.0167 | 0.0676 | 0.5356 | 0.0812 | 0.3518 | 0.1417 | 0.4761 |
BFO_HA | BFO_FR | 0.0224 | 0.8104 | 0.0237 | 0.2662 | 0.2099 | 0.0471 | 0.0816 |
BFO_HA | BFO_EL | 0.2871 | 0.0028 | 0.0001 | 0.1861 | 0.1862 | 0.0292 | 0.7619 |
BFO_HA | BFO_BR | 0.0058 | 0.0055 | 0.0134 | 0.1719 | 0.0450 | 0.0185 | 0.3954 |
BFO_HA | AFO_MN | 0.1612 | 0.9330 | 0.2395 | 0.2790 | 0.0200 | 0.6211 | 0.1104 |
BFO_HA | AFO_BA | 0.3801 | 0.1175 | 0.7059 | 0.8151 | 0.3973 | 0.1023 | 0.6299 |
BFO_HA | AFO_JA | 0.2851 | 0.5953 | 0.6602 | 0.1439 | 0.2956 | 0.9352 | 0.6379 |
BFO_HA | AFO_CA | 0.3445 | 0.7335 | 0.2003 | 0.0535 | 0.4257 | 0.7987 | 0.7599 |
BFO_HA | AFO_BR | 0.6436 | 0.1962 | 0.9308 | 0.7785 | 0.1203 | 0.4216 | 0.5435 |
BFO_HA | AFO_HR | 0.0362 | 0.6174 | 0.1711 | 0.0156 | 0.2814 | 0.1582 | 0.0599 |
BFO_CL | BFO_TU | 0.5961 | 0.6283 | 0.5619 | 0.1265 | 0.0000 | 0.8120 | 0.0929 |
BFO_CL | BFO_JA | 0.1294 | 0.1236 | 0.0498 | 0.0182 | 0.0129 | 0.7315 | 0.8010 |
BFO_CL | BFO_FR | 0.0848 | 0.9880 | 0.3444 | 0.9082 | 0.0001 | 0.0008 | 0.4897 |
BFO_CL | BFO_EL | 0.1677 | 0.0034 | 0.0001 | 0.0667 | 0.0571 | 0.7023 | 0.4701 |
BFO_CL | BFO_BR | 0.0489 | 0.0708 | 0.3673 | 0.6235 | 0.0000 | 0.0004 | 0.0337 |
BFO_CL | AFO_MN | 0.3269 | 0.9241 | 0.7662 | 0.7581 | 0.0000 | 0.0631 | 0.4395 |
BFO_CL | AFO_BA | 0.3296 | 0.1964 | 0.3054 | 0.4243 | 0.0000 | 0.7863 | 0.7680 |
BFO_CL | AFO_JA | 0.4911 | 0.3936 | 0.5403 | 0.8276 | 0.0135 | 0.1133 | 0.4945 |
BFO_CL | AFO_CA | 0.4133 | 0.5021 | 0.7952 | 0.5235 | 0.0002 | 0.0790 | 0.7057 |
BFO_CL | AFO_BR | 0.7020 | 0.2264 | 0.1506 | 0.1050 | 0.0315 | 0.3070 | 0.6009 |
BFO_CL | AFO_HR | 0.0910 | 0.5960 | 0.9389 | 0.4850 | 0.0681 | 0.0104 | 0.0010 |
BFO_TU | BFO_JA | 0.2404 | 0.0053 | 0.0810 | 0.7371 | 0.0001 | 0.9213 | 0.2781 |
BFO_TU | BFO_FR | 0.0052 | 0.4615 | 0.1048 | 0.0762 | 0.0171 | 0.0018 | 0.0107 |
BFO_TU | BFO_EL | 0.3277 | 0.0009 | 0.0001 | 0.6593 | 0.0000 | 0.5430 | 0.3569 |
BFO_TU | BFO_BR | 0.0065 | 0.2628 | 0.2252 | 0.0810 | 0.2388 | 0.0005 | 0.7083 |
BFO_TU | AFO_MN | 0.0244 | 0.2456 | 0.7230 | 0.2342 | 0.3132 | 0.0477 | 0.0260 |
BFO_TU | AFO_BA | 0.7812 | 0.0593 | 0.3814 | 0.7851 | 0.0869 | 0.7293 | 0.0934 |
BFO_TU | AFO_JA | 0.1048 | 0.2020 | 0.7954 | 0.1180 | 0.0003 | 0.2247 | 0.4693 |
BFO_TU | AFO_CA | 0.2572 | 0.2207 | 0.5960 | 0.0134 | 0.0051 | 0.2893 | 0.2474 |
BFO_TU | AFO_BR | 0.5742 | 0.0821 | 0.2215 | 0.4847 | 0.0001 | 0.5475 | 0.3110 |
BFO_TU | AFO_HR | 0.0132 | 0.7617 | 0.4821 | 0.0098 | 0.0002 | 0.0230 | 0.0841 |
BFO_JA | BFO_FR | 0.0000 | 0.0273 | 0.0001 | 0.0168 | 0.0607 | 0.0003 | 0.2326 |
BFO_JA | BFO_EL | 0.7839 | 0.1532 | 0.0031 | 0.6756 | 0.6665 | 0.2554 | 0.5177 |
BFO_JA | BFO_BR | 0.0001 | 0.0001 | 0.0009 | 0.0069 | 0.0022 | 0.0000 | 0.0929 |
BFO_JA | AFO_MN | 0.0051 | 0.0174 | 0.0225 | 0.0129 | 0.0030 | 0.0447 | 0.2076 |
BFO_JA | AFO_BA | 0.6808 | 0.8758 | 0.5786 | 0.3059 | 0.0334 | 0.9008 | 0.8212 |
BFO_JA | AFO_JA | 0.0351 | 0.3617 | 0.0845 | 0.0178 | 0.9625 | 0.1615 | 0.6546 |
BFO_JA | AFO_CA | 0.0049 | 0.3464 | 0.0100 | 0.0000 | 0.1491 | 0.0760 | 0.9725 |
BFO_JA | AFO_BR | 0.1825 | 0.7925 | 0.6562 | 0.1436 | 0.2863 | 0.6541 | 0.9066 |
BFO_JA | AFO_HR | 0.0004 | 0.0369 | 0.0121 | 0.0001 | 0.2403 | 0.0116 | 0.0014 |
BFO_FR | BFO_EL | 0.0007 | 0.0024 | 0.0000 | 0.0430 | 0.0243 | 0.0001 | 0.1026 |
BFO_FR | BFO_BR | 0.7396 | 0.0562 | 0.9728 | 0.8804 | 0.1931 | 0.6061 | 0.0028 |
BFO_FR | AFO_MN | 0.3402 | 0.6681 | 0.0838 | 0.6928 | 0.3363 | 0.1337 | 0.9534 |
BFO_FR | AFO_BA | 0.0126 | 0.1318 | 0.0079 | 0.1637 | 0.8240 | 0.0013 | 0.3970 |
BFO_FR | AFO_JA | 0.2461 | 0.1639 | 0.0327 | 0.7905 | 0.0506 | 0.0260 | 0.1602 |
BFO_FR | AFO_CA | 0.1324 | 0.2862 | 0.1430 | 0.7652 | 0.6541 | 0.0653 | 0.3283 |
BFO_FR | AFO_BR | 0.0162 | 0.2348 | 0.0047 | 0.1717 | 0.0226 | 0.0027 | 0.1385 |
BFO_FR | AFO_HR | 0.8957 | 0.6338 | 0.4099 | 0.6166 | 0.0155 | 0.7785 | 0.0001 |
BFO_EL | BFO_BR | 0.0002 | 0.0000 | 0.0000 | 0.0345 | 0.0002 | 0.0000 | 0.2260 |
BFO_EL | AFO_MN | 0.0089 | 0.0052 | 0.0000 | 0.0447 | 0.0011 | 0.0072 | 0.2265 |
BFO_EL | AFO_BA | 0.5790 | 0.0911 | 0.0007 | 0.2429 | 0.0081 | 0.5451 | 0.7783 |
BFO_EL | AFO_JA | 0.0314 | 0.0710 | 0.0004 | 0.0732 | 0.3790 | 0.0230 | 0.8962 |
BFO_EL | AFO_CA | 0.0255 | 0.0345 | 0.0000 | 0.0043 | 0.0191 | 0.0171 | 0.7502 |
BFO_EL | AFO_BR | 0.2964 | 0.1091 | 0.0012 | 0.3388 | 0.5868 | 0.3081 | 0.9016 |
BFO_EL | AFO_HR | 0.0059 | 0.0015 | 0.0000 | 0.0008 | 0.9453 | 0.0018 | 0.0038 |
BFO_BR | AFO_MN | 0.1362 | 0.0636 | 0.1382 | 0.8231 | 0.8852 | 0.0465 | 0.0013 |
BFO_BR | AFO_BA | 0.0080 | 0.0013 | 0.0367 | 0.2202 | 0.3267 | 0.0017 | 0.2366 |
BFO_BR | AFO_JA | 0.1207 | 0.0005 | 0.0533 | 0.4999 | 0.0005 | 0.0140 | 0.2834 |
BFO_BR | AFO_CA | 0.2180 | 0.0028 | 0.1815 | 0.5277 | 0.1003 | 0.0143 | 0.1146 |
BFO_BR | AFO_BR | 0.0054 | 0.0005 | 0.0088 | 0.1892 | 0.0003 | 0.0023 | 0.1244 |
BFO_BR | AFO_HR | 0.8312 | 0.0631 | 0.4024 | 0.7129 | 0.0006 | 0.5031 | 0.1892 |
AFO_MN | AFO_BA | 0.0373 | 0.1171 | 0.2233 | 0.3143 | 0.2948 | 0.0933 | 0.2197 |
AFO_MN | AFO_JA | 0.4770 | 0.2520 | 0.4602 | 0.9951 | 0.0019 | 0.3867 | 0.1078 |
AFO_MN | AFO_CA | 0.6662 | 0.3075 | 0.7797 | 0.2243 | 0.0789 | 0.7451 | 0.3375 |
AFO_MN | AFO_BR | 0.0855 | 0.1120 | 0.1801 | 0.7379 | 0.0004 | 0.1670 | 0.0964 |
AFO_MN | AFO_HR | 0.5652 | 0.5427 | 0.5334 | 0.3501 | 0.0007 | 0.3941 | 0.0001 |
AFO_BA | AFO_JA | 0.2087 | 0.6851 | 0.4934 | 0.1770 | 0.0507 | 0.2085 | 0.4477 |
AFO_BA | AFO_CA | 0.0610 | 0.4705 | 0.1079 | 0.0151 | 0.4383 | 0.0909 | 0.9597 |
AFO_BA | AFO_BR | 0.6675 | 0.9350 | 0.9289 | 0.8694 | 0.0200 | 0.5074 | 0.7100 |
AFO_BA | AFO_HR | 0.0035 | 0.0610 | 0.0722 | 0.0110 | 0.0037 | 0.0246 | 0.0025 |
AFO_JA | AFO_CA | 0.9262 | 0.7176 | 0.3245 | 0.2117 | 0.1669 | 0.7737 | 0.7711 |
AFO_JA | AFO_BR | 0.4600 | 0.3814 | 0.7048 | 0.3363 | 0.6374 | 0.2145 | 0.5911 |
AFO_JA | AFO_HR | 0.2319 | 0.1083 | 0.4105 | 0.2188 | 0.5315 | 0.1147 | 0.0131 |
AFO_CA | AFO_BR | 0.2065 | 0.4646 | 0.1812 | 0.0153 | 0.0282 | 0.2662 | 0.7071 |
AFO_CA | AFO_HR | 0.2848 | 0.1267 | 0.9936 | 0.9962 | 0.0449 | 0.3155 | 0.0077 |
AFO_BR | AFO_HR | 0.0258 | 0.0622 | 0.0782 | 0.0312 | 0.9389 | 0.0195 | 0.0071 |
Percentage of affected pair | 37.67% | 27.67% | 35.33% | 31.00% | 46.00% | 42.33% | 58.67% |