To appear in IEEE Transactions on Visualization and Computer Graphics.
xx.xxxx/TVCG.201x.xxxxxxx E. Oral, M Wijkstra and E.Dimara are with Utrecht
University, Netherlands Email<EMAIL_ADDRESS>R. Chawl and N. Mahyar are with
University of Massachusetts Amherst, United States

# From Information to Choice: A Critical Inquiry Into
Visualization Tools for Decision Making

Emre Oral0000-0001-9924-2576 Ria Chawla  Michel Wijkstra  Narges Mahyar  and
Evanthia Dimara0000-0001-5212-7888

###### Abstract

In the face of complex decisions, people often engage in a three-stage process
that spans from (1) exploring and analyzing pertinent information
(intelligence); (2) generating and exploring alternative options (design); and
ultimately culminating in (3) selecting the optimal decision by evaluating
discerning criteria (choice). We can fairly assume that all good
visualizations aid in the “intelligence” stage by enabling data exploration
and analysis. Yet, to what degree and how do visualization systems currently
support the other decision making stages, namely “design” and “choice”? To
further explore this question, we conducted a comprehensive review of
decision-focused visualization tools by examining publications in major
visualization journals and conferences, including VIS, EuroVis, and CHI,
spanning all available years. We employed a deductive coding method and in-
depth analysis to assess whether and how visualization tools support design
and choice. Specifically, we examined each visualization tool by (i) its
degree of visibility for displaying decision alternatives, criteria, and
preferences, and (ii) its degree of flexibility for offering means to
manipulate the decision alternatives, criteria, and preferences with
interactions such as adding, modifying, changing mapping, and filtering.  Our
review highlights the opportunities and challenges that decision-focused
visualization tools face in realizing their full potential to support all
stages of the decision making process. It reveals a surprising scarcity of
tools that support all stages, and while most tools excel in offering
visibility for decision criteria and alternatives, the degree of flexibility
to manipulate these elements is often limited, and the lack of tools that
accommodate decision preferences and their elicitation is notable. Based on
our findings, to better support the choice stage, future research could
explore enhancing flexibility levels and variety, exploring novel
visualization paradigms, increasing algorithmic support, and ensuring that
this automation is user-controlled via the enhanced flexibility levels.  Our
curated list of the 88 surveyed visualization tools is available in the OSF
link (https://osf.io/nrasz/?view_only=b92a90a34ae241449b5f2cd33383bfcb).

###### keywords:

Decision making, visualization, state of the art, review, survey, design,
interaction, multi-criteria decision making, MCDM.

##  1 Introduction

Information understanding can take various forms depending on the end goal.
Consider a city planner making critical decisions about urban development and
a researcher analyzing the broader impacts of such projects. Both consider
environmental, social, and economic factors, but their approaches differ. The
city planner’s workflow may direct focus to key factors, breaking down complex
information, synthesizing alternative plans, weighing pros and cons, and
combining data to _choose_ the optimal plan. In contrast, the researcher may
embark on a more exploratory journey, diving deep into various other aspects
of urban development and analyzing them in broader contexts. They may
synthesize findings, identify trends, and draw connections between components,
without the constraints of a specific decision making goal.

Given the distinct needs and objectives of decision makers and other analysts
[26, 62], they would likely benefit from different visualization systems
tailored to their unique information processing requirements. A city planner
may require a visualization system emphasizing key decision criteria and
preferences with interactive capabilities for iterating over decision
alternatives and adapting to decision making needs [21, 102]. In contrast, an
analyst might prefer a system enabling extensive data exploration and the
discovery of connections and patterns without being constrained or guided by
specific criteria or goals [48, 83]. This paper investigates how visualization
design can evolve to accommodate the distinct needs of decision making.

Decision making has undoubtedly been studied in the context of visualization,
including choice experiments based on one [16], two [103, 22] or multiple
criteria [23, 105], and visualizations proposed to assist decision making for
general purposes [15, 68, 35, 98] or within applications such as engineering
[19], urban planning [29], advertisement [57], or sport management [14].
However, recent findings suggest that visualization research may not yet fully
support real-world decision making [26, 25, 86, 13, 3, 52]. Evidence indicates
that professional decision makers are an under-supported community by
visualization [26, 86, 13]. An analysis of 940 commercial tools reported by
the participants revealed that there is no "decision making" visualization
tool that they can use during their decision making workflow [26]. Meanwhile,
decision makers report a desire to improve their understanding of the data
involved in their decisions since coordinating with the data analysis team is
not always effective [26, 39, 47]. Notably, neither the dedicated decision
support software reviews [95, 31] report visualization design as an inherent
component of their functionality (although they inevitably do contain some
visual representations). In addition, research on more complex visual analytic
systems reports on a similar trend. Two recent visualization surveys in the
context of explainable AI [17, 82] revealed that visualization design focuses
on displaying the inner workings of models, but not on how end-users can
involve the model in their decision making process. However, it remains
unclear whether this issue is inherent to AI models or also applies to all
visualization tools designed for decision support. Furthermore, the extent to
which the primary design goal of all aforementioned systems was explicitly
aimed at facilitating human decision making, as well as the specific design
features employed to support the decision making process, is still uncertain.

Additionally, another recent survey [25] revealed that visualization tools
have not been evaluated based on their effectiveness in supporting decision
making tasks. This methodological challenge has been attributed to the lack of
foundations and guidance from decision theory in the visualization
literature[25]. However, this survey [25] focused solely on the choice of
evaluation tasks by the researchers and not on whether and how visualization
designs support decision making.

While evidence suggests a misalignment between decision making goals and
potential visualization support, no survey paper has examined to what degree
and how visualization design aids decision making. Conducting such a survey
entails overcoming certain challenges, particularly the need for a reliable
method to identify ’decision-focused’ visualization tools. Establishing this
distinction is crucial, as a lack of clear criteria could lead to the
oversimplified assumption that all effective visual analytic systems
inherently support decisions. Moreover, even after deriving a list of
decision-focused visualization tools, it remains unclear which criteria,
relevant to decision making, should be used for their systematic assessment.
Addressing both questions requires a deeper understanding of decision making
and its components that can be aided by visualization. Our paper grounds its
investigation on the foundations of decision sciences, evaluating the extent
to which visualizations support specific aspects of the decision making
process.

To examine the extent and manner in which current visualization tools support
decision making, we review 88 papers mentioning the word “decision” in their
titles or abstracts, from leading visualization venues like VIS, EuroVis, and
CHI, covering all available years. Drawing from the literature on decision
making and visualization, we devise an approach to identify "decision-focused"
visualization tools by relying on established stages of decision making,
namely intelligence, design, and choice, along with their embedded properties.
Employing a deductive coding method, we begin with preconceived metrics
derived from the literature that are associated with properties of interactive
visualization design relevant to decision making. Specifically, we assess each
visualization tool based on (i) its degree of visibility, which pertains to
the display of alternative options, criteria, and preferences, and (ii) its
degree of flexibility, which involves offering means to manipulate alternative
options, criteria, and preferences through interactions such as adding,
modifying, changing mappings, and filtering. We ultimately explore and discuss
the opportunities and challenges faced by visualization tools in realizing
their full potential to support every stage of the decision making process.

##  2 Background: Decision Metrics & Tags Rationale

This section discusses literature from decision sciences and visualization to
provide a deeper understanding of what decision making is and why and how it
can be potentially aided by data visualization.

###  2.1 What is Decision Making?

In the introductory example, we outlined differences between decision makers
and other analysts on their information processing workflow [73]. Such
differences have been identified in the literature for decision making,
including _attention_ , _simplification_ , _evaluation_ , and _integration_.

Regarding _attention_ , when processing information for decision making,
attention must often be directed toward key factors relevant to the decision,
filtering out less pertinent information [71, 70]. In our example, the
decision maker may need a visualization tool that emphasizes factors critical
to urban development, like traffic patterns and environmental regulations, and
zoning restrictions. In contrast, the analyst may need to delve into a broader
array of factors, investigating social demographics, economic impacts, and
architectural considerations, all without the constraint of prioritizing
certain factors over others.

In _simplification_ , decision makers often simplify complex information by
breaking it down into manageable chunks, categorizing data, and identifying
patterns or themes [76, 70]. This helps to reduce the cognitive load and make
the decision process more efficient [76, 70]. In contrast, the analyst may not
appreciate prompts for the simplification of complex information into
manageable chunks from a visualization tool in the same way. They might need
tools to probe data complexity, analyze thoroughly, and reveal hidden
relationships.

In _evaluation_ , decision makers evaluate and compare information relevant to
the decision against other alternatives to determine its value or importance
[49]. This involves weighing the pros and cons of various options, assessing
the likelihood of outcomes, and considering the risks and benefits of each
choice [49]. In comparison, the evaluation process for the analyst may be less
focused on weighing the costs and benefits of specific alternatives, and more
on assessing the quality of the data, the robustness of the analysis
methodologies, and the overall significance of their findings for the broader
field of urban development.

In _integration_ , decision makers integrate the information they gathered and
evaluated into a coherent, actionable plan [38, 36]. The city planner may need
a visualization tool that helps identify trade-offs, reconciles conflicting
information, and makes a final decision that balances competing demands and
priorities. For the analyst, integration may be less about reconciling
conflicting information to make a final decision and more about synthesizing
findings to generate new insights, identify trends, and contribute to the
existing body of knowledge.

We previously contrasted decision making with other data analysis types. Now,
we discuss Simon’s three stages of decision making [79]— (1)intelligence, (2)
design, and (3) choice—to help further identify distinct characteristics of
decision making tasks and guide the development of visualization tools
tailored to support decision makers effectively.

####  2.1.1 Decision making stage-1: INTELLIGENCE

In the intelligence stage of Simon’s model [79], the decision maker explores
and analyzes pertinent information. This stage is closely related to the
concept of _attention_ , with the decision maker focusing on key factors
relevant to the decision, filtering out irrelevant or distracting information,
and directing attention towards key data sources.

####  2.1.2 Decision making stage-2: DESIGN

In the design stage of Simon’s model[79], the decision maker generates and
explores alternative options. This stage requires _simplification_ of the
previous stage, breaking complex information down into manageable chunks or
categories. Formal methods for the design stage can be found in the field of
Decision Analysis, which draws on decision theory, operations research,
statistics, and behavioral sciences to provide a structured approach to making
decisions in complex and uncertain situations, developing decision models and
representations of the problem, and identifying decision criteria and
alternatives [49]. Yet, while visualization systems incorporating formal
methods can be effective, formal methods may not fully support all the
creative activities of the design stage that a decision maker engages in, such
as brainstorming, scenario planning, mental simulations, and intuitive
judgments, all of which play a role in generating and exploring alternative
options [18, 51].

####  2.1.3 Decision making stage-3: CHOICE

In the choice stage of Simon’s model[79], the decision maker selects the
optimal decision by evaluating discerning criteria. This stage is closely
related to the _evaluation_ of options generated in the previous stage,
including considerations of weights and preferences, costs and benefits, and
potential outcomes of each option, as well as likelihood assessments of
success. A level of _integration_ occurs in all three stages of Simon’s model
but is especially true during choice, where the decision maker integrates the
information they have gathered and evaluated into a coherent plan or decision.
They may need to reconcile conflicts, prioritize factors, integrate various
data types, and consider decision context.

choice stage studies involving a small set of criteria, such as risk or
profit, are common in visualization [77, 46, 103]. The choice stage with many
criteria is known in visualization literature as a “multi-attribute choice
task” [23] or “preferential choice”, introduced in the context of multi-
criteria decision making visualization tools, such as Value Charts [15, 11],
Attribute Explorer [81], SmartClient [72] and EZChooser [99]. Multi-criteria
decision making (mcdm) is a sub-discipline of operations research that builds
upon decision theory [50] and formalizes and develops methods to aid multi-
attribute choice tasks [102]. mcdm involves the decision maker’s judgment [21]
as input for various mcdm methods for solving decision problems [89]. Examples
of mcdm methods include PROMETHEE, which focuses on ordering criteria, MAUT,
which requires extensive input values from the decision maker, or SAW, which
needs precise weighting operations in each step [89]. According to the mcdm
formalizations, in the choice stage of decision making, the following three
elements are considered:

  * •

Decision _alternatives_ : the options to address a specific problem or goal.
For the city planner, the alternatives might include building a new highway or
improving public transportation.

  * •

Decision _criteria_ : the factors used to evaluate and compare the
alternatives. For the city planner, the decision criteria could involve
environmental impact, cost, and traffic congestion.

  * •

Decision _preferences_ or weights: the importance assigned to each criterion
based on the decision maker’s priorities. For the city planner, the
preferences would depend on their emphasis on sustainability, budget
constraints, or reducing traffic bottlenecks.

####  2.1.4 Conclusion for ’decision-focused’ visualizations

Our previous discussions revolved around decision making stages and
properties. To identify “decision-focused” visualization tools, we follow the
requirement proposed by Dimara & Stasko [25]: a decision-focused visualization
tool should at minimum support the choice stage, visualizing some decision
alternatives. Inclusion of intelligence and design stages can be optional but
advantageous. For the choice stage, mcdm methods and properties share a common
design requirement: flexible iterations for comparisons [89], supported by
transparency to establish the “one-to-one correspondence” between criteria
values and the evaluation of decision alternatives [89].  Yet, it is essential
to acknowledge that excluding tools lacking a choice stage may not fully
encompass the value of visualization in other decision stages. For instance,
medical diagnostic tools play a significant role in the intelligence stage by
aiding in the identification of problems and specifying symptoms. Therefore,
future research should explore the role of visualizations across all stages
and types of decision making.

Figure 1: An illustration of a decision matrix of a decision maker in urban
development consisting of _criteria_ , _alternatives_ , and _preferences_.
While the new highway may be more cost-effective and reduce congestion to a
greater extent, the improved public transportation has less negative
environmental impact, reflecting the decision maker’s preference (i.e. weight
function) for environmental considerations.

###  2.2 Decision VISIBILITY

Visualization research has studied decision making in critical application
domains such as engineering design [19] and civics [45] as well as in domain-
agnostic setups of personal [27, 23], and group [41, 61] decision making. A
consistent observation among these works is the need for transparency by means
of direct retrieval of the decision _criteria_ , _alternatives_ , and often
_preferences_ (or weights) as illustrated in Fig. 1.

The visibility of a system’s functionalities is crucial for effective
interface design, as emphasized by HCI and Design expert [67, 66]. Nielsen’s
first usability heuristic suggests that system design should keep users
informed and provide timely feedback [66], while Norman highlights visibility
as a basic design principle. For decision making, visible criteria and
transparent factors enhance informed decisions, trust, and satisfaction [92,
40, 44]. Researchers note that the lack of transparency in machine learning
algorithms, obscuring decision criteria, negatively impacts users [17, 82].
Transparent computational approaches further help decision makers to figure
out conflicts and differences in opinions and build common ground [61].

Applying transparency to complex problems and system design is not a trivial
task. One great counterexample is using reduction techniques such as
dimensionality reduction. Dimensionality reduction (dr) techniques help with
visual exploration of high-dimensional data by projecting the data into low-
dimensional space, mostly 2D or 3D [85]. For example, [84] implemented a
method to present the points on low-dimensional space to represent the trade-
offs of the decision space. However, despite the important benefits of such
reduction techniques, decision makers see them unfavorably, even in domains
with complex Pareto optimization problems because they hinder the direct
retrieval of any criteria value from any alternative [19].

The importance of supporting transparency amplifies in sociopolitical decision
making such as elections, urban planning, and environmental and health
policies where various stakeholders have high stakes in decisions [60].
Especially due to the inherently political nature of civic decisions,
transparency in decision making becomes increasingly important [10]. Yet,
given the size and complexity of data and supporting the visibility of
features, various functionalities and criteria in complex systems remain a
challenge that has not been explored yet.

This paper examines the extent to which decision-focused visualization tools
(Sec. 2.1.4) support transparency, particularly in terms of decision
_criteria_ , _alternatives_ , and _preferences_ , by providing methods for
making them visible and comprehensible to all stakeholders.

Figure 2: Classes of interactions of our flexibility metric based on the
Dimara & Perin interaction taxonomy [24].

###  2.3 Decision FLEXIBILITY

Enhancing visualization interactivity has been suggested to counteract
decision biases. By examining the interaction logs of decision makers,
researchers can identify behavioral signs of cognitive bias in visual
analytics [90]. A strategy influenced by the mcdm method “elimination by
aspects” has demonstrated its effectiveness in reducing decision biases, as it
prompts the exclusion of irrelevant information from the display [22].
Decision makers across various roles, organizational sizes, and sectors (e.g.,
commercial, nonprofit, health, government, education) have expressed concerns
over the limited interactivity of current visualization tools, calling for
more data input options and a more adaptable organization of their data within
customized layouts [26]. We now unpack what this reported lack of flexibility
entails.

Figure 3: Our methodology for paper collection and content analysis of
“decision-focused” visualization tools in the final 27 papers.

Flexibility in visualization has been connected to the degree of interactivity
offered by a visualization system. Flexibility is defined as “the number of
distinct, allowable actions of a person on the interface, as well as the
number of interaction means with which the person can perform each action”, a
concept based on Gibson’s idea of affordance, which refers to “what the
environment offers the individual” [24]. To measure the flexibility of a
visualization system, researchers have proposed an interaction taxonomy that
includes seven classes of interactions: input (interactions with raw data) or
their functions), processing (interactions with functions over raw data),
mapping (changing or choosing data representation), presentation (modifying
the specific data representation), as well as meta, social and interface (non-
data interactions) [24] (see Fig. 2).

Among these, presentation is the most prevalent interaction class, enabling
users to alter the specific data presentation through features such as
highlighting, filtering and zooming (i.e., standard “interaction techniques”
[101])). Another class is processing, which allows users to interact with
functions over the raw data, like modifying the underlying models or their
parameters. mapping, a less common class, deals with whether users can alter
or even select the way data are represented. mapping interactions, as
demonstrated by sketch-based data visualization [100], are essential for
decision makers to enable representations that align with their mental models
of the decision space [26]. input interactions, which are usually the least
implemented in visualization systems [43], are particularly crucial for
decision makers [26]. input interactions involve systems permitting decision
makers to incorporate their knowledge by editing or adding new criteria not
present in the dataset, following design recommendations by Huron & Willet on
using visualizations as input methods [42].

This paper assesses visualization tools on their flexibility for decision
making, encompassing all the aforementioned interaction classes. Fig. 2.b
introduces three additional “non-data” interaction classes, with social [26]
being particularly critical for group-level decisions [8]. Our focus is on
_data_ comprising criteria, alternatives, and preferences, which are critical
for implementing the choice stage. The three extra interaction classes do not
explicitly operate on these data elements but rather on external components
[24]. Thus, we have excluded these classes from our flexibility metric,
concentrating our analysis on the interactions that directly impact the
decision making process.

To address decision makers’ requests for increased interactivity [26], we
examine the extent to which current decision-focused visualization tools are
designed to support flexibility by providing methods for making them
accessible to all stakeholders.

##  3 Methodology

In this study, we conducted a detailed qualitative content analysis of
visualization tools that support decision making. This section outlines our
paper collection process (Fig. 3), coding process, and the final derived
metrics and tags.

###  3.1 Paper Collection Process

We collected papers from all years in all major visualization venues, TVCG,
VIS, InfoVis, VAST, EuroVis, CGF, and CHI, using their respective libraries,
IEEE (computer.org/csdl/), ACM (dl.acm.org), Eurographics (diglib.eg.org) and
Google Scholar. We considered only full papers with the keyword “decision” in
the title or abstract, as it encompasses a wide range of decision-related
concepts, such as “decision making support”, “decision aid/assistance”,
“multi-criteria decision making” and “decision analysis”. This broad query
aimed to capture the largest possible number of papers focusing on tools
designed to support decision making. We use the term “tool” to refer to all
types of contributions of visualization solutions based on the IEEE VIS area
model, such as new visual representations and techniques, systems as well as
application-specific solutions. We then removed the papers where the word
“decision” referred to designer decisions, algorithmic decisions, or did not
contain data visualizations (most common in CHI entries). Our choice of
visualization venues, search engines, and keywords are identical to a survey
on visualization and decision making tools [25]. However, we excluded
empirical, theoretical or BELIV workshop papers because our focus was on
papers that proposed a tool.

From the initial 88 papers, we carefully reviewed each manuscript, focusing
particularly on the abstract, introduction, design rationale, and conclusion.
This allowed us to narrow down our selection to 60 papers where the authors
claimed that their tool supports decision making. Ultimately, we identified 27
papers on the design of “decision-focused” visualization tools for further
analysis (see Fig. 3 and Sec. 2.1.4 where the term “decision-focused” was
clarified).

###  3.2 Coding Process

Our coding approach was primarily deductive incorporating elements of
inductive methods through iterative refinement of metric definitions and tag
applications [28]. We employed a team-based coding approach [59] involving
five coders (the authors of this paper) with distinct roles and
responsibilities. The assessment of visualization tools was based on the
design section of each paper, additional figures, video demos, supplementary
materials (if available), and online searches for web-based versions of the
tools or video demonstrations provided by the authors or found on YouTube and
Vimeo.

Two independent primary coders[59] were responsible for initially coding the
tools using the predefined metrics of visibility and flexibility. To calculate
inter-rater reliability, we determined Cohen’s Kappa [20] after each
iteration. While there are no strict guidelines on interpreting Cohen’s Kappa
K value, a score of K $\leq$ 0.7 is considered satisfactory for inter-coder
reliability, and a score of K $\geq$ 0.75 is considered excellent [32]. The
first iteration consisted of verbal discussion and rough guidelines for
scoring tools. The two coders independently scored 10 tools, obtaining a score
of K = 0.55. This score represented moderate agreement, which was deemed
unsatisfactory. In the next iteration, after evaluating discrepancies in
scoring, the guidelines were strengthened and revised. The two coders
independently scored 10 different tools, reaching a reliability score of K =
0.81, justifying the use of a single coder to review the remaining tools. A
different lead coder [59] reviewed all the tools coded by the independent
coders, expanded the metric applications, and applied the additional tags.
This lead coder was not independent, as they had seen the scores of the
independent coders. Two consulting coders reviewed tool subsets and met
regularly with the other coders [59] to discuss unclear points, refinements of
tags and metrics, and provide insights.

###  3.3 Final Tags & Metrics Scoring System

This section presents the final specifications for tags and metrics based on
our analysis of the literature from decision sciences and visualization (Sec.
2) and our content analysis and coding process (Sec. 3.2). Our analysis uses
23 tags. The first tag is the dm-goal ({Yes, No}), determining if the authors
claim their tool supports decision making (applied to 88 tools). If dm-goal is
’Yes,’ the decision making stage tags (See definitions in Sec. 2) are applied
to the remaining 60 tools: intelligence, design, and choice ({Yes, No} for
each). If choice is ’Yes,’ the visualization design assessment tags are
applied to the 27 tools. These tags are subdivided into three categories:

  1. 1.

Visibility Tags: _criteria_ , _alternatives_ , and _preferences_ ({Yes,
Partly, No} for each), assessing the visibility of these components in the
visualization.

  2. 2.

Flexibility Tags: input, mapping, processing, and presentation ({Yes, No} for
each of the _criteria_ , _alternatives_ , and _preferences_), evaluating the
tool’s modifiability of these elements. See interaction classes definitions in
Sec. 2.3.

  3. 3.

Other Tags: mcdm ({Yes, No}), indicating if the tool supports multi-criteria
decision making; dr/da ({Yes, No}), determining if the tool offers
dimensionality reduction or criteria aggregation; ai ({Yes, No}), denoting if
the choice stage is aided by AI or ML; and generalizability ({Yes, No}),
identifying if the authors claim that their tool applies to generic datasets.

To identify the user allowable tasks, the coders used the definitions from the
interaction framework reviewed in Sec. 2.3, but several other interaction
taxonomies could have been used (see [24] for a review). These tasks include
input, where the decision maker can add, edit, or delete alternatives (self-
decided or based on a recommendation), criteria, and preferences (exact
numbers or weight approximations). The decision maker can also perform mapping
to switch to or create a new representation of alternatives, criteria, or
preferences. In processing, the decision maker can modify or add new functions
for alternatives (e.g., a new alternatives creation model), criteria (e.g.,
aggregation or model parameters), and preferences (e.g., mcdm weight
function). Finally, the decision maker can adjust the presentation of
alternatives, criteria, and preferences through specific representations
(e.g., select, highlight, navigate, or zoom).

Our simplified coding scheme with three visibility levels, results from the
detail level in the tool descriptions from some papers. Coders couldn’t
extract more granularity due to insufficient explanations. Future research
could benefit from additional granularity in metrics, especially if used
during the design phase of a new visualization.

Next, we discuss the scoring system of our metrics derived from the tags
visibility and flexibility. The visibility metric ranges from 0 to 12 and
assigns a score of 4 for ’Yes,’ 2 for ’Partly,’ and 0 for ’No’ for each of the
three visibility tags (e.g., for _criteria_). The sum of these scores for all
visibility tags yields the total visibility score. The flexibility metric also
ranges from 0 to 12, with each ’Yes’ for a specific class (e.g., input) and
decision element (e.g., _criteria_) receiving a score of 1. The sum of these
scores for all flexibility tags yields the total flexibility score.

We note that the numerical scores assigned to flexibility and visibility
metrics do not carry any inherent meaning, and their absolute values should
not be used for direct comparison. To prevent any misinterpretation of the
importance of one metric over the other, we assigned a maximum of 12 points to
both flexibility and visibility. These scores provide a means to compare the
relative performance of tools within their respective categories. For
instance, a higher flexibility score may suggest a tool supports a larger
number of interaction classes or more decision elements. Furthermore, tools
with identical scores may still differ in the specific areas of flexibility or
visibility in which they excel. For example, one tool may offer more
flexibility in criteria, while another may have greater flexibility in
weights.

Figure 4:  Summary of the 60 visualization tools explicitly designed to
facilitate decision making, categorized based on their support for each
decision making stage, flexibility, visibility, and additional tags and
metrics. Notably, only 27 tools support the choice and 11 the design stage.
Figure 5:  A mosaic of representative figures for the 27 decision-focused
tools supporting the choice stage, arranged alphabetically. The diverging
color scale highlights the visibility and flexibility scores, with greener
shades indicating above-average scores (i.e., 6/12) and red shades indicating
below-average scores. In Sec. 2, we discuss how our metrics (visibility and
flexibility) and our analysis per decision making stages are grounded in
decision sciences literature. The presence of a dark blue G square denotes
generalizability, indicating that the authors suggest their tool can be
applied to generic datasets, while a light blue tag suggests the tool is more
specific in its application. This mosaic aims to provide a comprehensive
visual overview of how visualization design supports the choice stage in
decision making processes. However, it is important to note that these scores
simply indicate the number of various features offered by each tool, and
should not be interpreted as negative or positive judgments. In certain domain
applications, having too many features may hinder efficiency and
understandability.

##  4 Results

This section presents the results of our content analysis of the 60
visualization tools designed to support decision making, from the 88 decision-
relevant papers published in major visualization journals and conferences over
all available years. Fig. 4 displays our tagging, as described in Sec. 3
(black square for ’Yes’, white for ’No’) and metric scores (black bar charts).
Fig. 5 showcases representative figures of the 27 decision-focused tools that
support the choice stage, using a diverging color scale. The complete curated
list of the 88 tools is available in the supplementary materials, for which we
have provided the OSF link in the abstract section. For each visualization
tool, we will use the notation ([citation], column number in Fig. 4, letter in
Fig, 5).

###  4.1 How visible are the decision-focused visualizations?

####  4.1.1 Visibility of Decision Alternatives

All 27 decision-focused tools show alternatives in a fully visible manner.
While most of them facilitate the comparison of alternatives on the same
screen, Andrienko _et al._ ([4], 21, c) is an exception, which presents
alternatives sequentially. Andrienko _et al._ represents the parameters of an
evacuation plan, and a planner can explore each criterion and monitor how the
new plans perform based on the number of people rescued, the capacity of the
vehicles at any time, and the status of nearby cities in which people are
evacuated. The sequential presentation allows for a step-by-step exploration
of each dimension and the creation of a new plan based on the changing
parameters. Overall, the high level of visibility of alternatives in the
reviewed tools indicates that visualization designers recognize the importance
of making all decision alternatives clearly visible to decision makers.

Besides visibly presenting alternatives, some tools provide features to
enhance scalability and user experience with numerous alternatives. For
example, BNVA ([97], 11, d) allows users to expand and collapse alternative
sets based on shared features, such as bus routes with shared stops, to
improve the scalability of the system for larger datasets. Similarly,
ManyPlans ([94], 6, l) and Ribicic _et al._ ([74], 7, q) enable users to
collapse different connected branches into a single track while still
providing the ability to expand the original branches, enhancing the user
experience for growing alternative sets.

####  4.1.2 Visibility of Decision Criteria

Out of the 27 decision making tools we analyzed, criteria are fully visible in
20, while the remaining tools only partially display criteria due to
deliberate design choices. Afzal et al. ([1], 19, a) acknowledged their design
limitation, as users can monitor only one criterion on the decision history
tree’s y-axis. Ribicic et al. ([74], 7, q) and WorldLines ([93], 8, y) showed
a subset of criteria in their evaluation view, even though the flood manager
user requested to balance cost, time, and effectiveness. AHP Treemaps ([5],
10, b) allows users to see only the title of criteria in a treemap and not
their values, while Hi-Trees ([63], 27, i) displays facts about alternatives
(cons, pros, etc.) without a clear set of criteria. Pelt et al. ([88], 20, n)
show only treatment-focused criteria on their 3D layered model view, and
Ziegler et al. ([106], 25, z) present risk and assets performance on a
Dominance Plot.

Sec. 2.2 suggested that criteria obscuring concerns are often associated with
dimensionality reduction or data aggregation. However, some reviewed tools
with the dr/da tag, like SRVis ([96], 9, t) and Skylens ([104], 16, r),
facilitate the exploration of similarities between alternatives while still
maintaining full visibility of criteria. In addition to the projection view
that shows alternatives in 2D space, they offer other representations such as
tabular views (SRVis, Skylens) and radar charts (Skylens) displaying all
criteria transparently. Moreover, there were tools with data aggregation that
offered full criteria visibility, like FairSight ([2], 14, e) and Kreiser _et
al._ , ([53], 26, j) . Although FairSight only shows aggregated utility and
fairness measures of each ranking on the ranking list view, a user can see the
details of all criteria in the feature inspection view. Similarly, although
Kreiser _et al._ acknowledged the importance of focusing user attention on
treatment-specific information, their design choice kept both decision-
critical aggregated swallow data and individual data on the same screen while
highlighting decision-critical information with highly saturated colored
regions.

####  4.1.3 Visibility of Decision Preferences

Despite most tools displaying alternatives and criteria with full visibility,
as discussed in Sec. 4.1.1 and Sec. 4.1.2, only 7 of them fully show decision
preferences. The most common method, found in 5 of these 7 tools, including
Podium ([91], 2, o) , LiteVis ([80], 3, k) , ReACH ([98], 4, p) , WeightLifter
([68], 5, x) and SRVis ([96], 9, t) , enables users to rank alternatives and
observe the outcome weights on stacked bar graphs. Notably, WeightLifter also
features a unique triangular visual representation that facilitates
sensitivity analysis of weight changes. In contrast, AHP treemaps ([5], 10, b)
and Zooids ([54], 1, aa) present decision preferences using different
representations. AHP treemaps allows users to input their decision problem,
criteria titles, and alternatives in a hierarchical treemap structure with
entities coded as rectangles. Decision makers can adjust rectangle sizes to
express their preference for different alternatives and criteria. Zooids,
which are mini robots, demonstrated a use case for an admission committee
selecting the best candidate to enroll. Committee members can load candidate
data into candidate zooids by placing an empty zooid onto candidate data on a
tablet screen. Similarly, they can place a magnet zooid on a criterion on the
tablet. To assign different weights to various criteria, users adjust the
attraction force of magnet zooids by rotating them clockwise or
counterclockwise, encoding different criteria values and eliciting preference
weights for each criterion.

###  4.2 How flexible are the decision-focused visualizations?

####  4.2.1 Flexibility of Decision Alternatives

Concerning the presentation of alternatives, our analysis shows that all
tools, except 6, enable users to modify the alternatives’ presentation using
at least one interaction medium (like hovering, brushing and linking,
selecting and highlighting, filtering, adjusting visual marks, or panning and
zooming). Although our binary scoring solely captures the availability of at
least one of these interaction techniques, our further design analysis reveals
that some tools provide multiple techniques (e.g., WeightLifter ([68], 5, x) ,
WorldLines ([93], 8, y) , Pelt et al. ([88], 20, n) . Tools lacking
alternatives presentation flexibility, like FairSight ([2], 14, e) , show
alternatives rankings in a tabular view with static sliders representing
different criteria utility and fairness values, without any interaction for
manipulating the presentation of rankings.

FairSight ([2], 14, e) is the only tool enabling interaction at the processing
level, allowing to modify functions or add models to create alternatives (see
Sec.,3.3). While 11 tools offer ways to create new alternatives using existing
functions (e.g., FinVis ([75], 12, f) uses a menu to input criteria values and
a button to add the new alternative to the tabular view and line graph), they
do not provide an interactive way to modify or add another function.

Like the presentation-level flexibility for alternatives, we found that most
tools provide users the ability to change mapping by switching or creating new
visual representations of alternatives. For example, Pelt et al. ([88], 20, n)
built a visualization system in which a zoomable multilayered 3D model of a
brain vessel shows several stent configurations (alternatives) in different
visual representations, such as plain disks, flower glyphs, or radar charts at
each zoom level. FinVis ([75], 12, f) enables users to switch between a
tabular view and a line chart to monitor alternative investments in different
visual representations. In contrast to Pelt et al., most tools assist
flexibility of alternatives mapping only by switching between different
windows (e.g., SmartClient Travels ([72], 17, s) , ReACH ([98], 4, p) , Podium
([91], 2, o) , ManyPlans ([94], 6, l) ). Furthermore, Zooids ([54], 1, aa)
assists users with a broad range of possibilities, allowing to place mini
robots in any configuration , such as freely placing them on the table and
using magnet zooids to see the position of alternative zooids with respect to
magnet (i.e., criteria) or creating a scatterplot formation of alternatives.

On alternatives input, we found only 11 tools allowing to interactively add
alternatives, crucial for the design decision stage. Among them, LiteVis
([80], 3, k) , ManyPlans ([94], 6, l) , Ribicic et al. ([74], 7, q) and
WorldLines ([93], 8, y) support alternative creation by clicking nodes on a
tree view, inputting parameters using 2D or 3D sketch views, or selecting a
region in a 3D simulation view and inputting simulation parameters in another
steering view. FairSight ([2], 14, e) uses a machine-learning algorithm where
users select different parameter priorities to create ranking alternatives
from a candidate list, enabling fair and unbiased decisions. Afzal et al.
([1], 19, a) allow inputting various epidemic countermeasures using a pop-up
menu to enter values for individual measures via text fields. Zooids ([54], 1,
aa) offers a highly flexible method for creating alternatives by placing a
zooid on a tablet’s touch screen to load a candidate’s data.

####  4.2.2 Flexibility of Decision Criteria

Almost all tools offer flexible criteria at the presentation level, with the
exceptions of Kreiser et al. ([53], 26, j) and Hi-Trees ([63], 27, i) . The
remaining tools enable criteria flexibility at the presentation level by
simply highlighting STRATOS ([6], 24, u), navigating (e.g., changing the angle
of the 3D model in Pelt et al. ([88], 20, n) ), filtering criteria values
using sliders (e.g., Vismon ([12], 18, w); Free Flight ([7], 15, g)), Guo et
al ([37], 23, i) ), checkboxes in Afzal et al. ([1], 19, a), slider bar on a
histogram in BNVA ([97], 11, d), dragging and dropping criteria values to
change their order in AHP Treemaps ([5], 10, b), and brushing on parallel
coordinates (e.g., WeightLifter ([68], 5, x)); adding stickers onto physical
mini robots in Zooids ([54], 1, aa), collapsing and expanding line ensembles
for different simulation parameters in ManyPlans ([94], 6, l) and Ribicic _et
al._ ([74], 7, q). In contrast to alternatives, where FairSight ([2], 14, e)
offers flexible alternatives at the processing level, we did not find any
tools that assist processing level interaction of criteria.

On mapping, most tools are interactive via switching to different windows
(e.g., Podium ([91], 2, o), LiteVis ([80], 3, k), ReACH ([98], 4, p) , BNVA
([97], 11, d)). For some tools, it was possible to switch to other visual
representations of criteria by using a zoom function (Pelt et al. ([88], 20,
n)). Zooids ([54], 1, aa) enables users to pick up and place robots anywhere
in any form of visual representation, demonstrating the flexibility of
physicalization to facilitate decision makers in evaluating criteria from
different perspectives.

Only 9 tools support flexible criteria input. ReACH ([98], 4, p) users can
input reachability constraints, such as departure and arrival times, affecting
reachability scores visualized as a heatmap. ManyPlans ([94], 6, l) , Ribicic
_et al._ ([74], 7, q) and WorldLines ([93], 8, y) users can modify criteria
for all alternatives, like water velocity or embankment type. Urbane ([30],
13, v) users can adjust criteria, such as building location and height, using
select and steer options in 3D mode to evaluate urban development suitability
in each region.

####  4.2.3 Flexibility of Decision Preferences

We found limited flexibility of preferences compared to alternatives and
criteria. At the presentation level, only 3 tools enable flexibility of
preferences. Zooids ([54], 1, aa) lets users modify the attraction force of a
magnet zooid (criteria) and observe changes in the light’s brightness. Podium
([91], 2, o), allows toggling the visibility of bars representing criterion
weight. In WeightLifter ([68], 5, x) , users can filter weight values (e.g.,
setting a minimum weight for a particular criterion). No tools were found that
support processing-level interaction for preferences. Importantly, only Podium
provides mapping flexibility, enabling users to view preferences in various
visual representations by switching between views (e.g., vertical lines in
tabular view and red/green bars in the control panel under the attributes
tag).

Only 7 tools support input flexibility of preferences. Most have a ranking
view and feature stacked bars in a tabular layout, allowing to add preferences
by changing the size of rectangles representing criteria (e.g., LiteVis ([80],
3, k) , ReACH ([98], 4, p) , WeightLifter ([68], 5, x) SRVis ([96], 9, t)) .
Instead, to input preferences, Podium ([91], 2, o) , allows to drag and drop
rows in the tabular view, and the model calculates weights based on users’
manual ranking. Users can also change their preferences by adjusting the size
of horizontal bars in the control panel. With Zooids ([54], 1, aa) , users can
modify the attraction force of magnet zooids to input their preferences for a
criterion. AHP treemaps ([5], 10, b) enable users to input their preferences
for criteria and alternatives using the hook and pump tools.

###  4.3 AI Assistance, Domains & Lack of Choice Support

Only 5 decision-focused tools provide algorithmic assistance (e.g., for
preference elicitation, ranking, scheduling, event sequence prediction, and
treatment recommendation). Podium ([91], 2, o) utilizes Ranking SVM to
facilitate users’ preference elicitation process. Podium users can drag and
drop rows to manually rank the alternatives, and the model calculates inferred
weights for criteria. Andrienko _et al._ ([4], 21, c) use a genetic algorithm
to automatically create an evacuation schedule and FairSight ([2], 14, e)
implemented their own machine learning model based on top-k ranking to enable
users to identify possible biases (e.g., on gender) and also select different
ranking algorithms such as RankSVM, Logistic Regression, and SVM. Muller et
al. ([65], 22, m) utilize BNs to create the optimal treatment option for
cancer treatment. Guo et al ([37], 23, i) utilize TRNN to predict the event
sequence, the outcome of which is later shown in a disk glyph. Most of these
tools lack the ability to enable users to change model parameters to support
the design stage of their decision making process.

Different domains, as shown in Fig. 4 Gen, exhibit various design patterns and
focus on different aspects of the decision making process. In domains where
trade-off analysis is critical, more emphasis is placed on the design stage,
and interactive sketching is utilized to explore alternative solutions.
Examples include flood management ( Ribicic _et al._ ([74], 7, q) , ManyPlans
([94], 6, l) ), urban planning ( Urbane ([30], 13, v) ), and lighting design (
LiteVis ([80], 3, k) ). In the finance domain, tools such as FinVis ([75], 12,
f) and Ziegler et al. ([106], 25, z) , tend to have simpler designs focusing
on selecting assets and portfolios. In the health domain, designers
prioritized displaying decision-critical criteria. Understanding domain-
specific challenges guides the creation of tailored decision making tools.
Despite the importance of focusing doctors on key information, the paper
didn’t clarify why access to all criteria was restricted. Interestingly, our
top-2 flexible tools are domain-general tools (Zooids and Podium).

The other 33 tools, which assert support for decision making, focus solely on
data exploration and analysis. For instance, MatchPad offers real-time
analysis of match statistics and events on a tablet interface, claiming to aid
coaches in decision making. While exploring data is useful, decision makers
may require more support in the design and choice stages for informed
decisions. Similarly, tools like Zhang _et al._ and TaxiVis support urban data
analysis but lack alternatives. Nguyen _et al._ approaches decision making
more closely by facilitating multi-faceted decision making in cybersecurity,
but ultimately, focuses on identifying and investigating unusual action
sequences.

##  5 Discussion

Our extensive search for visualization tools designed to support decision
making uncovered a scarcity of decision-focused tools (see Sec. 2.1.4).
Despite the vastness of the decision making domain, we were surprised to find
a number of tools in our corpus comparable to that of visualization surveys on
more specialized and recent topics, such as Bitcoin visualization [87]. In
these tools, we found that support for decision preferences and flexibility
was inadequate, particularly in terms of input and processing interactions, as
well as preferences mapping, despite the visibility of alternatives and
criteria. Next, we delve deeper into our findings and their implications for
decision makers.

###  5.1 Implications of Missing Features in Decision Tools

Missing certain features in visualization tools can have implications for
decision effectiveness. Our findings revealed limited support for automating
certain parts of the decision making process, including AI assistance,
dimensionality reduction, and decision guidance. This is surprising given the
complexity of modern decisions, the abundance of data, and the increasing
number of criteria to consider. The integration of AI in visualization systems
has been widely explored [17, 82], highlighting its potential also for
decision making. Future research can focus on facilitating the choice stage,
especially when dealing with numerous alternatives or attributes, by
incorporating recommendations with user control via enhanced flexibility
levels of processing interactions. This in turn will help enhance the
scalability of tools. For instance, categorical colors can be used to encode
decision criteria, such as stacked bar visualizations [68, 80]. Additionally,
AI assistance in preference elicitation, an area where MCDM research can
provide valuable insights [102], can shed light on the complexity of the
preference elicitation process. Assisting decision makers in this area not
only benefits their decision making process but also supports the design of
evaluation protocols that rely on ground truth metrics [23]. Decision
strategies also need to be recommended to decision makers [22]. Interestingly,
our review did not identify any tool that suggests a way of deciding, an area
where MCDM could offer valuable guidance. Inadequate automation support for
human-driven decision making may fuel arguments for complete automation, given
the potential for human errors due to data complexity.

In addition to the absence of automation support, the lack of input features
in many of these tools further restricts opportunities for decision makers to
incorporate their own knowledge [56], despite prior user feedback demanding
such features in existing works [64, 23]. Furthermore, the scarcity of input
features also affects the stages of decision making they support, particularly
the design stage, which is currently only addressed by a few tools. During
this stage, scenario simulations necessitate the input of new alternatives,
potentially explaining the limited number of tools that incorporate
simulations and 3D model views showcasing their corresponding effects [93, 94,
74, 30]. To address this limitation, visualization research could explore the
use of visualization as data inputs, as suggested by Huron and Willet [42].

Furthermore, it’s not just whether certain features are supported but also
their quality. For example, we observed standard mapping features that allow
decision makers to switch between existing and effective in terms of precision
representations, like tabular layouts [9, 23], stacked bars [78, 35], parallel
coordinates [33, 23], and spatial layouts for specific domains [97]. Yet,
these representations were predetermined by the designers. Decision makers
could only view their decision space through the eyes of the designer. In
previous research, when asked to draw decision-related information, decision
makers seemed to create different layouts, like flow diagrams, pros and cons
lists, and unique trees [26]. This underlines the importance of visualization
research investigating how to allow decision makers to view their decision
space within their own _mental model_ [58], e.g., via abstracted components,
sketching, or physical manipulation like Zooids [54]. Such flexibility would
be crucial for the decision making process, which, besides support from formal
decision analysis methods [49], can be enhanced with brainstorming,
collaboration, synthesis, and planning features [18, 51].

###  5.2 Extending Decision Support Features: An Example

In Sec. 1 and 2, we discussed the information processing requirements of
decision makers, using the city planner scenario as an example (Fig. 1). Now,
let’s illustrate how visualization tools can be extended to incorporate
decision support features, continuing with the city planner scenario. In our
scenario, the decision maker, using the urban planning-relevant tool BNVA
(from our review, Fig. 4.11, Fig. 5.d), addresses a traffic issue. Currently,
the tool already enables a good range of features, including support for all
decision stages and an interactive map for determining the impact of different
bus route implementations on traffic conditions.

To enhance BNVA, we propose adding input functionalities to encompass a
broader range of alternative solutions, such as new bike lanes or highways.
This expansion would also enable city planners to include decision criteria
beyond pre-defined sets, permitting iterative adjustments to solution rankings
based on new evaluation criteria. To create a flexible decision space, BNVA
could transcend traditional formats like tabular layouts and embrace sketch-
based capabilities [100]. Mirroring the flexibility offered by Zooids [54],
within a digital whiteboard, decision makers can project their mental models
onto visual representations, annotating and comparing solutions, such as
constructing a new highway or improving public transportation. Planners could
then employ data-aware sticky notes to annotate the new highway solution with
remarks like “reduced travel times” or “increased road capacity”. Infusing
BNVA with automation capabilities could guide planners further by spotlighting
solutions aligning with their preferences, akin to Podium [91], facilitating
revisions within the decision space.

However, it is important to note that not all decision making tools must
incorporate each feature identified by our review’s tags and metrics to be
considered effective. Understanding the specific needs and context of decision
making is crucial. In some cases, simplicity and speed may be paramount, while
in others, complex features are needed to address domain complexity. However,
we found papers often lack discussions of design trade-offs in the context of
decision making tasks. The next section highlights the importance of the
underlying design process to identify decision makers’ needs in various
contexts.

###  5.3 Design Process for Tailored Decision Tools

Effective design involves not only determining which features to include but
also following a suitable methodology to derive and refine those features. For
that, designers need to be able to concretize the decision problem at hand.
However, during our survey, we encountered difficulties in extracting concrete
information about the decision elements, such as alternatives, criteria, and
preferences from most papers. We are concerned that this may reflect a less
tailored design process followed for the creation of decision making tools.
Tool designs should be influenced by user profiles, encompassing decision
makers with diverse goals, domains, and varying data analysis and
visualization proficiency [10]. Prior research suggests that novices would
need simplified functionalities, whereas experts might prefer advanced
features, customization, and enhanced visual control. In our survey, we
noticed unexpected patterns between user profiles and flexibility scores. Most
tools assumed expert users and provided limited flexibility, while a few were
designed for novices, such as Zooids and ReACH offering highly flexible
features. We observed insufficient involvement of decision makers in the
design of decision-focused tools. Even when decision makers were involved, it
was usually post-initial prototyping or during the final evaluation.
Consistent with prior reviews [25], few studies asked participants to
undertake decision tasks. This lack of early involvement and minimal attention
to decision makers and tasks during the requirement analysis phase can lead to
poorly defined decision problems. To optimize tool effectiveness, early
engagement with real decision makers is vital, delivering insights into needs,
preferences, and practices and ensuring context alignment [60]. We suggest
adopting comprehensive methods, including interviews, requirements, and task
analysis for a thorough delineation of the decision making problem.

##  6 Limitations

Our study has limitations that should be considered when interpreting our
findings. Our visibility metric is based on what the coders could
independently agree and code, with the limited value range in our tags due to
the lack of detailed descriptions in some of the papers. Moreover, our
flexibility metric emphasizes the diversity of interactions across classes
rather than the number of interactions within each class. These scores
indicate the number of various features offered by each tool and should not be
interpreted as negative or positive judgments. Moreover, our study does not
assess the real effectiveness of interactions or representations, and we hope
our metrics, tags, and analysis can help design evaluations in future
research. Additionally, papers without the word “decision” in abstracts or
titles might still be relevant tools [35] or concepts like comparison [34] for
decision making. Venues other than those we reviewed such as from the GIS [55]
or PubMed [69] community, may also have pertinent papers.

##  7 Conclusion

Our survey underscores a surprising scarcity of visualization tools that
support all stages of the decision making process. We suggest future research
augment decision making tools’ flexibility, involving exploration of novel
paradigms like physical visualizations or immersive technologies, catering to
diverse flexibility levels, enabling decision makers to navigate complex
decisions. Concurrently, designers could intertwine algorithmic support with
user control via the enhanced flexibility levels, thereby aiding preference
articulation and oversight. We hope that the metrics and review of decision
sciences presented in this study serve as valuable resources for designing
novel visualization tools. Designers can leverage these metrics to scrutinize
and enrich their designs with additional flexibility and visibility levels
while addressing domain-specific challenges by considering decision
alternatives and criteria preferences explicitly. Our study aims to empower
all decision makers with effective visualization tools, from the everyday
citizen to the highest levels of leadership, to make better-informed decisions
and move seamlessly from _information_ to _choice_.

###### Acknowledgements.

We thank M. Jasim for his precious feedback and the authors of the papers in
Fig. 5 who kindly and swiftly provided their own figures.

## References

  * [1] S. Afzal, R. Maciejewski, and D. S. Ebert.  Visual analytics decision support environment for epidemic modeling and response evaluation.  In Proc. VAST, pp. 191–200. IEEE, Piscataway, 2011. doi: 10 . 1109/VAST . 2011 . 6102457
  * [2] Y. Ahn and Y.-R. Lin.  Fairsight: Visual analytics for fairness in decision making.  IEEE TVCG, 26(1):1086–1095, 2020. doi: 10 . 1109/TVCG . 2019 . 2934262
  * [3] T. Alves, T. Delgado, J. Henriques-Calado, D. Gonçalves, and S. Gama.  Exploring the role of conscientiousness on visualization-supported decision-making.  Comput. Graphics, 111:47–62, 2023. doi: 10 . 1016/j . cag . 2023 . 01 . 010
  * [4] G. Andrienko, N. Andrienko, and U. Bartling.  Visual analytics approach to user-controlled evacuation scheduling.  In Proc. VAST, pp. 43–50. IEEE, 2007. doi: 10 . 1109/VAST . 2007 . 4388995
  * [5] T. Asahi, D. Turo, and B. Shneiderman.  Visual decision-making: using treemaps for the analytic hierarchy process.  In Proc. CHI, pp. 405–406. ACM, 1995. doi: 10 . 1145/223355 . 223747
  * [6] B. A. Aseniero, T. Wun, D. Ledo, G. Ruhe, A. Tang, and S. Carpendale.  Stratos: Using visualization to support decisions in strategic software release planning.  In Proc. CHI, pp. 1479–1488. ACM, New York, 2015. doi: 10 . 1145/2702123 . 2702426
  * [7] R. Azuma, H. Neely, M. Daily, and M. Correa.  Visualization of conflicts and resolutions in a" free flight" scenario.  In Proc. VIS, pp. 433–436. IEEE, Piscataway, 1999.  Color plate on page 557. doi: 10 . 1109/VISUAL . 1999 . 809923
  * [8] S. Bajracharya, G. Carenini, B. Chamberlain, K. Chen, D. Klein, D. Poole, H. Taheri, and G. Öberg.  Interactive visualization for group decision analysis.  Int. J. Inf. Technol. Decis. Making, 17(6):1839–1864, 2018. doi: 10 . 1142/S0219622018500384
  * [9] L. Bartram, M. Correll, and M. Tory.  Untidy data: The unreasonable effectiveness of tables.  IEEE TVCG, 28(1):686–696, 2022. doi: 10 . 1109/TVCG . 2021 . 3114830
  * [10] E. P. Baumer, M. Jasim, A. Sarvghad, and N. Mahyar.  Of course it’s political! a critical inquiry into underemphasized dimensions in civic text visualization.  Comput. Graphics Forum, 41(3):1–14, 2022. doi: 10 . 1111/cgf . 14518
  * [11] J. Bautista and G. Carenini.  An integrated task-based framework for the design and evaluation of visualizations to support preferential choice.  In Proc. AVI, pp. 217–224. ACM, 2006. doi: 10 . 1145/1133265 . 1133308
  * [12] M. Booshehrian, T. Möller, R. M. Peterman, and T. Munzner.  Vismon: Facilitating analysis of trade-offs, uncertainty, and sensitivity in fisheries management decision making.  Comput. Graphics Forum, 31(3pt3):1235–1244, 2012. doi: 10 . 1111/j . 1467-8659 . 2012 . 03116 . x
  * [13] M. Brehmer and R. Kosara.  From jam session to recital: Synchronous communication and collaboration around data in organizations.  IEEE TVCG, 28(1):1139–1149, 2022. doi: 10 . 1109/TVCG . 2021 . 3114760
  * [14] A. Cao, J. Lan, X. Xie, H. Chen, X. Zhang, H. Zhang, and Y. Wu.  Team-builder: Toward more effective lineup selection in soccer.  IEEE TVCG, pp. 1–16, 2022.  Early Access. doi: 10 . 1109/TVCG . 2022 . 3207147
  * [15] G. Carenini and J. Loyd.  Valuecharts: analyzing linear models expressing preferences and evaluations.  In Proc. AVI, pp. 150–157. ACM, New York, 2004. doi: 10 . 1145/989863 . 989885
  * [16] S. C. Castro, P. S. Quinan, H. Hosseinpour, and L. Padilla.  Examining effort in 1d uncertainty communication using individual differences in working memory and nasa-tlx.  IEEE TVCG, 28(1):411–421, 2022. doi: 10 . 1109/TVCG . 2021 . 3114803
  * [17] A. Chatzimparmpas, R. M. Martins, I. Jusufi, K. Kucher, F. Rossi, and A. Kerren.  The state of the art in enhancing trust in machine learning models with the use of visualizations.  Comput. Graphics Forum, 39(3):713–756, 2020. doi: 10 . 1111/cgf . 14034
  * [18] T. J. Chermack.  Scenario planning in organizations: how to create, use, and assess scenarios.  Berrett-Koehler Publishers, San Francisco, 2011.
  * [19] L. Cibulski, H. Mitterhofer, T. May, and J. Kohlhammer.  Paved: Pareto front visualization for engineering design.  Comput. Graphics Forum, 39(3):405–416, 2020. doi: 10 . 1111/cgf . 13990
  * [20] J. Cohen.  A coefficient of agreement for nominal scales.  EPM, 20(1):37–46, 1960. doi: 10 . 1177/001316446002000104
  * [21] R. K. Dhurkari.  Mcdm methods: Practical difficulties and future directions for improvement.  Rairo Oper. Res., 56(4):2221–2233, 2022. doi: 10 . 1051/ro/2022060
  * [22] E. Dimara, G. Bailly, A. Bezerianos, and S. Franconeri.  Mitigating the attraction effect with visualizations.  IEEE TVCG, 25(1):850–860, 2019. doi: 10 . 1109/TVCG . 2018 . 2865233
  * [23] E. Dimara, A. Bezerianos, and P. Dragicevic.  Conceptual and methodological issues in evaluating multidimensional visualizations for decision support.  IEEE TVCG, 24(1):749–759, 2018. doi: 10 . 1109/TVCG . 2017 . 2745138
  * [24] E. Dimara and C. Perin.  What is interaction for data visualization?  IEEE TVCG, 26(1):119–129, 2020. doi: 10 . 1109/TVCG . 2019 . 2934283
  * [25] E. Dimara and J. Stasko.  A critical reflection on visualization research: Where do decision making tasks hide?  IEEE TVCG, 28(1):1128–1138, 2022. doi: 10 . 1109/TVCG . 2021 . 3114813
  * [26] E. Dimara, H. Zhang, M. Tory, and S. Franconeri.  The unmet data visualization needs of decision makers within organizations.  IEEE TVCG, 28(12):4101–4112, 2022. doi: 10 . 1109/TVCG . 2021 . 3074023
  * [27] B. Dy, N. Ibrahim, A. Poorthuis, and S. Joyce.  Improving visualization design for effective multi-objective decision making.  IEEE TVCG, 28(10):3405–3416, 2022. doi: 10 . 1109/TVCG . 2021 . 3065126
  * [28] S. Elo and H. Kyngäs.  The qualitative content analysis process.  JAN, 62(1):107–115, 2008. doi: 10 . 1111/j . 1365-2648 . 2007 . 04569 . x
  * [29] Z. Feng, H. Li, W. Zeng, S.-H. Yang, and H. Qu.  Topology density map for urban data visualization and analysis.  IEEE TVCG, 27(2):828–838, 2021. doi: 10 . 1109/TVCG . 2020 . 3030469
  * [30] N. Ferreira, M. Lage, H. Doraiswamy, H. Vo, L. Wilson, H. Werner, M. Park, and C. Silva.  Urbane: A 3d framework to support data driven decision making in urban development.  In Proc. VAST, pp. 97–104. IEEE, Piscataway, 2015. doi: 10 . 1109/VAST . 2015 . 7347636
  * [31] J. Figueira, S. Greco, and M. Ehrgott.  Multiple criteria decision analysis: State of the art surveys.  Springer, New York, 2005. doi: 10 . 1007/b100605
  * [32] J. L. Fleiss, B. Levin, and M. C. Paik.  Statistical methods for rates and proportions.  John Wiley & Sons, Inc., New Jersey, 2003. doi: 10 . 1002/0471445428
  * [33] J. Gettinger, E. Kiesling, C. Stummer, and R. Vetschera.  A comparison of representations for discrete multi-criteria decision problems.  Decis. Support Syst., 54(2):976–985, 2013. doi: 10 . 1016/j . dss . 2012 . 10 . 023
  * [34] M. Gleicher.  Considerations for visualizing comparison.  IEEE TVCG, 24(1):413–423, 2018. doi: 10 . 1109/TVCG . 2017 . 2744199
  * [35] S. Gratzl, A. Lex, N. Gehlenborg, H. Pfister, and M. Streit.  Lineup: Visual analysis of multi-attribute rankings.  IEEE TVCG, 19(12):2277–2286, 2013. doi: 10 . 1109/TVCG . 2013 . 173
  * [36] L. A. Greening and S. Bernow.  Design of coordinated energy and environmental policies: use of multi-criteria decision-making.  Energy Policy, 32(6):721–735, 2004. doi: 10 . 1016/j . enpol . 2003 . 08 . 017
  * [37] S. Guo, F. Du, S. Malik, E. Koh, S. Kim, Z. Liu, D. Kim, H. Zha, and N. Cao.  Visualizing uncertainty and alternatives in event sequence predictions.  In Proc. CHI, pp. 573:1–573:12. ACM, New York, 2019. doi: 10 . 1145/3290605 . 3300803
  * [38] C. Ham and J. Oldham.  Integrating health and social care in england: lessons from early adopters and implications for policy.  JICA, 17(6):3–9, 2009. doi: 10 . 1108/14769018200900040
  * [39] Y. Han.  Kicking analysts out of the meeting room: Supporting future data-driven decision making with intelligent interactive visualization systems.  In Proc. TREX, pp. 16–21. IEEE, Piscataway, 2022. doi: 10 . 1109/TREX57753 . 2022 . 00007
  * [40] A. Handler, N. Mahyar, and B. O’Connor.  Clioquery: Interactive query-oriented text analytics for comprehensive investigation of historical news archives.  ACM TiiS, 12(3):22:1–22:49, 2022. doi: 10 . 1145/3524025
  * [41] E. Hindalong, J. Johnson, G. Carenini, and T. Munzner.  Abstractions for visualizing preferences in group decisions.  ACM HCI, 6(CSCW1):49:1–49:44, 2022. doi: 10 . 1145/3512896
  * [42] S. Huron and W. Willett.  Visualizations as data input?  In Proc. VIS. IEEE, 2021.  presented at alt.VIS workshop, https://altvis.github.io/2021.html.
  * [43] Y. Jansen and P. Dragicevic.  An interaction model for visualizations beyond the desktop.  IEEE TVCG, 19(12):2396–2405, 2013. doi: 10 . 1109/TVCG . 2013 . 134
  * [44] M. Jasim, C. Collins, A. Sarvghad, and N. Mahyar.  Supporting serendipitous discovery and balanced analysis of online product reviews with interaction-driven metrics and bias-mitigating suggestions.  In Proc. CHI, pp. 9:1–9:24. ACM, New York, 2022. doi: 10 . 1145/3491102 . 3517649
  * [45] M. Jasim, E. Hoque, A. Sarvghad, and N. Mahyar.  Communitypulse: Facilitating community input analysis by surfacing hidden insights, reflections, and priorities.  In Proc. DIS, pp. 846–863. ACM, New York, 2021. doi: 10 . 1145/3461778 . 3462132
  * [46] A. Kale, M. Kay, and J. Hullman.  Visual reasoning strategies for effect size judgments and decisions.  IEEE TVCG, 27(2):272–282, 2021. doi: 10 . 1109/TVCG . 2020 . 3030335
  * [47] E. Kandogan, A. Balakrishnan, E. M. Haber, and J. S. Pierce.  From data to insight: work practices of analysts in the enterprise.  IEEE CG&A, 34(5):42–50, 2014. doi: 10 . 1109/MCG . 2014 . 62
  * [48] D. Keefe.  Integrating visualization and interaction research to improve scientific workflows.  IEEE CG&A, 30(2):8–13, 2010. doi: 10 . 1109/MCG . 2010 . 30
  * [49] R. L. Keeney.  Decision analysis: an overview.  Oper. Res., 30(5):803–838, 1982. doi: 10 . 1287/opre . 30 . 5 . 803
  * [50] R. L. Keeney and H. Raiffa.  Decisions with multiple objectives: preferences and value trade-offs.  Cambridge University Press, Cambridge, 1993.
  * [51] G. Klein.  Naturalistic decision making.  Hum. Factors, 50(3):456–460, 2008. doi: 10 . 1518/001872008X288385
  * [52] R. Kosara.  Notebooks for data analysis and visualization: Moving beyond the data.  IEEE CG&A, 43(1):91–96, 2023. doi: 10 . 1109/MCG . 2022 . 3222024
  * [53] J. Kreiser, A. Hann, E. Zizer, and T. Ropinski.  Decision graph embedding for high-resolution manometry diagnosis.  IEEE TVCG, 24(1):873–882, 2018. doi: 10 . 1109/TVCG . 2017 . 2744299
  * [54] M. Le Goc, C. Perin, S. Follmer, J.-D. Fekete, and P. Dragicevic.  Dynamic composite data physicalization using wheeled micro-robots.  IEEE TVCG, 25(1):737–747, 2019. doi: 10 . 1109/TVCG . 2018 . 2865159
  * [55] N. J. Lim, S. A. Brandt, and S. Seipel.  Visualisation and evaluation of flood uncertainties based on ensemble modelling.  Int. J. Geogr. Inf. Sci., 30(2):240–262, 2016. doi: 10 . 1080/13658816 . 2015 . 1085539
  * [56] H. Lin, D. Akbaba, M. Meyer, and A. Lex.  Data hunches: Incorporating personal knowledge into visualizations.  IEEE TVCG, 29(1):504–514, 2023. doi: 10 . 1109/TVCG . 2022 . 3209451
  * [57] D. Liu, D. Weng, Y. Li, J. Bao, Y. Zheng, H. Qu, and Y. Wu.  Smartadp: Visual analytics of large-scale taxi trajectories for selecting billboard locations.  IEEE TVCG, 23(1):1–10, 2017. doi: 10 . 1109/TVCG . 2016 . 2598432
  * [58] Z. Liu and J. Stasko.  Mental models, visual reasoning and interaction in information visualization: A top-down perspective.  IEEE TVCG, 16(6):999–1008, 2010. doi: 10 . 1109/TVCG . 2010 . 177
  * [59] K. M. MacQueen, E. McLellan, K. Kay, and B. Milstein.  Codebook development for team-based qualitative analysis.  Field Methods, 10(2):31–36, 1998. doi: 10 . 1177/1525822X980100020301
  * [60] N. Mahyar, M. Jasim, and A. Sarvghad.  Designing technology for sociotechnical problems: challenges and considerations.  IEEE CG&A, 40(6):76–87, 2020. doi: 10 . 1109/MCG . 2020 . 3017405
  * [61] N. Mahyar, W. Liu, S. Xiao, J. Browne, M. Yang, and S. P. Dow.  Consesnsus: Visualizing points of disagreement for multi-criteria collaborative decision making.  In Proc. CSCW, pp. 17–20. ACM, New York, 2017. doi: 10 . 1145/3022198 . 3023269
  * [62] N. Mahyar, D. V. Nguyen, M. Chan, J. Zheng, and S. P. Dow.  The civic data deluge: Understanding the challenges of analyzing large-scale community input.  In Proc. DIS, pp. 1171–1181. ACM, New York, 2019. doi: 10 . 1145/3322276 . 3322354
  * [63] K. Marriott, P. Sbarski, T. van Gelder, D. Prager, and A. Bulka.  Hi-trees and their layout.  IEEE TVCG, 17(3):290–304, 2011. doi: 10 . 1109/TVCG . 2010 . 45
  * [64] N. McCurdy, J. Gerdes, and M. Meyer.  A framework for externalizing implicit error using visualization.  IEEE TVCG, 25(1):925–935, 2019. doi: 10 . 1109/TVCG . 2018 . 2864913
  * [65] J. Müller, M. Cypko, A. Oeser, M. Stoehr, V. Zebralla, S. Schreiber, S. Wiegand, A. Dietz, and S. Oeltze-Jafra.  Visual assistance in clinical decision support.  In Proc. EuroVis. Eurographics, Geneva, 2021. doi: 10 . 2312/evm . 20211075
  * [66] J. Nielsen.  Ten usability heuristics.  Website, 2020.  Accessed: 30 June 2023, https://www.nngroup.com/articles/ten-usability-heuristics/.
  * [67] D. Norman.  The design of everyday things: Revised and expanded edition.  Basic Books, New York, 2013.
  * [68] S. Pajer, M. Streit, T. Torsney-Weir, F. Spechtenhauser, T. Möller, and H. Piringer.  Weightlifter: Visual weight space exploration for multi-criteria decision making.  IEEE TVCG, 23(1):611–620, 2017. doi: 10 . 1109/TVCG . 2016 . 2598589
  * [69] S. Park, B. Bekemeier, A. Flaxman, and M. Schultz.  Impact of data visualization on decision-making and its implications for public health practice: a systematic literature review.  Inform. Health Soc. Care, 47(2):175–193, 2022. doi: 10 . 1080/17538157 . 2021 . 1982949
  * [70] J. W. Payne and J. R. Bettman.  Walking with the scarecrow: The information-processing approach to decision research.  In Blackwell Handbook of Judgment and Decision Making, chap. 6, pp. 110–132. John Wiley& Sons, Inc, Newark, 2004. doi: 10 . 1002/9780470752937 . ch6
  * [71] J. W. Payne, J. R. Bettman, and E. J. Johnson.  Adaptive strategy selection in decision making.  JEP:LMC, 14(3):534–552, 1988. doi: 10 . 1037/0278-7393 . 14 . 3 . 534
  * [72] P. Pu and B. Faltings.  Enriching buyers’ experiences: the smartclient approach.  In Proc. CHI, pp. 289–296. ACM, New York, 2000. doi: 10 . 1145/332040 . 332446
  * [73] V. F. Reyna and C. J. Brainerd.  Dual processes in decision making and developmental neuroscience: A fuzzy-trace model.  Dev. Rev., 31(2-3):180–206, 2011. doi: 10 . 1016/j . dr . 2011 . 07 . 004
  * [74] H. Ribicic, J. Waser, R. Gurbat, B. Sadransky, and M. E. Gröller.  Sketching uncertainty into simulations.  IEEE TVCG, 18(12):2255–2264, 2012. doi: 10 . 1109/TVCG . 2012 . 261
  * [75] S. Rudolph, A. Savikhin, and D. S. Ebert.  Finvis: Applied visual analytics for personal financial planning.  In Proc. VAST, pp. 195–202. IEEE, Piscataway, 2009. doi: 10 . 1109/VAST . 2009 . 5333920
  * [76] J. E. Russo and B. A. Dosher.  Strategies for multiattribute binary choice.  JEP:LMC, 9(4):676–696, 1983. doi: 10 . 1037/0278-7393 . 9 . 4 . 676
  * [77] A. Savikhin, H. C. Lam, B. Fisher, and D. S. Ebert.  An experimental study of financial portfolio selection with visual analytics for decision support.  In Proc. HICSS, pp. 1–10. IEEE, Piscataway, 2011. doi: 10 . 1109/HICSS . 2011 . 54
  * [78] H. Siirtola.  Bars, pies, doughnuts & tables–visualization of proportions.  In Proc. BCS-HCI, pp. 240–245. BCS, Swindon, 2014. doi: 10 . 14236/ewic/hci2014 . 30
  * [79] H. A. Simon.  The new science of management decision. Harper & Brothers, New York, 1960. doi: 10 . 1037/13978-000
  * [80] J. Sorger, T. Ortner, C. Luksch, M. Schwärzler, E. Gröller, and H. Piringer.  Litevis: integrated visualization for simulation-based decision support in lighting design.  IEEE TVCG, 22(1):290–299, 2016. doi: 10 . 1109/TVCG . 2015 . 2468011
  * [81] R. Spence and L. Tweedie.  The attribute explorer: information synthesis via exploration.  Interact. Comput., 11(2):137–146, 1998. doi: 10 . 1016/S0953-5438(98)00022-8
  * [82] F. Sperrle, M. El-Assady, G. Guo, R. Borgo, D. H. Chau, A. Endert, and D. Keim.  A survey of human-centered evaluations in human-centered machine learning.  Comput. Graphics Forum, 40(3):543–568, 2021. doi: 10 . 1111/cgf . 14329
  * [83] C. Stolte, D. Tang, and P. Hanrahan.  Polaris: A system for query, analysis, and visualization of multidimensional relational databases.  IEEE TVCG, 8(1):52–65, 2002. doi: 10 . 1109/2945 . 981851
  * [84] A. K. A. Talukder and K. Deb.  Paletteviz: A visualization method for functional understanding of high-dimensional pareto-optimal data-sets to aid multi-criteria decision making.  IEEE CIM, 15(2):36–48, 2020. doi: 10 . 1109/MCI . 2020 . 2976184
  * [85] Z. Tian, X. Zhai, G. van Steenpaal, L. Yu, E. Dimara, M. Espadoto, and A. Telea.  Quantitative and qualitative comparison of 2d and 3d projection techniques for high-dimensional data.  Information, 12(6):239:1–239:21, 2021. doi: 10 . 3390/info12060239
  * [86] M. Tory, L. Bartram, B. Fiore-Gartland, and A. Crisan.  Finding their data voice: Practices and challenges of dashboard users.  IEEE CG&A, 43(1):22–36, 2023. doi: 10 . 1109/MCG . 2021 . 3136545
  * [87] N. Tovanich, N. Heulot, J.-D. Fekete, and P. Isenberg.  Visualization of blockchain data: a systematic review.  IEEE TVCG, 27(7):3135–3152, 2021. doi: 10 . 1109/TVCG . 2019 . 2963018
  * [88] R. van Pelt, R. Gasteiger, K. Lawonn, M. Meuschke, and B. Preim.  Comparative blood flow visualization for cerebral aneurysm treatment assessment.  Comput. Graphics Forum, 33(3):131–140, 2014. doi: 10 . 1111/cgf . 12369
  * [89] M. Velasquez and P. T. Hester.  An analysis of multi-criteria decision making methods.  IJOR, 10(2):56–66, 2013.
  * [90] E. Wall, L. M. Blaha, L. Franklin, and A. Endert.  Warning, bias may occur: A proposed approach to detecting cognitive bias in interactive visual analytics.  In Proc. VAST, pp. 104–115. IEEE, Piscataway, 2017. doi: 10 . 1109/VAST . 2017 . 8585669
  * [91] E. Wall, S. Das, R. Chawla, B. Kalidindi, E. T. Brown, and A. Endert.  Podium: Ranking data using mixed-initiative visual analytics.  IEEE TVCG, 24(1):288–297, 2018. doi: 10 . 1109/TVCG . 2017 . 2745078
  * [92] R. Wang, R. Bush-Evans, E. Arden-Close, E. Bolat, J. McAlaney, S. Hodge, S. Thomas, and K. Phalp.  Transparency in persuasive technology, immersive technology, and online marketing: Facilitating users’ informed decision making and practical implications.  Comput. Hum. Behav., 139:107545:1–107545:15, 2023. doi: 10 . 1016/j . chb . 2022 . 107545
  * [93] J. Waser, R. Fuchs, H. Ribičič, B. Schindler, G. Blöschl, and E. Gröller.  World lines.  IEEE TVCG, 16(6):1458–1467, 2010. doi: 10 . 1109/TVCG . 2010 . 223
  * [94] J. Waser, A. Konev, B. Sadransky, Z. Horváth, H. Ribičić, R. Carnecky, P. Kluding, and B. Schindler.  Many plans: Multidimensional ensembles for visual decision support in flood management.  Comput. Graphics Forum, 33(3):281–290, 2014. doi: 10 . 1111/cgf . 12384
  * [95] H. R. Weistroffer and Y. Li.  Multiple criteria decision analysis software.  In Multiple criteria decision analysis: state of the art surveys, chap. 29, pp. 1301–1341. Springer, 2016. doi: 10 . 1007/978-1-4939-3094-4_29
  * [96] D. Weng, R. Chen, Z. Deng, F. Wu, J. Chen, and Y. Wu.  Srvis: Towards better spatial integration in ranking visualization.  IEEE TVCG, 25(1):459–469, 2019. doi: 10 . 1109/TVCG . 2018 . 2865126
  * [97] D. Weng, C. Zheng, Z. Deng, M. Ma, J. Bao, Y. Zheng, M. Xu, and Y. Wu.  Towards better bus networks: A visual analytics approach.  IEEE TVCG, 27(2):817–827, 2021. doi: 10 . 1109/TVCG . 2020 . 3030458
  * [98] D. Weng, H. Zhu, J. Bao, Y. Zheng, and Y. Wu.  Homefinder revisited: Finding ideal homes with reachability-centric multi-criteria decision making.  In Proc. CHI, pp. 247:1–247:12. ACM, New York, 2018. doi: 10 . 1145/3173574 . 3173821
  * [99] K. Wittenburg, T. Lanning, M. Heinrichs, and M. Stanton.  Parallel bargrams for consumer-based information exploration and choice.  In Proc. UIST, pp. 51–60. ACM, New York, 2001. doi: 10 . 1145/502348 . 502357
  * [100] H. Xia, N. Henry Riche, F. Chevalier, B. De Araujo, and D. Wigdor.  Dataink: Direct and creative data-oriented drawing.  In Proc. CHI EA, pp. D413:1–D413:13. ACM, New York, 2018. doi: 10 . 1145/3170427 . 3186471
  * [101] J. S. Yi, Y. ah Kang, J. Stasko, and J. A. Jacko.  Toward a deeper understanding of the role of interaction in information visualization.  IEEE TVCG, 13(6):1224–1231, 2007. doi: 10 . 1109/TVCG . 2007 . 70515
  * [102] E. K. Zavadskas and Z. Turskis.  Multiple criteria decision making (mcdm) methods in economics: an overview.  Technol. Econ. Dev. Econ., 17(2):397–427, 2011. doi: 10 . 3846/20294913 . 2011 . 593291
  * [103] Y. Zhang, R. K. Bellamy, and W. A. Kellogg.  Designing information for remediating cognitive biases in decision-making.  In Proc. CHI, pp. 2211–2220. ACM, New York, 2015. doi: 10 . 1145/2702123 . 2702239
  * [104] X. Zhao, Y. Wu, W. Cui, X. Du, Y. Chen, Y. Wang, D. L. Lee, and H. Qu.  Skylens: Visual analysis of skyline on multi-dimensional data.  IEEE TVCG, 24(1):246–255, 2018. doi: 10 . 1109/TVCG . 2017 . 2744738
  * [105] Y. Zhao, F. Luo, M. Chen, Y. Wang, J. Xia, F. Zhou, Y. Wang, Y. Chen, and W. Chen.  Evaluating multi-dimensional visualizations for understanding fuzzy clusters.  IEEE TVCG, 25(1):12–21, 2019. doi: 10 . 1109/TVCG . 2018 . 2865020
  * [106] H. Ziegler, T. Nietzschmann, and D. A. Keim.  Relevance driven visualization of financial performance measures.  In Proc. EuroVis, pp. 19–26. Eurographics, Geneva, 2007. doi: 10 . 2312/VisSym/EuroVis07/019-026