# The Chart Excites Me! Exploring How Data Visualization Design Influences
Affective Arousal

Xingyu Lan, Yanqiu Wu, Qing Chen, and Nan Cao Xingyu Lan, Yanqiu Wu, Qing
chen, and Nan Cao are with Intelligent Big Data Visualization Lab at Tongji
University.
E-mail: {xingyulan96, wuyanqiu.idvx, jane.qing.chen<EMAIL_ADDRESS>Nan
Cao is the corresponding author.

###### Abstract

As data visualizations have been increasingly applied in mass communication,
designers often seek to grasp viewers immediately and motivate them to read
more. Such goals, as suggested by previous research, are closely associated
with the activation of emotion, namely affective arousal. Given this
motivation, this work takes initial steps toward understanding the arousal-
related factors in data visualization design. We collected a corpus of 265
data visualizations and conducted a crowdsourcing study with 184 participants
during which the participants were asked to rate the affective arousal
elicited by data visualization design (all texts were blurred to exclude the
influence of semantics) and provide their reasons. Based on the collected
data, first, we identified a set of arousal-related design features by
analyzing user comments qualitatively. Then, we mapped these features to
computable variables and constructed regression models to infer which features
are significant contributors to affective arousal quantitatively. Through this
exploratory study, we finally identified four design features (e.g.,
colorfulness, the number of different visual channels) cross-validated as
important features correlated with affective arousal.

###### Index Terms:

Data Visualization, Affective Design, Visual Communication, General Public,
User Experience.

##  1 Introduction

Data visualization is now widely used in mass communication in forms such as
infographics, data-driven articles, and data videos [1, 2, 3]. Due to the
traits of mass communication (e.g., information overload, impatient viewers),
visualization designers often need to activate viewers emotionally to grasp
them immediately and motivate them to read more. In research, this design goal
is closely related to a concept called affective arousal, which measures the
activation of emotion on a continuous scale [4]. Research from fields such as
marketing and communications has provided abundant evidence that affective
arousal can moderate people’s willingness to spend time and money on visual
media [5, 6]or lead to behaviors such as clicking and sharing [7], thereby
making it an important perspective of investigating visual communication [8].
In the visualization community, some practitioners have also stressed the
importance of affective arousal. For example, a data visualization that looks
exciting is usually more desirable in advertising as it can attract more
clicks, likes, and sharing [9]. For nonprofits, designers may want a data
visualization to elicit high-arousal affects in order to raise funds and call
for social collaboration [10].  In addition, previous work has shown that
design elements such as color and imagery can effectively influence arousal
levels [11, 8], which from another perspective confirms the possibility of
conducting research on the relationship between data visualization design and
affective arousal.

Although many researchers have acknowledged that affects are an indispensable
part of users’ experience with data [12, 13, 1], previous literature has
mostly viewed affects as discrete categories rather than examining them in a
continuous space. For example, Kennedy and Hill [14] investigated people’s
affective responses to data visualizations by conducting a diary study. The
affects reported by the participants were mostly categorical, such as surprise
and disgust. Boy et al. [15] focused on examining how data visualization
design influences one particular type of affect, empathy. Lan et al. [16]
proposed an affective animation design scheme for common charts to convey
positive affects such as joy and tenderness in data stories. In their recent
work about serious data storytelling [17], the researchers focused on
examining how design methods facilitate the communication of negative emotions
(e.g., anxiety, helplessness). The work by Bartram et al. [13] is more
relevant to affective arousal. They proposed a set of color palettes to help
encode eight types of affect that represent typical combinations of valence
and arousal (e.g., positive, negative, calm, exciting). However, this study
still simplified affects as categories and did not show how affects change in
the dimensional space continuously. To sum up, although the above work has
contributed to our understanding of how data visualization design influences
affects, as yet no systematic work has been carried out to address this
problem through the lens of affective arousal.

To fill this gap, this study first explores what data visualization design
factors may influence the perception of affective arousal. To begin with, we
prepared a corpus containing 265 data visualization images based on the
MASSVIS dataset [18], which provides single visualizations from balanced
sources such as infographics, news media, and governments. To prevent the bias
caused by semantics (because the content of the images can strongly influence
affects), we blurred all the texts on these images. Based on this corpus, we
conducted a crowdsourcing study involving 184 participants and asked the
participants to assess the affective arousal elicited by the designs in the
corpus. We then analyzed the experimental results and derived a set of design
features that the participants thought related to affective arousal by coding
their comments manually. These design features were grouped into three main
categories: color, graphics, and information. Next, we mapped these features
to computable variables and constructed regression models to identify the most
significant features and assess their importance. By comparing the results of
the models, we finally identified four design features that were cross-
validated as important features correlated with affective arousal, including
colorfulness, the number of different visual channels, the X and Y
coordinates, and the grid layout. Finally, based on all the quantitative and
qualitative analyses above, we discuss the implications that arise from our
work, reflect on the limitations of this work, and propose future research
opportunities.

##  2 Related Work

###  2.1 Data Visualization for Communication

In recent years, a notable phenomenon in the visualization community is that
data visualization has been massively applied in mass communication (i.e., the
process of creating, sending, imparting, and exchanging messages to a large
number of audiences) through media such as infographics, data-driven articles,
and data videos [19, 1, 3]. However, on the other hand, the features of
contemporary mass communication have posed new challenges for visualization
design. As found by research in communications et al. [20], the abundance of
information produced by mass media today has narrowed people’s attention span
and made them more and more impatient. To grasp and engage more viewers,
designers have to consider user experience and modify their design according
to how viewers might respond.

In response to such challenges, a growing number of studies in the
visualization community have adopted a user-centered perspective and examined
visualization design through the lens of user experience. For example, Saket
et al. [21] argued that more efforts should be put into understanding people’s
subjective responses to visualization design. Cawthon et al. [22] and Harrison
et al. [23] investigated how data visualization influences aesthetic feelings.
Amini et al. [24] and Lan et al. [25] used indicators such as likability and
enjoyment to evaluate data stories. Borkin et al. [26] evaluated a series of
visualization designs by assessing how memorable they were. However, so far,
existing studies have only addressed limited aspects of user experience. Many
important concepts are left underexplored.

Therefore, in this work, we choose to focus on affect arousal, a core
construct of user experience in mass communication, and examine how it
correlates with data visualization design. We hope our work can contribute a
new perspective to understanding data communication and user-centered
visualization design.

###  2.2 Affective Arousal

Affective arousal is about the activation of emotion [4] and is viewed as a
key construct of human affects. For example, the well-known circumplex model
[4] views affects as combinations of valence and arousal. While valence
describes whether an affect is positive or negative, arousal reflects the
intensity of the affect, ranging from calm to excitement. Along with the
conceptualization of affective arousal, many researchers began to conduct
studies to investigate how to elicit affective arousal specifically. For
example, Sundar and Kalyanaraman [27] examined the relationship between
animation and affective arousal using physiological indicators. Lang et al.
[28] and Gorn et al. [29] used self-report methods to gather ratings from
users and assess how color and pacing influenced perceived arousal.

Meanwhile, as affective arousal is closely related to people’s reactions,
behaviors, and information processing [8], it has been massively studied in
research about visual communication, such as web advertising and TV
broadcasting. For example, Singh and Hitchon [30] reviewed dozens of studies
conducted in the last century that examined arousal-centered topics, such as
how to arouse viewers most advantageously in a TV show and how to increase
consumers’ desire for a product by eliciting arousal. Given the fundamental
role of affective arousal, this thread of research continues to prosper until
today while dealing with trending topics such as how to design arousing games,
user interfaces, or immersive environments [27, 31, 32]. For example, Aoki et
al. [33] examined how to design chat balloons that represent various levels of
arousal in text chats. Cusveller et al. [32] explored how to evoke affective
arousal in computer games.

This work follows this research thread and studies affective arousal
specifically in the context of visualization design. We see this work as an
initial step towards exploring what design factors contribute to affective
arousal in data visualization.

###  2.3 Affective Design in Data Visualization

In recent years, researchers [13, 1] have introduced the idea of affective
design to the community of data visualization and aimed to understand how
design elements influence viewers’ affective feelings towards data
visualization. For example, Bartram et al. [13] identified a set of affective
color palettes that can elicit eight categories of affects, such as calmness
and positivity. Boy et al. [15] investigated one specific category of affects,
empathy, and found that using anthropomorphic pictograms to represent human
rights data does not help trigger empathy. Lan et al. [1] summarized a set of
affect-related design factors in infographics, such as the usability of the
design and the expressiveness of the design. They also explored the dynamic
design elements in data visualization design [16] and proposed an animation
design scheme, Kineticharts, for creating charts that express five positive-
valenced affects, such as joy, surprise, and amusement. After this work, they
continued to explore negative-valenced affects in serious data storytelling
[17].

However, although the aforementioned work has laid a foundation for
investigating how data visualization design influences affects, it has two
main limitations. First, previous work usually treats affects as discrete
categories, and this paradigm has largely overlooked the continuous aspects of
affects. Second, compared to other concepts such as valence, arousal has been
less studied, thus limiting our understanding of how affects are activated by
visualization design. To bridge this gap, this work investigates affective
arousal specifically. As a continuous measurement, affective arousal helps
capture the intensity of people’s affective experience with visualization,
thus contributing new knowledge around affective visualization design.

##  3 Dataset and Pilot Study

To identify design factors that contribute to affective arousal in data
visualization design, we prepared a corpus of data visualization designs and
conducted a crowdsourcing study with 184 participants. The participants were
asked to view the data visualizations randomly chosen from the corpus and rate
the affective arousal triggered by these images. Afterward, we analyzed the
data collected from the study to summarize a set of potential arousal-related
design features. The study materials can be accessed at
https://bit.ly/3Ntv5ck.

###  3.1 Dataset

We aimed to use a dataset that contains diverse data visualizations and
reflects the variety of data visualization design in the real world. A dataset
that meets such needs is the Massachusetts (Massive) Visualization Dataset
(MASSVIS) [18]. MASSVIS has collected thousands of data visualizations by
balancing four types of sources: infographic galleries, news media,
governments, and scientific journals. Due to its representativeness in
visualization design and public availability, MASSVIS has been used frequently
in previous studies [34, 35, 26, 18]. MASSVIS is constituted of several sub-
corpora with different data sizes and labels. For example, single2k is a
corpus that contains more the 2000 pieces of single data visualizations
labeled with basic meta-information such as sources and titles. Targets410,
another sub-corpus, contains 410 representative data visualizations selected
by experts from single2k. It contains a more suitable amount of images to be
used in a user study while maintaining the design diversity of single2k.
Besides, targets410 has been labelled with more design-related tags, such as
the visualization type and data-ink ratio. Therefore, targets410 suits our
study better.

Besides, according to previous literature [1, 14], the semantics carried by a
data visualization will remarkably influence people’s affects. Many people
would react affectively to the content or the topic of a data visualization.
Besides, embellishments such as an illustration of violence and a photograph
of a child can also trigger strong affects. Such affects brought by semantics
will be confounded with the affects brought by design and thus hinder our
understanding of the exact impact of visualization design. Therefore, to get
rid of the influence of semantics, we blurred all the texts and embellishments
on the images so that the participants in our studies could focus on
perceiving the affects triggered by design only.

###  3.2 Pilot study

We used the blurred version of targets410 as our stimuli and conducted a pilot
study to investigate whether it was appropriate to use this corpus in our
study. Based on the findings from the pilot study, we refined the corpus to
construct the final stimuli for our crowdsourcing study.

####  3.2.1 Methodology

We recruited 10 participants (7 females, aging from 21 to 26) for the pilot
study by posting an open call on social media platforms. Once entering the
online survey, the participants were presented with an introduction to our
research, such as the definition of affective arousal and how to use the
instrument for evaluating affects, namely Affective Slider (AS, see Fig. 1)
[36]. AS is a modern version of Self-Assessment Manikin (SAM)[37], which is
the traditional instrument for measuring valence and arousal based on the
circumplex model. SAM adopts a 9-point scale (e.g., for affective arousal, 1
denotes not at all aroused, and 9 denotes strongly aroused). AS inherits the
core spirit of SAM but stretches the scale to 100 by providing a movable
slider whose range is 0.00 to 1.00. Such a scale can produce more continuous
and precise data, which is more suitable for use in statistical modeling. Note
that although this work focuses on arousal, we asked the participants to rate
both arousal and valence during the study in case there was any interaction
between these two dimensions. After reading the introduction, the participants
viewed 20 data visualizations randomly chosen from our corpus, one at a time.
We also shuffled the order of the 20 stimuli to ensure the participants viewed
a random sequence of images. After viewing each of the images, the
participants rated their arousal and valence using AS. They were also asked to
provide reasons for their ratings and rate the likability of the design (“Do
you like this data visualization?”) using a 5-point Likert scale (1 denotes
strongly dislike, 5 denotes strongly like). The question about likability was
included because prior literature has suggested that high arousal may not
always be favorable [38]. Therefore, we followed their practice [29, 39] and
evaluated whether the arousal elicited by stimuli was liked by the
participants to facilitate the interpretation of arousal.

Figure 1: Affective Slider [36] used in our study to rate the affective
arousal (up) and valence (down) triggered by visualization design.

After viewing and rating all 20 images, the participants filled out a form to
answer four demographic questions (i.e., gender, age, education level,
country) and two summary questions to report arousal-related and valence-
related design features respectively (e.g., “Please recall the data
visualizations you have viewed and list the design factor(s) that helped
increase the arousal level.”). Last, we interviewed the participants with a
set of pre-prepared questions concerning the study design, such as “Do you
think the study instructed you well? Did you feel confused about anything?”,
“Do you think 20 is an appropriate number of images to view? Did you feel
tired or impatient during the study?”, “Do you think some images are
significantly different from others in terms of eliciting your affects?”. On
average, the participants spent about 20 minutes completing this study and 10
minutes on the interview.

####  3.2.2 Results

We collected both positive and negative feedback from the pilot study. For
example, most participants agreed that 20 images is an appropriate amount for
them to complete the study without losing patience. Also, the participants
confirmed that the study task was clear and the whole process was well-guided.
However, on the downside, several participants reported that they did not feel
confident in their ratings until they had viewed and rated several images
(e.g., “Maybe there should be several examples in the very beginning to help
me decide my criterion of rating.”). Accordingly, we refined the introduction
page by including three examples. The three examples were selected by a
visualization expert to demonstrate the diversity of the stimuli.

Figure 2: Scientific visualizations (a) and the comparison of the original
version and the blurred version of two embellished visualizations (b).

We also noticed that there were some problems with our stimuli. First, many
participants reported that the scientific visualizations (see Fig. 2) were
significantly different from others, since the participants were unfamiliar
with such designs and felt these images should only be viewed by professional
scientists or expert users (e.g., “This is not a visualization that I will
come across in my daily life, and I feel it is apparently different from the
other charts I saw.”). Second, we found that embellishments are inherently
rich in semantics and their semantics could hardly be erased by the blur
effect. For example, a participant commented that "I can guess there is a
McDonald hamburger through its shape and color. I love hamburgers!", and
another participant said "there is a gym shoe next to the chart and this
aroused me." These two embellishments were arousing not because of their
design, but because of what they represent (see Fig. 2). In other words, the
existence of embellishments is likely to trigger a different mechanism of
eliciting affective arousal and confound the effects of design with the
effects of semantics. Third, several participants reported their discomfort
with images that had a large aspect ratio. For example, one participant said,
“There was a lengthy chart, and I had to keep scrolling up and down to view it
completely. This made it difficult to focus on my emotion.”

Based on these findings, we removed scientific images, embellished images, and
images with an aspect ratio greater than 3:1 [26] from the targets410 corpus
and kept 202 qualified images. However, a consequent problem was that the
removal had changed the distribution of the image categories (i.e.,
Infographic, Government, News Media), since many images removed because of
embellishment belonged to the Infographic category. Thus, the sample sizes of
the three categories became unbalanced. To deal with this problem, we searched
the single2k corpus to identify more qualified infographic images. Finally, we
found 63 qualified images. After including these 63 images in our corpus
(which resulted in 265 images in total), the corpus contains 83 Infographic
images (31%), 82 News Media images (31%), and 100 Government images (38%). The
distribution of the image categories is similar to that of the targets410
corpus, thus ensuring the images are representative of the single
visualization designs in the wild. In our crowdsourcing study, we used this
refined corpus as our stimuli.

##  4 Crowdsourcing study

Then, we conducted our formal study. We chose crowdsourcing to collect user
data concerning the affective arousal elicited by data visualization design
and explore design-arousal relationships.

####  4.0.1 Methodology

We recruited 184 participants from Prolific who speak English as their first
language and have correct vision. We rejected the data from ten participants,
as they misunderstood the study task (e.g., failed to understand why the texts
were blurred and kept complaining about the legibility) or provided data with
poor quality (e.g., giving all the images the same ratings without explaining
why). The remaining 174 participants included 98 females, 75 males, and one
identified as non-binary. Their ages ranged from 18 to 78 (M = 38.47, SD =
13.79), and their educational levels varied (Less than a high school diploma:
0.57%, High school or equivalent: 31.03%, Bachelor or equivalent: 48.28%,
Master or equivalent: 16.09%, Doctoral or equivalent: 4.02%). The participants
were paid $9 per hour.

The study procedure of the crowdsourcing study was identical to the pilot
study. The participants first read an introduction page, then viewed 20 data
visualizations (one at a time), rated arousal and valence, wrote down reasons
for their ratings, and rated how they liked the designs. After viewing and
rating all 20 images, the participants filled out a form to provide their
demographic information and wrote down what design features they thought
contributed to arousal and valence, respectively. The participants spent 22.34
minutes on average completing this study.

###  4.1 Analyses and Results

We got 3480 valid ratings for the 265 images, including 1101 for the
Infographic category, 1052 for the News Media category, and 1327 for the
Government category. Each image received 13.13 ratings on average.

####  4.1.1 Descriptive Analysis

Figure 3: How the ratings of affective arousal distributed in different
groups, including image categories, gender, age, and education level. Dashed
lines represent means. Figure 4: Five designs that received the highest scores
in affective arousal (H1 to H5) and five designs that received the lowest
scores (L1 to L5) in our study. The scores range from 0 to 1, where 0 means
not aroused and 1 means strongly aroused. All texts were blurred.

##### Consistency of Ratings

We examined whether people formed consistent judgments on the arousing level
of the images. We used intraclass correlation coefficients (ICC) to calculate
the agreement in user ratings, and the result showed that the agreement was
moderate (ICC1k = .589, 95% CI = [.512, .658], p < .001), suggesting that
users held moderately similar judgments on affective arousal.

##### Means & Distributions Comparison

Overall, the average arousal score of all the images was 0.53 (SD = 0.23, 95%
CI = [0.52, 0.54]). The distribution of the arousing levels of the images was
close to a normal distribution. However, as shown in Fig. 3, the means and
distributions of affective arousal varied in different groups. For example,
the mean arousal of the Infographic category was 0.61, higher than that of the
News Media category (M = 0.53) and the Government category (M = 0.48). A
Kruskal-Wallis H Test further showed that the difference was significant (H(2)
= 211.866, p < .001). Dunn’s post-hoc test with a Bonferroni correction showed
that the mean arousal of Infographic was significantly higher than that of
News Media and Government (p < .001), and the mean arousal of News Media was
significantly higher than that of Government (p < .001).

We also examined the arousal scores set by different age, gender, and
education groups. First, a Mann-Whitney U Test (z = -.886, p = .376) showed
that there was no significant difference between the scores set by females and
by males ("Non-binary" was not included because its sample size was too
small). Then, we assigned the participants into 5 age groups (<=25, 26-35,
36-45, 46-55, >56) and compared the mean scores of arousal set by these
groups. As shown in Fig. 3, the mean arousal scores set by different age
groups interestingly decreased with the increasing age. A Kruskal-Wallis H
Test combined with a post-hoc test further showed that the difference was
significant (H(4) = 32.072, p < .001) and mainly caused by the lower mean (M =
0.48) of group 5 (age > 56). In other words, in our study, the younger
participants were more aroused by the stimuli while the elder participants
were generally calmer. A significant difference (H(3) = 22.838, p < .001) also
existed in the scores given by participants with different education levels
("Less than a high school diploma" not included because its sample size was
too small), mainly caused by the higher scores set by participants with a
"Doctoral or equivalent" degree (M = 0.60). However, this finding should be
taken cautiously since the sample size of "Doctoral or equivalent" was not big
(7 participants).

We also calculated the mean arousal score for each image. Fig. 4 shows the
five images that received the highest scores in affective arousal and the five
images that received the lowest scores. In line with the above findings, the
top ranked images all belong to the Infographic category (Fig. 4 H1-H5) while
the low-ranking images mostly belong to the Government and News Media
categories (Fig. 4 L2-L5).

Figure 5: The relationships between (a) arousal and valence ($\tau$ = .592),
(b) arousal and likability ($\tau$ = .539), and (c) valence and likability
($\tau$ = .736).

##### Correlation Analysis

Then, we examined the correlations between affective arousal, valence, and
likability to see whether an arousing design will be pleasing or favorable.
Fig. 5 plots the mutual relationships between these three variables. It can be
seen from the figure that they generally showed a positive correlation with
each other. Given that likability is an ordinal variable and all the three
variables are non-normal, we used Kendall’s Tau ($\tau$) to compute their
correlation coefficients and significance levels. The results confirmed that
arousal was positively correlated with likability ($\tau$ = .539, p < .001),
so as valence and likability ($\tau$ = .736, p < .001), whose positive
correlation was even stronger. Besides, arousal and valence also showed a
moderately positive correlation with each other ($\tau$ = .592, p < .001).
Note that these correlations still held when we conducted the correlational
tests within different image categories.

To sum up, in general, high arousal was likely to co-occur with pleasure and
high likability. However, there were also cases when arousal did not lead to
pleasure or when arousal hindered likability. For example, as shown in Fig. 5,
some participants set scores to "high arousal + displeasure" or "high arousal
+ low likability". We will discuss such situations in detail in Section 5.2.

Figure 6: Arousal-related features reported by the participants in our
crowdsourcing study. The codes about color, graphics, and information are
mapped to computable variables in Section 4.1.3.1. The high-level strategies
are discussed in Section 5.1.

####  4.1.2 Qualitative Analysis

As stated above, at the end of the user study, we asked the participants to
write down what design features had helped augment their affective arousal.
This resulted in 174 comments that suggest arousal-related design features
from the perspective of users.

Then, we conducted an in-depth qualitative analysis on these 174 comments
following the bottom-up procedure of thematic analysis [40]. Two of the
authors first went through all the user comments and took notes on the
messages deemed useful. Then, they coded the comments independently and
generated possible themes by grouping the codes. After that, they met to
compare codes and discuss mismatches. Through iterative discussions, they
generated three themes most commonly mentioned by the participants, including
color, graphics, and information. Then, they assigned the low-level codes into
these three themes and discussed whether to retain, modify, or merge the
codes. For example, when describing colors that had strong visual impact, the
participants used different expressions such as strong color, vibrant color,
and bold color. Therefore, these similar expressions were merged into one
code, namely strong color. It was also noticed that some codes described high-
level design strategies rather than specific design features. For example,
simplicity can mean simple colors, simple visualization design, or a small
amount of data. Unusualness can refer to unusual color, unusual shape, or
unusual visual encodings. Therefore, such codes were categorized into another
theme called high-level strategies. After iterative discussions, the coders
achieved a 100% agreement. At last, they generated 412 codes from the 174
comments in total. The frequencies of all the codes were analyzed and the
result visualized, as in Fig. 6. Note that for the clarity of presentation,
codes that occurred only once were excluded. Collectively, the codes
visualized in Fig. 6 constitute 92.72% of all the codes. Thus, they are highly
representative of all the arousal-related design features written down by the
participants.

As shown in Fig. 6, color and graphics were the most mentioned themes. Within
the theme of color, 49 codes only broadly referred to color as a contributor
to affective arousal. Apart from this, bright color (28), colorfulness (25),
strong color (14), and contrast (13) were the most frequently-mentioned
specific color features. Within the theme of graphics, visualization type
(54), layout (17), shape (16), and density (11) were the most frequently-
mentioned features. Within the theme of information, 11 codes talked about
information amount in a broad sense, while six codes specifically referred to
text amount and five codes specifically referred to data amount. Last, with
the theme of high-level strategies, unusualness (37), simplicity (17), and
easy reading (15) were most mentioned. Besides, clarity (9), aesthetics(9),
complexity (5), and boldness(4) were also repeatedly mentioned.

####  4.1.3 Inferential Analysis

Then, we sought to predict the user ratings for affective arousal using these
codes and identify the most significant design features. For each of the codes
about color, graphics, and information in Fig. 6, we reviewed previous
literature in relevant fields (e.g., computer vision, multimedia) to find the
mature and commonly-used methods to map these codes to computable variables.
Below we first introduce these features briefly, then report the modeling
process.

##### Design Features

The mappings are summarized in Table. I.

TABLE I: The mappings from qualitative codes to variables. For variable types,
N denotes nominal, C denotes continuous, and O denotes ordinal.

Color-based Features. To translate the code chromatic color, we used a nominal
variable whose value is yes or no to depict whether a visualization design is
achromatic. To translate the code background color in Fig. 6, we coded the
images’ background colors manually and found that white and black were the
dominant choices. Therefore, this code was translated into a nominal variable
whose value can be white, black, or others. For codes that can be measured as
continuous variables, such as bright color, strong color, colorfulness, and
contrast, we referred to prior literature [41, 42] and extracted corresponding
design features, such as calculating the mean and standard deviation of the S
(i.e., saturation) and V (i.e., brightness) channels in the HSV color space
using OpenCV, computing colorfulness using the algorithm proposed by Hasler et
al. [43], and calculating contrast using the RMS contrast algorithm [44]. Note
that according to some comments, the participants’ perception of brightness
and saturation was also influenced by the dominant color, we also extracted
the primary chromatic color of each image using Colorthief [45] and calculated
the mean and standard deviation of its saturation and brightness. Last, we
translated the code color hue using the algorithm proposed by van de Weijer et
al. [46]. This algorithm can identify and calculate the percentages of 11
nameable color hues (i.e., black, blue, brown, green, gray, orange, pink,
purple, red, white, yellow) in images.

Graphics-based Features. As two codes in this theme (visualization type,
shape) were closely related to how data was visualized into graphical
elements, we used five features to depict the visual encodings of the
visualizations. First, following the typology in [26], data visualizations
were categorized into 12 types (e.g., bar, area, map, circle) and a set of
subtypes (e.g., grouped bar chart and stacked bar chart in the category of
bar; pie chart and donut chart in the category of circle). In addition, we
used variables to characterize which data marks (e.g., bar, line, geoshape),
visual channels (e.g., position, size, shape), and layout (e.g., X and Y
coordinates, grid, polar coordinate) had been used according to existing
taxonomies [47, 48]. For example, H4 in Fig. 4 was characterized as a
visualization type called area and a subtype called proportional area chart.
The shape of its data marks is circle, and the overall shape of this
visualization is also circular because it organizes the marks using a polar
coordinate. Two visual channels (color and position) have been used to encode
data. For the remaining qualitative codes in this theme, we mapped
multiplicity, which defines whether the visualization is stand-alone or
somehow grouped with other visualizations, to categories such as single,
grouped, and multi-panel [26]. We mapped annotation and 3D to nominal
variables ([yes, no]) and computed density by calculating the white space (the
percentage of the background color) of each image.

Information-based Features. Information-based features concern the cognitive
load of viewing a data visualization and can be further split into two main
types: texts and data. First, although all the texts had been blurred, several
participants stated that they could still “feel” that there were a lot of
texts to be read and this had also impacted their arousal. Therefore, we used
Tesseract-OCR [49] to identify the texts on the images, manually revised
inaccurate identifications, and then counted how many words each image
contained and the word density on each image by dividing the word amount by
the size of the image. Next, to quantify the amount of data, we counted how
many visual channels were used in each image in total, as well as how many
distinct visual channels were used. We also counted the number of data marks
in each image and how many distinct data marks were used. For example, Fig. 4
H2 only has eight data marks while H4 contains more than 100 data marks, but
both of them only used one type of data mark, namely circle. In some images,
the data marks were too many to be counted precisely (e.g., thousands of
overlapping dots), therefore, we transformed this feature into a nominal
variable with three categories (<10, 10-100, >=100). Last, we used an ordinal
variable, data-ink ratio ([good, medium, bad]) [26], to characterize the
cognitive load brought by information that was neither texts nor data (e.g.,
axis ticks, gridlines).

Figure 7: Comparing the important features suggested by different models. Note
that the Multiple Linear Regression Model indicates importance through
statistical significance (p-values) and standardized coefficients, while
Random Forest and Decision Tree suggest key features through Gini importance.
${R}^{2}$ and MSE show the model performance, where ${R}^{2}$ (the coefficient
of determination) represents the proportion of the variance explained, and MSE
(mean squared error) measures the difference between the original and
predicted values.

##### Modeling affective arousal

Based on the extracted features, we tried five widely used predictive models
to explore important moderators of affective arousal. Three are linear models
(Multiple Linear Regression, Ridge Regression, Lasso Regression) and two are
ensemble models (Random Forest, Decision Tree). These models can help us
estimate the relationship between affective arousal and design features while
interpreting which features are contributing to the relationship. The
dependent variable is the mean arousal scores of the 265 images, and the
design features of the 265 images we extracted in Table. I are independent
variables. During the data pre-processing stage, first, we checked and revised
empty and mistaken values. Then, for all nominal variables, we used the one-
hot encoder in SciPy to transform them into a set of binary variables that use
1 and 0 to indicate the presence or absence of a category [50]. Next, we tried
re-scaling the raw arousal scores set by each participant before calculating
the mean arousal score received by each image. This step was done because we
noticed from both the pilot study and the crowdsourcing study that the
participants showed varied rating habits. For example, even with a very
similar judgment of a design, some participants tended to set higher scores to
arousal (e.g., 0.95) while some thought 0.70 or 0.80 were already high scores.
In other words, the participants’ criteria of rating affective arousal were
differently-ranged. Therefore, re-scaling the ratings helped mitigate the
influence of rating habits and revealed the real judgments of all the
participants. Specifically, we tried two common means of re-scaling, including
the z-score normalization and the Min-Max normalization [50]. When
constructing the models, we tested the performance of the two re-scaled
dependent variables as well the un-scaled version. Results showed that the
z-score normalization method performed the best. Therefore, we adopted this
version of dependent variable when constructing the final models.

After data pre-processing, we used Pearson correlation coefficients to get an
initial understanding of the correlations between the independent variables
and the dependent variable. Results showed that features such as the number of
different visual channels (r = 0.52, p < .001), colorfulness (r = 0.48, p <
.001), and standard deviation of saturation (r = 0.46, p < .001) had a
significantly positive relationship with affective arousal, while features
such as X-Y coordinates layout (r = -0.61, p < .001), table visualization (r =
-0.52, p < .001), and percentage of white (r = -0.49, p < .001) had a
significantly negative relationship with affective arousal. High correlations
also exist between some design features. For example, the percentage of white
was highly correlated with mean brightness (r = 0.82, p < .001). These
correlations were referred to when performing feature selection to prevent the
overfitting problem.

Next, we fed the variables into the five models one by one. To enhance the
validity of results, we adopted the idea of machine learning and split the
dataset into a training set (70%) and a test set (30%) using Sklearn. When
splitting the dataset, we also guaranteed that the distribution of the values
of arousal in the test set was similar to that of the training test, which was
close to a normal distribution. Then, we fed the training set to the models
and evaluated the goodness of fit using the test set.

Linear Models. For the Multiple Linear Regression, we fine-tuned the models by
(1) fitting the models using a stepwise approach to drop redundant features;
(2) conducting multicollinearity diagnostics using the Variance Inflation
Factor (VIF). A VIF above 10 implies serious multicollinearity and a VIF above
5 is usually deemed problematic [51]; (3) Ensuring that there was no
autocorrelation problem using the Durbin-Watson test (0 < DW < 2); (4)
Ensuring that the residuals were normally distributed by conducting the
Anderson-Darling test and plotting a histogram for residuals. For the Lasso
Regression and Ridge Regression, we used five-fold cross-validation to
identify the optimal values of their key parameter, alpha (i.e., the strength
of regularization). As a result, we found that in our case, the optimal alpha
values for these two models were 0, which made their results identical to that
of the Multiple Linear Regression.

As shown in Fig. 7, the Multiple Linear Regression yielded 11 statistically
significant features. Seven features showed a significantly positive
correlation with affective arousal, including colorfulness ($\beta$ = .329, t
= 7.536, p < .001), the number of different visual channels ($\beta$ = .299, t
= 5.828, p < .001), visualization subtype = heatmap ($\beta$ = .141, t =
3.013, p = .003), layout = geo ($\beta$ = .138, t = 2.588, p = .010),
percentage of black ($\beta$ = .131, t = 2.957, p = .004), visualization type
= circle ($\beta$ = .131, t = 2.886, p = .004), and number of data marks >=
100 ($\beta$ = .113, t = 2.531, p = .012). On the other hand, four features
showed a significantly negative correlation with affective arousal, including
layout = grid ($\beta$ = -.435, t = -7.279, p < .001), layout = X and Y
coordinates ($\beta$ = -.153, t = -2.360, p = .019), channel = texture
($\beta$ = -.136, t = -3.034, p = .003), and visualization subtype = stacked
bar ($\beta$ = -.085, t = -2.020, p = .045). The training set explained 72.6%
of the variance in the dependent variable (${R}^{2}$ = .726, MSE = .071), and
the test set explained 67.0% (${R}^{2}$ = .670, MSE = .099).

Ensemble Models. For ensemble models, we first identified the optimal
hyperparameters (e.g., max depth) for the models by performing five-fold
cross-validation within a grid of hyperparameter ranges. We also fine-tuned
the hyperparameters by observing the performance of the training set and the
test set to avoid overfitting. At last, the Random Forest model had a max
depth of 2. After iterating 1000 times, the average ${R}^{2}$ of the training
set was .644 (MSE = .092), and the average ${R}^{2}$ of the test set was .595
(MSE = .122). The Decision Tree model had a max depth of 3. The average
${R}^{2}$ of the training set was .668 (MSE = .085), and the ${R}^{2}$ of the
test set was .616 (MSE = .115). We also computed the Gini importance of the
features in these two models and visualized high-ranking features in Fig. 7.
The most important features in the Random Forest model included the number of
different visual channels (Gini importance = .220), percentage of green
(.189), layout = X and Y coordinates (.166), mean of saturation (.103), layout
= grid (.065), colorfulness (.048), percentage of white (.042), visualization
type = table (.041), mark shape = text (.017), and mark shape = circle (.015).
These 10 features collectively explained 90% of the total importance. The most
important features in the Decision Tree model included the number of different
visual channels (.424), layout = X and Y coordinates (.241), colorfulness
(.130), percentage of white (.075), mark shape = text (.064), and layout =
grid (.062). These six features collectively explained 99% of the total
importance.

Summary. By comparing the results in Fig. 7, we found that four design
features were collectively supported by all the models as the key moderators
of affective arousal, including the number of different visual channels,
colorfulness, layout = X and Y coordinates, and layout = grid. The first two
features showed positive effects on affective arousal in all models, while the
latter two features consistently showed a negative relationship with affective
arousal. In other words, in our exploratory study, we found that the
participants were more aroused by colorful visualizations with a certain
amount of visual complexity. However, conventional layouts that put data in an
upright and square space were less arousing. We also noticed that there were
features whose importance was not cross-validated by different models. For
example, the percentage of green showed high importance in the Random Forest
model, but it was not important in other models. Such features are worthy of
more investigation in the future.

##  5 Discussion

Below we discuss the design implications that arise from this work, the
possible negative consequences of eliciting high arousal, the challenges of
automatic visualization design evaluation, as well as limitations and future
work.

###  5.1 Design Implications

First, we contribute new knowledge for understanding how data encodings
influence affective arousal. An interesting finding of this work is that in
data visualization design, graphical elements that are related to data
encodings showed significant capability of eliciting affective arousal. For
the visualization community, this finding is significant in two main ways.
First, we have found new evidence that the appearance of data visualization
does affect user experience. In the past, Moere et al. [52] found that when
encoded with identical data, different chart types (e.g., treemaps and
sunburst diagrams) triggered different subjective feelings and preferences.
Borkin et al. [26] found that people remembered certain chart types better
than others. In other words, choosing data encodings is not only about how to
present data but also about manipulating user experience. Second, this work
also supplements the existing knowledge about user-centered visualization
design from the perspective of affective arousal. For example, in the study by
Borkin et al. [26], the most remembered visualization types were grid & matrix
and tree & network. However, in our study, visualizations that use a grid
layout (e.g., L1 in Fig. 4) were less arousing than others. This suggests that
viewing visualization design from the perspective of affective arousal may be
different from viewing it from perspectives such as memorability and
aesthetics (although these perspectives are all user-centered). In other
words, focusing on different aspects of user experience may also lead to
different focuses of design.

Second, we found colorfulness outweighs other color-based features in terms of
moderating affective arousal in the context of data visualization design.
Although the affective power of color has long been verified in various
domains, in different research contexts, the findings can be different. For
example, red was identified as the most arousing color in classic
psychological experiments [53], but some researchers also found evidence that
green can be more arousing than red [11]. Our work, however, found that
colorfulness is more related to arousal than specific color hues in
visualization design. According to our study, a reason may be that people’s
color perception would be influenced by their attempts to decode data when
viewing a data visualization. For example, some participants reported low
arousal to designs with a large amount of red because the color had hindered
reading (e.g., "it is too bright and dazzling"). Similarly, some participants
stated that the use of analogous color hues (e.g., red and orange) had made
the graphical elements on the charts indistinguishable, thus lowering arousal.
On the other hand, colorfulness can be both viscerally stimulating and useful
in presenting data, thus making it the most arousing color-based feature
compatible with data visualization design.

Third, data visualizations that belonged to the Infographic category elicited
stronger affective arousal than others. Infographics are often seen as a
visual form that pays more attention to data communication and appealing to
viewers subjectively than traditional visualizations [1, 54], and previous
work [55, 56] has often focused on studying the rich embellishment techniques
in infographics such as pictograms, illustrations, and visual metaphors.
However, another interesting finding from this work is that even after we
removed all embellished designs for our corpus, the images in the Infographic
category were still significantly more arousing than others. This finding
indicates that the specificity of infographic design may not only lie in
embellishment, but also in how it deals with color, graphics, and information.
As shown by an empirical study [57], when creating infographics, designers
often spend a lot of time choosing encoding approaches and polishing the
appearance of the visualization iteratively, because the designers not only
hope to present data clearly, but also to create something new and unique.
Such needs meet the unusualness strategy we mentioned above and may help
improve the arousing level of a design.

Fourth, the qualitative and quantitative analyses collectively imply the
strategies for manipulating arousal. In the crowdsourcing study, we identified
several high-level strategies for eliciting affective arousal by coding user
comments qualitatively (Fig. 6). After constructing models to predict arousal,
we found that the quantitative results suggested by the models resonate more
or less with the qualitative codes. For example, our models showed that
affective arousal is significantly higher when a visualization does not use
layouts such as Cartesian coordinates, and this resonates well with the most
mentioned high-level strategy by the participants, namely unusualness (e.g.,
"The design factors that helped increase the arousal was those that looked
interesting and new, something I’ve never seen before."). Another two
strategies mentioned by the participants (boldness and aesthetics) were partly
supported by the significance of colorfulness, as previous research has found
colorfulness is linked with the perception of energization and extraversion
[58], as well as an indicator of aesthetics [23, 42]. However, there seems to
be controversy over simplicity and complexity (we can see the co-existence of
these two strategies in Fig. 6). Our models, however, ended up leaning towards
the side of complexity because affective arousal showed a significantly
positive relationship with the number of different visual channels. However,
by referring to the raw comments written by the participants, we found that
simplicity and complexity may not necessarily contradict each other. For
example, when reporting that simple designs had increased their affective
arousal, some participants were in fact emphasizing the understandability of
the designs (e.g., “This diagram looks like a really simple way to display
data, even to a layperson”). On the other hand, when reporting that complex
designs were more arousing, some participants were emphasizing the richness of
visuals (e.g., "different graphs and charts - when there was more information
on the chart to concentrate on the arousal was higher"). In other words, for
designers, it may be beneficial to include an appropriate level of visual
complexity while ensuring that the data visualization is understandable.

###  5.2 The Negative Side of High Arousal

As we have discussed in Section 4.1.1.3, in our exploratory study, high
affective arousal did not always lead to high pleasure or high preference. In
some cases, the participants did not like or enjoy data visualization designs
that elicited high arousal, or they thought high arousal was harmful. By
analyzing such comments, we identified two main reasons:

First, high arousal may lead to physical discomfort. In our crowdsourcing
study, several participants reported high arousal but thought the
corresponding designs were too chaotic and over-stimulating (e.g., “It is a
very arousing visual in terms of color and design, however looking at it makes
your eyes go unwell so does not give great pleasure when looking at the
visual.”, "Unpleasant over-stimulating colours, chaotic graphic that is
difficult to decipher and is unpleasant to look at."). One participant
commented that "Too much red is making me feel nervous." These findings
resonate with prior psychological studies that intense affects may bring about
side effects [59] or even evolve into tension and anxiety [60]. Therefore, for
designers, designing an arousing data visualization is not equal to simply
adding up all stimulating design features. On the contrary, it is necessary to
find a balance between arousing affects and preventing discomfort.

Second, high arousal may be caused by inappropriate design. In some cases, the
participants were affectively aroused because the designs looked confusing or
ugly to them. For example, one participant said that a design "looks really
cool but I don’t know how easy it would be to interpret data using this".
Similarly, another participant wrote: "A very striking chart that looks like
it makes sense only to the person who created it." We also saw comments that
reported high arousal accompanied by negative affects such as anger and
irritation (e.g., "I don’t like the orange. It makes me angry.", "The red line
is irritating."). Under such circumstances, high arousal is not beneficial.

###  5.3 Towards Automatic Design Evaluation

Understanding what design features contribute to affective arousal has paved a
path toward automatically evaluating the arousing level of data visualizations
or automatically generating arousing data visualizations. For example, we can
envision a feedback system that can intelligently assess the arousing level of
a data visualization design by scoring its design from multiple arousal-
related aspects and identifying its strengths and weaknesses, thus guiding
designers to improve their work. To achieve such goals, extracting computable
features from data visualization designs automatically becomes a pursuit. In
this work, although about half of the features were calculated by computer, we
noticed that there are three main challenges if we want to fully automate the
process of feature extraction.

First, not all the design-related concepts can be easily transformed into
computable features. For example, so far, there is no recognized algorithm for
evaluating the data-ink ratio of a data visualization automatically. Besides,
high-level concepts such as unusualness are difficult to be measured
computationally, as they are usually feelings co-shaped by multiple design
elements and are moderated by the participants’ own knowledge, values, or
preferences. How to take advantage of the valuable signals that arise from
such expressions is a tricky problem.

Second, parsing the composition of a data visualization from an image can be
difficult. Understanding how a data visualization is composed of data,
graphical elements, and texts requires utilizing techniques such as object
detection, recognition, and classification. When the input is pixel-based
images, the accuracy of the outcomes may not be satisfying. For example, when
calculating the whitespace, the algorithm sometimes misunderstood which color
is the background color. When detecting texts from the images, the algorithm
frequently made mistakes. As a result, we had to adjust these results
manually. Therefore, to achieve automatic design evaluation, we still need to
incorporate more state-of-the-art techniques from computer vision and develop
more models and algorithms customized for data visualization design to improve
the accuracy of design feature extraction.

Third, how to assess the quality of a design remains a challenge. Currently,
we can use variables to describe the existence of a design feature or to what
extent it is used. However, we are not able to judge whether the feature is
used "appropriately". For example, in our study, some participants reported
low arousal to visualizations that were badly designed (e.g., “Not much
immediate clarity what the visualisation is about hence my interest is low and
disinterested.”). In other words, except for "quantifying" the design, the
"quality" of a design also counts; except for "describing" the design,
"assessing" the design also counts. This has posed another challenge for
future research.

###  5.4 Limitations and Future Work

This work is exploratory in nature so the findings of this work were mainly
limited to correlations rather than causal effects and were constrained by the
inherent limitations of exploratory studies. First, although we have carefully
constructed a corpus that contains diverse data visualization designs, the
corpus is still by no means exhaustive. Second, affective arousal is a
subjective feeling that is difficult to quantify. To mitigate this problem, we
excluded the influence of semantics, sampled diverse participants from a large
population pool through crowdsourcing, and normalized each participant’s
ratings when constructing the models. However, noise caused by subjectivity
may have inevitably entered and lowered the accuracy of prediction. Third, the
features we extracted in this work are not exhaustive and are limited in
characterizing some high-level subjective concepts such as aesthetics and
unusualness. Also, this work is short in revealing the fundamental impact of
low-granularity design elements (e.g., angles, roundness) on affective
arousal. To address such issues, more controlled experiments should be done to
examine the effects of a certain design condition while other variables are
rigorously controlled. We also hope a follow-up study can be done in the
future to test the generalizability of this study by collecting more samples
and systematically manipulating one or more of the identified features.

We see several directions for future research. First, more alternative methods
can be utilized to deepen our understanding of the design-affect relationship
in data visualization. For example, by excluding any semantics, this work did
not examine the interaction between content and visualization design. Future
work could fill this gap by first measuring the arousal caused by content and
then analyzing how much arousal is augmented or lessened by design. In
addition, future work could examine how the incorporation of embellishment
moderates the conveyance of semantics and the elicitation of affects. Second,
as our study has suggested that affective arousal may be related to the
complexity of data visualization, future work could further investigate how
people’s visualization literacy influences their perceived complexity and
arousal. Third, more work can be done to investigate another important
dimension of affects, namely valence, such as how valence-related design
features are different from arousal-related features and how they interact
with each other. Last, we hope there will be more research into how we should
take advantage of the benefits brought by affective arousal and how we should
avoid its negative side effects. This asks for the incorporation of more
empirical knowledge, such as understanding the practice of designers, to help
outline clearer guidelines for what should and should not be done.

##  6 conclusion

This work explores the relationship between visualization design and affective
arousal. First, we collected a corpus of 265 data visualizations and conducted
a crowdsourcing study with 184 participants to gather ratings on the affective
arousal elicited by their design as well as qualitative feedback. Based on the
study data, we first manually coded the arousal-related design features
reported by the participants, then mapped these features to computable
variables and constructed regression models to identify the most significant
features. By comparing the results of the models, we finally identified four
design features (e.g., colorfulness, the number of different visual channels)
that were cross-validated as important features correlated with affective
arousal. We hope this work will inspire more future research on affective
visualization design and user experience with data visualization.

## References

  * [1] X. Lan, Y. Shi, Y. Zhang, and N. Cao, “Smile or scowl looking at infographic design through the affective lens,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 27, no. 6, pp. 2796–2807, 2021.
  * [2] E. Segel and J. Heer, “Narrative visualization: Telling stories with data,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 16, no. 6, pp. 1139–1148, 2010.
  * [3] Y. Shi, X. Lan, J. Li, Z. Li, and N. Cao, “Communicating with motion: A design space for animated visual narratives in data videos,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2021, pp. 1–13.
  * [4] J. A. Russell, “A circumplex model of affect.” _Journal of personality and social psychology_ , vol. 39, no. 6, pp. 1161–1187, 1980.
  * [5] E. Sherman, A. Mathur, and R. B. Smith, “Store environment and consumer purchase behavior: mediating role of consumer emotions,” _Psychology & Marketing_, vol. 14, no. 4, pp. 361–378, 1997.
  * [6] V. D. Kaltcheva and B. A. Weitz, “When should a retailer create an exciting store environment?” _Journal of Marketing_ , vol. 70, no. 1, pp. 107–118, 2006.
  * [7] J. Berger, “Arousal increases social transmission of information,” _Psychological Science_ , vol. 22, no. 7, pp. 891–893, 2011.
  * [8] J. Storbeck and G. L. Clore, “Affective arousal as information: How affective arousal influences judgments, learning, and memory,” _Social and Personality Psychology Compass_ , vol. 2, no. 5, pp. 1824–1843, 2008.
  * [9] Lin, “Minimal yet powerful: Improving the text on your infographics,” https://dearcontent.com/minimal-yet-powerful-improving-the-text-on-your-infographics/, 2019\.
  * [10] N. Chibana, “The ultimate guide to infographics for nonprofits,” https://visme.co/blog/the-ultimate-guide-to-infographics-for-nonprofits/, 2015\.
  * [11] P. Valdez and A. Mehrabian, “Effects of color on emotions.” _Journal of Experimental Psychology: General_ , vol. 123, no. 4, p. 394, 1994.
  * [12] Y. Wang, A. Segal, R. Klatzky, D. F. Keefe, P. Isenberg, J. Hurtienne, E. Hornecker, T. Dwyer, and S. Barrass, “An emotional response to the value of visualization,” _IEEE Computer Graphics and Applications_ , vol. 39, no. 5, pp. 8–17, 2019.
  * [13] L. Bartram, A. Patra, and M. Stone, “Affective color in visualization,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2017, pp. 1364–1374.
  * [14] H. Kennedy and R. L. Hill, “The feeling of numbers: Emotions in everyday engagements with data and their visualisation,” _Sociology_ , vol. 52, no. 4, pp. 830–848, 2018.
  * [15] J. Boy, A. V. Pandey, J. Emerson, M. Satterthwaite, O. Nov, and E. Bertini, “Showing people behind data: Does anthropomorphizing visualizations elicit more empathy for human rights data?” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2017, pp. 5462–5474.
  * [16] X. Lan, Y. Shi, Y. Wu, X. Jiao, and N. Cao, “Kineticharts: Augmenting affective expressiveness of charts in data stories with animation design,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 28, no. 1, pp. 933–943, 2021.
  * [17] X. Lan, Y. Wu, Y. Shi, Q. Chen, and N. Cao, “Negative emotions, positive outcomes? exploring the communication of negativity in serious data stories,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2022, pp. 1–14.
  * [18] M. A. Borkin, Z. Bylinskii, N. W. Kim, C. M. Bainbridge, C. S. Yeh, D. Borkin, H. Pfister, and A. Oliva, “Beyond memorability: Visualization recognition and recall,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 22, no. 1, pp. 519–528, 2015.
  * [19] D. Shi, X. Xu, F. Sun, Y. Shi, and N. Cao, “Calliope: Automatic visual data story generation from a spreadsheet,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 27, no. 2, pp. 453–463, 2020.
  * [20] P. Lorenz-Spreen, B. M. Mønsted, P. Hövel, and S. Lehmann, “Accelerating dynamics of collective attention,” _Nature communications_ , vol. 10, no. 1, pp. 1–9, 2019.
  * [21] B. Saket, A. Endert, and J. Stasko, “Beyond usability and performance: A review of user experience-focused evaluations in visualization,” in _Proceedings of the Workshop on Beyond Time and Errors on Novel Evaluation Methods for Visualization_. ACM, 2016, pp. 133–142.
  * [22] N. Cawthon and A. V. Moere, “The effect of aesthetic on the usability of data visualization,” in _International Conference Information Visualization_. IEEE, 2007, pp. 637–648.
  * [23] L. Harrison, K. Reinecke, and R. Chang, “Infographic aesthetics: Designing for the first impression,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2015, pp. 1187–1190.
  * [24] F. Amini, N. H. Riche, B. Lee, J. Leboe-McGowan, and P. Irani, “Hooked on data videos: assessing the effect of animation and pictographs on viewer engagement,” in _Proceedings of the 2018 International Conference on Advanced Visual Interfaces_ , 2018, pp. 1–9.
  * [25] X. Lan, X. Xu, and N. Cao, “Understanding narrative linearity for telling expressive time-oriented stories,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2021, pp. 1–13.
  * [26] M. A. Borkin, A. A. Vo, Z. Bylinskii, P. Isola, S. Sunkavalli, A. Oliva, and H. Pfister, “What makes a visualization memorable?” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 19, no. 12, pp. 2306–2315, 2013\.
  * [27] S. S. Sundar and S. Kalyanaraman, “Arousal, memory, and impression-formation effects of animation speed in web advertising,” _Journal of Advertising_ , vol. 33, no. 1, pp. 7–17, 2004.
  * [28] A. Lang, P. Bolls, R. F. Potter, and K. Kawahara, “The effects of production pacing and arousing content on the information processing of television messages,” _Journal of Broadcasting & Electronic Media_, vol. 43, no. 4, pp. 451–475, 1999.
  * [29] G. J. Gorn, A. Chattopadhyay, T. Yi, and D. W. Dahl, “Effects of color as an executional cue in advertising: They’re in the shade,” _Management science_ , vol. 43, no. 10, pp. 1387–1400, 1997.
  * [30] S. N. Singh and J. C. Hitchon, “The intensifying effects of exciting television programs on the reception of subsequent commercials,” _Psychology & Marketing_, vol. 6, no. 1, pp. 1–31, 1989.
  * [31] I. Mavridou, E. Seiss, T. Kostoulas, C. Nduka, and E. Balaguer-Ballester, “Towards an effective arousal detection system for virtual reality,” in _Proceedings of the Workshop on Human-Habitat for Health (H3): Human-Habitat Multimodal Interaction for Promoting Health and Well-Being in the Internet of Things Era_ , 2018, pp. 1–6.
  * [32] J. Cusveller, C. Gerritsen, and J. d. Man, “Evoking and measuring arousal in game settings,” in _International Conference on Serious Games_. Springer, 2014, pp. 165–174.
  * [33] T. Aoki, R. Chujo, K. Matsui, S. Choi, and A. Hautasaari, “Emoballoon - conveying emotional arousal in text chats with speech balloons,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2022, pp. 1–16.
  * [34] M. Brehmer, B. Lee, B. Bach, N. H. Riche, and T. Munzner, “Timelines revisited: A design space and considerations for expressive storytelling,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 23, no. 9, pp. 2151–2164, 2016.
  * [35] D. H. Kim, V. Setlur, and M. Agrawala, “Towards understanding how readers integrate charts and captions: A case study with line charts,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2021, pp. 1–11.
  * [36] A. Betella and P. F. Verschure, “The affective slider: A digital self-assessment scale for the measurement of human emotions,” _PLOS ONE_ , vol. 11, no. 2, p. e0148037, 2016.
  * [37] M. M. Bradley and P. J. Lang, “Measuring emotion: the self-assessment manikin and the semantic differential,” _Journal of Behavior Therapy and Experimental Psychiatry_ , vol. 25, no. 1, pp. 49–59, 1994.
  * [38] D. E. Berlyne, _Conflict, arousal, and curiosity._ McGraw-Hill Book Company, 1960.
  * [39] M.-C. Lichtlé, “The effect of an advertisement’s colour on emotions evoked by attitude towards the ad: The moderating role of the optimal stimulation level,” _International Journal of Advertising_ , vol. 26, no. 1, pp. 37–62, 2007.
  * [40] V. Braun and V. Clarke, “Using thematic analysis in psychology,” _Qualitative Research in Psychology_ , vol. 3, no. 2, pp. 77–101, 2006.
  * [41] J. Machajdik and A. Hanbury, “Affective image classification using features inspired by psychology and art theory,” in _Proceedings of the ACM International Conference on Multimedia_. ACM, 2010, pp. 83–92.
  * [42] K. Reinecke, T. Yeh, L. Miratrix, R. Mardiko, Y. Zhao, J. Liu, and K. Z. Gajos, “Predicting users’ first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2013, pp. 2049–2058.
  * [43] D. Hasler and S. E. Suesstrunk, “Measuring colorfulness in natural images,” in _Human Vision and Electronic Imaging VIII_ , vol. 5007. SPIE, 2003, pp. 87–95.
  * [44] E. Peli, “Contrast in complex images,” _Journal of the Optical Society of America A_ , vol. 7, no. 10, pp. 2032–2040, 1990.
  * [45] L. Dhakar, “Color thief,” https://lokeshdhakar.com/projects/color-thief/.
  * [46] J. Van De Weijer, C. Schmid, and J. Verbeek, “Learning color names from real-world images,” in _IEEE Conference on Computer Vision and Pattern Recognition_. IEEE, 2007, pp. 1–8.
  * [47] A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer, “Vega-lite: A grammar of interactive graphics,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 23, no. 1, pp. 341–350, 2016.
  * [48] T. Munzner, _Visualization analysis and design_. Boca Raton, FL, US: CRC press, 2014.
  * [49] R. Smith, “An overview of the tesseract ocr engine,” in _International Conference on Document Analysis and Recognition_ , vol. 2. IEEE, 2007, pp. 629–633.
  * [50] S. García, J. Luengo, and F. Herrera, _Data preprocessing in data mining_. London, UK: Springer, 2015, vol. 72.
  * [51] G. James, D. Witten, T. Hastie, and R. Tibshirani, _An introduction to statistical learning_. Springer, 2013, vol. 112.
  * [52] A. V. Moere, M. Tomitsch, C. Wimmer, B. Christoph, and T. Grechenig, “Evaluating the effect of style in information visualization,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 18, no. 12, pp. 2739–2748, 2012.
  * [53] K. W. Jacobs and F. E. Hustmyer Jr, “Effects of four psychological primary colors on gsr, heart rate and respiration rate,” _Perceptual and motor skills_ , vol. 38, no. 3, pp. 763–766, 1974.
  * [54] J. Lankow, J. Ritchie, and R. Crooks, _Infographics: The power of visual storytelling_. Hoboken, NJ, US: John Wiley & Sons, 2012.
  * [55] Y. Wang, H. Zhang, H. Huang, X. Chen, Q. Yin, Z. Hou, D. Zhang, Q. Luo, and H. Qu, “Infonice: Easy creation of information graphics,” in _Proceedings of the CHI Conference on Human Factors in Computing Systems_. ACM, 2018, pp. 1–12.
  * [56] L. Byrne, D. Angus, and J. Wiles, “Acquired codes of meaning in data visualization and infographics: beyond perceptual primitives,” _IEEE Transactions on Visualization and Computer Graphics_ , vol. 22, no. 1, pp. 509–518, 2015.
  * [57] A. Bigelow, S. Drucker, D. Fisher, and M. Meyer, “Reflections on how designers design with data,” in _Proceedings of the International Working Conference on Advanced Visual Interfaces_. ACM, 2014, pp. 17–24.
  * [58] A. D. Pazda and C. A. Thorstenson, “Color intensity increases perceived extraversion and openness for zero-acquaintance judgments,” _Personality and Individual Differences_ , vol. 147, pp. 118–127, 2019.
  * [59] M. R. Hyman and R. Tansey, “The ethics of psychoactive ads,” _Journal of Business Ethics_ , vol. 9, no. 2, pp. 105–114, 1990.
  * [60] R. B. Malmo, “Anxiety and behavioral arousal.” _Psychological Review_ , vol. 64, no. 5, p. 276, 1957.

|  Xingyu Lan received her Ph.D. degree in design from Tongji University,
China, in 2022. She is going to work as an assistant professor at the College
of Journalism, Fudan University, China. She conducts interdisciplinary
research across fields such as data visualization, data journalism, digital
communication, and design. Her research interests include data-driven
storytelling, visualization design, human-computer interaction, and
visualization for the masses.
---|---
|  Yanqiu Wu received her MA degree in design from Tongji University, China,
in 2022. She is currently a research assistant with Intelligent Big Data
Visualization Lab (iDVx Lab), the College of Design and Innovation, Tongji
University, Shanghai, China. Her research interests include data visualization
design, intelligent user interfaces, and human-computer interaction.
---|---
|  Qing Chen received her B.Eng degree from the Department of Computer
Science, Zhejiang University and her Ph.D. degree from the Department of
Computer Science and Engineering, Hong Kong University of Science and
Technology (HKUST). After receiving her PhD degree, she worked as a postdoc at
Inria and Ecole Polytechnique. She is currently an assistant professor at
Tongji University. Her research interests include information visualization,
visual analytics, humancomputer interaction, online education, visual
storytelling, intelligent healthcare and design.
---|---
|  Nan Cao received his Ph.D. degree in Computer Science and Engineering from
the Hong Kong University of Science and Technology (HKUST), Hong Kong, China
in 2012. He is currently a professor at Tongji University and the Associate
Dean of the Tongji College of Design and Innovation. He also directs the
Tongji Intelligent Big Data Visualization Lab (iDVx Lab) and conducts
interdisciplinary research across multiple fields, including data
visualization, human computer interaction, machine learning, and data mining.
Before his Ph.D. studies at HKUST, he was a staff researcher at IBM China
Research Lab, Beijing, China. He was a research staff member at the IBM T.J.
Watson Research Center, New York, NY, USA before joining the Tongji faculty in
2016.
---|---