# The Reality of the Situation: A Survey of Situated Analytics

Sungbok Shin, Andrea Batch, Peter W. S. Butcher,
Panagiotis D. Ritsos, and Niklas Elmqvist  Sungbok Shin, Andrea Batch, and
Niklas Elmqvist are with University of Maryland, College Park, MD, United
States. E-mail: {sbshin90, ajulca<EMAIL_ADDRESS>Peter W. S. Butcher and
Panagiotis D. Ritsos are with Bangor University in the United Kingdom. E-mail:
{p.butcher<EMAIL_ADDRESS>received XXX XX, 2022; revised
XXX XX, 2023.

###### Abstract

The advent of low-cost, accessible, and high-performance augmented reality
(AR) has shed light on a situated form of analytics where in-situ
visualizations embedded in the real world can facilitate sensemaking based on
the user’s physical location. In this work, we identify prior literature in
this emerging field with a focus on situated analytics. After collecting 47
relevant situated analytics systems, we classify them using a taxonomy of
three dimensions: situating triggers, view situatedness, and data depiction.
We then identify four archetypical patterns in our classification using an
ensemble cluster analysis. We also assess the level which these systems
support the sensemaking process. Finally, we discuss insights and design
guidelines that we learned from our analysis.

###### Index Terms:

situated analytics, situated visualization, augmented reality, immersive
analytics, data visualization.

##  1 Introduction

Humans are fundamentally embodied beings [128]. Our very own bodies, the
objects we surround ourselves with, and the physical environment we live in
all play a critical role in our thinking. People tend to remember things
better when they act them out bodily [133]; spatial concepts abound in
language (“I’m feeling up today,” or “I’m on top of this”), and thinking about
the future causes people to lean slightly forwards, whereas thinking about the
past causes them to lean backwards [102]. In fact, the theory of situated
action holds that human behavior can only be understood within its real-world
context [136]. It follows that a situated form of analytics [139], where
physical space can play as much a part of sensemaking as the technology
enabling it, could be particularly efficient for certain tasks. After all,
people routinely talk about being “in the situation” or “on the ground” as
benefits to seeing a problem and thinking about it more clearly. In some
cases, there is information that can only be obtained by being in-situ.

With the recent advances in Mixed/Augmented Reality (MR/AR) technology [104],
where computer-generated imagery is overlaid and anchored on the real world,
we may finally have reached a point where in-situ sensemaking is within reach
for the average person. This is the promise of situated visualization (SV),
where data relevant to a physical location is visualized directly in that
location [21, 144, 150]. Such situated visualizations have long been a
research topic in AR research, dating back to the origins of the field. The
latest progress on higher-order sensemaking in mixed reality using situated
visualization has now yielded the concept of situated analytics (SA) [48, 139]
(Fig. 1).

Recent years have seen a rise in SA tools and techniques for AR [48, 1, 145],
creating a mosaic of different solutions and approaches. However, there is
currently little consensus within the field of data visualization on the
precise characteristics of situated analytics. Some efforts, such as the work
of Willett et al. [150] revisit and extend terminology, whereas others, such
as Ens et al. [50] focus on challenges in the broader area of immersive
analytics (IA) [100], as an umbrella concept of SA. And the overarching area
of ubiquitous analytics [45] has been named as the future of a form of
analytics that can be conducted anywhere and anytime [44]. Recently, Bressa et
al. [21] identified different perspectives of situated visualization and
outlined future research directions. However, none of these surveys and
frameworks focus specifically on the situatedness of data and visual
representations and how the analytical process can be best integrated into the
real world during visual analytics.

In this paper, we address this gap in the literature by exploring AR
technologies and techniques developed for situated analytics. For this, we
limit our interest to data-driven visualization-based systems for sensemaking
using AR techniques based on the user’s physical location (i.e., situated
analytics systems). Then, we propose a design space to describe specific
situated analytics techniques. Using this design space, we study patterns of
implementation, techniques, and enabling technologies. To achieve this, we
surveyed 47 works on situated analytics from the human-computer interaction
(HCI), visualization, and immersive technologies literature, as well as from
other domains such as medicine and construction. We then describe four
archetypes of situated analytics systems identified using multiple cluster
analyses. We close the paper by discussing observations drawn from our
taxonomy.

Figure 1: Example of a situated analytics tool that uses Augmented Reality to
enable sensemaking in-situ. The user is exploring a set of visualizations that
are contextual to (but not embedded in) their current real-world position and
shown as 2D planes facing the user in the world.

##  2 Background

The notion of using technology to immerse humans in an alternate world dates
back to early cinematic screenings or theatrical performances. These
experiences were a precursor to the more contemporary, technology-driven,
interactive flavors of Virtual Reality (VR) and AR/MR, which underpin both IA
and SA. Nonetheless, creating these alternate realities—and immersing users in
them—requires a multifaceted approach, as the human perceptual and cognitive
system is incredibly complex.

Inevitably, these immersive interaction paradigms, often collectively referred
to as Extended Reality (XR, where X can be treated as an algebraic variable
for V, A, M, etc.), have been defined in several different ways. In an effort
to avoid ambiguity and to provide a framework for our analysis, we revisit
these definitions here.

###  2.1 Immersive Technologies

Virtual Reality is defined as a computer-generated, three-dimensional (3D)
environment that can be explored by a user that is predominantly isolated from
their physical surrounding. We prefer the term physical, as opposed to “real”
because the experience of VR can be very much real. VR is typically accessed
using head-mounted displays (HMDs), physical environments such as CAVEs, as
well as standalone “fishtank” VR displays. The term is often attributed to
Jaron Lanier [89], yet various similar definitions exist.

Augmented Reality [4] goes beyond VR by fusing computer-generated objects with
the physical world, enabling both the real and virtual worlds to be
experienced and interacted with by the user. According to Azuma [4], AR
environments have the following three characteristics: (1) combines real and
virtual, (2) interactive in real time, and (3) registered in three dimensions.

Mixed Reality [34] is often used interchangeably with AR, yet according to the
definition by Milgram and Kishino [103], it is a superset that includes AR and
Augmented Virtuality (AV) where a virtual environment is enhanced with
physical world information (essentially, the opposite of AR). According to
this definition, MR does not include VR and spans between the physical
(reality) and the entirely computer-generated (virtuality) environment. This
definition was elaborated further by Milgram et al. [104] in terms of the
extent of world knowledge, reproduction fidelity, and extent of presence
metaphor.

These fairly graphics-centric definitions imply how dependent XR flavors are
on the available interface technology, and consequently how the first VR
systems—and AR systems that followed—focused on graphics depiction. The advent
of smartphones has had a significant impact in the recent popularization and
democratization of XR. In particular, as highlighted by Barba et al. [10],
smartphones have shifted MR from the lab to the real world, where technology
becomes capability, space becomes place, and vision becomes perception. At the
same time, the related improvements in display technology, miniaturization,
power consumption, and mobile graphic processing power make contemporary XR
interfaces capable of delivering upon the vast potential of XR at the consumer
level [125]. Moreover, such systems often go beyond the visual modality and
add virtual (or simulated) information using, auditory, somatosensory, and
even olfactory modalities [111, 113].

Nevertheless, using 2D visualizations in immersive settings is challenging;
several efforts explore methods for visualization placement and interactivity
within such immersive information spaces [112, 87]. 3D visualization, on the
other hand, while initially popular in the research field [119], has been
shunned in visualization research because of inefficiencies arising from
occlusion, distortion, and navigation [46, 108]. In addition, inherent XR
challenges such as interaction, registration, calibration, tracking, and
visual coherence remain [125], yet contemporary devices have somewhat
ameliorated some of these issues [5, 90].

###  2.2 Ubiquitous and Immersive Analytics

Ubiquitous analytics (UA), initially proposed in 2013 [45], was one of the
first concerted research efforts from the visualization domain to go beyond
the regular desktop computer to encompass groups of networked mobile devices
for anytime and anywhere sensemaking. The approach is motivated by post-
cognitive frameworks such as distributed and embodied cognition, which
emphasize the role of the surrounding world and the user’s own body for
sensemaking [44]. In 2014, the approach was extended to encompass Mixed
Reality as the next logical step for data visualization [118].

Immersive analytics (IA), introduced in 2015 [30], is concerned with
investigating how immersive interaction and display technologies can be used
to support analytical reasoning and decision making [100]. IA research efforts
explore a plethora of challenges, such as user experiences in collaborative
and multi-user applications [37, 19], long-term deployment and usage [11], and
evaluation of visualization techniques applied in an IA context [53, 70, 132,
69, 86]. True to its tech-centric nature, a significant focus of IA is on
developing toolkits and frameworks, such as DXR [131], IATK [35], and VRIA
[25].

There also exist research efforts that use AR/MR but which does not situate
the information in the actual physical space where the data is located,
collected, or leveraged. Hence, these are not strictly SA examples. These
include, for example, the study of visual cues that help spatial perception of
datasets in AR-based immersive environments [97], 3D spatial interaction using
mobile devices [23], effectiveness of different edge types in AR settings
[24], glyphs in AR [32], and studies of 3D exploration in AR [6].

###  2.3 Situated Analytics

In situated analytics, analytical reasoning is supported by embedding the
visual representations and interaction of the resulting data in a physical
environment using AR [48]. To make the distinction between SA and “mere” IA in
AR, as discussed in the prior section, we typically think of SA as also
requiring data and analytical tasks that are tied to the user’s current
physical location [139].

Inevitably, the real potential for SA is in supporting experts in the field,
as highlighted by Whitlock et al. [147]. Although the term SA is fairly
contemporary, systems that potentially fulfill this definition date back to
the Touring Machine [52] and similar mobile, wearable AR set-ups. A broad
range of research activities in immersive technologies has addressed issues
that can be challenging in SA context, such as cross-device synergies [88],
perceptual challenges [123], labeling [60], highlighting [47], etc. One
objective of this survey is to bring clarity to what constitutes an SA system
in both the past and the present.

###  2.4 A Survey of Surveys

There is a long history in the AR field on surveying and classifying AR/MR
technologies. One of the earliest surveys from Azuma [4], provides a widely
adopted definition of AR, accompanied by examples of applications, and a
discussion on the two main challenges of AR, registration and tracking.
Focusing on the latter, Zhou et al. [157] discuss AR techniques that have been
studied for a 10-year period, starting from 2008, and presented at the ISMAR
conference. Van Krevelen and Poelman [140] review a variety of applications of
AR, whereas Carmigniani et al. [28] classify AR systems and argue that they
need to be more socially acceptable to facilitate broad adoption. Another
strand of research focuses on specific areas (for example, assembly plants
[143]) and techniques (for example, tracking technique [116] where AR is used
as a key technique. Chatzopoulos et al. [31] focus on mobile AR, and the
influence of handheld systems in the domain, highlighting networking
implications. Finally, more inclusive surveys, such as from Billinghurst [18]
and Kim et al. [81] focus on AR techniques and their challenges.

Driven by the increased interest to explore the synergies of immersive
technologies and data visualization over the last five years, a number of
surveys focusing specifically on the intersection of visualization and AR, IA,
SV, and SA have emerged. For the broader IA domain, the work by Fonnet et al.
[55] explore systems that represent information related to its physical
location and analyze them from five main perspectives; technologies, data
mapping, interactions, collaboration, and types of user studies conducted.
They also propose three directions for the future of IA, in fostering multi-
sensory and embodied interactive IA, converging towards best practices, and
aiming at real life IA systems. Ens et al. [50] describe 17 grand challenges
in the broader domain of IA, deriving five specific challenges on the topic of
spatially situated data visualization.

Some surveys focus specifically on visualizations embedded into the physical
world. The work of Willett et al. [150], although not a survey per se, offers
a distinction between non-situated, situated, and embedded data
representations in terms of their relationship to physical referents. They
also review enabling technologies and potential applications. Zollmann et al.
[158] present a survey of visualization techniques used in visualizations that
deploy AR. Besancon et al. [14], in their review of spatial interfaces for 3D
visualization, discuss various interaction techniques that can be applied in a
SA/SV context. Bressa et al. [21] review prior work in terms of the
situatedness, presenting five perspectives on it, including space, time,
place, activity and community. They also note that many papers do not discuss
deeply their definition of SV, and instead rely on the definitions from White
and Feiner [144], and Willett et al. [150]. Furthermore, they also identify
the dominance of AR among other technologies used for SV.

To that end, understanding how AR techniques are used to present data in-situ
is a key component for understanding SA and SV. Recent work by Fröhler et al.
[57] discuss the concept of cross-virtuality analytics (XRA), which enable
visual analysis through a combination of different devices across the reality-
virtuality continuum. Their approach identifies existing challenges and future
opportunities for such cross-device visualization research. And finally,
Elmqvist [44] review the history of the ubiquitous analytics research agenda
and its influence on immersive and situated analytics, as well as future
challenges.

###  2.5 Summary and Comparison with Prior Art

While AR has been used in the context of data visualization for more than two
decades, not much focus has been given on analyzing the characteristics of AR
techniques utilized in situated analytics, especially those of in-situ
depiction. Our work aims to address this. We also believe that the fact that
situated analytics is based on a different platform from conventional
visualizations calls for different rules for effectively presenting data in a
situated context. Our work tries to disclose these rules from a corpus of
papers. Unlike prior work, our focus is on the synergy between AR technology
and SA: the interplay between the situatedness of data and its integration
into the real world. Compared to past surveys, we attempt to both be more
prescriptive in qualitatively describing archetypes of SA in the literature,
as well as more generative of new directions for research.

The situated visualization survey by Bressa et al. [21] is most related to the
present paper. In contrast to our work, Bressa et al.’s survey focuses on
situated visualization, whereas our goal here is to survey higher-order
analytics in situ based on the classic sensemaking loop by Pirolli and Card
[114]. We also mine our classification using dimensional reduction to extract
high-level archetypes of SA in the literature. Finally, Bressa’s work also
mostly ignores the rich body of situated visualizations proposed by the
Augmented Reality community prior to the introduction of situated analytics in
2016.

##  3 Design Space: Situated Analytics

Here we derive the design space for SA implemented using AR. We first give our
definition of situated analytics. This is followed by our survey methodology.
Then we present the design dimensions for classifying the surveyed papers.

###  3.1 Situated Analytics: Definition

Thomas et al. [139] define situated analytics (SA) as methods “supporting
analytic reasoning through the use of situated visualizations.” White and
Feiner [144] define situated visualization as a visualization that “is related
to and displayed in its environment.” Together these definitions entail that
SA is (1) data-driven, (2) uses interactive visualization, (3) is based on
Augmented Reality to integrate with the physical environment, (4) draws on the
user’s location, and (5) integrates analytical reasoning. Table I summarizes
these criteria.

In fact, we can use these criteria to distinguish between different forms of
analytics and visualization. Table II gives an overview of such a
classification. Note that this classification is not intended to represent
exclusive and disjoint categories; for example, it is widely accepted that IA
and SA are subsets of visual analytics, that SV is a subset of visualization,
and that all forms of analytics incorporate visualizations. In fact, this
mini-taxonomy also gives rise to the concept of immersive visualization;
visualizations that form the building blocks of IA applications.

It is also important to acknowledge the wealth of existing work on situated
visualization even prior to situated analytics being coined in 2016 [48, 125].
Most of this work was primarily done in the Augmented and Virtual Reality
research areas. While there exists some early work that fits the SA definition
(and is thus included in our survey), we note that many early situated
visualizations were characterized by being relatively simplistic; they tended
to either use labels or simple symbols, or 3D objects with a real-world
appearance. Support for higher-level analytical reasoning is the key factor
distinguishing SA from SV, similar to how visual analytics (VA) is
distinguished from visualization. We present a disambiguation of what
sensemaking support means in this context in Section 3.3.4.

TABLE I: Situated analytics criteria. These criteria were used for selecting papers to include in our survey; all must be fulfilled for a paper to be selected. # | Criteria
---|---
1 | Presents data
2 |  Uses interactive visualization
3 | Deployed using Augmented Reality techniques
4 |  Utilizes the user’s physical location
5 | Integrates analytical reasoning
TABLE II: Distinguishing analytics. Various forms of data analytics and visualizations classified based on their use of data (Data), visualization (Vis), computing platform (Platform), user’s physical location (Loc), and integration of the analysis process (AP). Topic | Data | Vis | Platform | Loc | AP
---|---|---|---|---|---
Visualization | ✓ | ✓ | Desktop | $\times$ | $\times$
Visual analytics | ✓ | ✓ | Desktop | $\times$ | ✓
Immersive visualization∗ | ✓ | ✓ | VR/AR | $\times$ | $\times$
Immersive analytics | ✓ | ✓ | VR/AR | $\times$ | ✓
Ubiquitous visualization | ✓ | ✓ | Any | ✓ | $\times$
Ubiquitous analytics | ✓ | ✓ | Any | ✓ | ✓
Situated visualization | ✓ | ✓ | AR | ✓ | $\times$
Situated analytics | ✓ | ✓ | AR | ✓ | ✓

###  3.2 Survey Methodology

Our paper selection process was performed in four steps: (1) collecting
candidate papers for potential inclusion in our taxonomy, (2) selecting the
candidate papers that match our inclusion criteria, (3) classifying the
selected papers, and (4) pruning papers that do not fit our definition of SA.

Collection. We first collect papers using the keywords listed in Table III
from conference proceedings or journals in AR/VR (for example, IEEE VR, IEEE
ISMAR), visualization (for example, IEEE VIS, IEEE TVCG, EuroVis, IEEE
PacificVis) and human-computer interaction (HCI) (for example, ACM CHI, ACM
UIST) communities. We restrict our search from 1995 to 2021. To retrieve
papers outside of the conferences described above, such as application papers
where SA is deployed, we also search the ACM Digital Library, IEEE Xplore,
Elsevier’s ScienceDirect, and Google Scholar using the same keywords. We also
include papers that are cited by the candidate papers that have relevant
titles or contain keywords from Table III to our list of candidates.

Using the above search process, we collected 312 candidate papers that
advanced to the next step.

Selection. We select papers that match our inclusion criteria (see below) from
the list of candidate papers obtained during the collection process. We use
the definitions in Section 3.1 to formulate the inclusion criteria in Table I.

TABLE III:  Keywords relevant to situated analytics. We used these keywords to
search for papers during the collection process. Relevant Keywords
---
situated visualization (with AR), situated analytics (with AR)
visualization (with AR), augmented reality visualization,
immersive analytics, immersive visualization, situated information,
situated information visualization, situated infovis, situation vis,
point of interest visualization, situated vis, immersive vis, AR vis,
poi vis, situation visualization, augmented reality vis,
in situ data visualization (with AR), in situ data analysis (with AR),
in situ analytics (with AR), in situ data vis (with AR),
situated analysis (with AR), situated data (with AR).

The selection process was conducted by three authors. A paper was selected
only if all three authors agreed that the paper met the four criteria. Note
that we include only one mention of each system; in other words, we eliminated
multiple versions of the same work by the same authors.

We selected 75 papers from the 312 candidate papers. The earliest work in our
taxonomy is the Touring Machine [52] by Feiner et al., published in 1997. The
latest work is MARVIS [88] by Langner et al., published in 2021.

Classification. We classified 75 papers using our taxonomy (Section 3.3); see
Table V. We also present changes in taxonomy over 5-year periods in Table VI.
Note that we consider all aspects of the surveyed papers beyond than those on
Table V. Please refer to Sec. 5 for an in-depth analysis of the surveyed
papers, including aspects such as interaction, usage scenarios, 2D vs. DD, and
development.

Pruning. Finally, we wanted to prune candidate papers that our classification
revealed to not be true situated analytics papers. More specifically, papers
that only support the very first Sensemaking task—read—are situated
visualizations and do not support higher-level analytical reasoning. We ended
up removing 24 such papers out of the total of 75 classified papers, resulting
in a remaining 51 papers.

###  3.3 Taxonomy Dimensions

Drawing on our literature survey (Section 2) as well as our corpus of selected
papers, we now describe the dimensions we use to characterize this novel
research area. We focus on what makes SA unique; the situatedness of the
visualizations, the data depictions, and the sensemaking support.

To this end, we use a 5W1H approach—what, where, when, what, who, and how—to
structure our inquiry. This approach yields several motivating questions that
we present in Table IV to describe how SA tools display data in the world
using AR. Note that we leave the user (who?) and the motivation for the task
(why?) outside the scope of our design space as they are strictly application-
specific; we want to be able to capture situations where two systems have
vastly different uses, but identical (or at least similar) properties in terms
of situatedness.

As can be seen from Table IV, this yields four design dimensions: Situating
Triggers, View Situatedness, Data Depiction, and Sensemaking Tasks. The first
two deal with how the data is situated in the world whereas the remaining deal
with their representation and analytical support. We describe each of these
dimensions in turn below.

TABLE IV: The 5W1Hs of our taxonomy. Using basic problem-solving to derive our design dimensions: what, who, where, when, why, and how. Questions | Dimensions
---|---
When does the system react? | Situating Triggers
Where is the view represented? | View Situatedness
What type of data is presented? | Data Depiction
How is sensemaking supported? | Sensemaking Support
Why is the data represented in space? | Out of scope
Who is interested in using the data? | Out of scope

####  3.3.1 Situating Triggers

Situating triggers are discrete events that lead to the instantiation or
modification of an SA tool. Can be classified as:

Referents: Spatial triggers that indicate a view should be instantiated or
modified. These can be further defined as:

Location: When the instantiation of the data view or visualization is
triggered by the user’s location. This is typically a physical place, such as
GPS location or a distance walked from a starting point in steps as indicated
by a pedometer. For example, the Bottari app [8] uses the user’s location
information to recommend nearby restaurants around them. BoreholeAR [92]
points exact locations of the boreholes in the user’s view via the GPS.

Object: Here the instantiation of the data view or visualization is triggered
by an object being detected by a sensor on the user’s device (for example,
marker, object recognition, loud sound originating near the user). PapARVis
[33] detects static information displays in the real world using QR codes and
extends them with virtual content in AR. ServAR estimates the volumetric
amount of food on a plate (identified by a fiducial marker) [120].

Context: These are situations when the instantiation of the data view or
visualization is triggered by semantic or conceptual information surrounding
the user at the current time and place. The trigger may or may not be spatial.

Event: These are situations triggered by a specific event. They can be
unplanned: this includes situational information identified by the system at a
given moment in time that triggers a view instantiation or modification. Tools
of this type deliver different signals based on the situation that the user is
in. For example, a collision detection system in a car may sound a warning
when the car approaches an obstacle [80], or assistive technologies for people
with hearing loss may provide different sounds to assist communication during
an ongoing conversation [73]. They can also be planned, such as for post-
mortem analysis of a dynamic sequence; for example, Lin et al. [95] propose a
tool for analyzing ball movements in basketball.

Setting: The type of space that the user is in and its state at the time they
are in it—for example, a construction site causing the appropriate SA software
to boot up [12]—that is not explicitly tied to the location of the user. Other
examples include a flood simulation not tied to a specific location [63] or a
surgical environment augmented by AR overlays [16]).

Non-situated: Here the view is spawned independent of the user’s locus,
context, or objects. A system may have a non-situated view creation command,
but in order for a view to be situated—that is, positioned relative to
objects, locations, or contexts around the user—a situating trigger is still
required. If neither the trigger nor the view are situated, the system is not
a situated analytics system. Consequently, we do not include non-situated
systems in Table V.

####  3.3.2 View Situatedness

View situatedness describes the integration of data views into the world,
potentially including the device viewport.

World-Registered: Here the data is presented so that it is positionally
anchored to the real world, such as an imitation of a real object (for
example, an amount of stacked sugar cubes representing the amount of sugar in
food items sitting on a table near those food items). We consider two levels:

World-absolute: the data is situated at the exact point of interest (for
example, a holographic representation of a person to reenact an event exactly
on top of a marker); what Willett et al. [150] called embedded data. In other
words, the data is anchored to a physical location, regardless of the user’s
position or orientation. This aligns well with Azuma’s [4] definition of AR
and in that regard it is one of the most popular flavors of AR, used for
navigation, tour guides, cultural heritage, etc. For example, the Touring
Machine overlays information about a university campus on the user’s view
[52].

World-relative: The data is situated in a location that is relevant to a point
of interest, but the actual layout of the visualizations is designed to
optimize user interpretation rather than to strictly mirror real-world object
positions (for example, a cluttered real-world view with labels that are
neatly organized but linked to real-world objects via indicator lines). The
optimization may include user-driven interactions within the view, such as
placing virtual furniture in a house [59] in the IKEA Place app [71], or
estimating size to serve patients in hospital [120].

Sticky: Sticky data views are linked to a specific location, but will persist
in the periphery of the user’s viewport even when this location falls outside
the current field of view. As illustrating examples, consider an arrow that
guides the user to a specific destination [107], or labels that refer to items
in the user’s vicinity even when the items are outside the field of view [60].

Device-Anchored: Here the data view is linked to the user’s own device,
regardless of where it is in the world. It is situated because it was spawned
by a situating trigger; for example, a notification on a smartphone when you
are near a point of interest. We consider two types of anchorings:

Device-relative: The data view changes, oftentimes optimized to the user in
absolute terms. Consider a GPS map on a smartphone that rotates and changes
based on the user’s position and orientation. Various systems that utilize
automated label placement algorithms [60] have the data anchored to device-
relative views, so that it does not disturb the user’s view.

Device-fixed (Overlay): When the data is represented in front of the user’s
view of the real world on their device, without being registered to the world.
The canonical example of a device-fixed display is a heads-up display (HUD),
as seen in modern (3rd generation and up) combat aircraft; the HUD is attached
to the aircraft in front of the pilot, but displays situated information on
top of the scene where the aircraft is pointing. Another example of a device-
fixed view is the information display in a viewfinder of a DSLR camera, which
stays fixed regardless of where the user points the camera. An example of such
views can be seen in an SA tool for shopping by Elsayed et al. [48]. In their
system, the object in the center of the view is detected, and the analysis
view is always shown on a particular area of the view.

Note that a view can be both absolute and relative. An object may be
positioned absolutely inside an automobile, for example, but the automobile is
a container that can move. Thus, the view is relative to the world.

####  3.3.3 Data Depiction

This dimension describes the forms that data representations take; that is,
the visual encoding and abstraction of data. This is relevant to the taxonomy
because of the way the data representations integrate with the real-world
environment.

Visual Encoding: The visual encoding is directly relevant to how the visual
representations are situated in the physical environment: as 2D, as 3D, or as
nonvisual objects.

2D: The data is depicted on a two-dimensional plane inside the 3D environment.
We include 2.5D SA systems in this category, where 2D data is depicted in 3D
space. Examples include standard 2D visualizations—such as scatterplots,
barcharts, and linecharts—as well as labels accompanying visualizations, in-
world noticeboards, and building maps. ARVino [82] presents an analysis of
viticulture data of a particular land using a 2D heatmap on a world-registered
environment.

3D: The data is depicted as a three-dimensional entity; the canonical situated
visualization. These can be volumetric data, 3D visualizations (e.g, cubic bar
charts, 3D scatter plots, etc.), as well as representations of physical
objects. If the data is presented using 3D cubes, then it belongs to this
category. SiteLens [144] and FieldView [147] embed 3D graphical cues (for
example, spheres, bars) about objects within the user’s view. Some 3D
visualizations are used to simulate real-world objects within the world [12,
124].

Non-visual: The data is depicted in a non-visual manner (for example, sound,
smell, etc.). In this case the non-visual data may be accompanying a visual
depiction, or may be adding context to the user’s situation (for example, via
narration or via signals). For example, different sound patterns are used to
deliver different information to aid a microscopic precision task [121], or to
communicate between those who are visually impaired [74].

Data Abstraction: These are the visual expressions used for data
representations. These can be classified as:

Abstract: Data is expressed using abstract graphical marks and channels that
are used to encode features and values of a dataset (for example, lines,
squares, spheres, etc. of varying color, position, size, etc). This approach
is most often used for data that is abstract in nature, that is, that does not
have a straightforward mapping to a visual representation. An example of this
is an analytics tool for collaboration [88]. People share the visualization
via a head-mounted device. Another model presents textual labels to pinpoint
borehole location on top of the view [92].

Physical: The data is expressed using the real world’s spatial dimensions to
represent spatial features of the data. This could be an artificial imitation
of an existing object/entity (for example, an imaginary chair in a place of
interest, a heart positioned in a cutaway over a human patient). For example,
the MARVisT [32] toolkit creates visualization glyphs of common real-world
objects for data representation.

Hybrid: Hybrid abstractions include components of both abstract and physical
data. A canonical example is a map, which is an abstraction of 3D space that
exists as an entity in the real world. While 2D maps are common and useful for
orientation in a 3D space, more esoteric approaches use 3D representations
such as a world-in-miniature (WIM) [135], which provides the user with an
easily accessible downsized version of the 3D environment. These maps present
an overview by locating the point of interest to the user during navigation
[43, 151, 3].

####  3.3.4 Sensemaking Support

Finally, according to Table I, what distinguishes situated analytics from
situated visualization is the support for analytical reasoning. While this is
a vague criteria—similar to the distinction between visualization and visual
analytics—we choose to base this taxonomy dimension on Pirolli and Card’s
sensemaking loop [114]. We model the following levels. Note that the
distinction is based on the role the system plays in the sensemaking process:

Read: The technique supports extracting data from the visual representation.
The techniques that belong to this category focus on delivering information
relevant to the situation. However, reading does not entail active interaction
by the user beyond merely navigating in physical space (i.e., walking and
moving your head). Representing the lowest abstraction level in the
sensemaking loop [114], this property is supported by both situated
visualization and analytics; in other words, it is not sufficient for a tool
to qualify as SA. Examples of this task include observing the yield data
overlaid on top of the vineyard as a heatmap [82], providing guidance or
information about tourist destinations (for example, attractions, restaurants
and so on [151, 8]), and providing information about where boreholes are
within a region [92].

Explore: Once a tool enables the user to interact with the data beyond mere
situated viewing, we consider it to support exploration. Exploration includes
interactions such as filtering, rearranging, zooming, panning, drilling down,
and other interactions that facilitate discovery. While the boundary between
SV and SA is blurry, we deem a tool that only supports “read” as situated
visualization, whereas one that also supports “explore” is as situated
analytics. For example, merely drawing a 3D arrow [107] or labeling to augment
the real world [60] does not qualify as exploration, whereas viewing hidden
features behind an occluding wall determined by the user’s interests [77], or
providing interactions, such as filtering and rearranging to help the user
locate their items of interest during shopping experience [48], do.

Schematize: Pirolli and Card [114] define finding a schema as a marshalling or
coordination action where evidence is organized into a “small-scale story” or
some more rigid structure. This is a step above exploration, where the system
helps the analyst to discern some underlying insight about the data being
explored. In other words, tools that belong to this category should be capable
of organizing the data inside the tool, such as creating clusters, annotating
data, and tagging items—i.e., creating schemas. For example, SiteLens [144]
enables the user to choose visual representation of air pollution depending on
the data and context, and MARVIS [88] allows the user to arrange both physical
and virtual AR displays in space depending on the structure of the underlying
data.

Report: Finally, at the top of the sensemaking loop, the analyst organizes the
evidence into an artifact that combines all of the component findings into a
coherent story that can be used to support or reject hypotheses. We also think
of this level as a dissemination of findings to stakeholders or other
analysts. An ideal example of this would be a system that not only helps users
organize evidence for a hypothesis, but also present this material in a
product suitable for dissemination.

TABLE V: Classification of Augmented Reality systems for situated analytics.
The bottom section includes papers that were included in our selection of
papers but omitted from the cluster analysis as they do not support
sensemaking levels beyond Read.

|  |  |  | situating triggers | view situatedness | data depiction |  | sensemaking
---|---|---|---|---|---|---|---|---
|  |  Key:  and  boxes denote dimension applies |  | referents | context | device- |  | world- | visual encoding | data abstraction |  | support
|  |  Cluster colors correspond to Figure 2 |  |  |  | anchored |  | registered |  |  |  |
cluster | no | name (shortened) | year |  location |  object |  event |  setting |  relative |  fixed |  sticky |  absolute |  relative |  2D |  3D |  nonvisual |  abstract |  hybrid |  physical |  |  read |  explore |  schematize |  report
| 1 |  Information Filtering for Mobile AR [76] | 2000 |  |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 2 |  AR for Military in Urban Terrain [96] | 2002 |  |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 3 |  The Virtual Mirror [15] | 2009 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 4 |  Underground Infrastructure Vis [124] | 2009 |  | ✗ |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 5 |  Situated Simulations [93] | 2009 |  |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  | ✗ |  |  ✗ |  |  |  |  | ✗ | ✗
| 6 |  ARtifact [141] | 2012 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ |  |  |  ✗ | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 7 |  Adaptive Ghost Views for AR [77] | 2013 |  |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 8 |  In-Situ Climate Change Vis [94] | 2014 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 9 |  Riverwalk [29] | 2016 |  |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
C1: Simulators | 10 |  Corsican Twin [115] | 2020 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  |  | ✗
| 11 |  Situated Documentaries [67] | 1999 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  |  |  |  |  | ✗ | ✗
| 12 |  MARS [68] | 1999 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  |  |  |  |  | ✗ | ✗
| 13 |  Virtual X-Ray Vision in Mobile AR [9] | 2004 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 14 |  Hypermedia Vis for AR/VR [62] | 2006 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 15 |  AR on Historic Sites [61] | 2006 |  | ✗ |  ✗ |  |  |  | ✗ |  | ✗ |  |  | ✗ |  |  |  |  |  |  | ✗ | ✗
| 16 |  SiteLens [144] | 2009 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  |  | ✗
| 17 |  CityViewAR [91] | 2012 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  | ✗ |  |  ✗ | ✗ |  |  |  |  |  |  | ✗ | ✗
| 18 |  Extended Overview Tech for Outdoor AR [142] | 2012 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 19 |  Wind and Uncertainty Vis [54] | 2014 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 20 |  Whistland [98] | 2017 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 21 |  Pedestrian Navigation AR [75] | 2017 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 22 |  The Urban Cobuilder [72] | 2018 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 23 |  AR for Supporting Aquaculture Farmers [153] | 2019 |  | ✗ |  ✗ | ✗ |  | ✗ | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  | ✗ | ✗
| 24 |  Agricultural Fieldwork Vis [156] | 2019 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 25 |  FieldView [147] | 2020 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  | ✗ |  |  | ✗ |  |  | ✗ |  |  |  |  | ✗
C2: Assistants | 26 |  Basketball AR [95] | 2021 |  | ✗ |  | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 27 |  AR for Manufacture Planning [41] | 2003 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 28 |  AR for Construction [12] | 2005 |  | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  |  | ✗
| 29 |  AR and Tangible Interface in Urban Design [127] | 2007 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  |  |  |  |  |  | ✗
| 30 |  Contextual Anatomic Mimesis [17] | 2007 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ |  |  |  |  | ✗ | ✗
| 31 |  AR for Building Acceptance [126] | 2008 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 32 |  Vis for Psychomotor-Cognitive Tasks [85] | 2009 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  |  ✗ | ✗ |  |  ✗ |  |  |  |  | ✗ | ✗
| 33 |  MR in OR [110] | 2012 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 34 |  In-Situ Flood Vis [63] | 2016 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
| 35 |  Cues for Social Contacts [109] | 2017 |  ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 36 |  HydrogenAR [146] | 2020 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ |  |  |  |  | ✗ | ✗
| 37 |  UpLift [51] | 2021 |  ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  |  |  | ✗ |  |  |  |  |  |  |  | ✗
C3: Planners | 38 |  MARVIS [88] | 2021 |  ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  |  |  | ✗ |  |  | ✗ |  |  |  |  | ✗
| 39 |  Healthy Grocery Shopping via Mobile AR [2] | 2013 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 40 |  Horus Eye [47] | 2016 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 41 |  SA for Shopping [48] | 2016 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 42 |  Crime Scene Investigation AR [38] | 2016 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 43 |  SA for Network Data [155] | 2017 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
| 44 |  Supporting Fine Details from AR [74] | 2020 |  ✗ |  |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
C4: Scanners | 45 |  Impact of 2D Visualizations in AR [123] | 2021 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ | ✗
O1 | 46 |  AR for Size Estimation [59] | 2008 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  |  | ✗ | ✗
O2 | 47 |  MARVisT [32] | 2019 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  | ✗ |  |  |  |  |  |  |  | ✗
— | 48 |  A Touring Machine [52] | 1997 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 49 |  Augmentable Reality [117] | 1998 |  ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  | ✗ |  ✗ |  ✗ |  |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 50 |  AR on Molecular Models [58] | 2004 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 51 |  ARVino [82] | 2005 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 52 |  AR for Opportunistic Controls [65] | 2010 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 53 |  AR with Paper Maps [106] | 2011 |  ✗ |  |  ✗ | ✗ |  | ✗ | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ |  | ✗ | ✗ | ✗
— | 54 |  Handheld AR for Indoor Navigation [107] | 2011 |  | ✗ |  ✗ | ✗ |  ✗ |  |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 55 |  AR for Maintenance and Repair [66] | 2011 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ | ✗ |  |  | ✗ |  |  ✗ |  |  |  | ✗ | ✗ | ✗
— | 56 |  Bottari [8] | 2012 |  | ✗ |  ✗ | ✗ |  | ✗ | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 57 |  Label Placement in AR [60] | 2012 |  ✗ |  |  ✗ | ✗ |  ✗ |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 58 |  AR for Outdoor Navigation [43] | 2012 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 59 |  Inattentional Blindness in Surgical AR [40] | 2013 |  ✗ |  |  ✗ |  |  ✗ |  | ✗ |  | ✗ |  ✗ |  ✗ |  |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 60 |  Orientation Measurement for Audio AR [64] | 2014 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  | ✗ |  ✗ |  ✗ |  |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 61 |  Hedgehog Labeling [138] | 2014 |  ✗ |  |  ✗ | ✗ |  | ✗ | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 62 |  HMD Vis for the Deaf [73] | 2015 |  ✗ | ✗ |  |  |  | ✗ | ✗ |  ✗ | ✗ |  ✗ |  ✗ |  |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 63 |  Rhema [137] | 2015 |  ✗ |  |  ✗ |  |  | ✗ | ✗ |  ✗ |  |  |  | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 64 |  Borehole AR [92] | 2015 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 65 |  Projector-based AR for Welding [42] | 2017 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  ✗ |  |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 66 |  ToARist [151] | 2017 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 67 |  iMAP-CampUS [3] | 2017 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  |  | ✗ |  | ✗ | ✗ | ✗
— | 68 |  SonifEye [121] | 2017 |  ✗ | ✗ |  |  |  ✗ |  | ✗ |  ✗ | ✗ |  ✗ |  ✗ |  |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 69 |  ServAR [120] | 2017 |  ✗ |  |  ✗ | ✗ |  ✗ | ✗ | ✗ |  ✗ |  |  ✗ |  | ✗ |  ✗ |  ✗ |  |  |  | ✗ | ✗ | ✗
— | 70 |  Collision Warning AR [80] | 2018 |  | ✗ |  | ✗ |  ✗ |  | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 71 |  Information Seeking Strategy in AR [26] | 2019 |  | ✗ |  ✗ | ✗ |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 72 |  Situated Storytelling [78] | 2019 |  | ✗ |  ✗ | ✗ |  ✗ |  | ✗ |  | ✗ |  ✗ |  | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 73 |  HypAR [49] | 2019 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  |  |  |  ✗ | ✗ |  |  | ✗ |  |  | ✗ | ✗ | ✗
— | 74 |  PapARVis [33] | 2020 |  ✗ | ✗ |  ✗ |  |  ✗ | ✗ | ✗ |  | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗
— | 75 |  Visualizing Air Pollution (AiR) [101] | 2021 |  | ✗ |  ✗ | ✗ |  |  | ✗ |  ✗ | ✗ |  |  ✗ | ✗ |  |  ✗ | ✗ |  |  | ✗ | ✗ | ✗

TABLE VI: Statistical summary of classified papers. Each value in the four
dimensions has been abbreviated.

|  | Situating triggers | View situatedness | Data Depiction |
---|---|---|---|---|---
|  | Referents | Context | Device- |  | World- | visual encoding | data abstraction | Sensemaking Support
|  |  |  | anchored |  | registered |  |  |
Year | Total |  loc |  obj |  time |  set |  rel |  fix |  sti |  abs |  rel |  2D |  3D |  nonv |  abs |  hybr |  phys |  read |  expl |  sche |  repo
bef. 2000 | 2 | 2 | 0 | 0 | 0 | 0 | 2 | 0 | 2 | 0 | 2 | 2 | 0 | 2 | 2 | 2 | 2 | 2 | 0 | 0
2000 - 2004 | 4 | 3 | 2 | 0 | 1 | 1 | 1 | 0 | 2 | 1 | 1 | 4 | 0 | 3 | 0 | 1 | 4 | 4 | 0 | 0
2005 - 2009 | 12 | 6 | 3 | 0 | 6 | 1 | 4 | 0 | 7 | 5 | 6 | 10 | 0 | 6 | 4 | 10 | 12 | 12 | 3 | 0
2010 - 2014 | 7 | 4 | 3 | 0 | 1 | 3 | 4 | 0 | 5 | 2 | 5 | 4 | 0 | 4 | 3 | 4 | 7 | 7 | 0 | 0
2015 - 2019 | 14 | 6 | 7 | 0 | 2 | 2 | 10 | 0 | 10 | 0 | 12 | 6 | 0 | 12 | 4 | 3 | 14 | 14 | 1 | 0
2020 - onw. | 8 | 2 | 3 | 1 | 3 | 2 | 5 | 0 | 5 | 3 | 6 | 6 | 0 | 7 | 3 | 3 | 8 | 8 | 4 | 0
Total | 47 | 23 | 18 | 1 | 13 | 9 | 26 | 0 | 31 | 11 | 32 | 32 | 0 | 34 | 16 | 23 | 47 | 47 | 8 | 0

##  4 Meta-Analysis of Taxonomy

Here we explore our classification of different SA systems through an ensemble
combination of multiple clustering algorithms. This allows us to identify four
archetypical patterns as well as a few outliers. We describe each of the four
clusters as well as the outliers, focusing on their characteristics and
elaborating on them using examples.

Figure 2: Ensemble combination of multiple cluster analyses performed on our
classification. The resulting cluster network has been visualized through a
variant of a self-organizing map [84]: it has been laid out using a force-
directed graph algorithm and the regions have been tesselated using a Voronoi
diagram. This exposes the structure of the design space and current research
in this area. Note that while the graph layout algorithm is non-deterministic,
this has no effect on the resulting clusters, only the visual appearance of
this self-organizing map. Thus, the specific layout does not impact the
validity of our work.

###  4.1 Analysis Method

Unlike traditional survey papers, where the authors manually cluster the
surveyed papers, we take a data-driven approach using an ensemble of multiple
cluster algorithms. We do this to reduce the impact of author bias and
increase the validity of our resulting clusters. Furthermore, by using an
ensemble combination of multiple cluster analyses, we can eliminate
idiosyncracies from individual cluster algorithms.

After the automatic cluster analysis, all authors validated the resulting
clusters and outliers. Any discrepancies raised by one or more authors were
discussed as a group and the classification redone. This manual validation
step allowed us to confirm the findings using our own expertise.

To perform cluster analysis, we first transform each surveyed paper into a
15-dimensional binary vector. Note that we do not use the dimension
“sensemaking support” in the analysis; it is used only to distinguish SA from
SVs. A value of each dimension in a paper is 1 if the marked value is “yes”
and 0 if the value is “no” in the taxonomy table. We have three goals in
selecting a clustering algorithm:

  1. 1.

Eliminate algorithm idiosyncracies. We need a method that combines multiple
cluster analyses.

  2. 2.

Yield high clustering quality. The quality of created clusters should be
optimal.

  3. 3.

Represents both large and small clusters. The algorithm should not directly
dismiss small clusters.

Accordingly, our strategy is to create an ensemble model of five clustering
algorithms, where each algorithm is optimized to yield high quality clusters.
The five different clustering methods are k-means, Hierarchical Clustering,
Spectral Clustering, Birch, and UMAP initialized with PCA.

Optimizing models. We optimize all hyperparameters except the number of
clusters (the $k$-value) via random grid search. To choose the optimal number
of clusters, we run each model for different number of clusters, from $k=2$ to
11, to avoid dismissing small but significant clusters. In choosing the
optimal number of clusters, we use a combination of three metrics: Silhouette
[122], Calinski-Harabasz [27], and Davies-Bouldin [39] scores. These metrics
are widely used for evaluating the quality of clusters that do not have
predetermined labels. Since they have different units, for each $k$ we rank
them ordinally, giving 10 to the highest and 1 to the lowest ones. Then, we
sum these three scores and select the one with the lowest score. As a result,
the optimal $k$ for k-means is 4, 11 for Hierarchical Clustering, 7 for
Spectral Clustering, and 4 for Birch.

UMAP is a model optimized for obtaining local structure. We find the optimal
hyperparameters via random grid search. We initialize UMAP using PCA [152], as
recent work [83] suggests that such initialization helps preserve global
structure. We categorize an element to a group if the Euclidean distance
between the element and any element of the group is less than the threshold of
0.1. That is, if an element has a neighbor within 0.1 distance, then it is
classified as the same group as that neighbor. This results in a total of
seven clusters.

Ensemble Model. We use an ensemble model to formulate clusters using the five
selected clustering algorithms. More specifically, we use a graph-based
consensus clustering method. First, we create five $47\times 47$ similarity
matrices $S_{i}~{}(i=\text{1 to 5})$, where each $i$ represent different
clustering algorithms. These similarity matrices assume that papers belonging
to the same cluster in a clustering algorithm are related. Accordingly, given
$p_{x}$ as the $x^{th}$ paper of the taxonomy, and $C_{i}$ as the clustering
results of $i^{th}$ cluster, each entry in the similarity matrix is defined as
follows:

$S_{i}(p_{x},p_{y})=\begin{cases}1&\text{if $C_{i}(p_{x})$ = $C_{i}(p_{y})$,
$x\geq y$.}\\\ 0&\text{otherwise.}\\\ \end{cases}$ (1)

By adding all $S_{i}$s, we formulate a consensus matrix [56]
$CSS(p_{x},p_{y})$ that represents the relationship between papers based on
clusters from five clustering algorithms. We do not define the lower
triangular part of the matrix, as it is redundant to the upper triangular one.
Then, we create a graph $G=(V,E)$ where the vertices $v_{x}$, $v_{y}$ are the
set of 47 papers and the edge $v_{ij}$ that connect between papers $p_{x}$ and
$p_{y}$ are defined as $CSS(p_{x},p_{y})$. In identifying the clusters, we
utilize the normalized cut algorithm [129] to the graph. This yields four
clusters representing common patterns.

Validation Process. Three of our authors validated the results from the final
cluster. To understand the topological structure of our graph, we created a
force-directed node-link diagram, as shown in Fig. 2. The diagram is drawn
using the same set of vertices and weighted edges of the graph $G$. Using the
diagram and the information from Table V, we checked papers within the
clusters and identified the similarities between them. We also distinguish
outliers and ambiguous works that do not clearly belong to a cluster.

Implementation. We conducted the clustering in a Jupyter Notebook with Python
3.9 environment, on a Macbook Pro 2021 version that has a M1 Chip, 1TB SSD,
and 16GB RAM. We provide details of our analysis—classification, cluster
analysis, and ensemble combination—in an OSF repository: https://osf.io/3wxv2/

###  4.2 Resulting Clusters

The graph structure of surveyed SA systems resulted in four clusters and four
outliers; see Fig. 2. Information about each paper is shown in Table V. We
also present the composition of these clusters in 5-year periods in Table VI.
Below we describe the characteristics of these clusters (C1-C4) in detail.

(a)

(b)

(c)

(d)

Figure 3: Prototypical examples of Simulators (C1).
LABEL:sub@subfig:corsicantwin Corsican Twin: a tool for designing situated
data visualizations [115]. LABEL:sub@subfig:riverwalk RiverWalk: a situated
analytics tool that superimposes historical images onto matching views [29].
LABEL:sub@subfig:virtual_mirror The Virtual Mirror [15]: an AR-based mirror
that lets users view part of a virtual object that is occluded in the user’s
view. LABEL:sub@subfig:ghost Adaptive ghosted view [77]: a technique that
helps users look at the interior of an object from outside, in a see-through
manner

.

(a)

(b)

(c)

(d)

Figure 4: Prototypical examples of Assistants (C2). LABEL:sub@subfig:mars
MARS: a situated analytics tool that helps the user to explore about different
locations of Columbia University campus [68]. LABEL:sub@subfig:xrayvision A
situated analytics tool showing a virtual X-Ray vision of a location [9].
LABEL:sub@subfig:sitelens SiteLens: a 3D visualization that directly
visualizes information related to the user’s physical location [144].
LABEL:sub@subfig:fieldview FieldView: a situated analytics tool that supports
professional activities conducted in the field [147].

#### C1 – Simulators

Situating trigger |  View
situatedness |  Visual
encoding |  Data
abstraction
---|---|---|---
referents/ location, object |  world-registered/ absolute |  3D/2D |  physical

The Simulators cluster presents simulated data in situ (Figure 3d). Such data
representations add authenticity and immersion for the user. To achieve this
effect, many systems avoid using device-anchored views and instead register
data to the world. These tools are absolute in their view situatedness. Works
that belong to this cluster are triggered by mainly referents, either by
objects or locations. The data in this cluster are presented as a form of 2D
or 3D glyph that emulates a real object. They are registered to the World in
an absolute manner. This may mean that current SA systems are more focused in
locating the data than providing changes, or interactions within the view.

One example of this is Corsican Twin [115], an SA tool for embedding data
visualizations for building maintenance purposes (Fig. 3d(a)). There are also
simulators that utilize 2D visualization by superimposing another processed
view on top of the scene the user is currently seeing. One example of this is
RiverWalk, a visualization that superimposes historical images onto matching
views of a city [29] (Fig. 3d(b)).

#### C2 – Assistants

Situating trigger |  View
situatedness |  Visual
encoding |  Data
abstraction
---|---|---|---
referents/ location |  device-anchored/ fixed |  2D/3D |  abstract/ hybrid
|  world-registered/ absolute |  |

Assistants aid the user with location-specific information (Fig. 4). The 19
works belonging to this cluster are all triggered by location. Assistants
attempt to support user’s decision making processes about by presenting data
relevant to the user’s location. These systems often use both 2D and 3D
components to represent data. The task is observational, either being fixed to
the device or being positioned in an absolute location in the world.

One example of this type is FieldView (Fig. 4(d)) [147], an SA tool for field
analysts that uses HMDs and mobile tablet PCs to communicate and investigate
field data. The system presents various types of data that would be helpful
for field analysts to conduct analysis of the land. The data is presented not
only as a chart in tablets, but also as a 3D object to the points of interest.
Another example is MARS [68] (Fig. 4(a)), which helps users explore locations
of interest at Columbia University. The tool provides advanced interactions
such as labeling places, superimposing past buildings at a location, linking
relevant websites, and so on.

#### C3 – Planners

Situating trigger |  View
situatedness |  Visual
encoding |  Data
abstraction
---|---|---|---
context/ setting |  world-registered/ relative |  3D |  physical

The third cluster, denoted as Planners, shares some similarities with
Simulators, yet its main difference is that the visualizations are not
registered in specific locations, but are relative to a setting, such as a
factory floor, a classroom or an operating theater. In that regard, they do
not merely simulate the physical space in its absolute location, but enhance
the sensemaking processes involved with that space.

One example is a visualization of construction sites by Behzadan et al. [12]
(Fig. 5(a)). The user can move around physical 3D cues around the space to
identify the optimal location for construction devices, such as cranes or
excavators. Other examples include medical operation planning [110] and real-
time feedback prompts for joint psychomotor-cognitive training [85].

#### C4 – Scanners

Situating trigger |  View
situatedness |  Visual
encoding |  Data
abstraction
---|---|---|---
referents/ object |  device-anchored/ fixed |  2D |  abstract

The Scanners cluster augments real-world objects with additional information,
akin to a science fiction “scanner” sensor (Figure 6). The cluster is composed
of 7 systems. Triggered by objects, systems in this group present abstract
information mainly in the form of 2D representations. The view is mainly fixed
to the device.

One example of a Scanner is a SA tool for shopping by Elsayed et al. (Fig.
6(c)) [48]. When the user points the device camera at a product, the system
displays the product’s nutritional values, tracks the total amount of calories
bought, and keeps a record of the amount of money spent based on a given
budget. Other Scanner examples include systems for crime scene investigation
[38] as well as for visualizing the invisible fine scientific details [74]
(Fig. 6(d)).

(a)

(b)

(c)

(d)

Figure 5: Prototypical examples for Planners (C3).
LABEL:sub@subfig:construction SA for construction site: a SA tool where the
user can find the optimal location for construction facilities prior to
proceeding the operation [12]. LABEL:sub@subfig:mimesis Contextual Anatomic
Mimesis: an in-situ visualization to improve depth perception of doctors
during medical surgery. [17]. LABEL:sub@subfig:buildacceptance SA for building
acceptance [126] A user can compare the condition of the building with a
prototype of the plan. LABEL:sub@subfig:marvis MARVIS combines mobile devices
and Augmented Reality for data analysis to enable collaboration between users
[88].

(a)

(b)

(c)

(d)

Figure 6: Prototypical examples of Scanners (C4).
LABEL:sub@subfig:groceryshopping A tool that helps grocery shopping by
presenting recommendations on customized healthy products [2].
LABEL:sub@subfig:horuseye HorusEye: A technique that imitates bird and snake
vision to highlight data of interest shown in objects [47].
LABEL:sub@subfig:shopping An SA tool that lets users choose products by
identifying and comparing products [48]. LABEL:sub@subfig:ar_temp A system
that shows changes in thermal images over time [74].

(a)

(b)

Figure 7: Examples of Outliers (O1-O2). LABEL:sub@subfig:sizeestimation A SA
tool that superimposes an augmented object at the user’s point of interest
[59]. LABEL:sub@subfig:marvist MARVisT: A technique that highlights the data
of interest by using virtual objects as glyphs and bing data on them [32].

###  4.3 Outliers

Beyond the four clusters discussed above, we also find two outliers that do
not conform to any specific cluster (see Table V). Below we describe these two
outliers.

Size estimation (O1). The first type of outliers is a size estimation tool.
Gomez and Figueroa [59] enable estimating the exact size of furniture by
visualizing it in the intended space using AR (Fig. 7(a)). This is also a type
of a simulation, but most simulations in Simulators (C1) are triggered by
referents anchored to an absolute position, without the ability to alter its
size or location. Such systems allow a form of spatial exploration.

Highlighters (O2). This outlier is capable of highlighting both abstract and
physical information about a context. MARVisT [32] effectively highlights the
data of interest by using virtual objects as glyphs and by binding data on
them (Fig. 7(b)).

##  5 Discussion

Here we discuss several notable observations drawn from our taxonomy as well
as the surveyed papers that would help future situated analytics research and
practice.

###  5.1 General Observations

Despite our efforts to collect historical situated analytics tools, almost
half of the SA systems (22/47) in this review are from 2015 or later (Table
VI). There was less research on interactive data exploration in AR before this
time, possibly because the AR equipment needed was highly specialized and
expensive, making it mostly inaccessible to the data visualization field. As a
case in point, the commercial release of the Microsoft HoloLens was in 2016
and the situated analytics concept grew out of the immersive analytics
movement in data visualization in that same year [48, 125]. Such improvement
of AR technology has also contributed to the proliferation of SA tools, as
most systems with schematizing support were only developed after 2015.

Overall, most of our surveyed SA systems are triggered by referents, whether
they are location or object; very few are triggered by context (see Table VI).
We speculate that this is because referents are generally easier to detect
than context. For view situatedness, systems have a slight preference for
device-anchored rather than world-registered views. Registration, or the
alignment of virtual objects with the real world, is a key challenge in AR,
which may explain the focus on device-anchored views that do not require
accurate registration. Note that whether device-anchored or world-registered,
systems tend to place their data in a fixed, absolute rather than a relative
location. Finally, abstract forms of data is the dominant type in data
abstraction.

###  5.2 When to Use Situated Analytics?

We may be approaching a future where wearable AR devices become ubiquitous.
Imagining such a future, it is clear that essentially all computing tasks,
from email to gaming, can be seen as being situated simply by virtue of being
conducted in the field and on the go (in fact, with today’s mobile computing
technologies, this is very much the truth). However, situated analytics comes
into its own when representing situated data. The question is, what are the
real “killer apps” of such situated computing capabilities?

Based on our survey, simulation—where a situated representation of data in the
world can contribute to immersion and understanding—is one such killer app.
Simulations are used to present data as a three-dimensional object that
imitates the form of its original physical shape. For example, MARVisT [32]
renders information using a glyph that looks like a real object. Such
imitation of real-world objects both increases the user’s immersion as well as
provides easy real-world references; for example, by showing the flood level
of a historical hurricane in an afflicted area [63]. To that end, simulation
techniques can be beneficial for training purposes especially for
sophisticated tasks involving manual dexterity [65], such as medical surgery
[16, 17], and construction [13], or in hazardous conditions [94], such as when
training first responders for an emergency or military personnel [96] for an
upcoming operation.

Furthermore, AR can also provide a see-through view not visible to our naked
eyes. This is in line with the “Enhanced Vision” concept proposed by Willett
et al. [149]. Examples include displaying underground infrastructure [124] on
top of the real world, or imagery of a patient’s internal organs superimposed
on their body to a surgeon.

At the same time, simulations are a canonical use-case of AR in general, and
so their “killer app” status is not surprising. The real question is what
other such critical and well-suited applications are in store for the future
of SA? Based on our survey of the literature, we think that future SA tools
will excel at truly situated tasks where place itself can be used as an index
into a dataset; for example, in space planning such as organizing the layout
of a factory floor based on usage data, or in real-time command and control
such as controlling emergency responders during a developing natural disaster.

###  5.3 Interaction in Situated Analytics

Interaction is a key component in visualization [154]. However, current SA
systems are more focused on displaying the data rather than providing powerful
ways for the user to interact with it. This is a microcosm of the history of
visualization, where interaction initially received short shrift largely
because visual challenges were dominant. Similarly, for situated analytics,
registration and depiction are still significant challenges in AR/MR research,
thus consuming much research attention. Furthermore, efficient 3D interaction
is generally an intractable problem, with an entire scientific community
devoted to such techniques.

Nevertheless, analyzing our surveyed corpus of situated analytics systems, we
find that many use innovative forms of interaction that leapfrog traditional
visualization interaction techniques. Some of this is due to the mobile
setting, which necessitates using interaction methods beyond the traditional
mouse and keyboard common with personal computers [118]. For example, many of
the surveyed systems use so-called “natural” user interfaces [148] such as
touch [48], gestures [88], and specialized controllers [96] to enable
selecting, filtering, and drilling into data.

At the same time, interaction is also a fundamental element when implementing
human input for sensemaking process. However, while many tools are focused on
providing interactions to read and explore the situation and their relevant
data, they lack the ability for generating hypotheses and validating them
(such as tools for authoring visualizations of the user’s interest or
understanding patterns from data in situ). We speculate that this could
inspire interesting future work. We envision that the development of automated
chart authoring tools [105], natural language interfaces [134, 79] for data
analysis, and recent large language models (for example, GPT-3 [22]) can help
develop SA tools that facilitate generating insights from the situation.

###  5.4 Anchoring to the Device or the World

AR technology is evolving to provide better immersion and visual quality while
increasing user mobility and decreasing unit weight. In that regard, device-
anchored views, whether fixed or relative, can be seen as an antiquated form
that estranges the user from being completely immersed into the environment.
Hence in the long term, we expect that the developers of situated analytics
will be inclined to use world-registered interfaces in places that previously
used device-anchored views. That said, we anticipate that device-anchored
views will still have a role over world-registered views, such as when (1)
reading text [78], and when (2) interacting with controls [48, 8, 147].

Reading text yields the best performance when the text is both (a) facing the
reader and (b) is a convenient size: not too large so that head movements are
needed for reading, and not too small so that letters are not legible. It
follows that a device-anchored view is optimal for this task because
fulfilling the above two conditions is trivial when the text is attached to
the device. In contrast, 3D text anchored to the world can run afoul of both
conditions. Using a billboard and scaling the text to mitigate them is not
realistic and will thus inevitably impact the user’s presence in the world.
This is doubly true for interface elements; a button placed in 3D not only
must be facing the user and be at an appropriate distance to be seen, it must
also not be obscured by other objects and be within reach for easy
manipulation. This is again trivial for a 2D button attached to the user’s own
device. In all of these cases, placing elements on the device will break
immersion, but can serve to remind the user that the view is “augmented” and
not fully “real.”

###  5.5 Utility of 3D in Situated Analytics

2D visualizations are the most common way of depicting data in our survey
corpus, but 3D is slowly gaining popularity. Standard visualizations tend to
be 2D in nature, and designing 3D representations that integrate well with the
surrounding world is a challenge. Furthermore, 3D is still seen with some
skepticism in the visualization community (and rightly so); consider, for
example, Munzner’s rule of thumb of “no unmotivated 3D” [108].

Many of our observations from the survey are consistent with the benefits of
presenting data in 3D by Willett et al. [99]. Intuitively, 3D representations
that mimic real-world objects provide better immersiveness for the user. This
allows users to identify the impact of the data onto the world, such as
estimating the impact of flood by different precipitation levels [63].
However, 3D visualization is considered a risky choice for representing data
because of its propensity for occlusion [46], legibility of text, and utility
on a 2D monitor screen, etc. Recently, Satkowski and Dachselt [123] found that
reading 2D charts in an AR environment is still effective, although slightly
less so than when using 2D dashboards. That said, data that is 3D in nature
can often benefit from a 3D representation [108].

Furthermore, a full 3D environment provides benefits not available in a 2D
workspace. For one thing, using AR as a new platform for visualization
provides freedom to move and arrange data at different angles, layouts, and
locations than on a 2D screen [99]. Furthermore, the immersive 3D space can
also facilitate a collaborative environment [88, 51] if the same chart can be
seen by multiple users. Stereographic effects may also be beneficial for
certain 3D views. Finally, switching from a 2D display to an augmented 3D
world opens up new possibilities in presenting 3D data embedded in the place
where it was collected.

###  5.6 Evolution of AR Development Platforms

The development ecosystems used for the situated analytics systems we have
surveyed in this paper have evolved radically over the last 25 years. Between
1997 and 2015, the platforms were mostly bespoke and custom-made for the
specialized AR equipment used (e.g., [52, 68, 67]).

However, since 2016, game engines such as Unity and Unreal Engine—that have
versatile mechanisms for usage with contemporary Virtual and AR/MR equipment
and are increasingly affordable—have become an important enabler for the
development of contemporary situated analytics systems (e.g., for IATK [35],
DXR [131], and ImAxes [36]). Game engines have strong synergies with mobile
platform development ecosystems (ARCore and ARKit), and therefore could be
considered a relatively unified development platform. In addition, depending
on the deployment hardware, they also support a variety of registration
mechanisms—a feature that often required bespoke, often hybrid,
implementations in the past.

However, commercial game engines have the drawback of being proprietary
software ecosystems controlled by a single vendor. While situated analytics
research has yet to make an en masse shift to open-standards web-based
technologies [118], such as that pioneered by Butcher et al. [25] (or by Badam
et al. [7] for ubiquitous analytics), we believe that this will eventually be
necessary. Unity supporters will note its performance and convenience; we note
that these are mostly engineering concerns. As a case in point, this
transition from desktop and native apps to web-based ones has already mostly
happened for regular data visualization [20].

###  5.7 Gaps and Opportunities

One of the key benefits of conducting a survey such as ours is the opportunity
to use the classification to not just identify current research trends, but
also to identify gaps where no research has been conducted. Such gaps may
represent opportunities for future research that will significantly enrichen
and advance the research area.

Based on the discussion above, we can see several opportunities for future
growth in this area:

  * •

Sensemaking support. One important observation from our survey concerns the
level of sensemaking support offered by most SA systems, with few offering
capabilities for schematizing and none in our survey enabling reporting. It is
clear that as SA technology matures, more systems will also include the
Schematize and Report levels of sensemaking support.

  * •

Increase analytical capabilities. We think there is significant potential for
creating more advanced analytical functionality in future situated analytics
tools. Human-centered Artificial Intelligence may help us create “supertools”
[130]; what about a situated Tableau?

  * •

Richer interaction. Most current situated analytics systems support only
limited interaction. We think that future situated analytics systems should
better harness the mobile setting and the touch, gestural, and voice-based
interaction modalities available there.

  * •

Richer data depictions. Similarly, there is significant potential for drawing
on existing research from AR/MR to create compelling visual representations,
potentially even embedding them into the real-world so that they are virtually
indistinguishable from real objects.

  * •

Interoperability. All of the situated analytics systems surveyed are designed
for exclusive and focused use. But what if you want to use more than one tool
at the same time? Future research in this space should focus on building a
unified platform where multiple SA systems can co-exist and interoperate.

  * •

Context matters. Few of the situated analytics systems we surveyed use event-
based context as a situating trigger—the majority use the setting, object, or
physical location to instantiate a situated visualization. To be clear,
detecting a specific context is a challenging prospect, but as situated and
contextual computing proliferates, we think that this will eventually become a
key utility for SA: data views that appear implicitly and on-demand without
the user having to request them.

###  5.8 Limitations

Our classification of situated analytics has several potential limitations.
For one thing, our four design dimensions are clearly not exhaustive, and nor
are their values. For example, our taxonomy assumes that all SA tools are
initiated by a trigger, often by a situation. However, there are tools that
utilize projection maps or physicalizations that present data near the point
of interest [150]; in other words, these are persistently on and not
triggered. We acknowledge that this is a possible scenario, and that our
design space does not distinguish these systems.

We also do not claim that our survey corpus is exhaustive; in particular,
there is a wealth of situated visualization in the AR domain, some of which
potentially could qualify for inclusion in our taxonomy. Our overall goal with
this work was to survey the field of situated analytics from a data
visualization viewpoint. For this reason, we opted to be representative rather
than exhaustive in surveying older work from AR prior to 2015.

##  6 Conclusion

Situated analytics in AR is increasingly within reach for a growing share of
households as technology improves and becomes more inexpensive. However, the
design space of analytics tools that utilize AR to support such situated
analytical tasks is still largely unexplored. In this paper, we have
elaborated on a rich design space, devised based on a classification of
situated analytics systems that involves their situating triggers, view
situatedness, data depiction, and data abstraction. Using this classification,
we have identified four archetypes of situated analytics systems. We also
discuss several design lessons that we derived from our survey of the
literature.

## Acknowledgments

We thank the anonymous reviewers for their feedback on this paper. We also
thank Sanghyun Hong and Hyeon Jeon for their comments on our work. This work
was partially supported by the U.S. National Science Foundation grant
IIS-1908605, and the DSP Centre, which has been partly funded by the European
Regional Development Fund through Welsh Government and also by the North Wales
Growth Deal through Ambition North Wales, Welsh Government and UK Government.
Any opinions, findings, and conclusions or recommendations expressed here are
those of the authors and do not necessarily reflect the views of the funding
agencies.

## References

  * [1] R. P. Abao, C. V. Malabanan, and A. P. Galido.  Design and development of FoodGo: A mobile application using situated analytics to augment product information.  Procedia Computer Science, 135:186–193, 2018. doi: 10 . 1016/j . procs . 2018 . 08 . 165
  * [2] J. Ahn, J. Williamson, M. Gartrell, R. Han, Q. Lv, and S. Mishra.  Supporting healthy grocery shopping via mobile augmented reality.  ACM Trans. Multimedia Computing, Communicatio, and Applications, 12(1s), 2015. doi: 10 . 1145/2808207
  * [3] H. Alqahtani and M. Kavakli.  IMAP-CampUS: Developing an intelligent mobile augmented reality program on campus as a ubiquitous system.  In Proceedings of the International Conference on Computer and Automation Engineering, pp. 1–5. ACM, New York, NY, USA, 2017. doi: 10 . 1145/3057039 . 3057062
  * [4] R. T. Azuma.  A survey of augmented reality.  Presence: Teleoperators and Virtual Environments, 6(4):355–385, 1997. doi: 10 . 1162/pres . 1997 . 6 . 4 . 355
  * [5] B. Bach, R. Dachselt, S. Carpendale, T. Dwyer, C. Collins, and B. Lee.  Immersive analytics: Exploring future interaction and visualization technologies for data analytics.  In Proceedings of the ACM Conference on Interactive Tabletops and Surfaces, pp. 529–533. ACM, New York, NY, USA, 2016. doi: 10 . 1145/2992154 . 2996365
  * [6] B. Bach, R. Sicat, J. Beyer, M. Cordeil, and H. Pfister.  The hologram in my hand: How effective is interactive exploration of 3D visualizations in immersive tangible augmented reality?  IEEE Transactions on Visualization and Computer Graphics, 24(1):457–467, 2018. doi: 10 . 1109/TVCG . 2017 . 2745941
  * [7] S. K. Badam, A. Mathisen, R. Rädle, C. N. Klokmose, and N. Elmqvist.  Vistrates: A component model for ubiquitous analytics.  IEEE Transactions on Visualization and Computer Graphics, 25(1), Jan. 2019.  (To appear). doi: 10 . 1109/TVCG . 2018 . 2865144
  * [8] M. Balduini, I. Celino, D. Dell’Aglio, E. Della Valle, Y. Huang, T. Lee, S.-H. Kim, and V. Tresp.  BOTTARI: an augmented reality mobile application to deliver personalized and location-based recommendations by continuous analysis of social media streams.  Journal of Web Semantics, 16:33–41, 2012. doi: 10 . 1016/j . websem . 2012 . 06 . 004
  * [9] R. Bane and T. Hollerer.  Interactive tools for virtual X-ray vision in mobile augmented reality.  In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 231–239, 2004. doi: 10 . 1109/ISMAR . 2004 . 36
  * [10] E. Barba, B. MacIntyre, and E. D. Mynatt.  Here We Are! Where Are We? Locating Mixed Reality in The Age of the Smartphone.  Proceedings of the IEEE, 100(4):929 –936, April 2012. doi: 10 . 1109/JPROC . 2011 . 2182070
  * [11] A. Batch, A. Cunningham, M. Cordeil, N. Elmqvist, T. Dwyer, B. H. Thomas, and K. Marriott.  There is no spoon: Evaluating performance, space use, and presence with expert domain users in immersive analytics.  IEEE Transactions on Visualization and Computer Graphics, 26(1):536–546, 2020. doi: 10 . 1109/TVCG . 2019 . 2934803
  * [12] A. Behzadan and V. Kamat.  Visualization of construction graphics in outdoor augmented reality.  In Proceedings of the Winter Simulation Conference. IEEE Computer Society, Los Alamitos, CA, USA, 2005. doi: 10 . 1109/WSC . 2005 . 1574469
  * [13] A. H. Behzadan, S. Dong, and V. R. Kamat.  Augmented reality visualization: A review of civil infrastructure system applications.  Advanced Engineering Informatics, 29(2):252–267, 2015. doi: 10 . 1016/j . aei . 2015 . 03 . 005
  * [14] L. Besançon, A. Ynnerman, D. F. Keefe, L. Yu, and T. Isenberg.  The state of the art of spatial interfaces for 3D visualization.  Computer Graphics Forum, 40(1):293–326, 2021. doi: 10 . 1111/cgf . 14189
  * [15] C. Bichlmeier, S. M. Heining, M. Feuerstein, and N. Navab.  The virtual mirror: A new interaction paradigm for augmented reality environments.  IEEE Transactions on Medical Imaging, 28(9):1498–1510, 2009. doi: 10 . 1109/TMI . 2009 . 2018622
  * [16] C. Bichlmeier, S. Holdstock, S. M. Heining, S. Weidert, E. Euler, O. Kutter, and N. Navab.  Contextual in-situ visualization for port placement in keyhole surgery: Evaluation of three target applications by two surgeons and eighteen medical trainees.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 173–174. IEEE, Piscataway, NJ, USA, 2009. doi: 10 . 1109/ISMAR . 2009 . 5336474
  * [17] C. Bichlmeier, F. Wimmer, S. M. Heining, and N. Navab.  Contextual anatomic mimesis hybrid in-situ visualization method for improving multi-sensory depth perception in medical augmented reality.  In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 129–138, 2007. doi: 10 . 1109/ISMAR . 2007 . 4538837
  * [18] M. Billinghurst, A. Clark, and G. Lee.  A survey of augmented reality.  Foundations and Trends in Human–Computer Interaction, 8(2-3):73–272, 2015. doi: 10 . 1561/1100000049
  * [19] S. Bonada, R. Veras, and C. Collins.  Personalized views for immersive analytics.  In Proceedings of the ACM Companion on Interactive Surfaces and Spaces, pp. 83–89. ACM, New York, NY, USA, 2016. doi: 10 . 1145/3009939 . 3009953
  * [20] M. Bostock, V. Ogievetsky, and J. Heer.  D3: Data-driven documents.  IEEE Transactions on Visualization and Computer Graphics, 17(12):2301–2309, 2011. doi: 10 . 1109/TVCG . 2011 . 185
  * [21] N. Bressa, H. Korsgaard, A. Tabard, S. Houben, and J. Vermeulen.  What’s the situation with situated visualization? A survey and perspectives on situatedness.  IEEE Transactions on Visualization and Computer Graphics, 28(1):107–117, 2022. doi: 10 . 1109/TVCG . 2021 . 3114835
  * [22] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei.  Language models are few-shot learners.  In Advances in Neural Information Processing Systems, pp. 1877–1901. Curran Associates, Inc., 2020.
  * [23] W. Büschel, P. Reipschläger, R. Langner, and R. Dachselt.  Investigating the use of spatial interaction for 3D data visualization on mobile devices.  In Proceedings of the ACM Conference on Interactive Surfaces and Spaces, pp. 62–71. ACM, New York, NY, USA, 2017.
  * [24] W. Büschel, S. Vogt, and R. Dachselt.  Augmented reality graph visualizations.  IEEE Computer Graphics & Applications, 39(3):29–40, 2019.
  * [25] P. W. S. Butcher, N. W. John, and P. D. Ritsos.  VRIA: A web-based framework for creating immersive analytics experiences.  IEEE Transactions on Visualization and Computer Graphics, 27(7):3213–3225, 2021. doi: 10 . 1109/TVCG . 2020 . 2965109
  * [26] G. Caggianese, V. Colonnese, and L. Gallo.  Situated visualization in augmented reality: Exploring information seeking strategies.  In Proceedings of the International Conference on Signal-Image Technology Internet-Based Systems, pp. 390–395. IEEE, Piscataway, NJ, USA, 2019. doi: 10 . 1109/SITIS . 2019 . 00069
  * [27] T. Caliński and J. Harabasz.  A dendrite method for cluster analysis.  Communications in Statistics, 3(1):1–27, 1974. doi: 10 . 1080/03610927408827101
  * [28] J. Carmigniani, B. Furht, M. Anisetti, P. Ceravolo, E. Damiani, and M. Ivkovic.  Augmented reality technologies, systems and applications.  Multimedia Tools and Applications, 51(1):341–377, 2011.
  * [29] M. Cavallo, G. A. Rhodes, and A. G. Forbes.  Riverwalk: Incorporating historical photographs in public outdoor augmented reality experiences.  In Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 160–165. IEEE, Piscataway, NJ, USA, 2016. doi: 10 . 1109/ISMAR-Adjunct . 2016 . 0068
  * [30] T. Chandler, M. Cordeil, T. Czauderna, T. Dwyer, J. Glowacki, C. Goncu, M. Klapperstueck, K. Klein, F. Schreiber, and E. Wilson.  Immersive analytics.  In Proceedings of the International Symposium on Big Data Visual Analytics, pp. 1–8. IEEE, Piscataway, NJ, USA, 2015. doi: 10 . 1109/BDVA . 2015 . 7314296
  * [31] D. Chatzopoulos, C. Bermejo, Z. Huang, and P. Hui.  Mobile augmented reality survey: From where we are to where we go.  IEEE Access, 5:6917–6950, 2017. doi: 10 . 1109/ACCESS . 2017 . 2698164
  * [32] Z. Chen, Y. Su, Y. Wang, Q. Wang, H. Qu, and Y. Wu.  MARVisT: authoring glyph-based visualization in mobile augmented reality.  IEEE Transactions on Visualization and Computer Graphics, 26(8):2645–2658, 2019. doi: 10 . 1109/TVCG . 2019 . 2892415
  * [33] Z. Chen, W. Tong, Q. Wang, B. Bach, and H. Qu.  Augmenting static visualizations with PapARVis designer.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 1–12. ACM, New York, NY, USA, 2020. doi: 10 . 1145/3313831 . 3376436
  * [34] Z. Chen, Y. Wang, T. Sun, X. Gao, W. Chen, Z. Pan, H. Qu, and Y. Wu.  Exploring the design space of immersive urban analytics.  Visual Informatics, 1(2):132–142, 2017. doi: 10 . 1016/j . visinf . 2017 . 11 . 002
  * [35] M. Cordeil, A. Cunningham, B. Bach, C. Hurter, B. H. Thomas, K. Marriott, and T. Dwyer.  IATK: An immersive analytics toolkit.  In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces, pp. 200–209. IEEE, Piscataway, NJ, USA, 2019. doi: 10 . 1109/VR . 2019 . 8797978
  * [36] M. Cordeil, A. Cunningham, T. Dwyer, B. H. Thomas, and K. Marriott.  ImAxes: Immersive axes as embodied affordances for interactive multivariate data visualisation.  In Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 71–83. ACM, New York, NY, USA, 2017.
  * [37] M. Cordeil, T. Dwyer, K. Klein, B. Laha, K. Marriott, and B. H. Thomas.  Immersive collaborative analysis of network connectivity: CAVE-style or head-mounted display?  IEEE Transactions on Visualization and Computer Graphics, 23(1):441–450, 2017. doi: 10 . 1109/TVCG . 2016 . 2599107
  * [38] D. Datcu, S. G. Lukosch, and H. K. Lukosch.  Handheld augmented reality for distributed collaborative crime scene investigation.  In Proceedings of the ACM Conference on Supporting Group Work, p. 267–276. ACM, New York, NY, USA, 2016. doi: 10 . 1145/2957276 . 2957302
  * [39] D. L. Davies and D. W. Bouldin.  A cluster separation measure.  IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-1(2):224–227, 1979. doi: 10 . 1109/TPAMI . 1979 . 4766909
  * [40] B. J. Dixon, M. J. Daly, H. Chan, A. D. Vescan, I. J. Witterick, and J. C. Irish.  Surgeons blinded by enhanced navigation: the effect of augmented reality on attention.  Surgical Endoscopy, 27(2):454–461, 2013. doi: 10 . 1007/s00464-012-2457-3
  * [41] F. Doil, W. Schreiber, T. Alt, and C. Patron.  Augmented reality for manufacturing planning.  In Proceedings of the Workshop on Virtual Environments, pp. 71–76. ACM, New York, NY, USA, 2003. doi: 10 . 1145/769953 . 769962
  * [42] A. Doshi, R. T. Smith, B. H. Thomas, and C. Bouras.  Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing.  The International Journal of Advanced Manufacturing Technology, 89:1279–1293, 2017.
  * [43] A. Dünser, M. Billinghurst, J. Wen, V. Lehtinen, and A. Nurminen.  Exploring the use of handheld AR for outdoor navigation.  Computers & Graphics, 36(8):1084–1095, 2012. doi: 10 . 1016/j . cag . 2012 . 10 . 001
  * [44] N. Elmqvist.  Anywhere & everywhere: A mobile, immersive, and ubiquitous vision for data analytics.  Communications of the ACM, 2023.  To appear.
  * [45] N. Elmqvist and P. Irani.  Ubiquitous analytics: Interacting with big data anywhere, anytime.  IEEE Computer, 46(4):86–89, 2013. doi: 10 . 1109/mc . 2013 . 147
  * [46] N. Elmqvist and P. Tsigas.  A taxonomy of 3D occlusion management for visualization.  IEEE Transactions on Visualization and Computer Graphics, 14(5):1095–1109, 2008. doi: 10 . 1109/TVCG . 2008 . 59
  * [47] N. A. M. ElSayed, R. T. Smith, and B. H. Thomas.  HORUS EYE: See the invisible bird and snake vision for augmented reality information visualization.  In Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 203–208. IEEE, Piscataway, NJ, USA, 2016. doi: 10 . 1109/ISMAR-Adjunct . 2016 . 0077
  * [48] N. A. M. ElSayed, B. H. Thomas, K. Marriott, J. Piantadosi, and R. T. Smith.  Situated analytics: Demonstrating immersive analytical tools with augmented reality.  Journal of Visual Languages & Computing, 36:13–23, 2016. doi: 10 . 1016/j . jvlc . 2016 . 07 . 006
  * [49] U. Engelke, C. Rogers, J. Klump, and I. Lau.  HypAR: Situated mineralogy exploration in augmented reality.  In Proceedings of the International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, New York, NY, USA, 2019. doi: 10 . 1145/3359997 . 3365715
  * [50] B. Ens, B. Bach, M. Cordeil, U. Engelke, M. Serrano, W. Willett, A. Prouzeau, C. Anthes, W. Büschel, C. Dunne, T. Dwyer, J. Grubert, J. H. Haga, N. Kirshenbaum, D. Kobayashi, T. Lin, M. Olaosebikan, F. Pointecker, D. Saffo, N. Saquib, D. Schmalstieg, D. A. Szafir, M. Whitlock, and Y. Yang.  Grand challenges in immersive analytics.  In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3446866
  * [51] B. Ens, S. Goodwin, A. Prouzeau, F. Anderson, F. Y. Wang, S. Gratzl, Z. Lucarelli, B. Moyle, J. Smiley, and T. Dwyer.  Uplift: A tangible and immersive tabletop system for casual collaborative visual analytics.  IEEE Transactions on Visualization and Computer Graphics, 27(2):1193–1203, 2021. doi: 10 . 1109/TVCG . 2020 . 3030334
  * [52] S. Feiner, B. MacIntyre, T. Höllerer, and A. Webster.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment.  Personal Technologies, 1(4):208–217, 1997.
  * [53] J. A. W. Filho, W. Stuerzlinger, and L. Nedel.  Evaluating an immersive space-time cube geovisualization for intuitive trajectory data exploration.  IEEE Transactions on Visualization and Computer Graphics, 26(1):514–524, 2020. doi: 10 . 1109/TVCG . 2019 . 2934415
  * [54] B. Fjukstad, J. M. Bjørndalen, and O. Anshus.  Uncertainty estimation and visualization of wind in weather forecasts.  In Proceedings of the International Conference on Information Visualization Theory and Applications, pp. 321–328, 2014.
  * [55] A. Fonnet and Y. Prié.  Survey of immersive analytics.  IEEE Transactions on Visualization and Computer Graphics, 27(3):2101–2122, March 2021. doi: 10 . 1109/TVCG . 2019 . 2929033
  * [56] A. L. Fred and A. K. Jain.  Combining multiple clusterings using evidence accumulation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6):835–850, 2005. doi: 10 . 1109/TPAMI . 2005 . 113
  * [57] B. Fröhler, C. Anthes, F. Pointecker, J. Friedl, D. Schwajda, A. Riegler, S. Tripathi, C. Holzmann, M. Brunner, H. Jodlbauer, H.-C. Jetter, and C. Heinzl.  A survey on cross-virtuality analytics.  Computer Graphics Forum, 2022. doi: 10 . 1111/cgf . 14447
  * [58] A. Gillet, M. Sanner, D. Stoffler, D. Goodsell, and A. Olson.  Augmented reality with tangible auto-fabricated models for molecular biology applications.  In Proceedings of the IEEE Visualization Conference, pp. 235–241. IEEE Computer Society, Los Alamitos, CA, USA, 2004. doi: 10 . 1109/VISUAL . 2004 . 7
  * [59] A. Gómez and P. Figueroa.  Size estimation in product visualization using augmented reality.  In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, p. 267–268. ACM, New York, NY, USA, 2008. doi: 10 . 1145/1450579 . 1450646
  * [60] R. Grasset, T. Langlotz, D. Kalkofen, M. Tatzgern, and D. Schmalstieg.  Image-driven view management for augmented reality browsers.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 177–186. IEEE, Piscataway, NJ, USA, 2012. doi: 10 . 1109/ISMAR . 2012 . 6402555
  * [61] S. Guven and S. Feiner.  Interaction techniques for exploring historic sites through situated media.  In Proceedings of the IEEE Symposium on 3D User Interfaces, pp. 111–118. IEEE Computer Society, Los Alamitos, CA, USA, 2006. doi: 10 . 1109/VR . 2006 . 87
  * [62] S. Guven and S. Feiner.  Visualizing and navigating complex situated hypermedia in augmented and virtual reality.  In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 155–158. IEEE Computer Society, Los Alamitos, CA, USA, 2006. doi: 10 . 1109/ISMAR . 2006 . 297807
  * [63] P. S. Haynes and E. Lange.  In-situ flood visualisation using mobile AR.  In Proceedings of the IEEE Symposium on 3D User Interfaces, pp. 243–244, 2016. doi: 10 . 1109/3DUI . 2016 . 7460061
  * [64] F. Heller, A. Krämer, and J. Borchers.  Simplifying orientation measurement for mobile audio augmented reality applications.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 615–624. ACM, New York, NY, USA, 2014. doi: 10 . 1145/2556288 . 2557021
  * [65] S. Henderson and S. Feiner.  Opportunistic tangible user interfaces for augmented reality.  IEEE Transactions on Visualization and Computer Graphics, 16(1):4–16, 2010. doi: 10 . 1109/TVCG . 2009 . 91
  * [66] S. Henderson and S. Feiner.  Exploring the benefits of augmented reality documentation for maintenance and repair.  IEEE Transactions on Visualization and Computer Graphics, 17(10):1355–1368, 2011. doi: 10 . 1109/TVCG . 2010 . 245
  * [67] T. Höllerer, S. Feiner, and J. Pavlik.  Situated documentaries: embedding multimedia presentations in the real world.  In Proceedings of the International Symposium on Wearable Computers, pp. 79–86, 1999. doi: 10 . 1109/ISWC . 1999 . 806664
  * [68] T. Hollerer, S. Feiner, T. Terauchi, G. Rashid, and D. Hallaway.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system.  Computers & Graphics, 23(6):779–785, 1999. doi: 10 . 1016/S0097-8493(99)00103-X
  * [69] N. Hube and M. Müller.  The data in your hands: Exploring novel interaction techniques and data visualization approaches for immersive data analytics.  In Proceedings of the ACM AVI Workshop on Visual Interfaces for Big Data Environments in Industrial Applications, pp. 12–21. ACM, New York, NY, USA, 2018.
  * [70] C. Hurter, N. H. Riche, S. M. Drucker, M. Cordeil, R. Alligier, and R. Vuillemot.  FiberClay: Sculpting three dimensional trajectories to reveal structural insights.  IEEE Transactions on Visualization and Computer Graphics, 25(1):704–714, 2019. doi: 10 . 1109/TVCG . 2018 . 2865191
  * [71] IKEA Corporation.  IKEA Place.  https://www.ikea.com/gb/en/customer-service/mobile-apps/, 2017.
  * [72] H. Imottesjo and J.-H. Kain.  The Urban CoBuilder - a mobile augmented reality tool for crowd-sourced simulation of emergent urban development patterns: Requirements, prototyping and assessment.  Computers, Environment and Urban Systems, 71:120–130, 2018. doi: 10 . 1016/j . compenvurbsys . 2018 . 05 . 003
  * [73] D. Jain, L. Findlater, J. Gilkeson, B. Holland, R. Duraiswami, D. Zotkin, C. Vogler, and J. E. Froehlich.  Head-mounted display visualizations to support sound awareness for the deaf and hard of hearing.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 241–250. ACM, New York, NY, USA, 2015. doi: 10 . 1145/2702123 . 2702393
  * [74] S. Jiang, X. Huang, C. Xie, S. Sung, and R. Yalcinkaya.  Augmented scientific investigation: Support the exploration of invisible ”fine details” in science via augmented reality.  In Proceedings of the ACM Conference on Interaction Design and Children, p. 349–354. ACM, New York, NY, USA, 2020. doi: 10 . 1145/3392063 . 3394406
  * [75] J. Joo-Nagata, F. Martinez Abad, J. García-Bermejo Giner, and F. J. García-Peñalvo.  Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: Evaluation of an educational program in Chile.  Computers & Education, 111:1–17, 2017. doi: 10 . 1016/j . compedu . 2017 . 04 . 003
  * [76] S. Julier, Y. Baillot, D. Brown, and M. Lanzagorta.  Information filtering for mobile augmented reality.  IEEE Computer Graphics and Applications, 22(5):12–15, Sep. 2002. doi: 10 . 1109/MCG . 2002 . 1028721
  * [77] D. Kalkofen, E. Veas, S. Zollmann, M. Steinberger, and D. Schmalstieg.  Adaptive ghosted views for augmented reality.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 1–9. IEEE, Piscataway, NJ, USA, 2013. doi: 10 . 1109/ISMAR . 2013 . 6671758
  * [78] S. Ketchell, W. Chinthammit, and U. Engelke.  Situated storytelling with SLAM enabled Augmented Reality.  In Proceedings of the International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, New York, NY, USA, 2019. doi: 10 . 1145/3359997 . 3365681
  * [79] D. H. Kim, V. Setlur, and M. Agrawala.  Towards understanding how readers integrate charts and captions: A case study with line charts.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 610:1–610:11. ACM, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3445443
  * [80] H. Kim, J. L. Gabbard, A. M. Anon, and T. Misu.  Driver behavior and performance with augmented reality pedestrian collision warning: An outdoor user study.  IEEE Transactions on Visualization and Computer Graphics, 24(4):1515–1524, 2018. doi: 10 . 1109/TVCG . 2018 . 2793680
  * [81] K. Kim, M. Billinghurst, G. Bruder, H. B.-L. Duh, and G. F. Welch.  Revisiting trends in augmented reality research: A review of the 2nd decade of ISMAR (2008–2017).  IEEE Transactions on Visualization and Computer Graphics, 24(11):2947–2962, 2018. doi: 10 . 1109/TVCG . 2018 . 2868591
  * [82] G. R. King, W. Piekarski, and B. H. Thomas.  ARVino — outdoor augmented reality visualisation of viticulture GIS data.  In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 52–55. IEEE Computer Society, Los Alamitos, CA, USA, 2005. doi: 10 . 1109/ISMAR . 2005 . 14
  * [83] D. Kobak and G. C. Linderman.  Initialization is critical for preserving global data structure in both t-SNE and UMAP.  Nature Biotechnology, 39(2):156–157, 2021. doi: 10 . 1038/nbt . 4314
  * [84] T. Kohonen.  The self-organizing map.  Proceedings of the IEEE, 78(9):1464–1480, 1990. doi: 10 . 1109/5 . 58325
  * [85] A. Kotranza, D. Scott Lind, C. M. Pugh, and B. Lok.  Real-time in-situ visual feedback of task performance in mixed environments for learning joint psychomotor-cognitive tasks.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 125–134. IEEE Computer Society, Piscataway, NJ, USA, 2009. doi: 10 . 1109/ISMAR . 2009 . 5336485
  * [86] M. Kraus, N. Weiler, D. Oelke, J. Kehrer, D. A. Keim, and J. Fuchs.  The impact of immersion on cluster identification tasks.  IEEE Transactions on Visualization and Computer Graphics, 26(1):525–535, 2020. doi: 10 . 1109/TVCG . 2019 . 293439
  * [87] O.-H. Kwon, C. Muelder, K. Lee, and K.-L. Ma.  A study of layout, rendering, and interaction methods for immersive graph visualization.  IEEE Transactions on Visualization and Computer Graphics, 22(7):1802–1815, 2016. doi: 10 . 1109/TVCG . 2016 . 2520921
  * [88] R. Langner, M. Satkowski, W. Büschel, and R. Dachselt.  MARVIS: Combining mobile devices and augmented reality for visual data analysis.  In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 5 2021. doi: 10 . 1145/3411764 . 3445593
  * [89] J. Lanier.  Virtually there.  Scientific American, 284(4):66–75, 2001.
  * [90] B. Lee, B. Bach, T. Dwyer, and K. Marriott.  Immersive analytics.  IEEE Computer Graphics & Applications, 39(3):16–18, 2019. doi: 10 . 1109/MCG . 2019 . 2906513
  * [91] G. A. Lee, A. Dünser, S. Kim, and M. Billinghurst.  CityViewAR: A mobile outdoor ar application for city visualization.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 57–64. IEEE, Piscataway, NJ, USA, 2012. doi: 10 . 1109/ISMAR-AMH . 2012 . 6483989
  * [92] S. Lee, J. Suh, and H.-D. Park.  BoreholeAR: a mobile tablet application for effective borehole database visualization using an augmented reality technology.  Computers and Geosciences, 76:41–49, 2015. doi: 10 . 1016/j . cageo . 2014 . 12 . 005
  * [93] G. Liestøl.  Situated simulations: A prototyped augmented reality genre for learning on the iphone.  International Journal of Interactive Mobile Technologies, 3:24–28, 2009. doi: 10 . 3991/ijim . v3i0 . 963
  * [94] G. Liestøl, A. Morrison, and T. Stenarson.  Visualization of climate change in situ.  In Proceedings of the International Conference on Virtual Systems Multimedia, pp. 251–256, 2014. doi: 10 . 1109/VSMM . 2014 . 7136657
  * [95] T. Lin, R. Singh, Y. Yang, C. Nobre, J. Beyer, M. A. Smith, and H. Pfister.  Towards an understanding of Situated AR visualization for basketball free-throw training.  In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3445649
  * [96] M. A. Livingston, L. J. Rosenblum, S. J. Julier, D. Brown, Y. Baillot, J. E. Swan II, J. L. Gabbard, D. Hix, et al.  An augmented reality system for military operations in urban terrain.  Technical report, Naval Research Laboratory, Advanced Information Technology Branch, Washington, DC, USA, 2002.
  * [97] M. Luboschik, P. Berger, and O. Staadt.  On spatial perception issues in augmented reality based immersive analytics.  In Proceedings of the ACM Companion on Interactive Surfaces and Spaces, pp. 47–53. ACM, New York, NY, USA, 2016. doi: 10 . 1145/3009939 . 3009947
  * [98] G. Luchetti, A. Mancini, M. Sturari, E. Frontoni, and P. Zingaretti.  Whistland: An augmented reality crowd-mapping system for civil protection and emergency management.  ISPRS International Journal of Geo-Information, 6(2), 2017. doi: 10 . 3390/ijgi6020041
  * [99] K. Marriott, J. Chen, M. Hlawatsch, T. Itoh, M. A. Nacenta, G. Reina, and W. Stuerzlinger.  Immersive Analytics: Time to Reconsider the Value of 3D for Information Visualisation, pp. 25–55.  Springer International Publishing, Cham, 2018. doi: 10 . 1007/978-3-030-01388-2_2
  * [100] K. Marriott, F. Schreiber, T. Dwyer, K. Klein, N. H. Riche, T. Itoh, W. Stuerzlinger, and B. H. Thomas, eds.  Immersive Analytics, vol. 11190 of Lecture Notes in Computer Science.  Springer International Publishing, Berlin, Germany, 2018. doi: 10 . 1007/978-3-030-01388-2
  * [101] N. S. Mathews, S. Chimalakonda, and S. Jain.  AiR: An augmented reality application for visualizing air pollution.  In Short Proceedings of the IEEE Visualization Conference, pp. 146–150, 2021. doi: 10 . 1109/VIS49827 . 2021 . 9623287
  * [102] L. K. Miles, L. K. Nind, and C. N. Macrae.  Moving through time.  Psychological Science, 21(2):222–223, 2010. doi: 10 . 1177/0956797609359333
  * [103] P. Milgram and F. Kishino.  A Taxonomy of Mixed Reality Visual Displays.  IEICE Transactions on Information Systems, E77-D(12):1321–1329, 1994.  pp. 1321–1329.
  * [104] P. Milgram, H. Takemura, A. Utsumi, and F. Kishino.  Augmented reality: A class of displays on the reality-virtuality continuum.  In Telemanipulator and Telepresence Technologies, vol. 2351, pp. 282–293. International Society for Optics and Photonics, Bellingham, WA, USA, 1995. doi: 10 . 1117/12 . 197321
  * [105] D. Moritz, C. Wang, G. L. Nelson, H. Lin, A. M. Smith, B. Howe, and J. Heer.  Formalizing visualization design knowledge as constraints: Actionable and extensible models in Draco.  IEEE Transactions on Visualization and Computer Graphics, 25(1):438–448, 2019. doi: 10 . 1109/TVCG . 2018 . 2865240
  * [106] A. Morrison, A. Mulloni, S. Lemmelä, A. Oulasvirta, G. Jacucci, P. Peltonen, D. Schmalstieg, and H. Regenbrecht.  Collaborative use of mobile augmented reality with paper maps.  Computers & Graphics, 35(4):789–799, 2011. doi: 10 . 1016/j . cag . 2011 . 04 . 009
  * [107] A. Mulloni, H. Seichter, and D. Schmalstieg.  Handheld augmented reality indoor navigation with activity-based instructions.  In Proceedings of the ACM Conference on Human-Computer Interaction with Mobile Devices and Services, p. 211–220. ACM, New York, NY, USA, 2011. doi: 10 . 1145/2037373 . 2037406
  * [108] T. Munzner.  Visualization Analysis and Design.  CRC Press, Boca Raton, FL, USA, 2014.
  * [109] A. Nassani, G. Lee, M. Billinghurst, T. Langlotz, and R. W. Lindeman.  Using visual and spatial cues to represent social contacts in AR.  In Proceedings of the SIGGRAPH Asia Conference on Mobile Graphics & Interactive Applications. ACM, New York, NY, USA, 2017. doi: 10 . 1145/3132787 . 3139199
  * [110] A. Okur, S.-A. Ahmadi, A. Bigdelou, T. Wendler, and N. Navab.  MR in OR: First analysis of AR/VR visualization in 100 intra-operative Freehand SPECT acquisitions.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 211–218. IEEE, Piscataway, NJ, USA, 2011. doi: 10 . 1109/ISMAR . 2011 . 6092388
  * [111] S. A. Panëels and J. C. Roberts.  Review of designs for haptic data visualization.  IEEE Transactions on Haptics, 3(2):119–137, 2010. doi: 10 . 1109/TOH . 2009 . 44
  * [112] J. H. Park, A. Kaufman, and K. Mueller.  Graphoto: Aesthetically pleasing charts for casual information visualization.  IEEE Computer Graphics & Applications, 38(6):67–82, 2018. doi: 10 . 1109/MCG . 2018 . 2879066
  * [113] B. Patnaik, A. Batch, and N. Elmqvist.  Information olfactation: Harnessing scent to convey data.  IEEE Transactions on Visualization and Computer Graphics, 25(1):726–736, 2019. doi: 10 . 1109/TVCG . 2018 . 2865237
  * [114] P. Pirolli and S. Card.  The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis.  In Proceedings of the International Conference on Intelligence Analysis, vol. 5, pp. 2–4, 2005.
  * [115] A. Prouzeau, Y. Wang, B. Ens, W. Willett, and T. Dwyer.  Corsican Twin: Authoring in situ augmented reality visualisations in virtual reality.  In Proceedings of the ACM Conference on Advanced Visual Interfaces. ACM, New York, NY, USA, 2020. doi: 10 . 1145/3399715 . 3399743
  * [116] I. Rabbi and S. Ullah.  A survey on augmented reality challenges and tracking.  Acta graphica: znanstveni časopis za tiskarstvo i grafičke komunikacije, 24(1-2):29–46, 2013.
  * [117] J. Rekimoto, Y. Ayatsuka, and K. Hayashi.  Augment-able reality: situated communication through physical and digital spaces.  In Proceedings of the International Symposium on Wearable Computers, pp. 68–75, 1998. doi: 10 . 1109/ISWC . 1998 . 729531
  * [118] J. Roberts, P. Ritsos, S. K. Badam, D. Brodbeck, J. Kennedy, and N. Elmqvist.  Visualization beyond the desktop—the next big thing.  IEEE Computer Graphics & Applications, 34(6):26–34, Nov 2014. doi: 10 . 1109/MCG . 2014 . 82
  * [119] G. G. Robertson, J. D. Mackinlay, and S. K. Card.  Cone trees: Animated 3D visualizations of hierarchical information.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 189–194. ACM, New York, NY, USA, 1991. doi: 10 . 1145/108844 . 108883
  * [120] M. E. Rollo, T. Bucher, S. P. Smith, and C. E. Collins.  ServAR: An augmented reality tool to guide the serving of food.  International Journal of Behavioral Nutrition and Physical Activity, 14(1), 2017. doi: 10 . 1186/s12966-017-0516-9
  * [121] H. Roodaki, N. Navab, A. Eslami, C. Stapleton, and N. Navab.  Sonifeye: Sonification of visual information using physical modeling sound synthesis.  IEEE Transactions on Visualization and Computer Graphics, 23(11):2366–2371, 2017. doi: 10 . 1109/TVCG . 2017 . 2734327
  * [122] P. J. Rousseeuw.  Silhouettes: A graphical aid to the interpretation and validation of cluster analysis.  Journal of Computational and Applied Mathematics, 20:53–65, 1987. doi: doi . org/10 . 1016/0377-0427(87)90125-7
  * [123] M. Satkowski and R. Dachselt.  Investigating the impact of real-world environments on the perception of 2D visualizations in augmented reality.  In Proceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3445330
  * [124] G. Schall, E. Mendez, E. Kruijff, E. Veas, S. Junghanns, B. Reitinger, and D. Schmalstieg.  Handheld augmented reality for underground infrastructure visualization.  Personal Ubiquitous Computing, 13(4):281–291, 2009. doi: 10 . 1007/s00779-008-0204-5
  * [125] D. Schmalstieg and T. Höllerer.  Augmented Reality: Principles and Practice.  Addison-Wesley, Boston, MA, USA, 2016.
  * [126] R. Schoenfelder and D. Schmalstieg.  Augmented reality for industrial building acceptance.  In Proceedings of the IEEE Virtual Reality Conference, pp. 83–90. IEEE, Piscataway, NJ, USA, 2008. doi: 10 . 1109/VR . 2008 . 4480755
  * [127] H. Seichter.  Augmented reality and tangible interfaces in collaborative urban design.  In Proceedings of Computer-Aided Architectural Design Futures, pp. 3–16. Springer Netherlands, Dordrecht, 2007.
  * [128] L. Shapiro.  Embodied Cognition.  Routledge, New York, NY, USA, 2010.
  * [129] J. Shi and J. Malik.  Normalized cuts and image segmentation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888–905, 2000. doi: 10 . 1109/34 . 868688
  * [130] B. Shneiderman.  Human-Centered AI.  Oxford University Press, Oxford, United Kingdom, 2022.
  * [131] R. Sicat, J. Li, J. Choi, M. Cordeil, W.-K. Jeong, B. Bach, and H. Pfister.  DXR: A toolkit for building immersive data visualizations.  IEEE Transactions on Visualization and Computer Graphics, 25(1):715–725, 2019. doi: 10 . 1109/TVCG . 2018 . 2865152
  * [132] M. Simpson, J. Zhao, and A. Klippel.  Take a walk: Evaluating movement types for data visualization in immersive virtual reality.  In Proceedings of the IEEE VIS Workshop on Immersive Analytics. IEEE, Piscataway, NJ, USA, 2017.
  * [133] A. Skulmowski and G. D. Rey.  Embodied learning: introducing a taxonomy based on bodily engagement and task integration.  Cognitive Research: Principles and Implications, 3(1), 2018. doi: 10 . 1186/s41235-018-0092-9
  * [134] A. Srinivasan, N. Nyapathy, B. Lee, S. M. Drucker, and J. Stasko.  Collecting and characterizing natural language utterances for specifying data visualizations.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 464:1–464:10. ACM, New York, NY, USA, 2021. doi: 10 . 1145/3411764 . 3445400
  * [135] R. Stoakley, M. J. Conway, and R. Pausch.  Virtual reality on a WIM: interactive worlds in miniature.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 265–272. ACM, New York, NY, USA, 1995. doi: 10 . 1145/223904 . 223938
  * [136] L. A. Suchman.  Plans and Situated Actions: The Problem of Human-Machine Communication.  Cambridge University Press, Cambridge, UK, 1987.
  * [137] M. I. Tanveer, E. Lin, and M. E. Hoque.  Rhema: A real-time in-situ intelligent interface to help people with public speaking.  In Proceedings of the ACM Conference on Intelligent User Interfaces, p. 286–295. ACM, New York, NY, USA, 2015. doi: 10 . 1145/2678025 . 2701386
  * [138] M. Tatzgern, D. Kalkofen, R. Grasset, and D. Schmalstieg.  Hedgehog labeling: View management techniques for external labels in 3D space.  In Proceedings of the IEEE Virtual Reality, pp. 27–32. IEEE, Piscataway, NJ, USA, 2014. doi: 10 . 1109/VR . 2014 . 6802046
  * [139] B. H. Thomas, G. F. Welch, P. Dragicevic, N. Elmqvist, P. Irani, Y. Jansen, D. Schmalstieg, A. Tabard, N. A. M. ElSayed, R. T. Smith, and W. Willett.  Situated analytics.  In Immersive Analytics, vol. 11190 of Lecture Notes in Computer Science, pp. 185–220. Springer, Berlin, Germany, 2018. doi: 10 . 1007/978-3-030-01388-2_7
  * [140] D. Van Krevelen and R. Poelman.  A survey of augmented reality technologies, applications and limitations.  International Journal of Virtual Reality, 9(2):1–20, 2010. doi: 10 . 20870/IJVR . 2010 . 9 . 2 . 2767
  * [141] D. Vanoni, M. Seracini, and F. Kuester.  Artifact: Tablet-based augmented reality for interactive analysis of cultural artifacts.  In Proceedings of the IEEE International Symposium on Multimedia, pp. 44–49. IEEE, Piscataway, NJ, USA, 2012. doi: 10 . 1109/ISM . 2012 . 17
  * [142] E. Veas, R. Grasset, E. Kruijff, and D. Schmalstieg.  Extended overview techniques for outdoor augmented reality.  IEEE Transactions on Visualization and Computer Graphics, 18(4):565–572, 2012. doi: 10 . 1109/TVCG . 2012 . 44
  * [143] X. Wang, S. K. Ong, and A. Y. Nee.  A comprehensive survey of augmented reality assembly research.  Advances in Manufacturing, 4(1):1–22, 2016.
  * [144] S. White and S. Feiner.  SiteLens: situated visualization techniques for urban site visits.  In Proceedings of the ACM Conference on Human Factors in Computing Systems, pp. 1117–1120. ACM, New York, NY, USA, 2009. doi: 10 . 1145/1518701 . 1518871
  * [145] M. Whitlock, S. Smart, and D. A. Szafir.  Graphical perception for immersive analytics.  In Proceedings of the IEEE Conference on Virtual Reality. IEEE, Piscataway, NJ, USA, 2020. doi: 10 . 1109/VR46266 . 2020 . 00084
  * [146] M. Whitlock, D. A. Szafir, and K. Gruchalla.  HydrogenAR: interactive data-driven presentation of dispenser reliability.  In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, pp. 704–712. IEEE, Piscataway, NJ, USA, 2020. doi: 10 . 1109/ISMAR50242 . 2020 . 00101
  * [147] M. Whitlock, K. Wu, and D. Szafir.  Designing for mobile and immersive visual analytics in the field.  IEEE Transactions on Visualization and Computer Graphics, 26(1):503–513, 2020. doi: 10 . 1109/TVCG . 2019 . 2934282
  * [148] D. Wigdor and D. Wixon.  Brave NUI World: Designing Natural User Interfaces for Touch and Gesture.  Morgan Kaufmann, Burlington, MA, 2011.
  * [149] W. Willett, B. A. Aseniero, S. Carpendale, P. Dragicevic, Y. Jansen, L. Oehlberg, and P. Isenberg.  Perception! Immersion! Empowerment! Superpowers as Inspiration for Visualization.  IEEE Transactions on Visualization and Computer Graphics, 28(1):22–32, 2022. doi: 10 . 1109/TVCG . 2021 . 3114844
  * [150] W. Willett, Y. Jansen, and P. Dragicevic.  Embedded Data Representations.  IEEE Transactions on Visualization and Computer Graphics, 23(1):461–470, 2017. doi: 10 . 1109/TVCG . 2016 . 2598608
  * [151] M. Williams, K. Yao, and J. R. C. Nurse.  ToARist: An augmented reality tourism app created through user-centred design.  In Proceedings of the British Human Computer Interaction Conference. BCS, London, United Kingdom, 2017. doi: 10 . 14236/ewic/HCI2017 . 1
  * [152] S. Wold, K. Esbensen, and P. Geladi.  Principal component analysis.  Chemometrics and Intelligent Laboratory Systems, 2(1-3):37–52, 1987.
  * [153] M. Xi, M. Adcock, and J. Mcculloch.  An end-to-end augmented reality solution to support aquaculture farmers with data collection, storage, and analysis.  In Proceedings of the International Conference on Virtual-Reality Continuum and its Applications in Industry. ACM, New York, NY, USA, 2019. doi: 10 . 1145/3359997 . 3365721
  * [154] J. S. Yi, Y. ah Kang, and J. Stasko.  Toward a deeper understanding of the role of interaction in information visualization.  IEEE Transactions on Visualization and Computer Graphics, 13(6):1224–1231, 2007. doi: 10 . 1109/tvcg . 2007 . 70515
  * [155] M. Zhao, Y. Su, J. Zhao, S. Chen, and H. Qu.  Mobile situated analytics of ego-centric network data.  In Proceedings of the SIGGRAPH Asia Symposium on Visualization. ACM, New York, NY, USA, 2017. doi: 10 . 1145/3139295 . 3139309
  * [156] M. Zheng and A. G. Campbell.  Location-based augmented reality in-situ visualization applied for agricultural fieldwork navigation.  In Adjunct Proceedings of the International Symposium on Mixed and Augmented Reality, pp. 93–97, 2019. doi: 10 . 1109/ISMAR-Adjunct . 2019 . 00039
  * [157] F. Zhou, H. B.-L. Duh, and M. Billinghurst.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR.  In Proceedings of the IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 193–202. IEEE, Piscataway, NJ, USA, 2008. doi: 10 . 1109/ISMAR . 2008 . 4637362
  * [158] S. Zollmann, T. Langlotz, R. Grasset, W. H. Lo, S. Mori, and H. Regenbrecht.  Visualization techniques in augmented reality: A taxonomy, methods and patterns.  IEEE Transactions on Visualization and Computer Graphics, 27(9):3808–3825, 2021. doi: 10 . 1109/TVCG . 2020 . 2986247

|  Sungbok Shin received the Master’s degree in Computer Science and
Engineering in 2019 from Korea University in Seoul, South Korea. He is a Ph.D.
student in the Department of Computer Science at the University of Maryland,
College Park, Maryland, USA. He is also a student member of the Human-Computer
Interaction Laboratory (HCIL) at UMD.
---|---
|  Andrea Batch received the Ph.D. degree in information studies in 2022 from
the University of Maryland, College Park, MD, USA. She is an Economist at the
U.S. Bureau of Economic Analysis. She is also a member of the Human-Computer
Interaction Laboratory (HCIL) at UMD.
---|---
|  Peter W. S. Butcher received the Ph.D. degree in 2020 from the University
of Chester, UK. He is a Lecturer in the School of Computer Science and
Electronic Engineering, Bangor University, UK. His research interests focus on
immersive analytics with WebXR.
---|---
|  Panagiotis D. Ritsos received the Ph.D. degree in 2006 from the University
of Essex in Colchester, UK. He is a Senior Lecturer in the School of Computer
Science and Electronic Engineering, Bangor University, UK. His research
interests include mixed and virtual reality, information visualization, visual
analytics, and wearable computing. He is a member of the IEEE and the IEEE
Computer Society.
---|---
|  Niklas Elmqvist received the Ph.D. degree in 2006 from Chalmers University
of Technology in Göteborg, Sweden. He is a professor in the College of
Information Studies, University of Maryland, College Park in College Park, MD,
USA. He is also a member of the Institute for Advanced Computer Studies
(UMIACS) and formerly the director of the Human-Computer Interaction
Laboratory (HCIL) at UMD. He is a senior member of the IEEE and the IEEE
Computer Society.
---|---