1129 Research Evaluation Haley Adams is with the Department of Computer
Science Vanderbilt University. E-mail<EMAIL_ADDRESS>Holly
Gagnon is with the Department of Psychology at University of Utah. E-mail:
<EMAIL_ADDRESS>Sarah Creem-Regehr is with the Department of
Psychology at University of Utah. E-mail<EMAIL_ADDRESS>Jeanine
Stefanucci is with the Department of Psychology at University of Utah
Research. E-mail<EMAIL_ADDRESS>Bobby Bodenheimer is with
the Department of Computer Science Vanderbilt University. E-mail:
<EMAIL_ADDRESS>Adams et al.: Shape & Shadow in XR

# Stay in Touch! Shape and Shadow Influence Surface Contact in
XR Displays

Haley Adams  Holly Gagnon  Sarah Creem-Regehr  Jeanine Stefanucci  and Bobby
Bodenheimer  Member, IEEE

###### Abstract

The information provided to a person’s visual system by extended reality (XR)
displays is not a veridical match to the information provided by the real
world. Due in part to graphical limitations in XR head-mounted displays
(HMDs), which vary by device, our perception of space may be altered. However,
we do not yet know which properties of virtual objects rendered by
HMDs—particularly augmented reality displays—influence our ability to
understand space. In the current research, we evaluate how immersive graphics
affect spatial perception across three unique XR displays: virtual reality
(VR), video see-through augmented reality (VST AR), and optical see-through
augmented reality (OST AR). We manipulated the geometry of the presented
objects as well as the shading techniques for objects’ cast shadows. Shape and
shadow were selected for evaluation as they play an important role in
determining where an object is in space by providing points of contact between
an object and its environment—be it real or virtual. Our results suggest that
a non-photorealistic (NPR) shading technique, in this case for cast shadows,
may be used to improve depth perception by enhancing perceived surface contact
in XR. Further, the benefit of NPR graphics is more pronounced in AR than in
VR displays. One’s perception of ground contact is influenced by an object’s
shape, as well. However, the relationship between shape and surface contact
perception is more complicated.

###### keywords:

OST AR, VST AR, VR, perception, shadow, geometry, shape, complexity, depth,
surface contact, shading

I.3.7Computer GraphicsThree-Dimensional Graphics and RealismVirtual Reality;
J.4Computer ApplicationsSocial and Behavioral SciencesPsychology  Three
geometric shapes were evaluated in Experiment 1. Each object was rendered with
either dark (photorealistic) or light (non-photorealistic) shadow shading
methods and was displaced vertically based on the viewer’s visual angle to the
target. Changes in vertical displacement were subtle since a psychophysical,
two-alternative forced choice (2AFC) approach was employed to evaluate surface
contact perception.

Introduction

Rendering methods for extended reality (XR) systems are improving, and many
displays can now render compelling virtual objects. Some difficulties still
remain, however, and one that is common in consumer-level augmented reality
(AR) is that objects may seem detached from their surroundings. For example, a
Pokémon in Pokémon Go, the popular mobile AR game, may seem to float above the
ground where it is placed. A virtual hat in an AR face filter, like those used
by Snapchat or Instagram, may appear detached from a person’s head. This
phenomenon is a strong indicator that the depth information presented by
virtual stimuli and the information presented by the real world environment do
not sufficiently match. In the current research, we evaluate how the way we
render cast shadows for objects of different geometric shapes and orientations
affects one’s perception of surface contact across different extended reality
(XR) displays.

Recent work on optical see-through (OST) AR displays leverages the human
visual system to create shadows [40, 29]. These devices create the illusion of
dark color values using simultaneous contrast illusion, which approximates a
shadow. Other methods have been used for video see-through (VST) AR displays
[43], and shadow generation in virtual reality (VR) is considered mature [56,
17]. Nonetheless, for generally deployable XR applications, it would be
desirable to have a clear and unified understanding of which techniques work
across widely available technologies. Therefore, in the present work we
investigate how surface contact is affected by cast shadows across a set of
display types.

Additionally, Steptoe et al. [58] recently made a compelling case for non-
photorealistic shadow applications in AR. Some prior work suggests that non-
photorealistic shadow shading techniques in AR may be beneficial for attaching
virtual objects to the surfaces beneath them [2]. Thus, we also investigate
how dark, photorealistic and light, non-photorealistic cast shadows affect
surface contact judgments in XR. The possibility of non-photorealistic shadows
offering benefits for surface contact judgments may provide additional
flexibility for XR application designers given the difficulty in displaying
dark shadows that exists in many AR devices.

In the current work, we make several contributions by evaluating how people’s
judgments are affected by virtual object shape, orientation, and shadow
shading in three different XR displays. We demonstrate that non-
photorealistic, light shadow shading can enhance surface contact judgments. We
show that these judgments are complex and affected by an object’s shape as
well as the orientation in which it is presented. Our results also lay
groundwork to encourage further study of non-photorealistic rendering
techniques to improve spatial perception in XR.

Figure 1: A recreation of the “ball-in-a-box” demonstration [31]. Although the
sphere is located at the same position in all four images, it appears closer
in the bottom images due to the placement its cast shadow. The influence of
cast shadow on an object’s position works regardless of whether the shadow is
rendered realistic (left) or nonrealistic (right) manner. Top: the sphere
seems to rest along the ground due to an anchoring shadow beneath it that
connects the sphere with the back of the box. Bottom: the sphere seems to
float above the ground with a detached shadow beneath it that connects it
towards the front of the box.

##  1 Related Work

###  1.1 Cast Shadows

Cast shadows—from which humans infer size [65, 64, 10], shape [12, 32], and
the distance between an object and adjacent surfaces[62, 27, 36]—play a
particularly important role in our perception of space. In fact, one’s
perception of the position of an object can shift dramatically depending on
the placement of its associated cast shadow [38, 42]. Kersten et al.’s “ball-
in-a-box” study provides an important example of this effect [31]. Kersten and
colleagues demonstrated that by changing the location of a shadow in space,
even a stationary object may appear to move (See Figure 1). Furthermore, their
research demonstrated that cast shadow shading could be manipulated with
light, photometrically incorrect shading, but still provide a cue to determine
spatial location. For a recent and thorough review of shadow perception, see
Casati and Cavanagh [11].

###  1.2 Shape & Shadows in XR

In traditional computer graphics, evidence shows that cast shadows function as
“visual glue” to attach virtual objects to surfaces [62, 36]. However, it is
unclear how to best create visual glue for augmented reality devices. AR
introduces new complexities to the graphics pipeline, especially for OST
displays, which rely on additive light for rendering. These displays cannot
remove light, and thereby darken, virtual or real objects. Dark color values
in these displays become progressively more transparent such that a pure black
is completely invisible. As a result, depth from shading cues are less
reliable and the visual position of rendered objects in space becomes
ambiguous.

The inability to produce dark color values with fidelity in OST AR is
especially problematic for rendering cast shadows. Manabe et al. [39, 40]
developed methods to produce more perceptually valid shadow rendering
techniques for OST devices. Ikeda et al. [29] used simultaneous contrast to
create the illusion of dark color values within the umbra of a shadow by
applying brighter color values outside of the shadow’s penumbra. In their
evaluation, they found that their illusory shadow method caused the umbra of
the shadow to be perceived as dark–despite the fact that nothing was rendered
within the shadow’s umbra itself. However, research on shadow generation
algorithms gives rise to another question: how much fidelity is needed for
accurate spatial perception in AR? Adams et al. [2] attempted to answer this
question by evaluating surface contact judgements for both perceptually
motivated shadow rendering methods, including a simple, dark gray shadow as
well as a variant of the method developed by Manabe, Ikeda, and colleagues
[40], and a non-photorealistic rendering (NPR) method for shadows.
Surprisingly, they found that more realistic rendering techniques for shadows
did not inherently benefit surface contact judgements. Further, people’s
judgements for the NPR method, which produced a white shadow, were more
accurate in both OST AR and VST AR head-mounted displays.

Shadows may play an important role in alleviating inaccurate depth perception
in augmented reality. As such, several evaluations on the influence shadow
position and object shape on depth judgements have been conducted in both
mobile AR [6, 59] and in OST AR [48, 22, 14, 54, 25]. The presence of cast
shadows can dramatically improve the accuracy of people’s judgements in
perceptual matching tasks in OST AR displays [6, 14, 22, 25]. In perceptual
matching paradigms, a person is asked to align a virtual object with a real
world marker, or referent, that is positioned at a distance away from the
viewer. As a result, perceptual matching may be considered a relative, or
exocentric, measure of distance perception.

In a recent study, Gao et al. [22] revealed that the shape of a shadow did not
affect the accuracy of people’s relative depth judgements in a perceptual
matching task with floating, virtual targets. In the same study, Gao and
colleagues also provided evidence that lighting misalignment can adversely
affect people’s accuracy when making relative depth judgements. However, prior
research has found that “drop shadows,” in which shadows are rendered
immediately below an object, regardless of the lighting in the environment
around them, are an exception [6, 14]. Drop shadows improve people’s accuracy
in perceptual matching experiments, despite their unrealistic positioning.

Yet the results of these prior experiments, when taken in isolation, are
somewhat limited since all of the aforementioned studies that evaluated
shadows only rendered them with targets that floated above the ground [6, 14,
48, 22, 25]. This is important because anchoring shadows and detached shadows
are perceived differently [9, 11]. Anchoring shadows are formed when an
occluding object and its shadow are connected (See Figure 1 (top)). And
detached shadows are formed when an occluding object and its cast shadow do
not touch (See Figure 1 (bottom)). For example, the shape of a shadow when
detached needs to only approximately conform to its object’s shape [38, 11].
Accordingly, it makes sense that Gao et al.’s experiment found no differences
between differently shape shadows, since they only evaluated hard, soft and
’round’ (incorrect shape) shadow conditions when the test object was a
rotating cube.

In contrast, an anchoring shadow may be more sensitive to mismatches between
the contours of a shadow and its associated object [37, 11]. However, as noted
in Do et al. [15], AR depth perception experiments rarely incorporate multiple
object shapes. Accordingly, in their mobile AR study they evaluated how color
and luminance interacted to affect depth perception across four objects: a
cube, a sphere, the Stanford bunny, and a low polygon version of the Stanford
bunny. Although they predicted that simple shapes (i.e., the cube and sphere)
would be more affected by color and luminance depth cues than complex shapes,
their predictions were only partially supported. The sphere was more affected
by luminance than any other shape, but the cube was not susceptible to either
color or luminance. Their results are reasonable considering that spheres have
previously proven more challenging to perceive in depth than more distinctly
faceted shapes, like cubes or icosahedrons, in VR [49].

Figure 2: A VR HMD with VST capabilities: the HTC Vive Pro with mounted Zed
Mini camera

Figure 3: An OST HMD: the Microsoft HoloLens 1.

##  2 Overview

We evaluated a viewer’s sense of perceived surface contact over two studies,
both of which were conducted on three immersive head-mounted display (HMD)
types: a virtual reality (VR) display, a video see-through augmented reality
display (VST AR), and an optical see-through augmented reality (OST AR)
display. In both studies, shadows were manipulated to have either dark
(photorealistic) or light (non-photorealistic) color values. In our first
study, which is discussed in Section 4, we manipulated both shadow shading
technique and object geometry to understand how the interplay between these
two factors affected perceived surface contact in XR. In our second study
(Section 5), we compared a simple geometric shape to a complex one, and we
displayed them at different orientations. From the findings of our two
studies, we were able to extrapolate XR developer and design guidelines, which
are discussed in Section 6. The following section (Section 3) discusses: our
hardware setup (Subsection 3.1), the software environment used for rendering
to each display as well as the shaders used for rendering shadows (Subsection
3.3), and the equation used to displace test stimuli vertically based on
visual angle (Subsection 3.4).

##  3 Methods

###  3.1 Technical Setup

The virtual reality environment was rendered using a wireless HTC Vive Pro
(Figure 3), which has a maximum per eye resolution of $1440\times 1600$ and an
approximate field of view of $110^{\circ}\times 113^{\circ}$. Position
tracking for this condition was performed using the Vive’s lighthouse tracking
system.

The video see-through device also relied on the HTC Vive Pro for rendering.
However, it created an augmented reality environment by combining virtual
overlays with real time video feed, which was captured using the Zed Mini
stereoscopic camera system. The Zed Mini was affixed to the front of the Vive
Pro (Figure 3). The Zed Mini’s camera feed restricted the HMD’s resolution and
field of view to $1280\times 720$ and $90^{\circ}\times 60^{\circ}$,
respectively. Position tracking was performed by the Zed Mini, which
integrated its own native, inside-out tracking solution with the HTC Vive’s
tracking system.

The optical see-through environment was rendered using the Microsoft HoloLens
1 (Figure 3). The OST display is unique since it relies on additive light to
render images and is therefore unable to render dark color values with
fidelity. Instead, the darker a color value is, the more transparent it
becomes until it turns completely transparent at black. The HoloLens has an
approximate resolution of $1268\times 720$ and field of view of
$30^{\circ}\times 17^{\circ}$. Although the augmented field of view (FOV) of
the HoloLens is narrow, outside of this viewing area users’ vision is not
occluded by the device. For our experiment, position tracking was performed
using the HoloLens’ native inside-out tracking solution.

For the VST AR condition, virtual target objects were positioned within the
real world environment using a Vive tracking puck before the start of each
experiment. Similarly, for the OST AR system, this position calibration
involved placing a HoloLens spatial anchor at the same predetermined position.
This step was unnecessary for the VR condition since both the environment and
the stimuli were virtual. A wireless computer mouse was employed for user
inputs for all devices. Figure 5 shows a participant wearing the wireless Vive
Pro setup with the Zed Mini stereoscopic camera plugged in for the VST AR
condition. The participant looks toward the spot on a table in front of them
where the virtual target is rendered while holding a mouse.

###  3.2 Physical & Virtual Environment

In contrast to the two augmented reality display conditions, the virtual
reality condition required a virtual reconstruction of the real world
environment used for testing. Figures of both the physical and virtual testing
environments can be seen in Figure 5. The virtual environment was created
using photographed images of the real testing space along the walls and custom
3D models designed to match important furniture and foam floor tiles along the
ground. In both the real and virtual environments, a table, on which
experimental stimuli were displayed, was placed in the middle of the room. A
disposable, azure blue cloth was draped along the top of the table with white
tape wrapped around its edges. The tape was included to improve the table’s
salience for the two AR systems, which relied on inside-out tracking.

###  3.3 Software & Shadows

Applications for each device were developed in Unity version 2017.4.4f1 with
the C# programming language. A virtual, directional light was positioned
within each program so that a target object’s shadow would lie behind and to
the right of the object. To accomplish this, the orientation of the virtual
light was set to $141^{\circ}$ along the x axis and $-141^{\circ}$ along the y
axis. The color of the light had a slightly yellow tint with an RGB color
value of (255, 244, 214). Shaders to render hard shadows were programmed using
a variant of the HLSL language that is compatible with the Unity game engine.
The shaders used to render target objects and their cast shadows used RGB
color values to specify color output. All target objects were rendered with a
middle gray RGB color value of 128. However, the cast shadow shader was
developed to render shadows with custom color values. For the current study,
we used grayscale RGB color values of 208 and 48 for the light and dark shadow
shading methods, respectively. Visually, this created an off-white or lightly
shaded appearance for the non-photorealistic (or light) shadow. And it
rendered a dark gray colored shadow for the photorealistic (or dark) shadow.
Because target objects were rendered with a middle gray color value of 128,
there was a difference of 80 RGB color values for the dark (128-80=48) and the
light (128+80=208) values used to inform the cast shadow shaders. Across all
three devices, the same scripts were used for both object and cast shadow
shading. The equivalent values in CIE L*a*b* color space, which more
accurately represents differences in lightness in perceptual space, are:
$48=[19.9,0,0]$, $128=[53.6,0,0]$, and $208=[83.5,0,0]$.

The photorealistic, dark shadow condition was the most traditional method
given it rendered shadows that adhered closely to the principle that shadows
represent the absence of light. For most devices it created a perceptually
valid impression of a shadow. In contrast, the non-photorealistic, light
shadow condition added white light to generate a shadow instead of subtracting
light. This created a both perceptually and photometrically incorrect cast
shadow. Depictions of the shadow conditions can be seen in Figures Stay in
Touch! Shape and Shadow Influence Surface Contact in  XR Displays, 9, and 10.

###  3.4 Vertical Displacement

Participants in our experiments judged whether targets were in contact with a
surface; therefore, target stimuli had to be presented both on and above a
surface for discrimination. We displaced objects vertically based on a
viewer’s visual angle to target. Given that participants conducted the study
while seated, the eye height of the viewer was calculated by summing the
average eye height of a person while seated and the height of the seat of the
chairs used in our setup [24]. Using the average eye height of a viewer,
denoted as $h_{e}$, and the distance to a given target, $d_{t}$, we were able
to solve for a series of three triangles from which we could extract the
degree of vertical displacement, $d_{v}$, for target objects placed above
surfaces. The trigonometric formulas used for this calculation are shown in
the equations below:

$\sigma=\tan^{-1}\left(\frac{d_{t}}{h_{e}}\right)+\omega$ (1)

$d_{v}=\left(\frac{\tan(\sigma)\;{h_{e}}-d_{t}}{\tan(\sigma)}\right)$ (2)

For equations 1 and 2, $\omega$ represents the degree to which viewing angle
was modified–in our case in increments of $0.6^{\circ}$–and $\sigma$
represents the updated viewing angle to the vertically displaced target
object. The viewing angle of $0.6^{\circ}$ was selected during preliminary
testing, where the value resulted in a sufficiently challenging degree of
displacement for the experimental task. Figure 4 shows each variable in
context for clarity.

Figure 4: Visual depiction of trigonometric solution for vertical
displacement.

##  4 Experiment 1: Shape and Shadow

In our first experiment, we evaluated how traditional (dark) and non-
photorealistic (light) shading techniques for cast shadows affect one’s
ability to perceive surface contact for a cube, an icosahedron, or a sphere
(Figure Stay in Touch! Shape and Shadow Influence Surface Contact in  XR
Displays). These three objects were selected for their distinctive geometric
properties. In particular, the shape, shading, and shadows of the cube and
sphere are recognizable and different. The icosahedron represents an in-
between case, with its indistinct shape, but shading closer to that of the
sphere and shadow closer to that of the cube. Our hypotheses for this
experiment are as follows:

H1: We predicted that people’s ability to correctly perceive ground contact
would be affected by an object’s geometry, especially along the bottom edge
that would be in contact with a surface. Specifically, we anticipated that
people’s likelihood of correct response would be lower for the sphere across
all devices. A perfect sphere has only a single point of contact with a ground
surface and that singular point is occluded when viewed from above. In
contrast, both the icosahedron and the cube were flat along the edge and
benefited from many points of contact with the surface beneath them, although
the surface area of the bottom of the icosahedron was smaller than that of the
cube.

H2: We anticipated that people would be more likely to correctly perceive
surface contact when presented with the light shadow shading method, in
comparison to the traditional dark shading method in both AR devices.
Although, our prediction may seem counterintuitive, some prior research has
shown that non-photorealistic shadows may function as well as photorealistic
shadows as a depth cue for spatial perception [31, 2].

H3: We also anticipated that there would be interactions between shape and
shadow shading methods. However, _a priori_ we were uncertain how these
interactions would be expressed.

###  4.1 Participants

Six individuals in total (3M, 3F) aged 23–30 volunteered to participate in our
first experiment, which used a well-founded psychophysical paradigm.
Psychophysics is a class of psychological methods that quantitatively measures
perceptual responses to changes in physical stimuli [19]. These methods can
use a small number of participants to make a large number of simple,
behavioral responses that reveal underlying perceptual processes, gaining
experimental power from a large number of observations. Psychophysical
paradigms have proven highly replicable and robust since they employ judgments
or adjustments with low individual variance [18, 57]. Psychophysics methods
have previously been employed successfully by other XR research groups [8, 53,
21, 36].

All participants had normal or corrected-to-normal vision. Our experimental
methods were approved by the local institutional review board, written consent
was obtained from all subjects prior to participation, and each participant
was paid 20 USD for 2-3 hours of their time.

Figure 5: A participant views the experiment in the VST AR condition (top). An
image of the virtual environment used for the VR condition (bottom).

###  4.2 Design

To address our hypotheses, we utilized a 3 (XR display) $\times$ 3 (target
shape) $\times$ 2 (shadow shading) $\times$ 6 (target height) within-subjects
design to evaluate the effects of object shape and shadow shading on surface
contact judgments for objects at different vertical displacements. Although we
conducted our experiment with three different XR displays, we did not make any
direct statistical comparisons across displays due to the high variance in
optical and graphical properties across devices. However, we evaluated the
same experimental conditions for each XR display. As such, this section
describes the experimental design that was used for each XR device.

Each participant evaluated 3 object shapes with 2 different shadow shading
conditions, which resulted in 6 unique combinations of experimental stimuli.
The target shapes evaluated were a cube, an icosahedron, and a sphere. The
shadows were shaded with either dark or light color values. Figure Stay in
Touch! Shape and Shadow Influence Surface Contact in  XR Displays displays all
of the target shape and shadow shading conditions. All targets were rendered
on a table 1 meter in front of the viewer (Figure 5). The size of the targets
was adjusted to be 11.2 cm wide. Accordingly, the cube had a length, width,
and height of 11.2 cm, the sphere’s diameter was 11.2 cm, and the icosahedron
was scaled so that its width was approximately 11.2 cm.

We employed a temporal two-alternative forced choice paradigm with method of
constant stimuli—a classic psychophysical method [19, 7]—to evaluate people’s
perception of surface contact. Two-alternative forced choice, or 2AFC, is a
method for assessing someone’s sensitivity to a change in stimuli. It involves
a _forced choice_ , because participants must chose between one of two
options. A temporal 2AFC paradigm may also be referred to as a two-interval
forced choice (2IFC) paradigm. After viewing both stimuli in a sequence,
participants in our experiment are then asked ”Which object is closer to the
ground?”. Compared targets were always of the same shape and shadow condition
for each trial.

How the six levels of vertical displacement were presented was dictated by
_method of constant stimuli_. As such, at least one of the target stimuli was
always placed in contact with the surface beneath it. The other object was
then displaced vertically by one of six heights between 0 and 6mm at regular
intervals of $.06^{\circ}$ changes in viewing angle. When displacement was 0,
both the first and second targets were positioned in contact with the surface.

Changes in vertical displacement were subtle. All six height displacements are
displayed in Figure Stay in Touch! Shape and Shadow Influence Surface Contact
in  XR Displays, and the calculations for vertical displacement are described
in detail in Section 3.4. Each height comparison was presented 20 times each,
except for when both the first and second object heights were equal (at 0). In
this case, the height conditions were only presented 10 times each. Stimulus
pairs were presented pseudo-randomly such that there were no repetitions of
shape $\times$ shadow $\times$ height combinations before all combinations
were presented once. In addition, the order of presented stimuli was balanced
so that the number of trials in which the target was presented in contact with
the ground first and the number of times it was presented second were
equivalent for each condition.

Trials for the task were blocked by device and order of device was
counterbalanced across subjects. For each XR device, a participant completed
660 2AFC trials. Each participant thus completed 1,980 trials total, and
11,880 data points were collected across all subjects. We collected data over
a large number of trials, which is common practice in psychophysical
paradigms, to increase accuracy and ensure low variance in our study [50]. By
evaluating a large number of simple behavioral responses on a small number of
participants, this family of paradigms is better able to evaluate perceptual
behaviors that have little variance between individuals [18, 57]. We further
confirmed that variance in our collected data was not due to between-
participant variation by calculating the intra-class correlation coefficient
(ICC), the calculation of which is reported in Section 4.4 (Results).

###  4.3 Procedure

Before the experiment began, each individual was informed about the study and
filled out a consent form. They were told that they may stop the experiment at
any time and that they were allowed to take breaks during the experiment if
needed. Both the experimenter and the volunteer wore face masks while
maintaining 2 m of separation between each other during the study.

The experiment was blocked by device. As such, the participant picked up one
of the three XR HMDs, which was predesignated based on a device ordering that
was counterbalanced across participants, to begin the experiment. They were
then verbally instructed on how to interact with the system and they performed
10-20 practice trials that were randomly selected trials from the experiment.
After the volunteer expressed that they were comfortable with the system and
experimental paradigm, the experiment began.

For the temporal 2AFC paradigm with method of constant stimuli, each stimulus
was presented for 600 ms. In between each stimulus pair, there was an interval
of 800 ms in which a random pattern was presented at the position of the
object to avoid visual aftereffects. After the presentation of each stimulus
pair, the participant was asked “Which stimulus is closer to the ground?” They
were told that at least one of the objects was positioned on the ground. They
then responded using the left mouse button to indicate the first stimulus and
the right button for the second stimulus. The experiment was self-paced, and
both the user’s response as well as their response time were recorded for each
trial. The next trial began 1000 ms after the participant responded to the
previous trial—unless the experiment was paused. After every 66 trials,
participants were presented with a visual prompt that asked if they needed to
take a break.

Between devices, participants were also required to take a break to prevent
fatigue. During the break, they filled out a brief survey that asked them to
describe any strategies that they used to determine ground contact. After
finishing the experiment in its entirety, they were asked to fill out a final
survey. Although we did not directly compare peoples’ performance across XR
devices in this study, due to the significant differences in display
properties, in the final survey we asked participants about their experiences
in each device. We believed this information would be informative for
interpreting our results. Therefore, in the post-experiment survey, they were
asked to rank the difficulty of the experimental task for each device and to
rank the displays’ quality of graphics. They were also asked if their strategy
for determining ground contact differed across devices.

Figure 6: Experiment 1 — Average percentage of correct responses for each
target object shape by display condition: optical see-through AR (left), video
see-through AR (center), and virtual reality (right). The effects of shape on
surface contact judgement were complex, with significant but different effects
of target shape for each display. However, people’s judgements with the cube
were significantly better than judgements with the sphere in all three
devices.

###  4.4 Results

Our experiment used a small-N design. Therefore, before performing additional
statistical analyses we confirmed that variance in our collected data was not
due to between-participant variation by calculating the intra-class
correlation coefficient (ICC). The ICC measures the amount of variance
accounted for by a grouping variable. For our analysis, individual participant
was selected as the grouping variable, where an ICC value of 1 indicated that
any variance in the data is between-participants while a value of 0 indicated
that no variance in the data is due to between-participant factors. We found
that $\tau_{ost}$ = .009 for data collected with the OST AR display, which
indicated negligible variance was caused by between-participants factors.
Similarly low ICC values were found for the VST AR and VR display data with
ICCs of $\tau_{vst}$ = .012 and $\tau_{vr}$ = .042, respectively.

##### Statistical Analyses

Participants were asked to make binary decisions about which object was
positioned closer to the ground (first or second object) in a 2AFC task.
Therefore, for our statistical analyses we used binary logistic regression
models, which are appropriate for dichotomous outcome variables, to analyze
participants’ judgments. We used the glm function from the stats package in R
[28] to conduct logistic regressions by specifying binomial errors and a logit
link function. Because we wanted to analyze people’s perception in each
device, we ran separate models for each of the XR displays. For each display,
we modeled binary outcomes (accuracy: correct (1) or incorrect (0)) for our
predictors: object shape, shadow shading, and height. Height was recorded in
millimeters then centered at zero and treated as continuous. Object shape (3
levels: cube, icosahedron, sphere) and shadow shading (2 levels: dark, light)
were treated as categorical factors. Interactions between shape and shadow
were included in the models.

We used these models to test three planned comparisons for each device to
understand how object shape and shadow influenced people’s surface contact
judgments. These comparisons were: 1) whether surface contact judgments
differed for different object geometries (i.e., H1 the main effect of object
shape); 2) the difference in surface contact judgments between dark and light
shadow shading across all other conditions (i.e., H2 the main effect of shadow
shading); and 3) interactions between object shape and shading (i.e., H3).

In order to examine whether there were main effects, we coded shadow and shape
factors using deviation coding (also known as effect or sum coding). For the
shape factor, the sphere was set as the reference group (i.e., coded as -1).
For the shadow factor, the light shadow condition was coded as .5 and dark
shadow was coded as -.5. Using this deviation coding also allowed us to
observe whether there was a main effect of height. The general logistic
regression equation is depicted in Equation 3 below.

$\begin{split}\log\left(\frac{p}{1-p}\right)=B_{0}&+B_{1}(\mbox{shadow})+B_{2}(\mbox{cube})\\\
&+B_{3}(\mbox{icosahedron})+B_{4}(\mbox{height})\\\
&+B_{5}(\mbox{shadow}\times\mbox{cube})\\\
&+B_{6}(\mbox{shadow}\times\mbox{icosahedron})\end{split}$ (3)

For the shape variable (H1), we were interested in whether the sphere lead to
lower accuracy than the other two shapes, collapsed across shadow and height.
In order to answer this question, we conducted planned contrasts on the
aforementioned model comparing each shape to one another, using the Bonferroni
correction to account for multiple comparisons.

In order to examine whether there was a main effect of shadow shading (H2),
the shadow regression coefficient represents the difference between dark and
light shadows averaging over shape and height. We were also interested in a
potential interaction between shape and shadow (H3). Specifically, was each
shape affected by the shadow shading? In order to answer this question, we
calculated the shadow simple slopes for each shape.

For the sake of simplicity, we discuss the implications of our findings as
well as which factors are significant in text, and we report the full details
of the effects of factors in our analyses in Table 2.

Figure 7: Experiment 1 — Average percentage of correct responses for each
target object shadow by display condition: optical see-through AR (left),
video see-through AR (center), and virtual reality (right). People were
significantly more accurate when shadow shading was light, rather than dark in
both AR device conditions.

####  4.4.1 Optical See-through AR

We show the average percent of correct response for each of the evaluated main
effects in Table 1. The results of our statistical analyses are reported in
Table 2. We expected participants to be more likely to correctly indicate
which object was on the ground as the height of target objects increased. The
improvement of participants’ performance as height increased is demonstrated
statistically by a main effect of height in our logistic regression model
($OR=1.09$, $p<.001$). Collapsed across all shape and shadow conditions, the
odds ratio ($OR$) of 1.09 indicates that for every 1mm increase in height, the
odds of correctly stating which object was closer to the ground increased by a
factor of 1.09.

##### H1: Does object shape matter?

People were least accurate when assessing surface contact given the sphere,
with 77% accuracy on average. In comparison, people were 80% accurate when
presented with the cube and 84% accurate when presented with the icosahedron
(See Figure 6). In our paired comparisons of people’s accuracy for each target
shape, when collapsed across shadow and height, we found that people’s
accuracy when they were shown the sphere was significantly worse than their
accuracy when shown the cube ($OR=1.42$, $p<.01$) or the icosahedron
($OR=1.67$, $p<.001$). In other words, participants were 1.42 times or 1.67
times more likely to choose the correct object with the cube or icosahedron,
respectively, than with the sphere. There was not a significant difference in
response accuracy between the cube and icosahedron ($OR=.85$, $p=.50$).

##### H2: Does shadow shading method matter?

People were 77% accurate when assessing surface contact when presented with a
dark shadow and 84% accurate when presented with a light shadow (See Figure
7).

The main effect for shadow ($OR=1.71$, $p<.001$) indicates that a correct
response was 1.71 times more likely when the object was presented with a light
shadow compared to a dark shadow.

##### H3: Is there an interaction between shape and shadow?

Analysis of the shadow by shape simple slopes indicated that the effect of
shadow was only significant for the cube ($OR=4.21$, $p<.001$), a finding that
indicates that people were more 4.21 times more likely to make a correct
response when the cube was rendered with a light shadow. The predicted
probabilities of correct response for each shape and shadow are displayed in
Figure 8 (left). On average, when presented with the cube, people were 70%
($SE=1.9\%$) accurate with the dark shadow was presented and they were 90%
($SE=1.2\%$) accurate with the light shadow.

Table 1:  Experiment 1 — The accuracy for each shape and shadow condition
tested reported with the standard errors.

Accuracy for Experiment 1
---
Condition |  | OST AR | VST AR | VR
Cube |  | 80% $\ [1.2\%]$ | 93% $\ [\ \ .7\%]$ | 95% $\ [\ \ .6\%]$
Icosahedron |  | 84% $\ [1.1\%]$ | 81% $\ [1.1\%]$ | 73% $\ [1.3\%]$
Sphere |  | 77% $\ [1.2\%]$ | 75% $\ [1.3\%]$ | 78% $\ [1.0\%]$
Dark |  | 77% $\ [\ \ .1\%]$ | 78% $\ [1.0\%]$ | 81% $\ [\ \ .9\%]$
Light |  | 84% $\ [\ \ .1\%]$ | 88% $\ [\ \ .8\%]$ | 83% $\ [\ \ .9\%]$

####  4.4.2 Video See-through AR

The average probabilities of correct response for each main condition are
shown in Table 1, and the results of our statistical analyses are reported in
Table 2. For VST AR we again found a main effect of height ($OR=1.07$,
$p<.001$). Accordingly, as the height of the vertically displaced object
increased, participants were more likely to correctly indicate which object
was on the ground.

##### H1: Does object shape matter?

People were most likely to produce a correct response when presented with the
cube with 93% accuracy, on average, and they were least likely to make a
correct response when presented with a sphere with 75% accuracy. People’s
accuracy for the icosahedron was 81% on average (Figure 6). Planned contrasts
for the shape variable indicated that the odds of providing a correct response
significantly differed between all shape comparisons. The cube was more likely
to yield a correct response than either the sphere ($OR=4.90$, $p<.001$) or
the icosahedron ($OR=2.93$, $p<.001$). And the odds of providing a correct
response for the icosahedron was higher than the odds for the sphere
($OR=1.67$, $p<.001$).

##### H2: Does shadow shading method matter?

The main effect for shadow ($OR=2.28$, $p<.001$) indicates that a correct
response was 2.28 times more likely when the object was presented with a light
shadow compared to a dark shadow. Overall, people were 88% accurate on average
when presented with the light shadow and 78% accurate when presented with the
dark shadow (See Figure 7).

##### H3: Is there an interaction between shape and shadow?

Analysis of the shadow by shape simple slopes indicated that the effect of
shadow was significant for all three shapes. That is, the probability of
providing a correct response was higher when a light shadow was presented
regardless of whether the object presented was a sphere ($OR=1.38$, $p<.05$),
a cube ($OR=2.40$, $p<.001$), or an icosahedron ($OR=3.58$, $p<.001$). The
predicted probabilities of correct response for each shape and shadow are
displayed in Figure 8 (center). The average accuracy for people’s judgments
with the cube was 90% $(SE=1.2)$ on average for the dark shadow condition and
95% $(SE=0.8)$ for the light shadow condition. For the icosahedron, people
were 73% $(SE=1.8)$ accurate with the dark shadow and 90% $(SE=1.2)$ accurate
for the light shadow. For the sphere people were 72% $(SE=1.8)$ accurate with
the dark shadow and 78% $(SE=1.8)$ accurate for the light shadow.

Table 2: Experiment 1 — Results of planned comparisons using binary logistic
regression models for each display condition are displayed. _B_ is the
regression coefficient, $SE_{B}$ is the standard error of the regression
coefficient, $OR$ is the odds ratio, and $CI_{OR}$ is the confidence interval
associated with the odds ratio. Negative values for _B_ indicate that the
first factor in the comparison was more accurate, whereas positive values
indicate that the second factor was more accurate.

Results for Experiment 1
---
|  |  OST AR |  VST AR |  VR
Predictor |  | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$ | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$ | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$
Shape | (cube vs ico) | $.16$ | .12 | $.85$ | $[\ \ .64,\ 1.13]$ |  $-1.08$*** | .15 | $2.93$ | $[2.06,\ 4.17]$ |  $-2.10$*** | .16 | $8.20$ | $[5.66,\ 11.9]$
| (cube vs sphere) |  $-.35$ ** | .11 | $1.42$ | $[1.09,\ 1.85]$ |  $-1.59$*** | .14 | $4.90$ | $[3.51,\ 6.84]$ |  $-1.78$*** | .16 | $8.20$ | $[4.08,\ 8.67]$
| (ico vs sphere) |  $-.52$ *** | .11 | $1.67$ | $[1.29,\ 2.17]$ |  $-.51$ *** | .11 | $1.67$ | $[1.29,\ 2.17]$ |  $.32$ ** | .20 | $1.06$ | $[\ \ .57,\ \ .92]$
Shadow | (dark vs light) |  $.54$ *** | .09 | $1.71$ | $[1.43,\ 2.05]$ |  $.82$ *** | .11 | $2.28$ | $[1.85,\ 2.83]$ | $.21$ | .11 | $1.23$ | $[\ \ .99,\ 1.54]$
Shape$\times$Shadow | (cube: dark vs light) |  $1.44$ *** | .17 | $4.21$ | $[3.02,\ 5.85]$ |  $.88$ *** | .24 | $2.40$ | $[1.49,\ 3.85]$ | $.27$ | .28 | $1.31$ | $[\ \ .76,\ 2.27]$
| (ico: dark vs light) | $.31$ | .16 | $1.36$ | $[\ \ .99,\ 1.88]$ |  $1.28$ *** | .17 | $3.58$ | $[2.58,\ 4.98]$ | $-.01$ | .13 | $.99$ | $[\ \ .76,\ 1.29]$
| (sphere: dark vs light) | $-.13$ | .14 | $.88$ | $[\ \ .66,\ 1.16]$ |  $.32$ * | .14 | $1.38$ | $[1.05,\ 1.81]$ |  $.36$ * | .14 | $1.43$ | $[1.08,\ 1.90]$
Height |  |  $.09$ *** | .01 | $1.09$ | $[1.08,\ 1.10]$ |  $.07$ *** | .01 | $1.07$ | $[1.06,\ 1.08]$ |  $.06$ *** | .01 | $1.06$ | $[1.05,\ 1.07]$
Intercept |  |  $1.63$ *** | .05 | $5.10$ | $[4.62,\ 5.63]$ |  $1.88$ *** | .06 | $6.55$ | $[5.87,\ 7.35]$ |  $1.85$ *** | .06 | $6.36$ | $[5.69,\ 7.16]$
|  |  $*p<.5$ $**p<.01$ $***p<.001$

Figure 8:  Experiment 1 — Predicted probability of correct response for shape
(cube, icosahedron, sphere) and shadow (dark, light) interactions with 95%
confidence. In VST AR, all shape x shadow interactions were significant. In
contrast, for OST AR the effect of shadow was only significant for the cube
and for VR it was only significant for the sphere.

####  4.4.3 Virtual Reality

The average rate of correct response for each main condition is reported in
Table 1, and the results of our statistical analyses are reported in Table 2.
For VR, we found a main effect of height ($OR=1.06$, $p<.001$) such that for
every 1mm increase in height, the odds of correctly stating which object was
closer to the ground increased by a factor of 1.06.

##### H1: Does object shape matter?

People were most likely to make a correct response when presented with the
cube (95% accuracy on average), and they were least likely to make a correct
response when presented with the icosahedron (73%). People’s accuracy for the
sphere was 78% (Figure 6). The probability of providing a correct response
significantly differed for all shape comparisons. Specifically, planned
contrasts, when collapsed across shadow and height, showed that participants
were significantly more likely to make a correct response when presented with
the cube than the icosahedron ($OR=8.20$, $p<.001$) or sphere ($OR=5.95$,
$p<.001$). They were then more likely to make a correct response given the
sphere over the icosahedron ($OR=.73$, $p<.01$).

##### H2: Does shadow shading method matter?

The average accuracy for surface contact judgments was 81% for dark shadows
and 83% for light shadows (See Figure 7). The main effect for shadow was not
significant ($OR=1.23$, $p=.07$), suggesting that, averaged across shape and
height, shadow shading did not have a significant effect on judgments of
ground contact in virtual reality.

##### H3: Is there an interaction between shape and shadow?

Analysis of the shadow by shape simple slopes indicated that the effect of
shadow was only significant for the sphere ($OR=1.43$, $p<.05$). The predicted
probabilities of correct response for each shape and shadow are displayed in
Figure 8 (right). On average, when presented with the sphere, people were 76%
($SE=1.8\%$) accurate with the dark shadow when presented and they were 81%
($SE=1.6\%$) accurate with the light shadow.

####  4.4.4 How difficult was each condition?

We examined post-experiment survey responses to better understand how
participants interpreted our experimental stimuli. Participants were asked to
rank the devices from easiest to hardest for determining ground contact, and
they were asked to rank devices on graphics quality from lowest to highest.
For difficulty rankings, they assigned each XR display to one of three
categories: easiest, middle, or hardest. For quality of graphics rankings,
they assigned each display to best, middle, or worst.

There was no clear easiest display, given that all three XR devices were rated
as the easiest display by two out of six (2/6) participants. However, the AR
devices were seen as more difficult than the VR condition, with both the OST
AR display (4/6) and the VST AR display (2/6) receiving ratings for being the
most difficult display. Participants also stated that the video see-through
display had the lowest perceived quality of graphics with three out of six
(3/6) worst votes and 3/6 median votes, and that the optical see-through
display had the highest quality graphics (5/6).

Peoples’ reported strategies in the post-experiment survey support the idea
that the curved lines of the sphere and its shadow increased the difficulty of
discerning surface contact. For discussion, we will refer to each participant
with an acronym (e.g., P1 for the first participant). P1, P3, and P4 reported
looking at shadows “beneath” the objects or “beneath the front side” of
objects. P3 also commented that they tried to remember the point of optical
contact for a given object. P2 and P5 explicitly stated that they looked for
the “edges” of shadows to make judgments. A reliance on shadow edges would
also explain why people’s responses for the sphere were generally less
accurate, regardless of XR display. Participants reported that they employed
the same strategies for discerning ground contact across devices.

###  4.5 Discussion

In our first experiment, we found that light shadows improved ground contact
judgments compared to dark shadows for both AR devices and that they performed
comparably to dark shadows in VR. These results encourage the use of non-
photorealistic rendering solutions for improving surface contact judgments
across XR displays. Perceived surface contact plays an important role in depth
perception and improving it may help alleviate the disconnected appearance
between virtual objects and real surfaces in AR.

In addition, we found that object shape influences surface contact judgments,
where judgments were more accurate for the cube than the sphere in all three
XR devices. Judgments for the icosahedron were more accurate than the sphere
in VST AR and OST AR. Given that there was only one condition where people’s
judgments were more accurate for the sphere than another target shape, our
current study provides some evidence that spheres may be less effective than
other shapes in establishing surface contact. This finding may have
implications for depth perception in XR studies, as spheres are a common
target object used in distance perception research for these devices [4, 22,
46, 47]. In addition, while our results suggest an overall advantage for light
shadows for AR, there were some differences in these effects for specific
shapes and devices. The benefit of light shadows was largely driven by the
cube in OST AR. But, in VST AR all shapes were significantly influenced by the
presence of a shadow.

There are limitations to the present experiment. All objects were viewed from
a single vantage point, which may affect how generalizable our results are to
other geometric shapes and perspectives in XR. This concern is supported by
prior research, which has demonstrated that spatial perception can be
influenced by viewing angle [34, 3]. Some prior behavioral studies in AR have
made conscious efforts to display objects at multiple orientations to subvert
any possible effects of orientation [22]. Others have displayed objects at
varying heights and distances [2]. Thus, to understand better how the
placement of an object may affect people’s ability to determine surface
contact, we will evaluate target objects at different orientations in the next
experiment.

A second limitation is that we do not evaluate any 3D objects that have a
comparable complexity to those that might be used in typical AR and VR
applications. All three objects that we evaluated were simple geometric
shapes. As highlighted by Powell et al. [49], an object’s geometry has the
ability to influence one’s perception of space. In Powell and colleagues’
study, researchers evaluated people’s ability to reach and grasp virtual
spheres, icosahedrons, and apples in VR. They found that the richer geometry
cues provided by the icosahedron and apple positively influenced reaching and
grasping behaviors. In a similar manner, Do et al. [15] evaluated the effect
of object color and luminance on objects of different shapes for depth
judgments in mobile AR. They found interactions with color and luminance on
depth perception. The findings of both of these studies motivate us to
evaluate a more complex geometric shape in our next experiment.

Figure 9: The cube from Experiment 1 is rotated by 45∘ and evaluated under the
same shadow shading methods and vertical displacements in Experiment 2. Here
both cubes are shown for visual comparison.

##  5 Experiment 2: Complexity and Orientation

In this experiment, we investigated if shape complexity and orientation
influenced the perception of surface contact when dark and light shadows were
present. Figures 9 and 10, which display the experimental stimuli used in
Experiment 2, present a strong visual argument for investigating orientation
given we can observe changes in shape and shading between an object and its
cast shadows in these images with rotation. To address these concerns, we
evaluated two target objects, a cube and the Stanford dragon, at two
orientations, rotated by $0^{\circ}$ and rotated by $45^{\circ}$, in our
second study. Our hypotheses for these experiments were as follows:

H4: In our first experiment, people’s surface contact judgements were more
accurate for the cube than the sphere across all XR devices. Although the
sphere was technically a more complex geometric shape [20, 15]—where geometric
complexity is defined by the number of polygons used to construct a given
object’s mesh—the resulting shape was nonetheless a geometric primitive. As
such, in our second experiment we evaluated a notably more complex object, the
Stanford dragon, which consisted of 113,000 polygons.

Although the dragon object provides additional depth from shading information
due to its complexity, we anticipated that people’s responses to the cube
would be more accurate, given the importance of contour junctions (e.g.,
T-junctions) as local cues for occlusion [52] when perceiving surfaces [41].
Accordingly, the straight, rectilinear edges of the cube should prove more
beneficial to ground contact judgments. This finding would provide evidence
against the idea that more complex geometries are inherently better for depth
perception [15, 49, 5].

H5: Based on the results from our first study, we once again predicted that
people’s surface contact judgments would be more accurate for the light shadow
condition than the dark shadow condition across both AR devices.

H6: We predicted that, overall, a change in object orientation would affect
participants’ surface contact judgments given that changes in orientation can
alter both the perceived shape of the surface contact and its associated cast
shadow for the viewer. Because the starting position ($0^{\circ}$) was
different for the cube and the dragon, it’s possible that the $45^{\circ}$
rotation might have deferentially affected the perception of the shapes.
Therefore, we also include an interaction between shape and orientation in our
analysis.

Figure 10: In Experiment 2, the Stanford dragon was also rendered in each
display and shown at two orientations. All other shading and displacement
factors were the same as in Experiment 1.

###  5.1 Participants

The same six individuals (3M, 3F) aged 23–30 volunteered to participate in our
second experiment. Our experimental methods were approved by the local
institutional review board, written consent was obtained from all subjects
prior to participation, and each participant was paid 20 USD for 2-3 hours of
their time.

Figure 11: Experiment 2 — Average percentage of correct responses for each
target object shape by display condition: optical see-through AR (left), video
see-through AR (center), and virtual reality (right). Object complexity did
not inherently benefit surface contact judgements. People’s judgements were
significantly more accurate when presented with the cube than the dragon in
both the VST AR and VR conditions. The opposite was true in OST AR.

###  5.2 Design

Our experimental design for the second evaluation mirrored the one employed by
our first, in which a temporal two-alternative forced choice (2AFC) paradigm
with a method of constant stimuli was employed. All experimental details were
the same, except for the independent variables. We used the same testing
environment, height displacement values (between 0 and 6mm), the same number
of comparisons, and the same number of trials. We employed two models, the
cube from Experiment 1, and the Stanford
Dragon,111http://graphics.stanford.edu/data/3Dscanrep/ tesselated to have
113,000 polygons.

Each participant evaluated two cast shadow conditions (dark and light) over
three different object + orientation conditions: the cube rotated by
$45^{\circ}$, the Stanford dragon rotated by $0^{\circ}$, and the Stanford
dragon rotated by $45^{\circ}$. This resulted in 6 unique combinations of
experimental stimuli. However, our analysis evaluated 8 combinations of
stimuli, as we included the non-rotated cube condition (with both shadow
conditions) from Experiment 1. The Experiment 1 data was included to prevent
participant fatigue from an increased number of trials and because the
participants were the same, so concerns about individual differences overall
are low. In the second experiment, 1,980 points of data were collected from
each volunteer for a total of 11,880 data points. When combined with the cube
data from the first experiment, each participant completed 2,640 trials for a
total of 15,840 trials across all participants.

Using the same approach that we employed for Experiment 1, we evaluated the
extent to which participant variance accounted for variance in our collected
data by calculating the intraclass correlation coefficient (ICC) for
Experiment 2. For the OST AR display condition we found $\tau_{ost}$ = 0.006
and for the VST AR display we found $\tau_{vst}$ = 0.028. For the VR display
condition $\tau_{vr}$ = 0.023. Because our ICC values were near zero, we
determined that our experimental findings were not significantly influenced by
between-participant variation.

###  5.3 Procedure

The procedure for the second experiment followed the same protocol used in
Experiment 1. However, this time volunteers did not undergo training for the
devices and experimental paradigm since they had already performed a very
similar experimental task in Experiment 1.

Table 3:  Experiment 2 — The accuracy for each shape and shadow condition
tested reported with the standard errors.  Accuracy for Experiment 2
---
Condition |  | OST AR | VST AR | VR
Cube |  | 85% $\ [\ .7\%]$ | 92% $\ [\ .5\%]$ | 94% $\ [\ .5\%]$
Dragon |  | 90% $\ [\ .6\%]$ | 81% $\ [\ .8\%]$ | 83% $\ [\ .8\%]$
Dark |  | 81% $\ [\ .8\%]$ | 83% $\ [\ .8\%]$ | 86% $\ [\ .7\%]$
Light |  | 94% $\ [\ .5\%]$ | 91% $\ [\ .6\%]$ | 90% $\ [\ .6\%]$
Origin ($0^{\circ}$) |  | 84% $\ [\ .7\%]$ | 87% $\ [\ .7\%]$ | 86% $\ [\ .7\%]$
Rotated ($45^{\circ}$) |  | 91% $\ [\ .6\%]$ | 87% $\ [\ .7\%]$ | 91% $\ [\ .6\%]$

###  5.4 Results

##### Statistical Analyses

For our analyses, we used the binary logistic regression model approach that
we employed in the first study (See Section 4.4) to analyze participants’ 2AFC
judgements for each of the three XR devices. Separate GLMs with logit link
functions were run for each device to understand how an object’s cast shadow
shading, geometric complexity, and orientation affected people’s surface
contact judgments in each device. Specifically, we modeled binary outcomes
(correct or incorrect) for our predictors: object shape, object orientation,
shadow shading, and height. Height was recorded in millimeters then centered
at zero and treated as continuous. Object shape (2 levels: cube and dragon),
orientation (2 levels: rotated by $0^{\circ}$ and by $45^{\circ}$), and shadow
shading (2 levels: dark and light) were treated as categorical factors. These
factors were deviation coded. For shadow shading, dark was coded as -.5 and
light was .5; for shape, cube was -.5 and dragon was .5; for orientation,
$0^{\circ}$ was -.5 and $45^{\circ}$ was .5.

We tested three planned comparisons for each device. These comparisons were:
1) whether surface contact judgments differed for different object geometries
(i.e., H4 the main effect of object shape); 2) the difference in surface
contact judgments between dark and light shadow shading across all other
conditions (i.e., H5 the main effect of shadow shading); and 3) whether an
object’s orientation influenced people’s surface contact judgments (i.e., H6
the main effect of orientation). We also included an interaction between
object shape and shading and an interaction between object shape and
orientation to better understand the relationship between these variables.

Table 4: Experiment 2 — Results of planned comparisons using binary logistic
regression models for each display condition are displayed. _B_ is the
regression coefficient, $SE_{B}$ is the standard error of the regression
coefficient, $OR$ is the odds ratio, and $CI_{OR}$ is the confidence interval
associated with the odds ratio. Negative values for _B_ indicate that the
first factor in the comparison was more accurate, whereas positive values
indicate that the second factor was more accurate.

Results for Experiment 2
---
|  |  OST AR |  VST AR |  VR
Predictor |  | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$ | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$ | _B_ | $SE_{B}$ | $OR$ | $[CI]_{OR}$
Shape | (cube vs dragon) |  $.32$ ** | .11 | $1.38$ | $[1.12,\ 1.70]$ |  $-1.15$ *** | .10 | $.32$ | $[\ .26,\quad.39]$ |  $-1.31$ *** | .11 | $.27$ | $[\ \ .22,\ \ \ .33]$
Shadow | (dark vs light) |  $1.31$ *** | .10 | $3.70$ | $[3.02,\ 4.55]$ |  $.91$ *** | .10 | $2.47$ | $[2.03,\ 3.04]$ |  $.52$ *** | .11 | $1.68$ | $[1.36,\ 2.08]$
Orien | ($0^{\circ}$ vs $45^{\circ}$) |  $.67$ *** | .10 | $1.96$ | $[1.62,\ 2.38]$ | $-.06$ | .10 | $.94$ | $[\ \ .78,\ 1.14]$ | $-.13$ | .11 | $.88$ | $[\ \ .71,\ 1.08]$
Shape$\times$Shadow |  |  $-.90$ *** | .21 | $.41$ | $[\ \ .27,\ \ \ .61]$ |  $-.50$ * | .21 | $.61$ | $[\ \ .40,\ \ \ .91]$ |  $-0.45$ * | .21 | $.64$ | $[\ \ .42,\ \ \ .97]$
| (cube: dark vs light) |  $1.76$ *** | .15 | $5.79$ | $[4.35,\ 7.71]$ |  $1.15$ *** | .17 | $3.17$ | $[2.25,\ 4.46]$ |  $.74$ *** | .18 | $2.11$ | $[1.47,\ 3.02]$
| (dragon: dark vs light) |  $.87$ *** | .15 | $2.37$ | $[1.77,\ 3.17]$ |  $.66$ *** | .11 | $1.93$ | $[1.55,\ 2.40]$ |  $.29$ ** | .11 | $1.34$ | $[1.08,\ 1.67]$
Shape$\times$Orien |  |  $-0.51$ ** | .20 | $.61$ | $[\ \ .41,\ \ .88]$ | $.20$ | .19 | $1.22$ | $[\ \ .84,\ 1.79]$ |  $.77$ *** | .21 | $2.16$ | $[1.43,\ 3.29]$
| (cube: $0^{\circ}$ vs $45^{\circ}$) |  $.93$ *** | .13 | $2.53$ | $[1.95,\ 3.27]$ | $-.16$ | .16 | $.85$ | $[\ \ .63,\ 1.16]$ |  $-.52$ ** | .18 | $.60$ | $[\ \ .42,\ \ \ .85]$
| (dragon: $0^{\circ}$ vs $45^{\circ}$) |  $.42$ ** | .14 | $1.52$ | $[1.15,\ 2.02]$ | $.04$ | .11 | $1.04$ | $[\ \ .84,\ 1.29]$ |  $.26$ * | .11 | $1.29$ | $[1.04,\ 1.61]$
Height |  |  $.12$ *** | .01 | $1.13$ | $[\ \ 1.11,\ \ 1.14]$ |  $.09$ *** | .01 | $1.09$ | $[1.08,\ \ 1.11]$ |  $.08$ *** | .01 | $1.08$ | $[\ 1.07,\ \ 1.10]$
Intercept |  |  $2.50$ *** | .07 | $12.13$ | $[10.71,13.83]$ |  $2.25$ *** | .06 | $9.50$ | $[8.51,10.67]$ |  $2.39$ *** | .06 | $10.88$ | $[\ 9.69,12.28]$
|  |  $*p<.5$ $**p<.01$ $***p<.001$

Figure 12: Experiment 2 — Average percentage of correct responses for each
target object shadow by display condition: optical see-through AR (left),
video see-through AR (center), and virtual reality (right). People were
significantly more accurate when shadow shading was light, rather than dark in
all three display conditions.

####  5.4.1 Optical See-through AR

We show the average percent of correct response for each of the evaluated main
effects in Table 3, and the results of the logistic regression for the OST AR
display data are presented in Table 4. We expected participants to be more
likely to correctly indicate which object was on the ground as the height of
target objects increased. The improvement of participants’ performance as
height increased is demonstrated statistically by a main effect of height in
our logistic regression model ($OR=1.13$, $p<.001$). An odds ratio ($OR$) of
1.13 indicates that for every 1mm increase in height, the odds of correctly
stating which object was closer to the ground increased by a factor of 1.13.

##### H4: Does an object’s complexity matter?

People were more accurate when assessing surface contact with the dragon than
the cube, with 90% and 85% accuracy, on average, overall (Figure 11). Our
statistical analysis revealed that this relationship was significant given
that—when averaged across shadow shading, orientation, and height—the dragon
shape in our model was more likely to elicit correct responses than the cube
with an odds ratio of 1.38 ($p<.01$). As such, people were 1.38 times more
likely to correctly assess surface contact when presented with the dragon.
This outcome differs from what we observed in both the VST AR and VR
conditions as reported in Sections 5.4.2 and 5.4.3. In these devices surface
contact judgments to the cube were more accurate than judgments to the dragon.

##### H5: Does shadow shading method matter?

The main effect for shadow ($OR=3.70$, $p<.001$) indicates that a correct
response was 3.70 times more likely when the object was presented with a light
shadow compared to a dark shadow, when collapsed across shape, orientation,
and height. On average, people were 94% accurate when presented with the light
shadow and 81% accurate when presented with the dark shadow (See Figure 12).
There was also a significant interaction between shape and shadow ($OR=.41$,
$p<.001$). Analysis of the shadow simple slopes by shape indicated that, for
both shapes, a light shadow was more likely to yield a correct judgment of
ground contact than a dark shadow (supporting the main effect of shadow). The
simple slopes are depicted in Figure 13. However, this effect was stronger for
the cube ($OR=5.79$, $p<.001$) than the dragon ($OR=2.37$, $p<.001$).

##### H6: Does object orientation matter?

There was a significant main effect of orientation ($OR=1.96$, $p<.001$)
indicating that, overall, participants were more likely to make a correct
judgment when the object was rotated $45^{\circ}$ from its starting position.
On average, people’s judgments to targets at their original orientation were
84% accurate and their judgments to rotated targets were 91% accurate in the
HoloLens. This main effect was qualified by an interaction between object
shape and orientation ($OR=.60$, $p<.01$). In order to interpret the
interaction, we calculated the orientation simple slopes for each shape. For
both shapes, the likelihood of correctly judging ground contact was higher for
the $45^{\circ}$ orientation than the $0^{\circ}$ orientation (supporting the
main effect of orientation). However, this effect was stronger for the cube
($OR=2.53$, $p<.001$) than for the dragon ($OR=1.52$, $p<.01$).

Figure 13:  Experiment 2 — Predicted probability of correct response for shape
(cube, dragon) and shadow (dark, light) interactions with 95% confidence. For
all three devices a light shadow was more likely to elicit a correct judgement
of surface contact than a dark shadow, regardless of shape. In addition, this
effect was stronger for the cube than for the dragon.

####  5.4.2 Video See-through AR

We show the average percent correct response for each of the evaluated main
effects in Table 3. The results of our statistical analyses are reported in
Table 4. For VST AR we again found a main effect of height ($OR=1.09$,
$p<.001$) such that for every 1mm increase in height, the odds of correctly
stating which object was closer to the ground increased by a factor of 1.09.
Accordingly, as the height of the vertically displaced object increased,
participants were more likely to correctly indicate which object was on the
ground.

##### H4: Does an object’s complexity matter?

The cube shape was more likely to elicit correct responses than the dragon
($OR=.32$, $p<.001$), suggesting that object complexity is not inherently
beneficial to surface contact perception. People’s judgments were 92%
accurate, on average, when assessing surface contact with the cube and 81%
accurate, on average, when assessing contact with the dragon (Figure 11).

##### H5: Does shadow shading method matter?

The main effect for shadow ($OR=2.47$, $p<.001$) indicates that a correct
response was 2.47 times more likely when the object was presented with a light
shadow compared to a dark shadow. On average, people were 91% accurate with
the light shadow and they were 83% accurate with the dark shadow when
evaluating surface contact (See Figure 12). In addition, there was a
significant interaction between shape and shadow ($OR=.61$, $p<.05$). Analysis
of the shadow by shape simple slopes indicated that the effect of shadow was
significant for both shapes (See Figure 13 (center)). For both shapes, the
probability of correct response was higher when a light shadow was presented
compared to a dark shadow. However, this effect was stronger for the cube
($OR=3.17$, $p<.001$) than the dragon ($OR=1.93$, $p<.001$).

##### H6: Does object orientation matter?

There was no main effect of orientation in VST AR. People’s surface contact
judgments to targets when positioned at their original orientations and when
rotated were 87% accurate, on average. The interaction between orientation and
shape was also insignificant. In other words, the probability of providing a
correct response was not significantly different between the two orientations
in VST AR. This outcome differs from the results of our analyses in the OST AR
and VR conditions, both of which found that people’s judgments were more
accurate on average to the rotated targets.

####  5.4.3 Virtual Reality

The average rate of correct response for each main condition is reported in
Table 3, and the results of our statistical analyses are reported in Table 4.
For VR, we found a main effect of height ($OR=1.06$, $p<.001$) such that for
every 1mm increase in height, the odds of correctly stating which object was
closer to the ground increased by a factor of 1.06.

##### H5: Does an object’s complexity matter?

The cube was more likely to elicit correct responses than the dragon
($OR=.27$, $p<.001$), suggesting that object complexity is not inherently
beneficial to surface contact perception. Participants’ judgments were 94%
accurate, on average, when assessing surface contact with the cube and 83%
accurate, on average, when assessing contact with the dragon (Figure 11).

##### H4: Does shadow shading method matter?

The main effect for shadow ($OR=1.68$, $p<.001$) indicates that a correct
response was 1.68 times more likely when the object was presented with a light
shadow compared to a dark shadow. Judgments of surface contact were 90%
accurate, on average, in VR with the light shadow and they were 86% accurate,
on average, with the dark shadow (See Figure 12). The interaction between
shape and shadow was also significant ($OR=.64$, $p<.05$). Unlike our results
for Experiment 1, the analysis of the shadow by shape simple slopes indicated
that the effect of shadow was significant for both shapes in VR (See Figure 13
(right)). The probability of correct response was higher when a light shadow
was presented regardless of the object presented, although the shadow effect
was stronger for the cube ($OR=2.11$, $p<.001$) than the dragon ($OR=1.34$,
$p<.001$).

##### H6: Does object orientation matter?

The main effect of orientation was not significant. However, there was a
significant interaction between shape and orientation ($OR=2.16$, $p<.001$).
On average, people’s judgments to targets at their original orientation were
86% accurate and their judgments to rotated targets were 91% accurate.
Analysis of the orientation by shape simple slopes indicated that a correct
response was more likely when the cube was presented at 0∘ than when the cube
was rotated 45∘ ($OR=.60$, $p<.01$). In contrast, the probability of a correct
response was higher when the dragon was rotated 45∘ than when it was presented
at 0∘ ($OR=1.29$, $p<.05$).

####  5.4.4 How difficult was each condition?

After completing the second experiment, volunteers were asked to rank each
device by difficulty and by quality of graphics as in Experiment 1. All
participants ranked the video see-through display as the most difficult device
(6/6) and the device with the lowest quality of graphics (6/6). In contrast,
the optical see-through display was most consistently rated as the easiest
display (4/6) and the display with the highest quality of graphics (5/6).
Ratings were also corroborated by comments at the end of the survey. Both P2
and P4 stated that the HoloLens “felt more clear,” and P1 commented that “the
resolution of the Zed was really painful for my eyes.”

Participants were also asked about the strategies they used to discern surface
contact. P3 commented that the dragon was “easier but more mentally
exhausting… since the dragon moved.” Their comment implied that they had to
visually search for a shadow before making a judgment. Their complaint is
reasonable since, as the shape of the dragon was asymmetrical, changes in
orientation were more pronounced (See Figure 10). P3 further bemoaned of the
dragon that “when too detailed it was overwhelming, [but] when not detailed
enough it became equally difficult to use.” P2 commented that the dragon
provided more “pockets of shadow,” which altered their strategy such that “ It
wasn’t just watching an edge, more like watching for shape and quantity of
shadow.” This comment hints that people may have been able to use additional
depth information provided by the dragon (e.g., shape from shading) to inform
their judgments of surface contact.

###  5.5 Discussion

In Experiment 2, for both AR display conditions, we again found that judgments
of surface contact benefited from the presence of non-photorealistic, light
cast shadows. However, in contrast to our first study, we also found that
light shadows improved surface contact judgments in VR. Looking at shape
$\times$ shadow interactions for the cube in Experiment 1 and Experiment 2 in
VR may provide some insight into this difference in shadow effectiveness for
VR between experiments. Both experiments evaluated the same cube object, but
the object was presented at a new orientation in Experiment 2. Because we
found a significant difference in orientation for the cube and because we
found an effect of shadow shading condition in Experiment 2, we infer that the
change in orientation for the cube made the light shadows more important in
VR.

Our hypothesis that surface contact judgments for the dragon would be worse
than the cube was partially supported. Although we found better performance
with the cube compared to the dragon in both VST AR and VR, we found better
performance with the dragon in OST AR. With the current experiment alone, we
are unable to explain why judgments for the dragon in the OST AR device were
significantly more accurate than those for the cube. However, the current
study provides evidence that more complex geometries do not inherently benefit
surface contact perception. Although complex 3D shapes can provide more depth
information via self-shading cues, it appears that other factors such as the
bottom edge of the object may be more important. However, additional future
research is needed to verify this claim.

Given the effects of orientation that we found in both the OST AR and VR
displays, it will be important to consider the orientation of objects in XR in
future application development. This finding is in line with prior research
which shows that the angle from which we view a 3D object may affect spatial
perception judgments related to that object[34, 3]. The lack of difference in
orientation for the video see-through conditions requires further
investigation.

##  6 General Discussion

Perception of surface contact is important for determining the scale of space
and for governing our interactions within space. Cast shadows provide
important information about surface contact, and thus provide important cues
to our visual system for perceiving and acting with our surroundings [62, 42,
2]. In certain types of XR displays, rendering cast shadows is difficult.
Therefore, this paper manipulated cast shadow rendering across different
object shapes in XR displays. Our goal was to see if a unified framework would
emerge to help designers better design for accurate spatial perception in XR,
given the difficulties with rendering realistic, dark shadows in current
technology.

We hypothesized that people would be more likely to correctly perceive surface
contact when shadows were illuminated with light rather than dark color
values. This hypothesis was based on prior work [31, 2]. We found support for
these hypotheses (H2 and H5) for two types of AR devices across two
experiments. We also hypothesized that surface contact judgments would be
complex and affected by the shape of an object associated with a given surface
(H1). This hypothesis was derived from the idea that if a shadow provides
visible glue with a surface, then the shape of the object to which that glue
is applied should interact with shadow effectiveness [62, 36]. We found
support for this hypothesis as well, although we evaluated relatively simple
object shapes in our first study.

In our second study, we directly investigated the effect of an object’s
complexity and orientation on surface contact judgments. Consistent with the
reasoning above, we hypothesized that orientation would affect the perception
of ground contact (H6). We found that orientation mattered for surface contact
judgments in OST AR and VR, but we did not find a difference in VST AR, only
partially confirming this hypothesis. Our results suggest that future
investigations of surface contact and depth perception may be better able to
generalize their findings if targets are presented at multiple orientations
like in Gao et al. [22].

Given prior research on the importance of contour junctions when perceiving
surfaces, we also hypothesized that a more complex geometric shape would not
inherently benefit surface contact judgements (H4). Instead, we predicted that
people’s contact judgements for a cube, which provides clear T-junctions
between itself and the surface beneath it, would be more accurate than
judgements for a shape with more complex contour junctions. Our hypothesis was
confirmed in VST AR and VR, but not for OST AR, where the opposite was true.
This outcome supports the idea that more complex shapes do not inherently
benefit surface contact perception, but it is less informative about what
properties of the two evaluated objects led to different outcomes for
different displays.

One general finding that we can extract from this set of experiments is that
the realistic shadows are not requisite for accurate surface contact
perception. Further, non-photorealistic (light) shadows will likely work well
across XR devices. Our findings give XR designers some latitude for general
design, as they provide evidence that more stylized graphics may used in XR
applications without detracting from people’s spatial perception. In addition,
there are many AR applications in which accurate spatial perception may be
more important than graphical realism (e.g., AR games [23, 30] or training
applications [26, 35]).

Our findings on shape, complexity, and orientation are more complicated. It
may be that they do not generalize well across XR devices and that general
guidelines are not appropriate here. Clearly, more work is needed on shape and
orientation, but our results do offer some initial suggestions. For object
shape, we found only one case across all three XR devices where judgments were
more accurate for the sphere than another target shape (e.g., sphere compared
to the icosahedron in VR). Do et al. [15] found that depth judgments to
spheres were more sensitive to changes in luminance than other shapes in
mobile AR. Both our results and those of Do et al. [15] may caution against
the use of spheres for AR applications that require accurate spatial
perception. Finally, our findings on orientation may have implications for the
development of XR applications. The angle from which we view a 3D object can
affect spatial perception judgments related to that object [34, 3]. For the
development of XR applications, however, understanding how specific shapes and
orientations of objects influence where people perceive them to be positioned
in space may be important.

###  6.1 Limitations

A limitation of the current work is that we did not manipulate object shading
in our experiments. Prior research on surface contact perception in XR has
evaluated the effect of differences in object and shadow shading on people’s
surface contact judgments [2]. Based on this prior research, we elected to use
a median gray value to shade all of our test objects to mitigate any effects
of target object shading, although we recognize that this reduces
generalizability. Further, neither the current work nor the prior work
conducted by Adams et al. [2] manipulated the color of background surfaces on
which target objects were displayed. Yet just as an object’s shading color may
influence our spatial perception of that object in space [15, 14, 48, 3], so
too may the backdrop upon which it is presented [45]. In our case, we used the
same, median blue tablecloth in both experiments. Further work will need to
test background surfaces of varying color and texture to fully understand how
these may interact with shadows in XR devices.

It is also possible that the decision to include cube data from Experiment 1
in our analysis for Experiment 2 may have affected our analysis of orientation
since cube orientation varied between the two sessions. Specifically, it is
possible that training effects could have occurred between sessions since the
same cube object was used in both experiments. However, on average people’s
surface contact ratings for the cube in the second session were less accurate
for both the VST AR ( $93\%\rightarrow 92\%$) and VR ($95\%\rightarrow 94\%$)
conditions. Accordingly, we believe that participants’ responses varied due to
changes in orientation—not due to training effects.

##  7 Conclusions

This paper demonstrates that there are advantages to non-photorealistic
rendering of cast shadows for surface contact judgments in XR. We found that
judgments were better with light shadows in two types of AR devices and in VR
under certain conditions. In arriving at this finding we experimented with a
variety of object shapes, orientations, and complexities. Our findings suggest
that under certain circumstances it may be desirable to use light shadows in
XR applications in order to denote surface contact.

Because the light shadows in our current study enhanced people’s accuracy in
surface contact judgments—it may be worthwhile to evaluate whether more
ambitious non-photorealistic rendering approaches facilitate spatial
perception in AR. Evaluating colorful shadows, like those designed by Ooi et
al. [44], for AR may be a desirable starting point. If more research findings
are able to confirm the benefits of non-photorealistic rendering for spatial
perception in AR, then we may encourage the use of more stylized graphical
elements for designers and developers of XR applications. Intriguing design
studies like the one conducted by Sun et al. [60] that enhance users’ ability
to position objects in 3D space using non-photorealistic surface contact cues
are already underway.

However, we must be cautious about arbitrarily applying non-photorealistic
effects to XR applications before understanding how they affect perception.
Our work and recent research studies conducted by other groups [60, 63, 2]
encourage the use of stylized graphics in AR, but these results do not imply
that all non-photorealistic rendering techniques will be similarly beneficial.
The results of Cidota et al. [13], in which blur and fade were shown to have
adverse effects on action-based depth judgments in AR within personal space,
provides some evidence towards the need for caution. Interestingly, much of
the theoretical work on perception in augmented reality has speculated the
graphics would need to match reality for accurate depth perception [16, 33,
1]. But more recent empirical findings provide counter evidence for this idea.

Our current findings may have ramifications for egocentric depth judgments in
XR displays given prior distance perception research in which people’s
egocentric distance estimates to targets were more accurate when cast shadows
were present [42]. In future research, it may be worthwhile to evaluate how
object shape and cast shadow shading manipulations affect more direct measures
of depth perception. Rosales et al. [51] demonstrated that, in the absence of
cast shadows, people perceive an object that is placed above the ground
incorrectly as farther away. This may help explain some of the effects of
overestimation found in prior AR depth perception research, especially given
that many studies use floating objects in their assessments [46, 61, 55]. In
future work, we intend to evaluate how photorealistic and non-photorealistic
shadows affect egocentric depth judgments in XR displays.

###### Acknowledgements.

The authors wish to thank William B. Thompson for his insight and Sam Halimi
for his wit. This work was supported by the Office of Naval Research under
grant N00014-18-1-2964

## References

  * [1] H. Adams.  Resolving cue conflicts in augmented reality.  In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), pp. 547–548. IEEE, 2020.
  * [2] H. Adams, J. Stefanucci, S. Creem-Regehr, G. Pointon, W. Thompson, and B. Bodenheimer.  Shedding light on cast shadows: An investigation of perceived ground contact in ar and vr.  IEEE TVCG [Accepted with Minor Revisions], 0(0):0, 2021.
  * [3] J.-g. Ahn, E. Ahn, S. Min, H. Choi, H. Kim, and G. J. Kim.  Size perception of augmented objects by different ar displays.  In C. Stephanidis, ed., HCI International 2019 - Posters, pp. 337–344. Springer International Publishing, Cham, 2019.
  * [4] C. Armbrüster, M. Wolter, T. Kuhlen, W. Spijkers, and B. Fimm.  Depth perception in virtual reality: distance estimations in peri-and extrapersonal space.  Cyberpsychology & Behavior, 11(1):9–15, 2008.
  * [5] R. J. Bailey, C. M. Grimm, and C. Davoli.  The real effect of warm-cool colors.  Report Number: WUCSE-2006-17. All Computer Science and Engineering Research, 2006.
  * [6] M. Berning, D. Kleinert, T. Riedel, and M. Beigl.  A study of depth perception in hand-held augmented reality using autostereoscopic displays.  In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 93–98, 2014.
  * [7] R. Bogacz, E. Brown, J. Moehlis, P. Holmes, and J. D. Cohen.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks.  Psychological review, 113(4):700, 2006.
  * [8] H. H. Bülthoff and H. A. van Veen.  Vision and action in virtual environments: Modern psychophysics in spatial cognition research.  In Vision and attention, pp. 233–252. Springer, 2001.
  * [9] R. Casati.  Methodological issues in the study of the depiction of cast shadows: A case study in the relationships between art and cognition.  The Journal of Aesthetics and Art Criticism, 62(2):163–174, 2004\.
  * [10] R. Casati.  Size from shadow: Some informational paths less traveled.  Ecological Psychology, 28(1):56–63, 2016.
  * [11] R. Casati and P. Cavanagh.  The visual world of shadows.  MIT Press, 2019.
  * [12] P. Cavanagh and Y. G. Leclerc.  Shape from shadows.  Journal of Experimental Psychology: Human Perception and Performance, 15(1):3, 1989.
  * [13] M. A. Cidota, R. M. S. Clifford, S. G. Lukosch, and M. Billinghurst.  Using visual effects to facilitate depth perception for spatial tasks in virtual and augmented reality.  In 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct), pp. 172–177, 2016.
  * [14] C. Diaz, M. Walker, D. A. Szafir, and D. Szafir.  Designing for depth perceptions in augmented reality.  In 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 111–122, Oct 2017. doi: 10 . 1109/ISMAR . 2017 . 28
  * [15] T. Do, J. J. LaViola Jr., and R. McMahan.  The effects of object shape, fidelity, color, and luminance on depth perception in handheld mobile augmented reality.  In 2020 IEEE International Symposium on Mixed and Augmented Reality(ISMAR), 2020.
  * [16] D. Drascic and P. Milgram.  Perceptual issues in augmented reality.  In M. T. Bolas, S. S. Fisher, M. T. Bolas, S. S. Fisher, and J. O. Merritt, eds., Stereoscopic Displays and Virtual Reality Systems III, vol. 2653, pp. 123 – 134. International Society for Optics and Photonics, SPIE, 1996. doi: 10 . 1117/12 . 237425
  * [17] E. Eisemann, U. Assarsson, M. Schwarz, and M. Wimmer.  Casting shadows in real time.  In ACM SIGGRAPH ASIA 2009 Courses, SIGGRAPH ASIA ’09. Association for Computing Machinery, New York, NY, USA, 2009. doi: 10 . 1145/1665817 . 1722963
  * [18] S. Epstein.  The stability of behavior: Ii. implications for psychological research.  American Psychologist, 35(9):790–806, 09 1980.
  * [19] B. Farell and D. G. Pelli.  Psychophysical methods, or how to measure a threshold and why.  Vision research: A practical guide to laboratory methods, 5:129–136, 1999.
  * [20] D. E. Fox and K. I. Joy.  On polyhedral approximations to a sphere.  In Proceedings. Computer Graphics International (Cat. No. 98EX149), pp. 426–432. IEEE, 1998.
  * [21] J. Frankenstein, F. Kessler, and C. Rothkopf.  Applying psychophysics to applied spatial cognition research.  In J. Šķilters, N. S. Newcombe, and D. Uttal, eds., Spatial Cognition XII, pp. 196–216. Springer International Publishing, Cham, 2020.
  * [22] Y. Gao, E. Peillard, J.-M. Normand, G. Moreau, Y. Liu, and Y. Wang.  Influence of virtual objects’ shadows and lighting coherence on distance perception in optical see-through augmented reality.  Journal of the Society for Information Display, 2019.
  * [23] M. Haller, C. Hanl, and J. Diephuis.  Non-photorealistic rendering techniques for motion in computer games.  Comput. Entertain., 2(4):11, Oct. 2004. doi: 10 . 1145/1037851 . 1037869
  * [24] C. R. Harrison and K. M. Robinette.  Caesar: Summary statistics for the adult population (ages 18-65) of the united states of america.  Technical report, Air Force Research Lab Wright-Patterson AFB OH Human Effectiveness Directorate, 2002.
  * [25] J. Hertel and F. Steinicke.  Augmented reality for maritime navigation assistance-egocentric depth perception in large distance outdoor environments.  In 2021 IEEE Virtual Reality and 3D User Interfaces (VR), pp. 122–130. IEEE, 2021.
  * [26] J. C. Ho.  Real-world and virtual-world practices for virtual reality games: Effects on spatial perception and game performance.  Multimodal Technologies and Interaction, 4(1):1, 2020.
  * [27] H. H. Hu, A. A. Gooch, W. B. Thompson, B. E. Smits, J. J. Rieser, and P. Shirley.  Visual cues for imminent object contact in realistic virtual environment.  In IEEE Visualization, pp. 179–185, 2000.
  * [28] R. Ihaka and R. Gentleman.  Development core team (2009). r: A language and environment for statistical computing. r foundation for statistical computing, vienna, austria.  URL http://www. R-project. org.(Accessed 16 October 2009), 1996\.
  * [29] S. Ikeda, Y. Kimura, S. Manabe, A. Kimura, and F. Shibata.  Shadow induction on optical see-through head-mounted displays.  Computers & Graphics, 91:141 – 152, 2020. doi: 10 . 1016/j . cag . 2020 . 07 . 003
  * [30] M. Keo.  Graphical style in video games.  Programme in Information Technology Riihimäki, 2017.
  * [31] D. Kersten, P. Mamassian, and D. C. Knill.  Moving cast shadows induce apparent motion in depth.  Perception, 26(2):171–192, 1997.
  * [32] D. C. Knill, P. Mamassian, and D. Kersten.  Geometry of shadows.  JOSA A, 14(12):3216–3232, 1997.
  * [33] E. Kruijff, J. E. Swan, and S. Feiner.  Perceptual issues in augmented reality revisited.  In 2010 IEEE International Symposium on Mixed and Augmented Reality, pp. 3–12, 2010.
  * [34] R. Lawson and H. H. Bülthoff.  Using morphs of familiar objects to examine how shape discriminability influences view sensitivity.  Perception & psychophysics, 70(5):853–877, 2008.
  * [35] J. Lopez-Moreno, J. Jimenez, S. Hadap, E. Reinhard, K. Anjyo, and D. Gutierrez.  Stylized depiction of images based on depth perception.  In Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering, NPAR ’10, p. 109–118. Association for Computing Machinery, New York, NY, USA, 2010. doi: 10 . 1145/1809939 . 1809952
  * [36] C. Madison, W. Thompson, D. Kersten, P. Shirley, and B. Smits.  Use of interreflection and shadow for surface contact.  Perception & Psychophysics, 63(2):187–194, 2001.
  * [37] P. Mamassian.  Impossible shadows and the shadow correspondence problem.  Perception, 33(11):1279–1290, 2004.
  * [38] P. Mamassian, D. C. Knill, and D. Kersten.  The perception of cast shadows.  Trends in cognitive sciences, 2(8):288–295, 1998.
  * [39] S. Manabe, S. Ikeda, A. Kimura, and F. Shibata.  Casting virtual shadows based on brightness induction for optical see-through displays.  In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 627–628, 2018.
  * [40] S. Manabe, S. Ikeda, A. Kimura, and F. Shibata.  Shadow inducers: Inconspicuous highlights for casting virtual shadows on ost-hmds.  In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1331–1332, March 2019. doi: 10 . 1109/VR . 2019 . 8798049
  * [41] K. Nakayama and S. Shimojo.  Experiencing and perceiving visual surfaces.  Science, 257(5075):1357–1363, 1992.
  * [42] R. Ni, M. L. Braunstein, and G. J. Andersen.  Perception of scene layout from optical contact, shadows, and motion.  Perception, 33(11):1305–1318, 2004.  PMID: 15693673. doi: 10 . 1068/p5288
  * [43] Z. Noh and M. S. Sunar.  A review of shadow techniques in augmented reality.  In 2009 Second International Conference on Machine Vision, pp. 320–324, 2009. doi: 10 . 1109/ICMV . 2009 . 41
  * [44] C. W. Ooi and J. Dingliana.  Colored cast shadows for improved visibility in optical see-through ar.  In SIGGRAPH Asia 2020 Posters, SA ’20. Association for Computing Machinery, New York, NY, USA, 2020. doi: 10 . 1145/3415264 . 3425442
  * [45] K. Ozkan and M. L. Braunstein.  Background surface and horizon effects in the perception of relative size and distance.  Visual cognition, 18(2):229–254, 2010.
  * [46] E. Peillard, F. Argelaguet, J. Normand, A. Lécuyer, and G. Moreau.  Studying exocentric distance perception in optical see-through augmented reality.  In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 115–122, 2019. doi: 10 . 1109/ISMAR . 2019 . 00-13
  * [47] E. Peillard, Y. Itoh, G. Moreau, J. M. Normand, A. Lécuyer, and F. Argelaguet.  Can retinal projection displays improve spatial perception in augmented reality?  In 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 80–89, 2020. doi: 10 . 1109/ISMAR50242 . 2020 . 00028
  * [48] J. Ping, B. H. Thomas, J. Baumeister, J. Guo, D. Weng, and Y. Liu.  Effects of shading model and opacity on depth perception in optical see-through augmented reality.  Journal of the Society for Information Display, 2020.
  * [49] V. Powell.  Visual properties of virtual target objects: implications for reaching and grasping tasks in a virtual reality rehabilitation context.  PhD thesis, University of Portsmouth, 2012.
  * [50] N. Prins et al.  Psychophysics: a practical introduction.  Academic Press, 2016.
  * [51] C. S. Rosales, G. Pointon, H. Adams, J. Stefanucci, S. Creem-Regehr, W. B. Thompson, and B. Bodenheimer.  Distance judgments to on- and off-ground objects in augmented reality.  In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 237–243, 2019.
  * [52] N. Rubin.  The role of junctions in surface completion and contour matching.  Perception, 30(3):339–366, 2001.
  * [53] I. Schuetz, T. S. Murdison, K. J. MacKenzie, and M. Zannoli.  An explanation of fitts’ law-like performance in gaze-based selection tasks using a psychophysics approach.  In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1–13, 2019.
  * [54] G. Singh, S. R. Ellis, and J. E. Swan.  The effect of focal distance, age, and brightness on near-field augmented reality depth matching.  IEEE Transactions on Visualization and Computer Graphics, 26(2):1385–1398, 2020.
  * [55] G. Singh, J. E. Swan, J. A. Jones, and S. R. Ellis.  Depth judgments by reaching and matching in near-field augmented reality.  In 2012 IEEE Virtual Reality Workshops (VRW), pp. 165–166. IEEE, 2012.
  * [56] M. Slater, M. Usoh, and Y. Chrysanthou.  The influence of dynamic shadows on presence in immersive virtual environments.  In Selected Papers of the Eurographics Workshops on Virtual Environments ’95, VE ’95, p. 8–21. Springer-Verlag, Berlin, Heidelberg, 1995\.
  * [57] P. L. Smith and D. R. Little.  Small is beautiful: In defense of the small-n design.  Psychonomic bulletin & review, 25(6):2083–2101, 2018.
  * [58] W. Steptoe, S. Julier, and A. Steed.  Presence and discernability in conventional and non-photorealistic immersive augmented reality.  In 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 213–218, 2014. doi: 10 . 1109/ISMAR . 2014 . 6948430
  * [59] N. Sugano, H. Kato, and K. Tachibana.  The effects of shadow representation of virtual objects in augmented reality.  In Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR ’03, p. 76. IEEE Computer Society, USA, 2003\.
  * [60] J. Sun, W. Stuerzlinger, and D. Shuralyov.  Shift-sliding and depth-pop for 3d positioning.  In Proceedings of the 2016 Symposium on Spatial User Interaction, SUI ’16, p. 69–78. Association for Computing Machinery, New York, NY, USA, 2016. doi: 10 . 1145/2983310 . 2985748
  * [61] J. E. Swan, G. Singh, and S. R. Ellis.  Matching and reaching depth judgments with real and augmented reality targets.  IEEE Transactions on Visualization and Computer Graphics, 21(11):1289–1298, 2015. doi: 10 . 1109/TVCG . 2015 . 2459895
  * [62] W. B. Thompson, P. Shirley, B. Smits, D. J. Kersten, and C. Madison.  Visual glue.  University of Utah Technical Report UUCS-98-007, 1998.
  * [63] K. Vaziri, P. Liu, S. Aseeri, and V. Interrante.  Impact of visual and experiential realism on distance perception in vr using a custom video see-through system.  In Proceedings of the ACM Symposium on Applied Perception, SAP ’17. Association for Computing Machinery, New York, NY, USA, 2017. doi: 10 . 1145/3119881 . 3119892
  * [64] L. Wanger.  The effect of shadow quality on the perception of spatial relationships in computer generated imagery.  In Proceedings of the 1992 symposium on Interactive 3D graphics, pp. 39–42, 1992.
  * [65] A. Yonas, L. T. Goldsmith, and J. L. Hallstrom.  Development of sensitivity to information provided by cast shadows in pictures.  Perception, 7(3):333–341, 1978.