# DiPPeST: Diffusion-based Path Planner for Synthesizing Trajectories
Applied on Quadruped Robots

Maria Stamatopoulou∗, Jianwei Liu∗, and Dimitrios Kanoulas The authors are
with the Department of Computer Science, University College London, Gower
Street, WC1E 6BT, London, UK. {maria.stamatopoulou.21,jianwei.liu.21,
<EMAIL_ADDRESS>contributionThis work was supported by the UKRI
Future Leaders Fellowship [MR/V025333/1] (RoboHike) and the CDT for
Foundational Artificial Intelligence [EP/S021566/1]. For the purpose of Open
Access, the author has applied a CC BY public copyright license to any Author
Accepted Manuscript version arising from this submission.

###### Abstract

We present DiPPeST, a novel image and goal conditioned diffusion-based
trajectory generator for quadrupedal robot path planning. DiPPeST is a zero-
shot adaptation of our previously introduced diffusion-based 2D global
trajectory generator (DiPPeR). The introduced system incorporates a novel
strategy for local real-time path refinements, that is reactive to camera
input, without requiring any further training, image processing, or
environment interpretation techniques. DiPPeST achieves $92\%$ success rate in
obstacle avoidance for nominal environments and an average of $88\%$ success
rate when tested in environments that are up to $3.5$ times more complex in
pixel variation than DiPPeR. A visual-servoing framework is developed to allow
for real-world execution, tested on the quadruped robot, achieving $80\%$
success rate in different environments and showcasing improved behavior than
complex state-of-the-art local planners, in narrow environments.

##  I INTRODUCTION

Mobile robots have been gaining increased popularity due to their multi-
purpose applications, varying from industrial production sites to search-and-
rescue tasks [1]. To successfully complete such tasks, real-time autonomous
navigation in complex environments is required, rendering path planning an
important open challenge [2]. Many works focus on generating global paths to
provide a comprehensive path from a start to a goal point, assuming fully
known and static environments [3, 4] however, these methods cannot adapt in
dynamic or unknown ones. Studies on local planners aim to address this
limitation by utilizing real-time sensory data, enabling robots to make
immediate decisions and adapt their paths in response to dynamic changes in
their surroundings. However, these approaches may sacrifice the optimality of
the paths due to their myopic decision-making process. Hence, the ideal
solution lies on the approach where the global planner sets an initial course
path, and the local planner refines this path in real-time, ensuring both
efficiency and adaptability. Learning from demonstration methods using image-
conditioned Diffusion has demonstrated promising results in trajectory
planning, primarily in the domain of manipulators [5, 6, 7], with few works
extending these principles to mobile robot path planning [8, 3]. Diffusion
policies iteratively calculate the action-score gradient based on visual
observations, allowing for multi-modal action distribution representation,
scalability to larger output spaces, and training stability while preserving
expressive distribution capabilities [5]. However, training a diffusion model
is computationally expensive and timely, requiring large datasets, which makes
their adoption more challenging than other methods.

In this paper, we present a novel approach to path planning by extending our
global Diffusion-based 2D path planner, DiPPeR [3]), through a zero-shot
adaptation that integrates an adaptive local path planner with no further
training or fine-tuning. We introduce a robot path planning framework
utilizing RGB data, complemented by a visual servoing pipeline for converting
planned paths into actionable robot movements. We test our method on a real-
world quadruped robot and compare the performance with state-of-the-art local
planners both in static and dynamic environments. Our primary contributions
include:

  1. 1.

A novel image-conditioned diffusion path planner for mobile robots for real-
time path refinements.

  2. 2.

A zero-shot adaptation of DiPPeR to incorporate local path planning without
re-training, fine-tuning, image pre-processing, or environment contextual or
geometric information (e.g., semantics, obstacle recognition).

  3. 3.

A visual servoing real-world deployment stack.

The remainder of the paper is structured as follows. In Sec. II, we briefly
introduce literature in local path planning relevant to our proposed method.
In Sec. III, we provide the necessary background knowledge, including a brief
description of our previously introduced system, DiPPeR. In Sec. IV, we
describe our proposed method, with our experimental results presented in Sec.
V. Finally, in Sec. VI, we summarize the results and conclude with some future
work.

##  II RELATED WORK

Local path planning and visual servoing are extensively researched areas, as
they enable mobile robots to make real-time decisions and adapt to dynamic
environments efficiently. We briefly review these methods from both
traditional and diffusion-based perspectives.

###  II-A Local Path Planning and Visual Servoing

Traditional local path planning methods typically represent the environment
using geometric primitives to identify traversal areas and obstacles [9, 10,
11]. However, these approaches often lack context awareness, characterized as
the algorithm’s inability to understand the environment beyond physical
obstacles and free spaces and assume a static environment. This leads to
limited environment interpretation and exponentially increasing computational
demands for more complex environments [12]. One commonly used variation of
geometric planners is the Dynamic Window Approach (DWA), where the robot’s
velocities are sampled to generate a trajectory optimized based on obstacle
avoidance, reaching the target, and maintaining base velocity [13, 14, 15].
Chang et al. [16] improved on the original DWA evaluation functions by
employing Q-learning to adaptively learn the parameters, enhancing global
navigation performance and efficiency. However, the method relies on the
balance between effectiveness, speed, training quality and the complexity of
the environment, which might limit its effectiveness in unpredictable or
complex scenarios. More recent works attribute contextual information to gain
a better understanding of the environment [17]. Roth et al. [12] introduced a
learned local path planning approach, utilizing both geometric and semantic
information to identify terrain traversability and obstacles effectively.
Crespo et al. [18] proposed a path-planning approach employing co-safe Linear
Temporal Logic over perception-based atomic predicates, enabling the
generation of uncertain semantic maps for navigation. While semantic-driven
sampling-based algorithms offer promising results, they heavily rely on the
accuracy of the semantic map and the quality of sensor data, and they suffer
from increased computational complexity under severe perceptual uncertainty.
Our method aims to address the limitations of both semantic and geometric
methods by not relying on any environment representation other than visual
input and avoiding scaling computational complexity, as demonstrated in our
previous work [3].

Visual servoing provides a means of adaptively navigating based on visual cues
[19]. Traditional visual servoing techniques have been integrated with visual
cognition towards path planning. For instance, closer to our method,
Rodziewicz-Bielewicz et al. [20] utilize YOLO and real-time image processing
from a central camera to determine the robot positions and orientations for
precise trajectory planning and execution in indoor environments. However, the
method is constrained by specific environmental conditions and the
generalization capabilities of YOLO in complex environments. Zhu et al. [21]
deploy a visibility-based search tree with a greedy search strategy that
segments results based on field-of-view (FOV) constraints, ensuring the
feature points remain within the camera’s view. The approach assumes
consistent visibility and the need to keep feature points within the FOV at
all times, which may not be feasible in highly dynamic environments or in
situations with occlusions, potentially leading to servo failures. Our
methodology relaxes these dependencies, facilitating global and local path
planning interaction. This ensures alignment with the overarching objective
while simultaneously allowing for local refinements.

###  II-B Diffusion for Path Planning

Diffusion methods have gained popularity in the domain of path planning, with
several works demonstrating promising results. Hong et al. [22] deploy
diffusion maps to find local paths for reaching a goal while avoiding
collisions with dynamic obstacles simultaneously, by computing transition
probabilities between grid points. Liu et al. [3] introduce an image and goal-
conditioned diffusion-based global path planner, trained using a CNN,
showcasing great generalization and constant inference speed for various map
types. Sridhar et al. [8] present a unified robotic navigation diffusion
policy that handles task-oriented navigation and task-agnostic exploration in
unseen environments, exhibiting significant performance improvements and
computational efficiency compared to existing approaches. We aim to leverage
the advancement made in this diffusion-based path planning realm to develop
our trajectory generator. While all these works require training of diffusion
models, which can be computationally expensive and require large training
datasets, our proposed approach does not require further training or fine-
tuning of a diffusion policy.

##  III Preliminaries

(a)

(b)

(c)

Figure 2: Examples of DiPPeR’s training dataset: $100\times 100$ random
solvable maps with examples of end-to-end trajectories, generated through
$A^{*}$.

Path planning involves computing a trajectory for a robot to follow, that is,
optimizing for properties such as finding a collision-free, shortest, and
smoothest route between start and goal positions. As elaborated in Sec. II,
there are several approaches to solving path planning. For relevance to our
method, we expand on our previous work in probabilistic diffusion-based path
planning (DiPPeR [3]), which serves as a baseline for the developed DiPPeST
system.

Figure 3: DiPPeST process of generating the local path and correcting global
path failure. DiPPeST takes an input camera frame (A) and utilizes DiPPeR to
generate a global path (B) and the waypoints. These are tracked at each input
frame (C), and the optimal waypoint is selected. At each frame a, ROI is
created around the current position and the waypoint and the optimal local
goal position (D) within the ROI is selected (red cross) to avoid assigning
the goal close to the obstacle due to (C) waypoint suboptimal positioning.
DiPPeR is then used to generate a path to the goal position (E). Low
resolution images are displayed representing the exact output of the diffusion
model.

DiPPeR is an image-based diffusion model designed to plan global paths from a
starting to a goal position, given a 2D map in which space is classified as
free or occupied. Trained on images of random mazes and ground truth
trajectories generated by $A^{*}$ path planning, DiPPeR is able to generate
paths via a Denoising Diffusion Probabilistic Model (DDPM) [23] – a set of 2D
path trajectories $A_{t}^{k}$ with added $\epsilon^{k}$ Gaussian noise and the
map image observation $O$ are given as input, $K$ denoising iterations are
performed via gradient descent, and a noise-free path trajectory $A_{t}^{0}$
is produced:

$A_{t}^{k-1}=\alpha(A_{t}^{k}-\gamma\epsilon_{\theta}(O,A_{t}^{k},k)+\mathcal{N}(0,\sigma^{2}I)),$
(1)

where $\epsilon_{\theta}$ is the noise prediction network (1D temporal CNN),
$\alpha,\gamma,\sigma$ and the noise $\epsilon^{k}$ define the scheduling
learning rate.

During training, Gaussian noise $\epsilon^{k}$ is iteratively added to the
clean trajectory $A_{t}^{0}$. The noise predictor $\epsilon_{\theta}$ is
trained using gradient decent of the mean squared error
$\mathcal{L}=MSE(\epsilon^{k},\epsilon_{\theta}(O,A_{t}^{0}+\epsilon^{k},k))$.
DiPPeR achieves an average speedup of $23\times$ compared to state-of-the-art
global path planning methods [3]. However, the method faces certain
constraints. Initially, the path step number required for the denoising
process must be empirically determined and manually selected for each
experiment. This process is sub-optimal as it requires trial and error to
determine the optimal length. Additionally, due to the inherent global
planning nature of the approach, the goal position is manually chosen and
remains unaltered throughout the experiment. This fixed setting proves
suboptimal in scenarios requiring the dynamism of a local planner or in cases
necessitating refinements.

##  IV METHOD

In this work, we utilize DiPPeR for global path and end-to-end trajectory
generation while addressing its existing limitations. We further modify the
framework to introduce DiPPeST, a diffusion-based local path planner for
adaptive real-time local refinements leveraging visual input. Additionally, we
incorporate a customized visual servoing strategy to ensure efficient
execution of real-world paths. DiPPeST is a zero-shot adaptation of DiPPeR,
requiring no further training or fine-turning while introducing a novel local
planning framework.

###  IV-A Global Planner

Firstly, our method involves the generation of a global path plan based on RGB
input camera data. This path is generated via DiPPeR, utilizing the first
captured frame from the robot’s onboard camera which is positioned at it’s
front, facing towards the goal. The model generates a feasible path that
avoids all obstacles while aiming to reach the set goal, by maintaining local
uniformity in trajectory generation and traversing through pixels with
consistent intensity levels. Unlike DiPPeR’s original method, a map of the
environment is not required, and planning happens directly through the RGB
frame of the camera. Despite significant variations in the input images,
DiPPeR demonstrates zero-shot adaptation to the robot’s perspective and RGB
image inputs despite being initially only trained on top-down grayscale maze
images (Fig. 2). This transferability likely stems from the models’ property
to prioritize the generated trajectory’s consistency and goal-reaching
condition and its ability to learn color gradient-based features through
ResNet, which are transferable from grayscale to RBG. Further exploration of
the model’s generalization capabilities is provided in Sec. V-A (also
demonstrated in Fig. 6).

###  IV-B Local Planner

Given the generated global path, frame-to-frame local path adjustments are
required to better navigate around obstacles and adapt to dynamic environments
as the robot progresses toward its goal. To address this, we introduce
DiPPeST, which takes an RGB image as input and outputs a 2D local goal
position along with a 2D feasible collision-free trajectory to be followed,
all within the camera frame. The method involves tracking the global path
waypoints and the global goal position within individual local camera frames.
At each frame, a strategy is devised to select the optimal waypoint, and a
region of interest (ROI) is created from the robot’s current position to the
selected waypoint position. Processing within this ROI determines the optimal
local goal position for the robot to reach in the current frame and DiPPeR
generates the feasible path to the goal. An overview of DiPPeST is illustrated
in Fig. 3.

####  IV-B1 Waypoint Tracking

We utilize the Lucas-Kanade optical flow estimation with the pyramidal
approach for tracking the global path features across frames [24]. The method
assumes that optical flow remains constant within a local neighborhood of
pixels. Let $I_{x}$ and $I_{y}$ denote the gradients of the image in the x and
y directions, respectively, and let $I_{t}$ represent the temporal gradient
between consecutive frames. The linear optical flow vector $(u,v)$ is given
by:

$I_{x}\cdot u+I_{y}\cdot v=-I_{t}$ (2)

This can be estimated by solving the system of linear equations for each pixel
via the pyramidal approach, i.e., building an image pyramid, where each level
is a down-sampled version of the previous one. The optical flow is estimated
at each level iteratively. The waypoints are then tracked by individual frames
as seen in Fig. 3-C.

####  IV-B2 Intermediate Waypoint Selection

At each frame, the optimal waypoint to follow is determined based on two
conditions: (a) the half-point distance $dist$ from the current position to
the furthest away waypoint and (b) the highest similarity $sim$ to the
direction vector dir of the goal position. This ensures that for each frame,
the planned trajectory will aim toward the goal position while allowing for a
significant look-ahead distance for refinements. For condition (a) the
Euclidean distance $dist$ is calculated between the current position
$\textbf{p}_{curr}$ and all $n$ waypoint positions $\textbf{p}_{way}$ as:

$dist=\sqrt{\sum_{i=1}^{n}(\textbf{p}_{curr}-\textbf{p}_{way}[i])^{2}}.$ (3)

The median distance is then selected as the optimal waypoint position.
Choosing the median distance waypoint helps avoid myopic local decisions that
may be sub-optimal on a global scale. For condition (b), the dot product
between the current position $\textbf{p}_{curr}$ and all $n$ waypoint
positions $\textbf{p}_{way}$ is calculated as follows:

$\displaystyle\textbf{dir}=\textbf{p}_{curr}-\textbf{p}_{way}$ (4)
$\displaystyle\textbf{p}_{norm}=\frac{\textbf{dir}}{\|\text{{dir}}\|_{2}},~{}~{}~{}~{}\textbf{g}_{norm}=\frac{\textbf{g}}{\|\textbf{g}\|_{2}}$
(5) $\displaystyle sim=\textbf{p}_{norm}\cdot\textbf{g}_{norm}^{T}$ (6)

where $\textbf{p}_{norm}$ is the norm direction, g represents the goal
position, $\textbf{g}_{norm}$ is the norm goal, and $\cdot$ the dot product.
Finally, the next waypoint $i$ ($\textbf{p}_{way}[i]$) with the median
distance $dist$ and highest similarity $sim$ is chosen as the next
intermediate goal.

####  IV-B3 ROI Processing and Goal Selection

To ensure that the local goal is set optimally within a traversable region and
to account for the possibility of a waypoint being incorrectly positioned
within an obstacle, we further condition its selection. A ROI is defined in
the camera frame, spanning between the current robot position and the selected
intermediate waypoint. This ROI segment is utilized to assess the variation in
pixel intensities. Let $px_{std}$ be the standard deviation of pixel values in
each color channel:

$px_{std}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}({ROI}_{i}-\overline{ROI})^{2}}$ (7)

where $N$ denotes the number of pixels, ${ROI}_{i}$ is the pixel value at
position $i$ and $\overline{ROI}$ is the mean pixel value in each color
channel. The normalized standard deviation ($px_{var}$) is then obtained by
dividing $px_{std}$ by the maximum possible pixel value, typically $255$ for
$8$-bit images. This serves as a quantitative measure of intensity variation
within the ROI. The value of $px_{var}$ is used in Sec. IV-B4 to determine the
number of path steps during the denoising process.

To determine the local optimal goal position, the most common pixel intensity
$px_{int}$ within the ROI is obtained by calculating the mean pixel intensity
$\overline{px}$ within the ROI. To maintain consistency across frames, the
previously selected pixel intensity $px_{pint}$ serves as a prior and is used
as a tolerance value $\tau$ within which the new $px_{int}$ should be set.

$px_{\text{int}}=\text{mode}\left(\\{px\mid px\in
ROI,|\overline{px}-px_{\text{pint}}|\leq\tau\\}\right)$ (8)

This prevents setting the local goal within an obstacle in a ROI that is
predominantly surrounded by obstacles while also maintaining local consistency
for the diffusion model and allowing the flexibility to navigate in
traversable regions of varying intensities. The optimal pixel $px^{*}$ needs
to have intensity $px_{int}$ while also being the closest to the waypoint
position, ensuring that the selected goal does not deviate significantly from
the global path. The $px^{*}$ is chosen as the next local goal, and its
coordinates are scaled back to the entire image frame. An example of this
correction is visualized by the red cross in Fig. 3-D.

####  IV-B4 Local Path Generation

After the new goal position is selected, DiPPeR is utilized to generate the
feasible path, involving path step selection and path correction. Firstly, the
path step number $ps$ is selected through a combination of the Euclidean
distance between the current position to the local goal position, $px_{d}$ and
the $px_{var}$ within the ROI following:

$ps=px_{d}+\alpha e^{(10\times px_{var})}$ (9)

This method addresses the manual path step number selection of DiPPeR and
allows it to increase exponentially as the variation of pixel intensity
increases, thereby providing adaptation space scalable to obstacle density. We
set $\alpha$, the scaling coefficient, equal to $4$, to agree with the input
dimensions of the network. In cases where the local trajectory fails to avoid
obstacles, a mitigating method is implemented to allow for re-planning,
ensuring that the robot follows only feasible trajectories. The generated
local trajectory is evaluated within the ROI to ensure that it does not
traverse pixels whose intensity falls outside of the tolerance range compared
to the local goal position pixel intensity.

###  IV-C Path Following

To execute the local paths generated by DiPPeST in the real world, the
trajectory coordinates need to be de-projected from 2D image coordinates
$(x,y)$ obtained by the camera mounted on the robot’s head into 3D world
coordinates.

Figure 4: Visual servoing framework for real-world robot path execution.
DiPPeST 2D trajectories within RGB frame are de-projected into 3D world
coordinates. The path follower translates the trajectories into velocity
commands, and the robot module into motor commands to drive the robot towards
the desired path.

Given the depth $d$ from the depth reading for pixel coordinate $(i,j)$ of the
camera:

$\text{Deproj}(i,j,d)=\left(d\cdot
U_{\text{Model}}\left(\frac{(i,j)-P}{F}\right),d\right)$ (10)

where $P=(p_{x},p_{y})$ is the principal point $F=(f_{x},f_{y})$ the focal
length, both intrinsic parameters of the camera, $U_{\text{Model}}$ represents
the model for lens distortion, and $d$ is the depth obtained directly from the
depth image. The resulting de-projected 3D local path is then executed by a
path follower module [11], which generates velocity commands from the input
path. These commands utilize odometry readings to adjust the robot’s movement
in real time, ensuring accurate adherence to the planned trajectory. Finally,
the robot driver module translates the velocity commands into motor commands,
considering the robot’s joint states to drive the hardware toward the desired
state as defined by the 3D trajectory. A twist-command correction is
implemented to ensure the robot’s view does not significantly divert from the
global goal position. An overview of the framework is depicted in Fig. 4.

##  V RESULTS

###  V-A Performance Evaluation

We conduct experiments to assess the robustness of DiPPeST against varying
input conditions and validate its performance and generalization capabilities
in out-of-distribution cases. These experiments are performed using the RGB
frames obtained through a RealSense D435i camera. We focus on a) variation of
the traversable region and obstacle pixel intensity, b) change of input image
size, and c) variation of camera point-of-view (PoV), as seen in Fig.6. We
quantify the effect as the average percentage of successful collision
avoidance attempts over $10$ trials for each case. DiPPeST is executed on a
computer with an Nvidia RTX 3090 GPU and has a constant inference time of
$0.42$s per frame, as it depends on DiPPeR’s denoising time of $0.4$s.

We examine the impact of varying the color of the traversable regions and
obstacles in DiPPeST trajectory generation performance, as DiPPeR was only
trained on black and white images (Fig. 2), with white representing the
feasible path and black the obstacles. The color of the traversable region is
varied by adding multi-color pads, introducing varying pixel intensities
within the ROI. The mean variance values are converted into percentage values
to standardize comparisons. DiPPeST’s performance is quantified by assessing
the correlation between changing the percentage ROI pixel variance and the
success rate. DiPPeST achieves a mean success rate of $85\%$ for generating
successful trajectories, showing good generalization capabilities when
considering the training dataset. It successfully avoids obstacles with a
$92\%$ success rate for an ROI variance of less than $30\%$, representing the
most common environmental scenario. Increasing pixel intensity variation
results in a $19\%$ decline for the edge case of ROI variance exceeding
$80\%$, indicating a slight negative trend. This trend could potentially be
corrected by introducing colorful maps into the training dataset.

Figure 5: The effect of variation of a) floor and obstacle color, b) input
image size, and c) camera PoV, over DiPPeST $\%$ success rate.

(a)

(b)

(c)

Figure 6: Illustration of DiPPeST generated trajectories for cases of
variation of a) of camera PoV: top-down, human-eye, robot-eye, b) input image
size: training dataset, iPhone 11, RealSense D435i, and c) edge cases. Low
resolution frames correspond to the compressed images as received for the
denoising process.

We further evaluate the impact of image PoV on the success rate, considering
that DiPPeR is trained on maps with a top-down PoV. DiPPeST should generalize
to input images of varying PoV, reflecting variations in camera angle and
heights in real-world scenarios. To assess this, we conduct the experiments on
recorded RGB frames obtained by changing the angle and height of the position
of the camera, as depicted in Fig.6(a). DiPPeST generates successful
trajectories with an average $87\%$ success rate for all PoV variation
experiments. There is a slight negative trend as the PoV angle changes from
top-down to robot-level with an overall $13\%$ decline in success rate, which
could be corrected by refining the training dataset.

We also validate DiPPeST’s performance for image inputs of variable sizes, as
DiPPeR is trained on maps of the same size. DiPPeST receives camera input,
which may have a variable FoV based on hardware specifications. This concern
is based on the compression ratio variations occurring when the image input is
passed on the encoder network. We examine images of varying sizes, including
those of the same size as the training data set $[100,100,3]$, images taken
from an iPhone 11 camera $[3264,2448,3]$ and a RealSense camera
$[720,1280,3]$, as seen in Fig.6(b). However, there is no significant effect
on the success rate, with all scenarios displaying a $92\%$ generalization
success rate.

Overall, DiPPeST achieved an average success rate of $92\%$ for standard
environments, surpassing DiPPeR’s validation performance. Additionally, it
achieves an overall generalization success rate of $88\%$. These results are
summarized in Fig. 5.

We also investigate the edge cases where the obstacles are similar in pixel
intensity to the traversable region. DiPPeST’s trajectory generation
capabilities for these cases are shown in Fig. 6(c). In the first subfigure of
Fig. 6(c), the obstacle and the floor have an average pixel intensity
difference of $26\%$ for all three channels, and on the second subfigure, a
difference of $28\%$. Given that the intensity difference of DiPPeR validation
maps is $97\%$, DiPPeST showcases a $73\%$ superiority in generalization
capabilities with an improvement factor of $3.5$.

###  V-B Real-World Evaluation

For real-world evaluation, the Unitree Go1 robot is used with DiPPeST, taking
as input images from an Intel RealSense D435i camera mounted on the front at
an angle of $10$ degree depression. For all experiments, the global plan is
generated from the first frame (Sec. IV-A) while the robot remains stationary.
We test DiPPeST’s performance for a) static environments and b) dynamic
environments by measuring the rate of successful attempts in avoiding
obstacles and reaching the goal position across all attempts. For static
environment experiments, we test our method in lab conditions by constructing
a course with stationary obstacles. For dynamic environments, walking robots
are added within the scene to introduce dynamic obstacles. Each experiment is
performed for $3$ different instances of each environment, with increasing
obstacle density and decreasing obstacle distance to showcase avoidance in
narrow environments. The results are presented in Table I.

Environment | sCA | sGR | dCA | dGR | %mSR
---|---|---|---|---|---
DiPPeST | 85 | 79 | 78 | 79 | 80.3
IPlanner | 76 | 82 | 67 | 78 | 75.6
NoMad | 83 | 71 | 83 | 73 | 77.5
TABLE I: This table presents DiPPeST real-world Collision Avoidance (CA) and
Goal Reaching (GR) mean % success rates (%mSR), compared to other SOTA
methods.

DiPPeST achieves an overall success rate of 80% for all environments, showing
improved behavior against IPlanner, a geometric-based local planner and NoMad,
a diffusion-based local planner. Both methods require training for optimal
planning as opposed to DiPPeST. IPlanner displayed capabilities in reaching
the goal, however it often stumbled upon obstacles while doing so. On the
other hand, NoMaD demonstrates successful obstacle avoidance in both dynamic
and static environments, but it would get stuck when reaching too close to a
non-traversable area. Both SOTA methods avoid the narrow passage and redirect
following a safer and less efficient path. An example of an executed DiPPeST
path is depicted in Fig.LABEL:fig:localpath. DiPPeST’s failure case in goal
reaching is attributed to the requirement that the global goal should not
significantly deviate from the view of each image frame, as well as
restrictions on re-planning speed, which might be limiting if the speed of the
dynamic obstacles is high.

##  VI CONCLUSION

In this work, we introduced DiPPeST, a novel image-guided diffusion-based
method for synthesizing both global and local paths for mobile robots. DiPPeST
achieves a $92\%$ success rate for simple environments and an average of
$88\%$ success rate in generalization and edge case handling for environments
up to 3.5 times harder than DiPPeR. We implement visual-servoing to allow
real-robot execution of the novel local planning framework, achieving $80\%$
success rate in different environments and showcasing better behavior than two
SOTA planners in a narrow passage experimental set-up. We identify that re-
planning speed depends on inference time, which we aim to speed up by
improving the denoising process. Additionally, goal-reaching is limited by the
requirement of the global goal being visible within each frame, which we plan
to address through the utilization of memory mechanism in the model or the use
of Visual Odometry to better track features beyond a single frame. Our next
steps focus on incorporating the kinodynamic properties of the robot during
the trajectory generation process to achieve feasible real-world plans, which
will enable the planner to navigate more complex environments with greater
accuracy and efficiency.

## References

  * [1] N. Kottege, L. Sentis, and D. Kanoulas, “Editorial: Towards real-world deployment of legged robots,” _Frontiers in Robotics and AI_ , vol. 8, 2022.
  * [2] K. Cai, C. Wang, J. Cheng, C. W. De Silva, and M. Q.-H. Meng, “Mobile Robot Path Planning in Dynamic Environments: A Survey,” _arXiv preprint arXiv:2006.14195_ , 2020.
  * [3] J. Liu, M. Stamatopoulou, and D. Kanoulas, “Dipper: Diffusion-based 2d path planner applied on legged robots,” in _IEEE International Conference on Robotics and Automation (ICRA)_ , 2024.
  * [4] J. Liu, S. Lyu, D. Hadjivelichkov, V. Modugno, and D. Kanoulas, “ViT-A*: Legged Robot Path Planning using Vision Transformer A,” in _IEEE-RAS 22nd International Conference on Humanoid Robots (Humanoids)_ , 2023, pp. 1–6.
  * [5] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion,” in _Robotics: Science and Systems (RSS)_ , 2023.
  * [6] J. Carvalho, A. T. Le, M. Baierl, D. Koert, and J. Peters, “Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Models,” _arXiv preprint arXiv:2308.01557_ , 2023.
  * [7] M. Janner, Y. Du, J. Tenenbaum, and S. Levine, “Planning with Diffusion for Flexible Behavior Synthesis,” in _International Conference on Machine Learning_ , 2022.
  * [8] A. Sridhar, D. Shah, C. Glossop, and S. Levine, “Nomad: Goal masked diffusion policies for navigation and exploration,” 2023.
  * [9] D. Kanoulas, N. G. Tsagarakis, and M. Vona, “Curved Patch Mapping and Tracking for Irregular Terrain Modeling: Application to Bipedal Robot Foot Placement,” _Robotics and Autonomous Systems_ , vol. 119, pp. 13–30, 2019.
  * [10] R. Saeed, D. R. Recupero, and P. Remagnino, “A boundary node method for path planning of mobile robots,” _Robotics and Autonomous Systems_ , vol. 123, p. 103320, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889018307310
  * [11] F. Yang, C. Wang, C. Cadena, and M. Hutter, “iplanner: Imperative path planning,” 2023.
  * [12] P. Roth, J. Nubert, F. Yang, M. Mittal, and M. Hutter, “Viplanner: Visual semantic imperative learning for local navigation,” 2023.
  * [13] D. H. Lee, S. S. Lee, C. K. Ahn, P. Shi, and C.-C. Lim, “Finite distribution estimation-based dynamic window approach to reliable obstacle avoidance of mobile robot,” _IEEE Transactions on Industrial Electronics_ , vol. 68, no. 10, pp. 9998–10 006, 2021.
  * [14] M. Missura and M. Bennewitz, “Predictive collision avoidance for the dynamic window approach,” in _2019 International Conference on Robotics and Automation (ICRA)_ , 2019, pp. 8620–8626.
  * [15] E. J. Molinos, Ángel Llamazares, and M. Ocaña, “Dynamic window based approaches for avoiding obstacles in moving,” _Robotics and Autonomous Systems_ , vol. 118, pp. 112–130, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0921889018309746
  * [16] L. Chang, L. Shan, C. Jiang, and Y. Dai, “Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment,” _Autonomous Robots_ , vol. 45, no. 1, pp. 51–76, 2021. [Online]. Available: https://doi.org/10.1007/s10514-020-09947-4
  * [17] Y. Kantaros, S. Kalluraya, Q. Jin, and G. J. Pappas, “Perception-based temporal logic planning in uncertain semantic maps,” _IEEE Transactions on Robotics_ , vol. 38, no. 4, pp. 2536–2556, 2022.
  * [18] J. Crespo, J. Castillo, O. Mozos, and R. Barber, “Semantic information for robot navigation: A survey,” _Applied Sciences_ , vol. 10, p. 497, 2020. [Online]. Available: https://doi.org/10.3390/app10020497
  * [19] J. K. Johnson, “Visual Servoing for Mobile Ground Navigation,” in _88th IEEE Vehicular Technology Conference, VTC Fall 2018, Chicago, IL, USA, August 27-30, 2018_ , 2018, pp. 1–5.
  * [20] J. Rodziewicz-Bielewicz and M. Korzeń, “Vision-based mobile robots control along a given trajectory,” in _Artificial Intelligence and Soft Computing_ , L. Rutkowski, R. Scherer, M. Korytkowski, W. Pedrycz, R. Tadeusiewicz, and J. M. Zurada, Eds. Cham: Springer Nature Switzerland, 2023, pp. 69–77.
  * [21] Z. Zhu, R. Wang, and X. Zhang, “Visible rrt*: Asymptotically optimal random search tree for visual servo tasks with the fov constraint,” in _2023 42nd Chinese Control Conference (CCC)_ , 2023, pp. 4633–4638.
  * [22] S. Hong, J. Lu, and D. P. Filev, “Dynamic Diffusion Maps-based Path Planning for Real-time Collision Avoidance of Mobile Robots,” in _IEEE Intelligent Vehicles Symposium (IV)_ , 2018, pp. 2224–2229.
  * [23] H. Ali, S. Murad, and Z. Shah, “Spot the Fake Lungs: Generating Synthetic Medical Images Using Neural Diffusion Models,” in _Artificial Intelligence and Cognitive Science_ , L. Longo and R. O’Reilly, Eds. Cham: Springer Nature Switzerland, 2023, pp. 32–39.
  * [24] N. Sharmin and R. Brad, “Optimal filter estimation for lucas-kanade optical flow,” _Sensors (Basel)_ , 2012.


# Fairness via AI: Bias Reduction in Medical Information

Shiri Dori-Hacohen<EMAIL_ADDRESS>1234-5678-9012 University of Connecticut
& AuCoDeUSA ,  Roberto Montenegro  Seattle Children’s HospitalUSA ,  Fabricio
Murai  Universidade Federal de Minas GeraisBrazil ,  Scott A. Hale  MeedanUSA
,  Keen Sung  AuCoDeUSA ,  Michela Blain  University of Washington School of
MedicineUSA and  Jennifer Edwards-Johnson  Michigan State University College
of Human MedicineUSA

fairness in AI, health misinformation, bias reduction

††copyright: none

## 1 Introduction

Most Fairness in AI research focuses on exposing biases in AI systems. A
broader lens on fairness reveals that AI can serve a greater aspiration:
rooting out societal inequities from their source. Specifically, we focus on
inequities in health information, and aim to reduce bias in that domain using
AI. The AI algorithms under the hood of search engines and social media, many
of which are based on recommender systems, have an outsized impact on the
quality of medical and health information online. Therefore, embedding bias
detection and reduction into these recommender systems serving up medical and
health content online could have an outsized positive impact on patient
outcomes and wellbeing.

In this position paper, we offer the following contributions: (1) we propose a
novel framework of Fairness via AI, inspired by insights from medical
education, sociology and antiracism; (2) we define a new term, bisinformation,
which is related to, but distinct from, misinformation, and encourage
researchers to study it; (3) we propose using AI to study, detect and mitigate
biased, harmful, and/or false health information that disproportionately hurts
minority groups in society; and (4) we suggest several pillars and pose
several open problems in order to seed inquiry in this new space. While part
(3) of this work specifically focuses on the health domain, the fundamental
computer science advances and contributions stemming from research efforts in
bias reduction and Fairness via AI have broad implications in all areas of
society.

## 2 Fairness via AI

The vast majority of Fairness in AI work focuses on exposing the bias in AI
system, in order to showcase where the AI system is biased. However, AI
systems will continue to be biased so long as the data they are receiving are
biased, according to “Bias In, Bias Out” principle (Mayson, 2018). So long as
significant structural inequalities exist in the real world, AI systems will
continue to replicate and exacerbate them. The dominant Fairness in AI
approach, then, risks engaging in a Sisyphean task of minimizing bias in AI,
with attempts to debias AI datasets, models and algorithms continually needing
to be “fixed” as they learn biased outcomes and are bound to hit a ceiling of
fairness: that of real world settings that are inherently biased. Under this
approach, our highest aspiration in designing AI systems seems to be one of
avoidance: tweaking our models to refrain from adding to society’s ills and
inequities.

Rooted in insights from medical education, sociology, and antiracism, we offer
a broader lens on fairness, revealing that AI can serve a far greater
aspiration: enabling important restorative work and rooting out societal
inequities from their source, with a deeper and more meaningful impact. In
this position paper, we therefore reverse the traditional direction of
fairness: rather than aiming to achieve Fairness in or of AI, we propose
focusing on Fairness via AI. With this approach, one can use AI to study,
detect, mitigate and remedy situations that are inherently unequal, unjust and
unfair in society. With this ambitious yet grounded approach, our potential
impact is unbounded, and can accelerate progress towards a more fair, equal
and just world. In other words, we can use AI to thoughtfully, carefully and
ethically debias the world, rather than simply trying to debias AI.
Specifically, the AI algorithms under the hood of search engines and social
media, many of which are based on recommender systems, have an outsized impact
on the quality of information available online. Therefore, embedding bias
detection and reduction directly into these recommender systems could have an
outsized positive impact on the information ecosystem.

## 3 Bisinformation

We coin the term bisinformation to represent biased information, referring to
a unique and challenging aspect of the information landscape. We are
particularly interested in health bisinformation, where bias and language
misuse have a detrimental impact on patient outcomes, though the term can
easily apply in any field. Bisinformation may overlap with, but is not
identical to, misinformation. The use of biased language or inappropriate
social identifiers in a medical context, for example, can be harmful even if
strictly true - consider the case of referring to the prevalence of an illness
in a racial category without contextualizing it in Social or Structural
Determinants of Health (SSDoH), such as systemic racism or income inequities
(Metzl and Roberts, 2019). On the other hand, certain types of bisinformation
are, in fact, also a form of misinformation, such as the long-debunked Salt
Gene Hypothesis (Pollock, 2012).

To the best of our knowledge, no studies have computationally studied health
bisinformation at a large scale.

Medical bisinformation (and misinformation). As an illustration of societal
inequities in dire need of the Fairness via AI approach, consider the field of
medicine and medical education. The field is marred by a long and painful
history of overt and covert forms of social injustice, bias, and racism, as
illustrated by the American Medical Association’s recent pledge to take action
to confront systemic racism (Saini, 2019; Madara, 2020). Studies continue to
demonstrate that physicians possess implicit biases in a number of different
areas such as race/ethnicity, gender, sex, age, weight, substance use and
mental illness (FitzGerald and Hurst, 2017). This comes into play
significantly in medical institutions, which continue to teach biased medicine
in preclinical years (see, e.g. Tsai et al., 2016). Many educators, for
example, continue to inappropriately use race as a proxy for genetics or
ancestry, or even as a “risk factor” for numerous health outcomes often
erroneously associated with race while ignoring SSDoH (Ali-Khan et al., 2011;
Hunt et al., 2013; Acquaviva and Mintz, 2010; Karani et al., 2017; Metzl and
Roberts, 2019). Many educators continue to inappropriately use gender and sex
terms and perpetuate the idea that sex and gender are binary and stagnant
(versus fluid). Likewise, most medical educators are unaware of the numerous
biases in the types of images they use in their lectures or assessment
materials as well (Doja et al., 2016; Fallin-Bennett, 2015; Lempp and Seale,
2004). By equating social identifiers to biology without social or structural
context, medical educators are unknowingly perpetuating a curriculum that can
have an adverse effect on health outcomes (Martimianakis et al., 2015; Braun
et al., 2007). Bias reduction in curricular and assessment content is critical
for educating future physicians in accurate evidence-based medicine (Le et
al., 2018; Ripp and Braun, 2017), but is a manual, costly and time consuming
effort. SOTA AI and NLP approaches can be used to scale up these efforts
significantly111Montenegro, Dori-Hacohen and Sung have recently been awarded a
grant from the National Board of Medical Examiners’ Stemmler Fund to reduce
bias in medical curricular content using NLP and ML approaches..

Naturally, bisinformation and misinformation that persist in the medical
establishment, are also disseminated and extensively present in online medical
resources, websites and news articles, and social media, with large negative
effects on historically underserved populations, also reinforcing biased
narratives and stereotypes about minority groups. Recommender systems play an
outsized role in serving up such content online, but improving these systems
to reduce bias will prove extremely challenging if we don’t understand the
underlying mechanisms in which such bias is perpetuated and disseminated.
Prior work has suggested that controversy online is highly unevenly
distributed, and that a population-sensitive model is needed in order to
properly model this (Jang et al., 2017); we hypothesize that the same approach
may be needed in the computational study of bisinformation and misinformation.
For example, the COVID-19 pandemic and its associated “infodemic” has brought
health mis- and disinformation to the forefront of national and scientific
attention. However, trust in the medical establishment may be understandably
low among African-Americans (Brandon et al., 2005) and other minority groups
(Jaiswal et al., 2020). Recent work suggests, moreover, that health mis- and
disinformation is qualitatively distinct in different population groups (e.g.
Prasad, 2021; Jaiswal et al., 2020). To cite just one example, COVID-19
vaccine hesitancy has been demonstrated to be higher among racial and ethnic
minorities (Nguyen et al., 2021; Jaiswal et al., 2020).

## 4 Pillars & Open Problems

A few guiding pillars underlie and drive our Fairness via AI research agenda,
which we encourage others to adopt. First, we argue that Fairness via AI is a
more effective and impactful marshalling of research resources than “standard”
Fairness in AI work (important though the latter may be). Second, we argue
that collaboration across disciplinary fields is critically needed in order to
effectively and ethically study and understand society’s biggest challenges,
to say nothing of mitigating them. Researchers in other fields, including but
in no way not limited to the social sciences, have immense expertise in
studying and addressing societal inequities; computer scientists cannot, and
should not, go this alone. Finally, we argue that biased information interacts
with false information in complex ways that must be studied carefully in order
to reduce bias in recommender systems and other information delivery systems
(such as search engines).

With these pillars firmly in mind, we pose the following open questions:

  1. Q1.

What are effective approaches to identify societal problems that are in most
need of, and lend themselves to, the Fairness via AI framework?

  2. Q2.

Which existing and/or novel AI approaches need to be deployed and developed in
order to address such societal issues?

  3. Q3.

How can we encourage collaboration across disciplinary boundaries in order to
leverage hard-won insights from other fields, and infuse our Fairness research
with them?

  4. Q4.

Specifically with respect to bisinformation, several research questions arise:

    1. a.

How and where does bisinformation spread online? Is information (including
mis- and bisinformation) distributed and disseminated differentially among
diverse population groups? If so, how?

    2. b.

Which categories or types of bisinformation and misinformation are most
problematic and harmful, and thus deserving the most diligent fact checking,
and countermessaging efforts? In other words, how should we triage mis- and
bisinformation, combining best practices in the public health and fact
checking spheres with state-of-the-art computational approaches?

## 5 Conclusions

In this position paper, we introduced a novel Fairness via AI framework;
coined a new term, bisinformation, to describe biased information, and
demonstrated that is is overlapping with yet distinct from misinformation;
briefly discussed the documented presence of bisinformation in medical
curricula and posit that this extends to other information environments, such
as online; and posed several open questions to guide research agendas on the
subject.

_Acknowledgements_. This material is based in part upon work supported by the
National Science Foundation under Grant No. 1951091. Any opinions, findings
and conclusions or recommendations expressed in this material are those of the
author(s) and do not necessarily reflect the views of the National Science
Foundation.

## References

  * (1)
  * Acquaviva and Mintz (2010) Kimberly D Acquaviva and Matthew Mintz. 2010.  Perspective: are we teaching racial profiling? The dangers of subjective determinations of race and ethnicity in case presentations.  _Academic Medicine_ 85, 4 (2010), 702–705.
  * Ali-Khan et al. (2011) Sarah E Ali-Khan, Tomasz Krakowski, Rabia Tahir, and Abdallah S Daar. 2011.  The use of race, ethnicity and ancestry in human genetic research.  _The HUGO journal_ 5, 1 (2011), 47–63.
  * Brandon et al. (2005) Dwayne T Brandon, Lydia A Isaac, and Thomas A LaVeist. 2005\.  The legacy of Tuskegee and trust in medical care: is Tuskegee responsible for race differences in mistrust of medical care?  _Journal of the National Medical Association_ 97, 7 (2005), 951\.
  * Braun et al. (2007) Lundy Braun, Anne Fausto-Sterling, Duana Fullwiley, Evelynn M Hammonds, Alondra Nelson, William Quivers, Susan M Reverby, and Alexandra E Shields. 2007.  Racial categories in medical practice: how useful are they?  _PLoS medicine_ 4, 9 (2007), e271.
  * Doja et al. (2016) Asif Doja, M Dylan Bould, Chantalle Clarkin, Kaylee Eady, Stephanie Sutherland, and Hilary Writer. 2016.  The hidden and informal curriculum across the continuum of training: a cross-sectional qualitative study.  _Medical teacher_ 38, 4 (2016), 410–418.
  * Fallin-Bennett (2015) Keisa Fallin-Bennett. 2015\.  Implicit bias against sexual minorities in medicine: cycles of professional influence and the role of the hidden curriculum.  _Academic Medicine_ 90, 5 (2015), 549–552.
  * FitzGerald and Hurst (2017) Chloë FitzGerald and Samia Hurst. 2017.  Implicit bias in healthcare professionals: a systematic review.  _BMC medical ethics_ 18, 1 (2017), 1–18.
  * Hunt et al. (2013) Linda M Hunt, Nicole D Truesdell, and Meta J Kreiner. 2013\.  Genes, race, and culture in clinical care: racial profiling in the management of chronic illness.  _Medical anthropology quarterly_ 27, 2 (2013), 253–271.
  * Jaiswal et al. (2020) J Jaiswal, C LoSchiavo, and DC Perlman. 2020.  Disinformation, misinformation and inequality-driven mistrust in the time of COVID-19: lessons unlearned from AIDS denialism.  _AIDS and Behavior_ 24 (2020), 2776–2780.
  * Jang et al. (2017) Myungha Jang, Shiri Dori-Hacohen, and James Allan. 2017\.  Modeling Controversy within Populations. In _Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval_ (Amsterdam, The Netherlands). Association for Computing Machinery, New York, NY, USA, 141–149.  https://doi.org/10.1145/3121050.3121067 ICTIR ’17.
  * Karani et al. (2017) Reena Karani, Lara Varpio, Win May, Tanya Horsley, John Chenault, Karen Hughes Miller, and Bridget O’Brien. 2017.  Commentary: racism and bias in health professions education: how educators, faculty developers, and researchers can make a difference.  _Academic Medicine_ 92, 11S (2017), S1–S6.
  * Le et al. (2018) Tao Le, Vikas Bhushan, Matthew Sochat, Kimberly Kallianos, Yash Chavda, Andrew Harrison Zureick, and Mehboob Kalani. 2018.  _First Aid for the USMLE Step 1 2018_.  McGraw-Hill Medical.
  * Lempp and Seale (2004) Heidi Lempp and Clive Seale. 2004.  The hidden curriculum in undergraduate medical education: qualitative study of medical students’ perceptions of teaching.  _Bmj_ 329, 7469 (2004), 770–773.
  * Madara (2020) J. Madara. 2020\.  America’s Health Care Crisis Is Much Deeper than COVID-19.  https://www.ama-assn.org/about/leadership/america-s-health-care-crisis-much-deeper-covid-19.
  * Martimianakis et al. (2015) Maria Athina Tina Martimianakis, Barret Michalec, Justin Lam, Carrie Cartmill, Janelle S Taylor, and Frederic W Hafferty. 2015.  Humanism, the hidden curriculum, and educational reform: a scoping review and thematic analysis.  _Academic Medicine_ 90, 11 (2015), S5–S13.
  * Mayson (2018) Sandra G Mayson. 2018\.  Bias in, bias out.  _YAle lJ_ 128 (2018), 2218.
  * Metzl and Roberts (2019) Jonathan M Metzl and Dorothy E Roberts. 2019.  Structural competency meets structural racism: race, politics, and the structure of medical knowledge.  In _The Social Medicine Reader, Volume II, Third Edition_. Duke University Press, 170–187.
  * Nguyen et al. (2021) Long H. Nguyen, Amit D. Joshi, David A. Drew, Jordi Merino, Wenjie Ma, Chun-Han Lo, Sohee Kwon, Kai Wang, Mark S. Graham, Lorenzo Polidori, Cristina Menni, Carole H. Sudre, Adjoa Anyane-Yeboa, Christina M. Astley, Erica T. Warner, Christina Y. Hu, Somesh Selvachandran, Richard Davies, Denis Nash, Paul W. Franks, Jonathan Wolf, Sebastien Ourselin, Claire J. Steves, Tim D. Spector, Andrew T. Chan, and on behalf of the COPE Consortium. 2021.  Racial and ethnic differences in COVID-19 vaccine hesitancy and uptake.  _medRxiv_ (2021).  https://doi.org/10.1101/2021.02.25.21252402 arXiv:https://www.medrxiv.org/content/early/2021/02/28/2021.02.25.21252402.full.pdf
  * Pollock (2012) Anne Pollock. 2012\.  4\. The Slavery Hypothesis beyond Genetic Determinism.  In _Medicating Race_. Duke University Press, 107–130.
  * Prasad (2021) Amit Prasad. 2021\.  Anti-science Misinformation and Conspiracies: COVID–19, Post-truth, and Science & Technology Studies (STS).  _Science, Technology and Society_ (2021).  https://doi.org/10.1177/09717218211003413
  * Ripp and Braun (2017) Kelsey Ripp and Lundy Braun. 2017.  Race/ethnicity in medical education: an analysis of a question bank for step 1 of the United States Medical Licensing Examination.  _Teaching and learning in medicine_ 29, 2 (2017), 115–122.
  * Saini (2019) Angela Saini. 2019\.  _Superior: the return of race science_.  Beacon Press.
  * Tsai et al. (2016) Jennifer Tsai, Laura Ucik, Nell Baldwin, Christopher Hasslinger, and Paul George. 2016\.  Race matters? Examining and rethinking race portrayal in preclinical medical education.  _Academic Medicine_ 91, 7 (2016), 916–920.