# ID-Blau: Image Deblurring by Implicit Diffusion-based reBLurring
AUgmentation

Jia-Hao Wu∗
National Yang Ming Chiao Tung University
<EMAIL_ADDRESS>Fu-Jen Tsai∗
National Tsing Hua University
<EMAIL_ADDRESS>Yan-Tsung Peng
National Chengchi University
<EMAIL_ADDRESS>Chung-Chi Tsai
Qualcomm Inc
<EMAIL_ADDRESS>Chia-Wen Lin
National Tsing Hua University
<EMAIL_ADDRESS>Yen-Yu Lin
National Yang Ming Chiao Tung University
<EMAIL_ADDRESS>
∗equal contribution

###### Abstract

Image deblurring aims to remove undesired blurs from an image captured in a
dynamic scene. Much research has been dedicated to improving deblurring
performance through model architectural designs. However, there is little work
on data augmentation for image deblurring. Since continuous motion causes
blurred artifacts during image exposure, we aspire to develop a groundbreaking
blur augmentation method to generate diverse blurred images by simulating
motion trajectories in a continuous space. This paper proposes Implicit
Diffusion-based reBLurring AUgmentation (ID-Blau), utilizing a sharp image
paired with a controllable blur condition map to produce a corresponding
blurred image. We parameterize the blur patterns of a blurred image with their
orientations and magnitudes as a pixel-wise blur condition map to simulate
motion trajectories and implicitly represent them in a continuous space. By
sampling diverse blur conditions, ID-Blau can generate various blurred images
unseen in the training set. Experimental results demonstrate that ID-Blau can
produce realistic blurred images for training and thus significantly improve
performance for state-of-the-art deblurring models.

##  1 Introduction

Figure 1: Examples of continuous reblurring by ID-Blau, where blur condition
maps represent pixel-wise blur information, consisting of blur orientations
and magnitudes, in a continuous space. ID-Blau can take a sharp image and a
blur condition map to synthesize a blurred image, even unseen in the training
set. Condition A is a blur condition map computed from the GoPro training set,
which can be used to reblur a sharp image to generate a blurred image as
provided in the training set. We can create Condition B and C based on A to
synthesize new reblurred images, where Condition B is Condition A with rotated
orientations, and Condition C is Condition A added with a camera motion blur.

Camera shake or object movements cause unpleasant motion blurs when we capture
images. Such blurs are usually non-uniform, leading to locally and globally
undesirable artifacts. Image deblurring aims to restore the sharpness of a
blurred image, a highly ill-posed problem that has remained challenging over
the past decades.

Image deblurring has reached remarkable progress with the rise of deep
learning. Numerous methods based on convolutional neural networks (CNNs) have
rendered success in deblurring. These methods mostly adopted recurrent-based
architectures, such as multi-scales [18, 28, 8], multi-patches [41, 39], and
multi-temporal [20]. In addition to CNN-based methods, several studies have
shown significant improvement using Transformers [3, 40, 31, 33, 13].
Transformers [32] utilize self-attention mechanisms to extract longer-range
features than CNNs but require more memory. To overcome the high memory usage
in the vanilla self-attention mechanism, some efficient attention mechanisms
have been proposed, such as the channel-wise [40] and strip-wise attention
[31] mechanisms.

Although previous methods have improved deblurring performance through
architectural designs, an effective data augmentation strategy for image
deblurring has been less studied. A common strategy [42] is to synthesize
blurred images through Generative Adversarial Networks (GANs) [10]. However,
GANs usually generate uncontrollable results [37, 25, 14], constraining the
potential of blur augmentation strategies. As a result, we resort to
developing a stable and controllable blur augmentation method based on the
inductive bias of motion blurs. Since continuous motions cause blur patterns
during exposure, we attempt to implicitly characterize blurs, such as blur
orientations and magnitudes, in a continuous space where we can sample blur
trajectories to generate various blurred images. This enables us to generate
various blurred images with specified blur conditions, devising a stable and
controllable augmentation strategy for image deblurring.

In this paper, we propose Implicit Diffusion-based reBLurring AUgmentation
(ID-Blau) that can produce diverse blur and sharp training pairs for training
to improve image deblurring performance. ID-Blau generates a realistic blurred
image from a sharp image and a blur condition map to control the blur patterns
through pixel-wise blur condition maps. Several examples are shown in Figure
1. To create blur condition maps, we implicitly represent blur information,
including blur orientations and magnitudes, in a continuous space, where we
can sample various blur conditions to simulate real blur trajectories during
exposure. Compared to the GoPro training set [18], which accumulates
consecutive sharp frames to generate explicit blurred images, our strategy can
generate various blurred images with sampled blur conditions in the continuous
blur conditoin field. In addition, motivated by the remarkable generation
ability of diffusion models [12], ID-Blau adopts diffusion models to take
implicit blur condition maps to generate realistic and controllable blurred
images. Our experimental results demonstrate that ID-Blau, the proposed
implicitly conditional diffusion model, can generate diverse blur and sharp
training pairs for training, even for those unseen in the training set. As a
result, it significantly improves the performance of existing deblurring
models and performs favorably against state-of-the-art augmentation methods in
this regard. Our contributions are summarized as follows:

  * •

We propose ID-Blau, a stable and controllable blur augmentation strategy for
enhancing dynamic scene image deblurring.

  * •

We model a continuous blur condition field to implicitly represent blur
orientations and magnitudes, where we can sample various pixel-wise blur
condition maps to generate diverse reblurred images not provided in the
training set.

  * •

The proposed ID-Blau integrates pixel-wise blur condition maps into a
diffusion model to generate high-quality reblurred images.

  * •

Experimental results show that ID-Blau significantly improves existing
deblurring models and performs favorably against state-of-the-art deblurring
methods.

##  2 Related Work

###  2.1 Image Deblurring

The development of CNNs has advanced image deblurring remarkably. Several
studies improved deblurring performance through recurrent networks, such as
scale-recurrent [18, 28] and patch-recurrent [41, 39] networks. Specifically,
Nah _et al_. [18] designed a scale-recurrent method to deal with image blur in
a coarse-to-fine manner. Zhang _et al_. [41] developed a patch-recurrent
method to leverage local information in a hierarchical deblurring network.
Other than recurrent designs, Cho _et al_. [6] proposed an efficient
deblurring model that rethinks the coarse-to-fine strategy in a single-forward
model to reduce latency. Recently, motivated by the success of Vision
Transformer (ViT) [7], several works [40, 31, 13] resorted to Transformer-
based architectures to deblur an image. For example, Zamir _et al_. [40]
utilized channel-wise attention to alleviate the tremendous memory load in the
vanilla transformer for self-attention. Tsai _et al_. [31] proposed strip
attention that exploits strip features to better extract blur features with
various orientations and magnitudes. Kong _et al_. [13] proposed a frequency-
based attention mechanism that utilizes element-wise multiplication in the
frequency domain to replace the dot product in the spatial domain.

###  2.2 Deblurring through Reblurring

Besides focusing on architectural designs, some works [42, 19, 5, 15] enhanced
deblurring performance through reblurring models. Chi _et al_. [5] and Liu _et
al_. [15] utilized a reblurring network in meta-learning to achieve test-time
adaptation. Nah _et al_. [19] proposed a reblurring network to amplify the
blurs in a deblurred image while keeping sharp parts unchanged. It can be used
to reinforce a deblurring network to generate a sharp result more accurately.
Another potential strategy to improve deblurring models is to augment training
pairs through reblurring models. Zhang _et al_. [42] exploited real-world
blurry images to synthesize blurred images through GANs. However, GANs usually
generate uncontrollable results, and GANs’ unstable optimization process often
increases the difficulty of training [17, 11, 2]. Besides, GANs’ poor mode
convergence [35, 30] makes them less effective in synthesizing diverse blurred
images. In contrast, we propose to represent blur conditions in a continuous
space implicitly and adopt a diffusion method to generate diverse, realistic,
and high-quality blurry images by considering arbitrarily enumerated blur
conditions.

###  2.3 Diffusion Models

Diffusion models [12, 26, 23] have demonstrated their outstanding ability for
image synthesis. Unlike GANs, diffusion models have a stable training strategy
by synthesizing images through forward diffusion and backward denoising
processes. The former gradually adds Gaussian noise to a clear image in
multiple steps and generates noisy images in a sequence. The latter
iteratively denoises the degraded images to restore the original image. Other
than image synthesis, several studies have successfully applied diffusion
models to low-level vision tasks, such as super-resolution [24, 9, 38],
inpainting [16, 36, 1], and deblurring [34, 21, 4]. Specifically, Saharia _et
al_. [24] conditioned the diffusion models on low-resolution images to
reconstruct their high-resolution versions with visually pleasant quality.
Whang _et al_. [34] conditioned the diffusion models on deterministic
deblurring results to improve perceptual quality. Instead of constructing a
restoration network based on diffusion models, we propose an implicitly
conditional diffusion reblurring model to generate high-quality training data
for improving deblurring performance for existing models. We can generate
various blur conditions in a continuous space implicitly to control the
proposed reblurring model to synthesize blurred images consistent with these
conditions. By sampling the blur condition space, a set of diverse and high-
quality blurred images can be generated to enrich the training set and enhance
the performance of existing deblurring models.

##  3 Proposed Method

This section presents the proposed ID-Blau that can turn a sharp image into a
blurred version based on a pixel-wise blur condition map. We characterize
blurs with their orientations and magnitudes in a continuous space, where we
can sample blur conditions for reblurring a sharp image. With the modeling, we
can simulate blur trajectories caused by continuous motion during exposure. It
allows us to implicitly manipulate blur conditions to generate diverse blurred
images unseen in the training set for data augmentation. In the data
augmentation scenario, we can use ID-Blau to produce additional training data
before optimizing deblurring models, which means that we do not need to run
ID-Blau when optimizing deblurring models. The following details the
computation of blur conditions and the training and sampling processes of ID-
Blau for blurred image generation.

Figure 2: Illustration of distributions of blur magnitudes (left) and
orientations (right) of the GoPro training set.  Figure 3: Reblurring process
with ID-Blau. As the pie chart shows the blur condition field with
orientations and magnitudes delineated in different colors, we visualize the
process of generating blurred images with a set of blur condition maps. A
sharp image paired with a blur condition map and a noise map is concatenated
and fed into ID-Blau to produce a blurred image, where an MLP is used to
encode the iteration index $t$ as Time Embedding. Using ID-Blau can augment an
image deblurring training set offline for optimizing a deblurring model and
improving its performance.

###  3.1 Blur Conditions

During exposure, continuous motion from the camera or scene objects may cause
blur artifacts to captured images. Thus, global and/or local blur trajectories
can be found in such blurred images. To synthesize blurred images from their
sharp counterparts, Nah _et al_. [18] proposed the GoPro dataset,
approximating the continuous exposure through a blur accumulation function as

$B=g(\frac{1}{T}\int_{t=1}^{T}V(t)dt)\simeq
g(\frac{1}{N}\sum_{n=1}^{N}V\left[n\right]),$ (1)

where $B$, $V(t)$, $T$, $N$, and $g$ denote the generated blurred frame, sharp
frame at time $t$ collected by a high-speed camera, exposure time, number of
sampled sharp frames, and camera response function, respectively. The
accumulation function aggregates a sharp image sequence
$\textbf{V}=\\{V\left[1\right],...,V\left[N\right]\\}$ to generate one blurred
image $B$. The center sharp frame $V[\frac{N+1}{2}]$ is assigned to the
ground-truth image $S$ for $B$.

Continuous motions of different objects may cause various blur patterns since
they can move independently. We propose to characterize the blurs in an image
into a blur condition map specifying a blur orientation and magnitude to each
pixel. The blur condition map depicts a specific blur scenario for the blurred
image by vectorizing blur for each pixel in a continuous field. By changing
the blur condition map, we can simulate various blur scenarios and augment
diverse blurred images to complement existing image deblurring datasets. To
obtain the corresponding blur condition from a blurred image, we choose to
average bidirectional optical flows from the sharp sequence V and then
aggregate them by summation as

$\mathcal{F}=\sum_{n=1}^{N-1}\frac{f_{\theta}(V\left[n\right],V\left[n+1\right])-f_{\theta}(V\left[n+1\right],V\left[n\right])}{2},$
(2)

where $f_{\theta}$ denotes the pre-trained flow estimation network [29],
$\mathcal{F}=[u;v]\in\mathbb{R}^{H\times W\times 2}$ denotes the overall
motion trajectories during exposure, where $H$ and $W$ are the image height
and width, respectively. The tensor $\mathcal{F}$ records the horizontal and
vertical motion trajectories in $u$ and $v\in\mathbb{R}^{H\times W}$.

Next, we normalize $\mathcal{F}$ to obtain unit motion vectors and append one
dimension of the magnitudes to make it a 3D tensor
$C=[x;y;z]\in\mathbb{R}^{H\times W\times 3}$, where
$x_{i,j}=\frac{u_{i,j}}{\sqrt{u_{i,j}^{2}+v_{i,j}^{2}}}$ and
$y_{i,j}=\frac{v_{i,j}}{\sqrt{u_{i,j}^{2}+v_{i,j}^{2}}}$ respectively
represent the horizontal and vertical blur orientations for a pixel $(i,j)$,
and $z_{i,j}=\frac{\sqrt{u_{i,j}^{2}+v_{i,j}^{2}}}{M}$ denotes its
corresponding blur magnitudes. Here, $M$ is set to the largest blur magnitude
in a set of considered blur conditions to make $z\in[0,1]$.

Let $C$ be the blur condition map for a blurred image $B$ in a normalized
continuous space, corresponding to its ground-truth image $S$. With the above
modeling, we can construct a set of $K$ blurred and sharp image pairs with
their computed blur condition maps $\\{B_{n},S_{n},C_{n}\\}^{K}_{n=1}$ to
train a reblurring model. It uses a sharp image $S_{n}$ and a blur condition
map $C_{n}$ to produce the corresponding blurred image $B_{n}$, correlating
the sharp image $S_{n}$ and its blurred version $B_{n}$ with our computed blur
condition map $C_{n}$.

For example, we show the distribution of the blur magnitude and orientation of
GoPro in Figure 2, indicating that the magnitude histogram peaks towards the
left side, meaning most images are not heavily blurred. In contrast, the
orientation has slight peaks around 0, 180, and 360 degrees, implying more
blurs tend to be horizontal, but cases in other orientations can also be
found. Training our reblurring model on such diverse data can correlate a
blurred image with its computed blur information implicitly in a continuous
blur condition field and empower the model to generate blurred images unseen
in the original data by sampling different blur magnitudes and orientations.
Next, we describe optimizing the proposed diffusion-based reblurring model in
ID-Blau.

###  3.2 ID-Blau

ID-Blau is based on a conditional diffusion model that takes a blur condition
map $C$ and a sharp image $S$ as inputs to generate a blurred image $B$, as
shown in Figure 3. Therefore, we can create various blur conditions to augment
data with realistic reblurred images unseen in the training set. To expedite
the inference, we adopt the denoising diffusion implicit model (DDIM) [26],
utilizing the same training procedure as DDPM [12] but accelerating the
inference via a non-Markovian inference process.

Figure 4: Illustration of reblurred images with ID-Blau. It takes a sharp
image $S$ and a blur condition map $C=[x;y;z]$ as inputs to generate the
corresponding blurred image. We show several generated blurred images by
altering $C$, such as unit horizontal or vertical blur orientations,
$C^{1}=[1;0;z]$ and $C^{2}=[0;1;z]$, horizontally inverse $C$ as
$C^{3}=[-x;y;z]$, and $C^{4}=[-x;y;2z]$ with twice the magnitude for $C^{3}$.
Figure 5: Visuals of blurred images with local motions. ID-Blau can generate
blurred images of moving objects, such as pedestrians.

#### Training Stage.

Following DDPM [12], we perform a forward diffusion process and a backward
denoising process to optimize ID-Blau. In the forward diffusion process, we
gradually add Gaussian noise to a blurred image $B$ over $T$ iterations for
generating a sequence of noisy images $\\{B_{1},...,B_{T}\\}$, where $T$ is
set to $2,000$. The sampling probability $q(B_{1:T}|B_{0})$ is defined as

$\begin{split}q(B_{1:T}|B_{0})&=\prod_{t=1}^{T}q(B_{t}|B_{t-1});\\\
q(B_{t}|B_{t-1})&=\mathcal{N}(B_{t};\sqrt{\alpha_{t}}B_{t-1},(1-\alpha_{t})\mathrm{I}),\end{split}$
(3)

where $B_{0}=B$ and $\alpha_{t}\in(0,1)$ is a set of hyperparameters that
controls the noise variance over a sequence of steps. Instead of iteratively
adding Gaussian noise to $B$, the forward diffusion process can be
reparameterized as

$B_{t}=\sqrt{\overline{\alpha}_{t}}B_{0}+(1-\sqrt{\overline{\alpha}_{t}})\epsilon,$
(4)

where $\overline{\alpha}_{t}=\prod_{i=1}^{t}\alpha_{i}$ and
$\epsilon\sim\mathcal{N}(0,\mathrm{I})$.

In the backward denoising process, the blurred image $B$ is iteratively
restored from the noisy image $B_{T}$ through a denoising model
$\epsilon_{\theta}$. When recreating a blurred image $B$ from a noisy image,
we condition the denoising model on its sharp counterpart $S$ and the blur
condition map $C$ computed based on $B$. Thus, the denoising model learns to
correlate a blurred image $B$ and its sharp image $S$ conditioned on the blur
condition map $C$. The reverse conditional probability is written as

$\begin{split}&p_{\theta}(B_{t-1}|B_{t},S,C)=\mathcal{N}(B_{t-1};\mu_{\theta}(B_{t},S,C,t),\sigma^{2}_{t}\mathrm{I});\\\
&\mu_{\theta}(B_{t},S,C,t)=\frac{1}{\sqrt{\alpha_{t}}}(B_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\overline{\alpha}_{t}}}\epsilon_{\theta}(B_{t},S,C,t)),\end{split}$
(5)

where the Gaussian density function is parameterized with the mean of
$\mu_{\theta}(B_{t},S,C,t)$ and variance of $\sigma^{2}_{t}=(1-\alpha_{t})$.
To train the denoising model, we use the following objective function:

$\mathcal{L}=\parallel\epsilon-\epsilon_{\theta}(\sqrt{\overline{\alpha}_{t}}B_{0}+\sqrt{(1-\overline{\alpha}_{t})}\epsilon,S,C,t)\parallel_{1},$
(6)

where
$\sqrt{\overline{\alpha}_{t}}B_{0}+\sqrt{(1-\overline{\alpha}_{t})}\epsilon$
is the noisy image $B_{t}$ at the $t$-th step. Figure 3 shows the architecture
of the denoising model $\epsilon_{\theta}$.

#### Inference Stage.

After optimizing the denoising model, ID-Blau can generate a blurred image $B$
from a pure noisy image conforming to a Gaussian distribution
$B_{T}\sim\mathcal{N}(0,\mathrm{I})$, a sharp image $S$, and a blur condition
map $C$ through the sampling procedure in DDIM. It can take any given blur
condition map implicitly created to produce diverse blurred images unseen in
the training set. The denoising process is given by:

$\begin{split}B_{t-1}=&\sqrt{\alpha_{t-1}}(\frac{B_{t}-\sqrt{1-\alpha_{t}}\epsilon_{\theta}(B_{t},S,C,t)}{\sqrt{\alpha_{t}}})\\\
&+\sqrt{1-\alpha_{t-1}-\sigma^{2}_{t}}\cdot\epsilon_{\theta}(B_{t},S,C,t)+\sigma_{t}z,\end{split}$
(7)

where $\sigma_{t}=0$, making the process deterministic. Through DDIM, we set
the number of iterations during sampling to $20$ without excessively
sacrificing the image generation quality.

Figure 4 shows examples of generating different blurred images from a sharp
image $S$ by ID-Blau. That is, given a blur condition map $C=[x;y;z]$ and a
sharp image $S$, ID-Blau can produce a blurred image $B=\textbf{ID-
Blau}(S,C)$. Additionally, we create four blur condition maps based on $C$ for
further demonstration, including unit horizontal or vertical blur
orientations, $C^{1}=[1;0;z]$ and $C^{2}=[0;1;z]$, horizontally inverse $C$ as
$C^{3}=[-x;y;z]$, and $C^{4}=[-x;y;2z]$ with twice the magnitude for $C^{3}$.
Furthermore, ID-Blau can also simulate motion blurs caused by object
movements, as shown in Figure 5. Besides, we have more examples to demonstrate
the generalizability of ID-Blau by utilizing sharp images not in the training
set to yield different blurred images in the supplementary material. These
visuals attest that ID-Blau can effectively generate controllable and
realistic blurred images. Thus, it can augment training data to improve image
deblurring performance.

Table 1: Evaluation results on the GoPro, HIDE, and RealBlur datasets, where “Baseline” and “+ID-Blau” denote the deblurring performances without and with ID-Blau, respectively. |  | GoPro | HIDE | RealBlur-J | RealBlur-R | Average Gain
---|---|---|---|---|---|---
Model | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM
MIMO-UNet+ | Baseline | 32.44 | 0.957 | 30.00 | 0.930 | 31.92 | 0.919 | 39.10 | 0.969 |  |
+ID-Blau | 32.93 (+0.49) | 0.961 | 30.68 (+0.68) | 0.938 | 31.96 (+0.04) | 0.921 | 39.38 (+0.28) | 0.971 | +0.37 | +0.004
Restormer | Baseline | 32.92 | 0.961 | 31.22 | 0.942 | 32.88 | 0.933 | 40.15 | 0.974 |  |
+ID-Blau | 33.51 (+0.59) | 0.965 | 31.66 (+0.44) | 0.947 | 33.11 (+0.23) | 0.937 | 40.31 (+0.16) | 0.974 | +0.36 | +0.003
Stripformer | Baseline | 33.08 | 0.962 | 31.03 | 0.940 | 32.48 | 0.929 | 39.84 | 0.974 |  |
+ID-Blau | 33.66 (+0.58) | 0.966 | 31.50 (+0.47) | 0.944 | 33.77 (+1.29) | 0.940 | 41.06 (+1.22) | 0.977 | +0.89 | +0.006
FFTformer | Baseline | 34.21 | 0.969 | 31.62 | 0.946 | 32.62 | 0.933 | 40.11 | 0.973 |  |
+ID-Blau | 34.36 (+0.15) | 0.970 | 31.94 (+0.32) | 0.949 | 32.88 (+0.26) | 0.934 | 40.45 (+0.34) | 0.975 | +0.27 | +0.002
Average Gain | +0.45 | +0.003 | +0.48 | +0.005 | +0.46 | +0.005 | +0.50 | +0.002 | - | -

##  4 Experiments

###  4.1 Implementation Details

#### ID-Blau.

we adopt the GoPro training set [18], which contains $2,103$ blurred and sharp
images for training ID-Blau. The training also requires the blur condition
maps computed based on the blurred images. We use a batch size of $32$ and
randomly crop images with the size of $128\times 128$ for training $5,000$
epochs. We adopt the Adam optimizer and maintain a fixed learning rate of
$1e^{-4}$. After optimizing ID-Blau, we utilize sharp images in the GoPro
training set and create extra blur condition maps to produce additional
$10,000$ $1280\times 720$ blurred images by randomly modifying blur
orientations and magnitudes based on the originally computed blur condition
maps. The newly generated $10,000$ blurred and sharp pairs can largely enrich
the GoPro training set to enhance the performance of a deblurring model. ID-
Blau has 9.5 million parameters, and it takes 3.4 seconds to generate a
$1280\times 720$ blurred image on an Nvidia 3090.

#### Deblurring models.

We adopt four prominent deblurring models, including MIMO-UNet+ [6], Restormer
[40], Stripformer [31], and the state-of-the-art FFTformer [13] to demonstrate
the effectiveness of ID-Blau. Following previous works, we evaluate deblurring
performance on the GoPro, HIDE [27], and RealBlur [22] datasets. The GoPro
dataset comprises $2,103$ training pairs and $1,111$ testing pairs. The HIDE
dataset contains $2,025$ image pairs only for testing. The RealBlur dataset
contains $3,758$ training pairs and $980$ testing pairs. We pre-train each
deblurring model on the generated $10,000$ training pairs for $500$ epochs.
After that, we fine-tune each deblurring model on the GoPro training set
following its default training setting and then evaluate it on the GoPro
testing set and the HIDE dataset. Besides, we fine-tune each deblurring model
on the RealBlur training set to demonstrate the effectiveness of ID-Blau on
real-world blurred images. Since all compared methods utilized the GoPro and
RealBlur training sets for evaluating the RealBlur testing set, ID-Blau
ensures a fair comparison without relying on extra training data.

###  4.2 Experimental Results

#### Quantitative Results.

We compare the deblurring performance of four baselines and their ID-Blau-
powered versions in Table 1, where “Baseline” and “+ID-Blau” denote the
deblurring performance without and with ID-Blau, respectively. Table 1 shows
that ID-Blau significantly improves four prominent deblurring models,
including MIMO-UNet+ [6], Restormer [40], Stripformer [31], and the state-of-
the-art FFTformer [13], on the GoPro, HIDE, and RealBlur datasets. ID-Blau
enhances those models’ performance in PSNR by $0.45$dB, $0.48$dB, $0.46$dB,
and $0.50$dB averagely on GoPro, HIDE, RealBlur-J, and RealBlur-R,
respectively. In particular, with ID-Blau, the state-of-the-art performance
has been advanced to $34.36$dB ($+0.15$dB) on GoPro, $31.94$dB ($+0.32$dB) on
HIDE, $33.77$dB ($+0.89$dB) on RealBlur-J, and $41.06$dB ($+0.72$dB) on
RealBlur-R, where the enhanced FFTformer achieves new SOTA results on GoPro
and HIDE, and the enhanced Stripformer performs the best on ReaBlur-J and
RealBlur-R. These comprehensive quantitative results demonstrate that ID-Blau
significantly improves the performances of the four deblurring models on four
widely-used datasets, showcasing ID-Blau’s effectiveness and robustness as a
data augmentation scheme for deblurring.

#### Qualitative Results.

Figure 6 compares the visual qualities of the results with the four deblurring
baselines (denoted “Baseline”) and their ID-Blau-powered versions (denoted
“+ID-Blau”) on GoPro and HIDE. The results show that ID-Blau significantly
enhances deblurring visual qualities compared to the Baselines. Besides, we
demonstrate visuals of deblurring models on RealBlur-J in Figure 7. We also
demonstrate deblurring results on real-world blurry images [42] in the
supplementary material.

Figure 6: Qualitative results on the GoPro testing set (left) and the HIDE
dataset (right).  Figure 7: Qualitative results on the RealBlur-J testing set.

###  4.3 Ablation Studies

We analyze the impact of ID-Blau on deblurring performance using GoPro. For
efficiency, we train MIMO-UNet [6], a compact version of MIMO-UNet+, for
$1,000$ epochs for analysis. If not specifically mentioned, “Baseline” denotes
MIMO-UNet trained on the GoPro training set without using any additional
augmented data samples.

#### Effect of Augmenting Blur Orientations.

Table 2 shows the effect of augmenting blur orientations with different angles
on deblurring performance. To this end, with the original blur magnitudes, we
rotate the blur orientations $[x;y]$ by four fixed angles ($30^{\circ}$,
$60^{\circ}$, $90^{\circ}$, and $120^{\circ}$) plus a randomly selected angle
for each produced blurred image (denoted “Random”). The results indicate that
blur augmentation with all specific/random angles achieves similar performance
improvements over Baseline. This suggests no need to employ specific angles in
blur condition maps for augmentation. For implementation convenience, we
randomly alter blur orientations $[x;y]$ to $[-x;y]$, $[x;-y]$, or $[-x;-y]$
to augment blurred images with different orientations, achieving a similar
performance improvement as well.

Table 2: Impact of modifying blur orientations on deblurring performance in PSNR (dB). | Baseline | $30^{\circ}$ | $60^{\circ}$ | $90^{\circ}$ | $120^{\circ}$ | Random
---|---|---|---|---|---|---
PSNR | 31.22 | 31.99 | 32.00 | 31.97 | 31.95 | 31.96

#### Effect of Adjusting Blur Magnitudes.

Figure 8 illustrates the effect of modifying blur magnitudes on the deblurring
performance. We first investigate the distribution of blur magnitudes within
GoPro, in which most blurred images have blur magnitudes around with the peak
at $0.07$. Subsequently, we augment blurred images with different blur
magnitudes by shifting the peak of magnitude distribution to different values,
including $0.07$, $0.15$, $0.2$, $0.3$, and $0.4$. The results demonstrate
that shifting the distribution’s peak to a larger value causes decreased
performance, indicating that generating blurred images that deviate from the
original distribution of the dataset leads to a performance drop. Therefore,
we modify blur magnitudes but keep the peak of blur magnitudes at around
$0.07$.

#### Effect of the Number of Augmented Samples.

The left plot of Figure 9 shows the impact of the number of augmented training
pairs by ID-Blau on the deblurring performance. The performance gain of ID-
Blau over that of the baseline, trained on GoPro without augmentation,
increases from $0.53$dB to $0.87$dB with the number of augmented data samples
(from $2,500$ to $20,000$), demonstrating ID-Blau’s ability to improve the
deblurring performance without the need of collecting additional training data
samples. Augmenting $10,000$ additional samples (476% of the original data
size) achieves a reasonable tradeoff between the performance gain and training
complexity. Since ID-Blau does not require extra training samples but consumes
additional training complexity, we also investigate whether the performance
improvements come from the additional training complexity. As shown in the
right plot of Figure 9, ID-Blau stably outperforms the baseline with the same
training complexities (i.e., the numbers of training iterations), where both
the pre-training and fine-tuning process of ID-Blau are considered. Both the
above experiments verify the performance gain with ID-Blau-based augmentation.

Figure 8: Left: Distribution of blur magnitudes within the GoPro training set.
Right: Illustration of the effect of altering the distribution’s mode at
different values on the deblurring performance. Figure 9: Deblurring
performance (PSNR) versus the number of augmented samples by ID-Blau (left)
and versus the number of training iterations (right).

#### Effect of the Continuous Reblurring by ID-Blau.

Figure 10 shows two blurred images from GoPro and their reblurred versions by
ID-Blau. Since GoPro’s blurred images are generated by blur accumulation as in
(1), they contain unnaturally overlapping artifacts. In contrast, ID-Blau can
generate similar blurred images by simulating continuous motion trajectories,
yielding more realistic blurred images without such discontinuous overlapping
artifacts. Therefore, ID-Blau can generate more realistic blurred images to
improve the deblurring models’ performances.

#### Performance Gain with a Diffusion Process.

In ID-Blau, we adopt the diffusion process in [12] to generate blurred
versions of sharp images with specified blur conditions. However, without the
diffusion process, we can still utilize the same reblurring model to learn the
mapping from a sharp image plus a computed blur condition map to the
corresponding blurred image with the typical L1 loss function. To analyze the
impact of ID-Blau with or without the diffusion process on deblurring
performance, in Table 3, we compare Baseline (the same as in Figure 9), “w/
ID-Blau†,” (ID-Blau without diffusion), and “w/ ID-Blau” (our model). The
results show that “w/ ID-Blau†” and “w/ ID-Blau” improve the Baseline by
$0.57$dB and $0.80$dB, respectively, and the diffusion model achieves $0.23$dB
additional performance gain.

Figure 10: Left: Blurry image in GoPro dataset. Right: Blurry image generated by ID-Blau. ID-Blau can generate blurry images without overlapped artifacts compared to the GoPro dataset.  Table 3: Effectiveness of using a diffusion model in ID-Blau on MIMO-UNet’s performance in PSNR (dB), where “ID-Blau†” denotes the proposed ID-Blau without its diffusion process. Model | Baseline | w/ ID-Blau† | w/ ID-Blau
---|---|---|---
PSNR | 31.22 | 31.79 | 32.02
Table 4: Comparison between ID-Blau and BGAN regarding the DBGAN’s performance in PSNR (dB), where ID-Blau† denotes the proposed ID-Blau without its diffusion process. Model | Baseline | w/ BGAN | w/ ID-Blau† | w/ ID-Blau
---|---|---|---|---
PSNR | 30.43 | 31.10 | 31.57 | 31.67

#### Comparison between ID-Blau- and BGAN-based Augmentation.

Besides, we compare the performance gains of ID-Blau and that of BGAN [42], a
representative reblurring GAN for augmenting blurred images to improve the
DBGAN deblurring model. To fairly compare ID-Blau with BGAN, we utilize ID-
Blau to improve DBGAN’s deblurring performance. In Table 4, “Baseline” denotes
DBGAN’s performance trained on GoPro, and “w/ BGAN” denotes DBGAN’s
performance with BGAN-augmented data. “w/ ID-Blau†” and “w/ ID-Blau” denote
DBGAN’s performance using ID-Blau-augmented data without and with the
diffusion process, respectively. Note that GAN-based methods tend to generate
unpredictable and uncontrollable blurred images, whereas ID-Blau can generate
controllable results based on specified blur conditions. As a result, “w/ ID-
Blau†” and “w/ ID-Blau” improve the PSNR performance of DBGAN by $1.14$dB and
$1.24$dB. Thanks to the proposed continuous blur condition map with ID-Blau,
the simplified “w/ ID-Blau†” without the diffusion process can still offer an
additional gain of $0.47$dB over that with “w/ BGAN.” This verifies the
effectiveness of ID-Blau compared to the GAN-based reblurring model BGAN.

##  5 Conclusion

We proposed a diffusion-based reblurring model that can take a sharp image and
a controllable pixel-wise blur condition map to synthesize a blurred image. To
train the reblurring model, we parameterized the blur patterns of a blurred
image with their orientations and magnitudes and implicitly represented them
in a continuous blur condition field. With the model, we presented ID-Blau, an
effective data augmentation scheme for image deblurring, where we sample
various blur conditions from the field to produce diverse, realistic blurred
images and enrich the training set. Experimental results have shown that ID-
Blau can significantly improve the performance of state-of-the-art deblurring
models.

## References

  * Anciukevičius et al. [2023] Titas Anciukevičius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy J. Mitra, and Paul Guerrero.  Renderdiffusion: Image diffusion for 3d reconstruction, inpainting and generation.  In _CVPR_ , 2023.
  * Arjovsky et al. [2017] Martin Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein generative adversarial networks.  In _ICML_ , 2017.
  * Chen et al. [2021] Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao.  Pre-trained image processing transformer.  In _CVPR_ , 2021.
  * Chen et al. [2023] Zheng Chen, Yulun Zhang, Liu Ding, Xia Bin, Jinjin Gu, Linghe Kong, and Xin Yuan.  Hierarchical integration diffusion model for realistic image deblurring.  In _NeurIPS_ , 2023.
  * Chi et al. [2021] Zhixiang Chi, Yang Wang, Yuanhao Yu, and Jin Tang.  Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning.  In _CVPR_ , 2021.
  * Cho et al. [2021] Sung-Jin Cho, Seo-Won Ji, Jun-Pyo Hong, Seung-Won Jung, and Sung-Jea Ko.  Rethinking coarse-to-fine approach in single image deblurring.  In _ICCV_ , 2021.
  * Dosovitskiy et al. [2021] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _ICLR_ , 2021.
  * Gao et al. [2019] Hongyun Gao, Xin Tao, Xiaoyong Shen, and Jiaya Jia.  Dynamic scene deblurring with parameter selective sharing and nested skip connections.  In _CVPR_ , 2019.
  * Gao et al. [2023] Sicheng Gao, Xuhui Liu, Bohan Zeng, Sheng Xu, Yanjing Li, Xiaoyan Luo, Jianzhuang Liu, Xiantong Zhen, and Baochang Zhang.  Implicit diffusion models for continuous super-resolution.  In _CVPR_ , 2023.
  * Goodfellow et al. [2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In _NeurIPS_ , 2014.
  * Gulrajani et al. [2017] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.  Improved training of wasserstein gans.  In _NeurIPS_ , 2017.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In _NeurIPS_ , 2020.
  * Kong et al. [2023] Lingshun Kong, Jiangxin Dong, Jianjun Ge, Mingqiang Li, and Jinshan Pan.  Efficient frequency domain-based transformers for high-quality image deblurring.  In _CVPR_ , 2023.
  * Kowalski et al. [2020] Marek Kowalski, Stephan J. Garbin, Virginia Estellers, Tadas Baltrušaitis, Matthew Johnson, and Jamie Shotton.  Config: Controllable neural face image generation.  In _ECCV_ , 2020.
  * Liu et al. [2022] Po-Sheng Liu, Fu-Jen Tsai, Yan-Tsung Peng, Chung-Chi Tsai, Chia-Wen Lin, and Yen-Yu Lin.  Meta transferring for deblurring.  In _BMVC_ , 2022.
  * Lugmayr et al. [2022] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool.  Repaint: Inpainting using denoising diffusion probabilistic models.  In _CVPR_ , 2022.
  * Mao et al. [2017] Xudong Mao, Qing Li, Haoran Xie, Raymond Y.K. Lau, Zhen Wang, and Stephen Paul Smolley.  Least squares generative adversarial networks.  In _ICCV_ , 2017.
  * Nah et al. [2017] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee.  Deep multi-scale convolutional neural network for dynamic scene deblurring.  In _CVPR_ , 2017.
  * Nah et al. [2022] Seungjun Nah, Sanghyun Son, Jaerin Lee, and Kyoung Mu Lee.  Clean images are hard to reblur: Exploiting the ill-posed inverse task for dynamic scene deblurring.  In _ICLR_ , 2022.
  * Park et al. [2020] D. Park, D. U. Kang, J. Kim, and S. Y. Chun.  Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training.  In _ECCV_ , 2020.
  * Ren et al. [2023] Mengwei Ren, Mauricio Delbracio, Hossein Talebi, Guido Gerig, and Peyman Milanfar.  Multiscale structure guided diffusion for image deblurring.  In _ICCV_ , 2023.
  * Rim et al. [2020] Jaesung Rim, Haeyun Lee, Jucheol Won, and Sunghyun Cho.  Real-world blur dataset for learning and benchmarking deblurring algorithms.  In _ECCV_ , 2020.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In _CVPR_ , 2022.
  * Saharia et al. [2023] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi.  Image super-resolution via iterative refinement.  _IEEE TPAMI_ , 2023.
  * Shoshan et al. [2021] Alon Shoshan, Nadav Bhonker, Igor Kviatkovsky, and Gérard Medioni.  Gan-control: Explicitly controllable gans.  In _ICCV_ , 2021.
  * Song et al. [2021] Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  In _ICLR_ , 2021.
  * Su et al. [2017] Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang.  Deep video deblurring for hand-held cameras.  In _CVPR_ , 2017.
  * Tao et al. [2018] Xin Tao, Hongyun Gao, Xiaoyong Shen, Jue Wang, and Jiaya Jia.  Scale-recurrent network for deep image deblurring.  In _CVPR_ , 2018.
  * Teed and Deng [2020] Zachary Teed and Jia Deng.  Raft: Recurrent all-pairs field transforms for optical flow.  In _ECCV_ , 2020.
  * Thanh-Tung et al. [2019] Hoang Thanh-Tung, Truyen Tran, and Svetha Venkatesh.  Improving generalization and stability of generative adversarial networks.  In _ICLR_ , 2019.
  * Tsai et al. [2022] Fu-Jen Tsai, Yan-Tsung Peng, Yen-Yu Lin, Chung-Chi Tsai, and Chia-Wen Lin.  Stripformer: Strip transformer for fast image deblurring.  In _ECCV_ , 2022.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In _NeurIPS_ , 2017.
  * Wang et al. [2022] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li.  Uformer: A general u-shaped transformer for image restoration.  In _CVPR_ , 2022.
  * Whang et al. [2022] Jay Whang, Mauricio Delbracio, Hossein Talebi, Chitwan Saharia, Alexandros G. Dimakis, and Peyman Milanfar.  Deblurring via stochastic refinement.  In _CVPR_ , 2022.
  * Xiao et al. [2022] Zhisheng Xiao, Karsten Kreis, and Arash Vahdat.  Tackling the generative learning trilemma with denoising diffusion GANs.  In _ICLR_ , 2022.
  * Xie et al. [2023] Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang.  Smartbrush: Text and shape guided object inpainting with diffusion model.  In _CVPR_ , 2023.
  * Yu et al. [2020] Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, and Mario Fritz.  Inclusive gan: Improving data and minority coverage in generative models.  In _ECCV_ , 2020.
  * Yue et al. [2023] Zongsheng Yue, Jianyi Wang, and Chen Change Loy.  Resshift: Efficient diffusion model for image super-resolution by residual shifting.  In _NeurIPS_ , 2023.
  * Zamir et al. [2021] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao.  Multi-stage progressive image restoration.  In _CVPR_ , 2021.
  * Zamir et al. [2022] Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang.  Restormer: Efficient transformer for high-resolution image restoration.  In _CVPR_ , 2022.
  * Zhang et al. [2019] Hongguang Zhang, Yuchao Dai, Hongdong Li, and Piotr Koniusz.  Deep stacked hierarchical multi-patch network for image deblurring.  In _CVPR_ , 2019.
  * Zhang et al. [2020] Kaihao Zhang, Wenhan Luo, Yiran Zhong, Lin Ma, Bjorn Stenger, Wei Liu, and Hongdong Li.  Deblurring by realistic blurring.  In _CVPR_ , 2020.


# A Novel Method for Lane-change Maneuver in Urban Driving Using Predictive
Markov Decision Process

Avinash PRABU, Niranjan RAVI, and Lingxi LI The authors are with the
Department of Electrical and Computer Engineering, also with the
Transportation and Autonomous Systems Institute (TASI), Purdue School of
Engineering and Technology, Indiana University-Purdue University Indianapolis
(IUPUI), 723 West Michigan Street, SL-160, Indianapolis, IN 46202, USA. Email:
<EMAIL_ADDRESS>
###### Abstract

Lane-change maneuver has always been a challenging task for both manual and
autonomous driving, especially in an urban setting. In particular, the
uncertainty in predicting the behavior of other vehicles on the road leads to
indecisive actions while changing lanes, which, might result in traffic
congestion and cause safety concerns. This paper analyzes the factors related
to uncertainty such as speed range change and lane change so as to design a
predictive Markov decision process for lane-change maneuver in the urban
setting. A hidden Markov model is developed for modeling uncertainties of
surrounding vehicles. The reward model uses the crash probabilities and the
feasibility/distance to the goal as primary parameters. Numerical simulation
and analysis of two traffic scenarios are completed to demonstrate the
effectiveness of the proposed approach.

###### Index Terms:

Markov decision process, lane-change maneuver, hidden Markov models.

##  I Introduction

Autonomous vehicles (AVs) have a great potential to improve road safety,
reduce road congestion, and lower emissions in rural and urban scenarios [1].
Typical AVs are equipped with a variety of sensors including cameras, Lidar,
and radar, etc., to enable advanced driver-assistance system (ADAS) functions
and autonomous driving capabilities [2]–[8]. However, noisy sensor data also
poses challenges, and it is difficult to handle a variety of driving
scenarios, such as lane keeping, vehicle following, overtaking, and avoiding
static and dynamic objects [9]. The driver’s behavior, driving style, and
intention should also be taken into consideration. Furthermore, the
interaction between the ego vehicle and surrounding vehicles provides
additional information in the decision-making process. All these factors help
the driver/AV to navigate on the road safely and efficiently [10]. This has
led to approaches that use the real-time traffic information and historical
traffic data in order to predict trajectories of surrounding vehicles.

Several efforts on predicting vehicle trajectories focus on motion planning
algorithms. Motion planning consists of finding a path, searching for the
safest maneuver, and determining the most feasible trajectory. For instance,
the authors of [11] and [12] proposed approaches that are based on machine
learning and neural networks for motion planning. However, the computation
cost of the entire motion planning strategy can be reduced. The past
trajectory of the vehicle can be used to train the network to predict the
future trajectory [12]. Other than these approaches, researchers have also
used Markov decision process (MDP) to investigate this problem. In [8], the
authors developed a MDP and defined the reward per step. The optimal policy is
then deduced using dynamic programming. A Partially Observable MDP (POMDP) was
proposed in [13] to model driving scenarios at unsignalized intersections,
which introduces uncertain factors and intention of drivers. The authors of
[14] proposed a lane-level intersection estimation method using Markov Chain
Monte Carlo sampling, which eliminates the dependence on maps. This estimation
is based on trajectories of other traffic participants. Reinforcement learning
(RL) has also been used in decision-making for AVs. The authors of [15] used
the RL approach with value function approximation to build a decision-making
system that is based on simulation. In [10], the authors considered the
interaction between the AV and its environment, then a stochastic MDP was
developed to learn the driving style of the driver. The desired driving
behavior is then obtained using RL approaches. However, in a real-world
scenario, heavy traffic can be expected during rush hours. The complex driving
environment with multiple vehicles poses a big challenge to predict behavior
of other vehicles and can lead to indecisive actions while changing lanes.

Lane-change maneuver has always been a challenge for drivers/AVs on congested
roads, which is a crucial factor that affects road safety [16]. This action
also affects the surrounding vehicles which entail constant change of their
speed and positions. With the intention of minimizing disturbance and
maintaining road safety, there is an urgent need for developing a method that
can calculate and predict safety spacing for lane changing. The method also
needs to consider the lane changing policy of the merging AV and its effect on
both lateral and longitudinal motion.

Toward this goal, this paper proposes a predictive, reward-based MDP for lane-
change maneuver of vehicles on an urban road. The main goal is to plan a path
from the start position to the end position by considering both crash
probabilities with other vehicles and the length of travel. A hidden Markov
model is also developed for crash prediction and avoidance [17]. This paper
focuses on two uncertainties, i.e., vehicle speed range change and lane
change. The proposed approach is validated through numerical analysis and
simulation. The main contributions of the paper are summarized as follows.

  1. 1.

A MDP was developed to find all paths from the start position to the end
position and assign rewards based on the lengths of paths;

  2. 2.

A Markov model with two layers of uncertainties was proposed;

  3. 3.

A cumulative reward model was developed considering both the length of the
path and crash probabilities with other vehicles;

  4. 4.

A numerical simulation was conducted to show the effectiveness of the proposed
approach on two designed traffic scenarios.

##  II Preliminaries and Notation

###  II-A Markov Chains

Markov chain is a discrete-event system and has the memory-less property [18].
It is a stochastic process that defines a set of states and the state
transition probabilities among them.

Figure 1: A Simple Markov Chain Example.

A Markov chain can be described as follows:

  1. 1.

State Space $\chi$.

  2. 2.

Initial state probability vector that assigns each state a probability
measure: $p_{0}(x)=P[X_{0}=x]$, $\forall x\in\chi$.

  3. 3.

State transitions probabilities $p(x,x^{\prime})$ where $x$ is the current
state and $x^{\prime}$ is the next state.

The state transition probability is computed as follows:

$p_{ij}(k)=P[X_{k+1}=S_{j}|X_{k}=S_{i}],$ (1)

where $S_{i}$ is the current state, $S_{j}$ is the next state, $S_{i},\
S_{j}\in\chi$, and $k=0,1,2,...$. The state transition probabilities are
generally represented in the matrix form.

A simple Markov model is shown in Fig. 1. There are three states: A, B, and C.
The arcs connecting these states are transitions and the numbers on these arcs
represent transition probabilities from one state to another. It can also be
observed that the sum of all outgoing transition probabilities at a state sums
up to 1, i.e.,

$\sum_{j}p_{ij}=1.$ (2)

The state transition probability matrix, $P$, of the Markov chain shown in
Fig. 1 can be obtained as:

$P=\begin{bmatrix}0.5&0.25&0.25\\\ 0.4&0.4&0.2\\\ 0.3&0.3&0.4\end{bmatrix}$
(3)

.

###  II-B Hidden Markov Models

A hidden Markov Model (HMM) is a complex stochastic process [19], which finds
its extension from Markov Chains. An HMM generally consists of two layers of
Markov process, with one layer having unobservable or hidden states and with
the other layer whose behavior depends on the first layer. In most practical
applications, the states of the first layer are usually known or can be
assumed. The observation on the second layer (output) depends on the first
layer.

A HMM can be defined using the parameters below:

  * •

$N$ is the number of states.

  * •

$M$ is the number of observations at a state.

  * •

$P=[p_{ij}]$ is the state transition probability matrix.

  * •

$b_{i}$ is the observation sequence probability at state $S_{i}$, where
$i=1,2,3....N$.

In this paper, we use a similar Markov model design with two layers for
vehicle speed range change and lane change, respectively. This will be
explained in detail in Section 2.

###  II-C Markov Decision Processes

A Markov decision process (MDP) is a stochastic control process which is used
for decision-making in uncertain scenarios where the outcomes are random [20]
[21]. In this paper, we will focus on making decisions based on the cumulative
reward over a finite time (until the goal is reached). This is formally known
as a Markov reward process. The Markov reward process can be defined as a
tuple $(S,A,P,R,\gamma)$ where:

  * •

$S$ is the set of states.

  * •

$A$ represents the set of actions available at each state.

  * •

$P$ is the state transition probability matrix.

  * •

$R$ is the reward function and defined as
$R_{s}=\mathbb{E}[R_{t+1}|S_{t}=s]$, $s\in S$.

  * •

$\gamma\in[0,1]$ is the discount factor for future rewards.

The cumulative reward for the process (starting from an initial state and
ending at the goal state) is the total discounted reward until the process
reaches the goal state. The reward model is presented in Equation (4) as:

$G_{t}=R_{t+1}+R_{t+2}+...=\sum_{k}\gamma(k)R_{t+k+1}.$ (4)

###  II-D Depth-first Search (DFS)

Depth-first search (DFS), also known as backtracking [22], is an algorithm
that has been widely used in the field of artificial intelligence. It checks
recursively for all possible paths between the source and destination in a
graph or a tree. A graph is different from a tree since it can have a cycle in
which the same node can be visited twice. DFS starts with source node and
finds a nearest edge to traverse. The nearest node to source can have multiple
edges. DFS chooses one of those edges and traverses further. DFS saves the
nodes visited in a memory. Once it reaches the next node, it checks if it is
the destination node. If the destination is found, it prints the path
traversed. This action continues in a recursive manner until all edges of all
nodes are explored.

##  III Problem Formulation and Research Methods

In this paper, we consider a road segment and partition it into different
grids. One simple example of such grids is shown in Fig. 2 below. The ego
vehicle is in the start grid (marked in orange), other vehicles on the road
are depicted in white grids, and the goal state is marked in green. The main
goal of the ego vehicle is to find a path from the start state to the goal
state with the minimum crash probabilities with other vehicles on the road.

Figure 2: A Simple Example of Road Grids.

###  III-A Geometry of Road Sections

The average lane width on the road in the U.S. is about 4 meters wide. So,
each grid width is set as 4 meters. The average length of a passenger vehicle
on the road is about 5 meters. To give vehicles enough space to take actions,
the length of each grid is set as 10 meters. The basic geometry of the grid
design is depicted in Fig. 3. Note that the ego can also drive diagonally, the
distance to the diagonal grid is also calculated.

Figure 3: Geometry of a Road Grid.

###  III-B Permissible Actions for the Ego Vehicle

In this paper, we assume that there are five predominant actions that the ego
vehicle can take in a road section. This is depicted in Fig. 4. In particular,
the possible actions of the ego vehicle are described below: 1) No lane
change. Move forward to the grid in front; 2) Lane change to the immediate
right grid; 3) Lane change to the immediate left grid; 4) Lane change to the
diagonal right-front grid; and 5) Lane change to the diagonal left-front grid.

Figure 4: Permissible Actions for the Ego Vehicle.

Note that by setting the length of each grid as 10 meters (twice the length of
the vehicle length), the vehicle can operate to its immediate left or right
grid, as depicted in Actions 2 and 3. However, we prohibit these actions
(taking immediate left or immediate right grid) from happening twice in a row
for safety considerations.

###  III-C Grid Structure for Modified DFS

A modified DFS algorithm is used in this paper. The initial step is to convert
road grids into a connected graph. The modifications were made to the graph
edges. An un-directed grid graph was converted to a directed grid graph by
adding only the edges which represent the permissible actions. Each column in
the grid structure represents a lane. Assuming that $i$ represents rows and
$j$ represents columns. The permitted actions for the ego vehicle in the grid
are depicted as follows for appropriate $i$ and $j$ (corresponding to the
actions shown in Fig. 4).

  * •

$(i,j)$ $\rightarrow$ $(i+1,j)$

  * •

$(i,j)$ $\rightarrow$ $(i,j+1)$

  * •

$(i,j)$ $\rightarrow$ $(i,j-1)$

  * •

$(i,j)$ $\rightarrow$ $(i+1,j+1)$

  * •

$(i,j)$ $\rightarrow$ $(i+1,j-1)$

The grid structure for the road grids shown in Fig. 2 is depicted in Fig. 5.
The newly obtained graph is fed as an input to the modified DFS. All paths
from the start state to the goal state are obtained and stored as a list of
paths. In order to ensure that $(i,j)$ $\rightarrow$ $(i,j+1)$ and $(i,j)$
$\rightarrow$ $(i,j-1)$ are not performed consecutively, the list of paths are
filtered to remove forbidden actions.

Figure 5: Grid structure for DFS for the Road Grid shown in Fig. 2.

###  III-D Reward Model

In this paper, we consider a reward model that prioritizes the shortest length
of the ego vehicle travel path.

Assume that the total number of paths for DFS is $n$ and $T$ is the set of all
paths, i.e.,

$\\{{T_{s},T_{1},T_{2},T_{3},...,T_{l}}\\}\in T,$

where $T_{s}$ is the shortest path and $T_{l}$ is the longest path.

For a path $T_{n}$ (where $n\in{s,1,2,...,l}$), we define:

$T_{n}=\\{T_{n_{1}},T_{n_{2}},T_{n_{3}},...,T_{n_{m}}\\},$

where $\\{T_{n_{1}},T_{n_{2}},T_{n_{3}},...,T_{n_{m}}\\}$ are waypoints of the
path $T_{n}$ from the start state to the goal state. An illustrative example
of waypints for the road grid shown in Fig. 2 is depicted in Fig. 6.

Figure 6: An Illustrative Example of Waypoints for the Road Grid shown in Fig.
2.

Let $L$ be the function that defines the length of each path. For instance,
$L(T_{n})$ gives the length of path $T_{n}$. Assuming the reward for the
shortest path $T_{s}$ is 1 and defined as:

$r_{t}(s)=1.$

Then we can define the reward for the $n^{th}$ path $T_{n}$ as:

$r_{t}(n)=1-(L(T_{n})-L(T_{s}))/L(T_{s}).$ (5)

Equation (5) ensures that paths have a longer length than the shortest path
receive a reward less than 1 and the paths that are more than twice the length
of the shortest path have a negative reward. This reward model considers the
length of the path travelled by the ego vehicle. The reward for the length of
the path constitutes about one-third of the cumulative reward that the ego
vehicle gets for taking a particular path.

###  III-E Markov Models for Other Vehicles

Figure 7: Markov Model for Speed Range Change.

We firstly design a speed range change Markov model for other vehicles except
the ego vehicle using historical traffic data.

Assuming that there are $n$ other vehicles in a typical traffic scenario,
referred as $\\{C_{1},C_{2},C_{3},.....,C_{n}\\}$. Let $\\{A,B,C,...,L\\}$ be
the speed ranges of these vehicles. We partition each speed range as a 5 mph
(miles per hour) increment, for instance, $A\rightarrow(0-5)\
mph,B\rightarrow(5-10)\ mph,\ldots,L\rightarrow(55-60)\ mph$. The speed is
restricted to be below 60 mph since we consider an urban driving scenario.

$P_{m}=\setcounter{MaxMatrixCols}{12}\begin{bmatrix}0.85&0.15&0&0&0&0&0&0&0&0&0&0\\\
0.01&0.8&0.19&0&0&0&0&0&0&0&0&0\\\ 0&0&0.99&0.01&0&0&0&0&0&0&0&0\\\
0&0&0.01&0.98&0.01&0&0&0&0&0&0&0\\\ 0&0&0&0.01&0.98&0.01&0&0&0&0&0&0\\\
0&0&0&0&0.01&0.98&0.01&0&0&0&0&0\\\ 0&0&0&0&0&0.01&0.98&0.01&0&0&0&0\\\
0&0&0&0&0&0&0.01&0.98&0.01&0&0&0\\\ 0&0&0&0&0&0&0&0.01&0.98&0.01&0&0\\\
0&0&0&0&0&0&0&0&0.01&0.98&0.01&0\\\ 0&0&0&0&0&0&0&0&0&0.01&0.98&0.01\\\
0&0&0&0&0&0&0&0&0&0&0.01&0.99\end{bmatrix}$ (6)

The speed-range change Markov chain is depicted in Fig. 7. There are 12 states
in the model so the state transition matrix is computed as a $12\times 12$
matrix. An example of the speed-range change transition probability matrix,
$P_{m}$, is given in Equation (6). Note that states in $P_{m}$ correspond to
vehicles $\\{C_{1},C_{2},C_{3},.....,C_{n}\\}$. For example, $P_{5}[3,4]$
gives the probability of vehicle $C_{5}$ to go from speed range $(10-15)$ mph
to $(15-20)$ mph.

The lane change Markov model has been designed as a second layer to the speed-
range change Markov model. For each speed-range, a lane change Markov model is
developed. Each lane change Markov model has five states to represent the five
lanes considered in Fig. 2. The connection between speed-range change model
and lane change model are illustrated in Fig. 8. The state transition
probability matrix, $Q_{m}(V)$, has a dimension of $5\times 5$, where
$m=\\{1,2,3,...,n\\}$ that corresponds to vehicles
$\\{C_{1},C_{2},C_{3},.....,C_{n}\\}$, respectively, $V=\\{A,B,C,...,L\\}$. An
example of the lane change state transition probability matrix is shown in
Equation (7).

Figure 8: Markov Model for Lane Change.

$Q_{m}(V)=\begin{bmatrix}0.8&0.2&0&0&0\\\ 0.1&0.7&0.2&0&0\\\
0&0.3&0.6&0.1&0\\\ 0&0&0.2&0.7&0.1\\\ 0&0&0&0.4&0.6\\\ \end{bmatrix}$ (7)

###  III-F Calculation of Crash Probabilities

After the two Markov models are developed, the crash probabilities can be
calculated at every waypoint in the feasible paths. Consider the path $T_{n}$
with a set of waypoints
$\\{T_{n_{1}},T_{n_{2}},T_{n_{3}},\ldots,T_{n_{m}}\\}$. Let the speed of the
ego vehicle be $V_{e}$ and the speed of the other vehicles be
$\\{V_{C_{1}},V_{C_{2}},V_{C_{3}},...\\}$. We first find the time taken for
the ego vehicle to reach a waypoint. Let $d_{e}$ be the distance from the
start state to a waypoint $T_{n_{q}}\in T_{n}$, then the travel time $t_{e}$
can be calculated as

$t_{e}=d_{e}/V_{e}.$ (8)

The next step is to find the probability of other vehicles
$\\{C_{1},C_{2},...\\}$ to reach the waypoint at the same time as the ego
vehicle. Consider vehicle $C_{m}$, the speed required by the vehicle to reach
a waypoint, $T_{n_{q}}\in T_{n}$, at the same time as the ego vehicle is given
by

$V_{CC_{m}}=d_{c_{1}}/t_{e}.$ (9)

The speed change state transition probability matrix for vehicle $C_{m}$ at
time $t_{e}$ is $P_{m}^{t_{e}}$. The next step is to find the transition
probability from $V_{C_{m}}$ to $V_{CC_{m}}$. Assuming that $V_{C_{m}}$ is in
the speed range $i$ and $V_{CC_{m}}$ is in the speed range $j$, where
$(i,j)\in\\{A,B,C,\ldots,L\\}$. The transition probability from $i$ to $j$ is
given by $P_{m}^{t_{e}}[i,j]$, which is the element at $i^{th}$ row, $j^{th}$
column position of matrix $P_{m}^{t_{e}}$. Let this transition probability be
$P_{C_{m}}$.

Now we use the second-layer lane change Markov model at the required speed of
$V_{CC_{m}}$ to find the probability of vehicle $C_{m}$ to be at the same lane
as the way-point $T_{n_{q}}$. At the speed range $j$ of $V_{CC_{m}}$, the lane
change state transition probability matrix at time $t_{e}$ is
$Q_{m}^{t_{e}}(j)$, where $j\in\\{A,B,C,\ldots,L\\}$. Assuming that vehicle
$C_{m}$ is in lane $m$ and the waypoint $T_{n_{q}}$ is in lane $n$, the
probability of transition from $m$ to $n$ is given by $Q_{m}^{t_{e}}(j)[m,n]$.
Let this transition probability be $P_{L_{m}}$.

Now we have the probability of vehicle $C_{m}$ being at the speed needed to be
at the same waypoint of the ego vehicle and the probability of being at the
same lane as the waypoint. The crash probability at $T_{n_{q}}$ can be
calculated as:

$P_{CC_{m}}=P_{C_{m}}\times P_{L_{m}}.$

Similarly, the crash probabilities of all other surrounding vehicles at
waypoint $T_{n_{q}}$ can be calculated as
$\\{P_{CC_{1}},P_{CC_{2}},P_{CC_{3},\ldots}\\}$. The probabilities of no
crashes will be given by
$\\{(1-P_{CC_{1}}),(1-P_{CC_{2}}),(1-P_{CC_{3}}),....\\}$.

###  III-G Reward Formulation

The reward for moving to a waypoint $T_{n_{q}}$ for the ego vehicle is the sum
of all non-crash probabilities from all surrounding vehicles, that is,

$r_{C_{n}}(T_{n_{q}})=\frac{1}{w}\sum^{w}_{k=1}(1-P_{CC_{k}}).$ (10)

Note that $r_{C_{n}}(T_{n_{q}})$ is the reward for the ego vehicle to be at
the waypoint $T_{n_{q}}$. A factor of $\frac{1}{w}$ is used to normalize the
reward calculation across all scenarios, where $w$ is the number of
surrounding vehicles. The rewards for all waypoints along the travel path
$T_{n}$ can be calculated in a similar fashion as
$\\{r_{C_{n}}(T_{n_{2}}),r_{C_{n}}(T_{n_{3}}),r_{C_{n}}(T_{n_{4}})...\\}$.
Thus, the cumulative reward for the ego vehicle to take a particular travel
path is given as:

$R_{T_{x}}=(\frac{r_{t}(x)}{3}+\frac{1}{g}\sum^{g}_{k=2}\gamma^{k}r_{c_{x}}(T_{x_{k}}))\times
100$ (11)

In Equation (11), $x$ is the path ID. For instance, path $T_{3}$ has an ID 3.
$g$ is the number of waypoints in path $x$. $r_{c_{x}}(T_{x_{k}})$ is the
reward for waypoint $k$ in path $x$. $r_{t}(x)$ is the reward considering the
length of the path as explained in Equation (5). In order to prioritize
immediate predictions, a discount factor, $\gamma$, is introduced for future
predictions. A factor of $\frac{1}{g}$ is used to normalize the reward
calculations. Note that different paths can have different lengths and the
path with the highest reward is chosen as the safest path to travel for the
ego vehicle.

##  IV SIMULATION AND RESULTS

The numerical simulation is completed using historical data from the U.S.
Department of Transportation’s (US-DOT) Intelligent Transportation Systems
(ITS) [23]. The data was collected using an instrumented vehicle driven by
three drivers between 3-6 pm for 20 days. Two traffic scenarios were designed
in our simulation considering different speeds for the ego vehicle and other
vehicles. The road geometry is considered to be a 5-lane segment with a length
of 60 meters, as shown in Fig. 2. Assuming that the ego vehicle is initially
at grid $(0,0)$ and its goal state is at grid $(5,4)$. Firstly, all feasible
paths from $(0,0)$ to $(5,4)$ are determined with the permissible actions
using the two Markov models developed and modified DFS. A total of 1,921 paths
are identified. The rewards for these paths (with possibly different lengths)
are given in Fig. 9. The highest reward is 1 and the least reward is 0.22.
These paths are then used to find the best crash-free path using the method
proposed in Section 2.

Figure 9: Rewards for Paths with Different Lengths.

###  IV-A Scenario 1

Assuming that the ego vehicle (marked in Blue in Fig. 11) is currently at grid
$(0,0)$ and its goal state is at grid $(5,4)$. Assume that the speed of the
ego vehicle is 30 mph. Let other vehicles (marked in Black in Fig. 11) on the
road be $\\{{C_{1}},{C_{2}},{C_{3}}\\}$. Assume that they are initially at
grids $(1,0)$, $(1,2)$, and $(2,3)$, respectively, and their speed is given by
$V_{C_{1}}$=30 mph, $V_{C_{2}}$=35 mph, and $V_{C_{3}}$=30 mph, respectively.

The decision of the ego vehicle on which path to take depends on the
cumulative rewards of all waypoints in a particular path, which is depicted in
Fig. 11. The maximum reward for Scenario 1 was found to be 46.21 and the
minimum reward is 20.34.

Figure 10: Results for Scenario 1. Figure 11: Cumulative Rewards for Paths in
Scenario 1.

The result of the simulation is shown in Fig. 10, which identifies each
waypoint of the safest path at each time step. In particular, the path is
given by $T=[(0,0),(1,1),(1,2),(2,3),(3,4),(4,4),(5,4)]$. The reward at each
waypoint (calculated by Equation (10)) in $T$ is given in Table I.

TABLE I: Total Rewards at Each Waypoint in Scenario 1. Waypoint | Reward
---|---
(0,0) | 1
(1,1) | 0.898
(1,2) | 0.809
(2,3) | 0.729
(3,4) | 0.656
(4,4) | 0.590
(5,4) | 0.531

###  IV-B Scenario 2

Figure 12: Results for Scenario 2.

The second scenario is set up in the same way as Scenario 1 except that the
positions of surrounding vehicles and their speed are different. Assume that
other vehicles on the road are initially at grids $(0,1)$, $(0,2)$, and
$(2,3)$, respectively, and their speed is given by $V_{C_{1}}=$ 20 mph,
$V_{C_{2}}$=15 mph, and $V_{C_{3}}$=10 mph, respectively.

Figure 13: Cumulative Rewards for Paths in Scenario 2.

Assume that the initial speed of the ego vehicle is 15 mph for this scenario.
The cumulative rewards of all waypoints in a path are calculated using
Equation (11), which are plotted in Fig. 13. The maximum reward that a path
can achieve in this scenario is 43.58 and the minimum reward for this scenario
is 19.91. The highest reward possible in this scenario is lower than that in
Scenario 1 because the ego vehicle is forced to take a longer path due to the
initial positions of other vehicles on the road. The results of the simulation
is shown in Fig. 12, which identifies each waypoint of the safest path at each
time step. In particular, the path is given by
$T=[(0,0),(0,1),(1,2),(1,3),(2,4),(3,4),(4,4),(5,4)]$. The reward at each
waypoint (calculated by Equation (10)) in $T$ is given in Table II.

TABLE II: Total Rewards at Each Waypoint in Scenario 2. Waypoint | Reward
---|---
(0,0) | 1
(0,1) | 0.898
(1,2) | 0.799
(1,3) | 0.729
(2,4) | 0.656
(3,4) | 0.590
(4,4) | 0.531
(5,4) | 0.478

##  V Conclusion

In this paper, a Markov decision process was developed for studying lane-
change maneuver of drivers/AVs in an urban setting. A hidden Markov model was
built to predict the crash probabilities of the ego vehicle with surrounding
vehicles and formulate a way to calculate the reward for taking a particular
travel path. A numerical simulation was conducted on two designed traffic
scenarios. The results showed that a practically viable and safe path can be
identified in both cases using the proposed approach.

One of the future research directions is to implement a vehicle dynamic model
into the system and design a controller concerning the vehicle movement. It is
also interesting to study different reward models based on underlying
applications.

## References

  * [1] X. Li, Z. Sun, D. Cao, Z. He, and Q. Zhu, “Real-time trajectory planning for autonomous urban driving: Framework, algorithms, and verifications,” IEEE/ASME Transactions on Mechatronics, vol. 21, no. 2, pp. 740-753, April 2016.
  * [2] Y. Chen and L. Li. Advances in Intelligent Vehicles. ISBN-10: 0123971993, ISBN-13: 978-0123971999, Academic Press, December 2013.
  * [3] L. Chen, Q. Ding, Q. Zou, Z. Chen, and L. Li, “DenseLightNet: A light-weight vehicle detection network for autonomous driving,” IEEE Transactions on Industrial Electronics, vol. 67, no. 12, pp. 10600-10609, December 2020.
  * [4] X. Wang, S.-S. Han, L. Yang, T.-T. Yao, and L. Li, “Parallel internet of vehicles: ACP-based system architecture and behavioral modeling,” IEEE Internet of Things Journal, vol. 7, no. 5, pp. 3735-3746, May 2020.
  * [5] Y. Chen, Y. Lv, X. Wang, L. Li, and F.-Y. Wang, “Detecting traffic information from social media text with deep learning approaches,” IEEE Transactions on Intelligent Transportation Systems, vol. 20, no. 8, pp. 3049-3058, August 2019.
  * [6] R. Tian, K. Ruan, L. Li, J. Le, J. Greenberg, and S. Barbat, “Standardized evaluation of camera-based driver state monitoring systems,” IEEE/CAA Journal of Automatica Sinica, vol. 6, no. 3, pp. 716-732, May 2019.
  * [7] A. Lopez, S. Chien, L. Li, Q. Yi, Y. Chen, and R. Sherony, “Certainty and critical speed for decision making in tests of pedestrian automatic emergency braking systems,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 6, pp. 1358-1370, June 2017.
  * [8] Y. Guan, S.-E. Li, J. Duan, W. Wang, and B. Cheng, “Markov probabilistic decision making of self-driving cars in highway with random traffic flow: A simulation study,” Journal of Intelligent and Connected Vehicles, vol. 1, no. 2, pp. 77-84, June 2018.
  * [9] X. Xu, L. Zuo, X. Li, L. Qian, J. Ren, and Z. Sun, “A reinforcement learning approach to autonomous decision making of intelligent vehicles on highways,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 50, no. 10, pp. 3884-3897, October 2020.
  * [10] C. You, J. Lu, D. Filev, and P. Tsiotras, “Autonomous planning and control for intelligent vehicles in traffic,” IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 6, pp. 2339-2349, June 2020.
  * [11] A. Artuñedo, G. Corrales, J. Villagra, and J. Godoy, “Machine learning based motion planning approach for intelligent vehicles,” in Proc. 2020 IEEE Intelligent Vehicles Symposium, pp. 963-970, Las Vegas, NV, USA, October 2020.
  * [12] R. S. Tomar, S. Verma, and G. S. Tomar, “Neural network based lane change trajectory predictions for collision prevention,” in Proc. 2011 International Conference on Computational Intelligence and Communication Networks, pp. 559-564, Gwalior, India, 2011.
  * [13] D. Kye, S. Kim, and S. Seo, “Decision making for automated driving at unsignalized intersection,” in Proc. 15th International Conference on Control, Automation and Systems, pp. 522-525, Busan, South Korea, 2015.
  * [14] A. Meyer, J. Walter, and M. Lauer, “Fast lane-level intersection estimation using Markov chain monte carlo sampling and B-spline refinement,” in Proc. 2020 IEEE Intelligent Vehicles Symposium, pp. 71-76, Las Vegas, NV, USA, October 2020.
  * [15] Rui Zheng, Chunming Liu, and Qi Guo, “A decision-making method for autonomous vehicles based on simulation and reinforcement learning,” in Proc. 2013 International Conference on Machine Learning and Cybernetics, pp. 362-369, Tianjin, China, 2013.
  * [16] A. Kanaris, E. B. Kosmatopoulos, and P. A. Loannou, “Strategies and spacing requirements for lane changing and merging in automated highway systems,” IEEE Transactions on Vehicular Technology, vol. 50, no. 6, pp. 1568-1581, November 2001.
  * [17] A. Prabu, L. Li, B. King, and Y. Chen, “A method for crash prediction and avoidance using hidden Markov models,” in Proc. 2019 IEEE International Conference on Service Operations and Logistics, and Informatics, pp. 146-151, Zhengzhou, China, November 2019.
  * [18] C. G. Cassandras and S. Lafortune, Introduction to Discrete Event Systems. 2nd edition. Springer, 2008.
  * [19] L. R. Rabiner, “A tutorial on hidden Markov models and selected applications in speech recognition,” Proceedings of the IEEE, vol. 77, no. 2, pp. 257–286, February 1989.
  * [20] R. Bellman, “A Markovian decision process,” Journal of Mathematics and Mechanics, vol. 6, no. 5, pp. 679-684, 1957.
  * [21] R. Bellman, “Dynamic Programming,” Science 153, no. 3731, pp. 34-37, 1966.
  * [22] T. H. Cormen, C. E, Leiserson and R. Rivest, and C. Stein, Introduction to Algorithms. 3rd Edition. MIT Press, 2009.
  * [23] U.S. Department of Transportation Intelligent Transportation Systems Joint Program Office. (2016). Active Transportation Demand Management (ATDM) Trajectory Level Validation. [Dataset]. Provided by ITS DataHub through Data.transportation.gov. Accessed 2021-01-10 from http://doi.org/10.21949/1504468.

|  Avinash Prabu received the B.E. degree in Electrical and Electronics
Engineering from Anna University, Chennai, India, in 2015, M.S. degree in
Electrical and Computer Engineering from Indiana University-Purdue University
Indianapolis (IUPUI), USA, in 2019. He is now pursuing the Ph.D. degree in
Electrical and Computer Engineering at IUPUI. His main research interests
include motion planning, control, and optimization of connected and automated
vehicles, active safety systems, and human-machine interaction.
---|---
|  Niranjan Ravi received his bachelors from Anna University, Chennai, India,
2015 in the field of Electronics and Instrumentation Engineering. He completed
his master’s degree from Indiana University-Purdue University, Indianapolis,
in 2019, with his research in embedded systems and IoT. Currently, he is
pursuing his Ph.D. in field of Machine learning. His research interests
include Embedded Systems, Internet of Things, and UAV systems.
---|---
|  Lingxi Li received the B. E. degree in Automation from Tsinghua University,
Beijing, China, in 2000, the M. S. degree in Control Theory and Control
Engineering from the Institute of Automation, Chinese Academy of Sciences,
Beijing, China, in 2003, and the Ph. D. degree in Electrical and Computer
Engineering from the University of Illinois at Urbana-Champaign, in 2008.
Since August 2008, he has been with Indiana University-Purdue University
Indianapolis (IUPUI) where he is currently an Associate Professor of
Electrical and Computer Engineering. Dr. Li’s current research focuses on the
modeling, analysis, control, and optimization of complex systems; intelligent
transportation systems; connected and automated vehicles; active safety
systems; and human factors. He has authored/co-authored over one book and 100+
research articles in refereed journals and conferences and received two U.S.
patents. Dr. Li received a number of awards including three best/remarkable
paper awards, outstanding research contributions award, outstanding editorial
service award, and university research/teaching awards. He is currently
serving as an associate editor for four IEEE Transactions/Journals and the
Vice President of the IEEE Intelligent Transportation Systems Society. Dr. Li
is a Senior Member of the IEEE and Chinese Association of Automation, and a
Member of the SAE and AAAS.
---|---