# Learning-based Collision-free Planning on Arbitrary Optimization Criteria in
the Latent Space through cGANs

Tomoki Andoa, Hiroto Iinoa, Hiroki Moria, 1, Ryota Torishimab, 2,
Kuniyuki Takahashic, Shoichiro Yamaguchic,
Daisuke Okanoharac, and Tetsuya Ogataa CONTACT<EMAIL_ADDRESS>{iino<EMAIL_ADDRESS><EMAIL_ADDRESS>
{takahashi, guguchi<EMAIL_ADDRESS><EMAIL_ADDRESS>Waseda University
& AIST, Tokyo, Japanb SoftBank Corp., Tokyo, Japanc Preferred Networks, Inc.,
Tokyo, Japan1 H. Mori was a researcher in Cergy-Pontoise University when he
came up with the idea.2 This work was an achievement while he was at Waseda
University & AIST.

###### Abstract

We propose a new method for collision-free planning using Conditional
Generative Adversarial Networks (cGANs) to transform between the robot’s joint
space and a latent space that captures only collision-free areas of the joint
space, conditioned by an obstacle map. Generating multiple plausible
trajectories is convenient in applications such as the manipulation of a robot
arm by enabling the selection of trajectories that avoids collision with the
robot or surrounding environment. In the proposed method, various trajectories
that avoid obstacles can be generated by connecting the start and goal state
with arbitrary line segments in this generated latent space. Our method
provides this collision-free latent space, after which _any_ planner, using
_any_ optimization conditions, can be used to generate the most suitable paths
on the fly. We successfully verified this method with a simulated and actual
UR5e 6-DoF robotic arm. We confirmed that different trajectories could be
generated depending on optimization conditions. 333An accompanying video is
available at the following link:
https://www.youtube.com/watch?v=IJUxdmaSwy0

###### keywords:

Learning-based Collision-free Planning; cGANs; Learning from Experience;
Representation Learning;

††articletype: Full paper Figure 1:  Collision-free planning for robot arm
using latent space of cGANs. Latent variables $\bm{z}_{s}$ and $\bm{z}_{g}$
that correspond to the start $\bm{\theta}_{s}$ and goal $\bm{\theta}_{g}$
joint angles of the robot arm. Any path in the latent space $\bm{z}_{s:g}$ is
mapped to a collision-free path in the joint space $\bm{\theta}_{s:g}$ using
generator ${G}$ with condition $\bm{c}$ as obstacle information. On the other
hand, $E$ is an inverse transformation of the generator ${G}$, in which the
joint angles map to the latent variables.

##  1 Introduction

Collision-free planning is essential for robots working in various
environments. Multiple (potentially infinite) paths exist from a given start
to a goal. It is necessary to calculate the optimal path according to certain
desired criteria, such as the minimization of the velocities, acceleration, or
jerk for the robot’s movements ( i )  _Customizability_). There are other two
important factors in collision-free planning: ii )  _Adaptability_ and iii )
_Scalability of computation_. ii ) Robots need to adapt quickly to a new
environment, which requires appropriate planning for the placement of
untrained obstacles. iii ) These planning operations should be calculable,
even when there are many obstacles, since it generally takes a long time to
collision-check for a large number of obstacles. In other words, calculation
time should scale well with the number of obstacles.

It is challenging to meet these three requirements using existing methods (see
Section 2). Contrary to traditional planning in Cartesian or joint space, we
propose to plan trajectories within a new collision-free space as follows. Our
method consists of two steps: 1) construction of latent space and 2) planning
using this latent space. Conditional Generative Adversarial Networks (cGANs)
[1, 2] are used to map joint space to latent space corresponding to its
collision-free areas, such that the robot does not collide with obstacles if a
path is planned within this latent space (See Fig. 1). That is, selecting any
point in the latent space yields a particular robot pose that does not collide
with obstacles. There are several methods for acquiring such a latent space.
We use GANs since they offer the advantage of arbitrarily determining the
distribution of the latent space. The uniform distribution of $[0,1]$ as a
latent space allows the region to be clearly defined and is a convex space.
The mappings from the latent space to joint space _adapts_ to various
environments by changing according to obstacle information given to the cGANs
as a condition. The latent space is collision-free; any point on a line
connecting any two points is also within this latent space (since this is a
convex space). Thus, a collision-free path can be generated by connecting the
start and goal states with any arbitrary line or curve in the latent space
within a domain of definition. Then, the joint trajectory corresponding to the
trajectory planned in latent space is constructed through the learned
mappings. Since we separated the learning of the mappings and the actual
planning (or trajectory selecting), we can generate any trajectory we want on
the fly for any optimization parameters that we want without considering
collisions, making our method highly _customizable_. Furthermore, since
planning is performed in the latent space without colliding with obstacles,
ideally there is no need for collision-check for obstacles. However, in
practical use, it is challenging to guarantee 100% obstacle avoidance using
only learning-based methods. The learning method can generate a trajectory but
may slightly collide with obstacles. A collision check is performed for the
generated trajectory, and only the collision points are modified using the
existing planning method (in our case RRT Connect [3]). The computation time
for path planning can be reduced compared to existing planning methods for all
trajectories. The computation time for our proposed learning method does not
depend on the number of obstacles, making it _scalable_ for complex
environments. The most significant advantage compared to existing methods is
_customizability_ , where trajectories satisfying arbitrary optimization
criteria can be easily generated in this latent space. The adaptability of
trajectory generation to changes in the environment and the computational time
for the 6-DoF robot were also evaluated, showing the potential for future
expansion.

##  2 Related Work

There are mainly two planning methods: model-based and learning-based. The
following two model-based methods are the most common: Design functions for
obstacles and goals (e.g., potential fields [4, 5] and Riemannian motion
policies with improved potential fields [6]), search and optimization (e.g.,
RRTs [7, 3, 8, 9], $A^{\ast}$ [10], and gradient-based approach [11]). Methods
that combine these are also proposed and generally show improved results [12,
13, 14, 15, 16]. While model-based methods can reliably avoid obstacles, their
adaptability to various environments in real-time is limited since these
methods require specific function design and adjustment of parameters for each
situation in advance, not to mention the enormous computational searching
cost. As model-based methods are usually calculated according to certain
conditions/criteria, such as shortest traveling distance in end-effector space
or joint space [17] or minimum jerk change [18], other calculations must be
performed when these criteria change. In other words, model-based methods lack
_scalability_ and _customizability_.

The data collected by the model-based methods can be used to train learning-
based algorithms, particularly deep learning [19, 20, 21, 22, 23, 24, 25, 26,
27]. These algorithms can infer a path for a new environment in a short time
if it has been trained sufficiently in advance. However, learning-based
methods have the challenge that only one or a few paths can be generated, and
what kind of paths are generated depends on the training data. For example, if
naive RRT is used as training data, only collision-free paths to the goal will
be generated during inference, usually without considering any additional
constraints that naive RRT also does not. Usually, learning-based methods lack
_customizability_.

In [28, 29], the authors studied the generation of multiple trajectories.
Since the target of [28] was to generate various trajectories in environments
with no obstacles, obstacle avoidance was out of their scope. Another method
is a learning-based approach with deep learning to perform dimensionality
compression and planning in latent space to handle high-dimensional
information [30]. Their approach requires collision detection for each
planning process. Our proposed method is to plan paths in a collision-free
space which are mapped from the latent space to joint space. Hence, our method
does not require a collision check for each planning process. Since the
trajectory of [29] is fixed once it is generated, at best, only the optimal
trajectory among the ones generated can be selected, which is not necessarily
the best for the situation at hand. Thus, they have to generate trajectories
until one of them satisfies the criteria necessary for the situation, but they
are generated randomly, and the method does not provide a way to define
optimality. Our method does not directly output the trajectories but provides
a collision-free space after which any planner, using any optimization
conditions, can generate the most suitable paths.

The contribution of this research is to realize optimized planning with the
three factors; i )  _Customizability_ , ii )  _Adaptability_ , and iii )
_Scalability of computation_.

Figure 2: Structure of collision-free planning model using cGANs. The models
are optimized with the four objective functions with coefficients $\lambda$s
of each $\mathcal{L}$.

##  3 Method

Our proposed method consists of the following two steps: 1) Construction of a
latent space corresponding to the joint space to avoid collision for
_Adaptability_ (Section 3.1), and 2) planning according to the objective using
the constructed latent space for _Customizability_ and _Scalability of
computation_ (Section 3.2).

###  3.1 Training cGANs

We propose a method that maps the latent space of cGANs to the collision-free
area of the robot’s joint space so that the robot learns not to collide with
obstacles. Thus, any planned path in that latent space can be associated with
a collision-free path in joint space. The mapping from the latent space to the
joint space _adapts_ accordingly to the obstacle information given to cGANs as
a condition. The correspondence from the latent space to joint space is
trained by cGANs, which uses a min-max game between a generator $G$ and a
discriminator $D$. Also, encoder $E$ is trained to approximate an inverse
transformation of $G$; i.e., $E$ is trained to be equivalent to $G^{-1}$.

$\min_{G,E}\max_{D}V\left(D,G,E\right)$

These models are optimized alternately with the following objective function
with coefficients $\lambda$s of each $\mathcal{L}$:

$\begin{split}V\left(D,G,E\right)=&\,\lambda_{\mathrm{GAN}}\,\mathcal{L}_{\mathrm{GAN}}\left(D,G\right)+\,\lambda_{\mathrm{rec}}\,\mathcal{L}_{\mathrm{rec}}\left(G,E\right)\\\
&+\lambda_{\mathrm{map}}\,\mathcal{L}_{\mathrm{map}}\left(G\right)+\lambda_{\mathrm{col}}\,\mathcal{L}_{\mathrm{col}}\left(D\right)\end{split}$
(1)

  1. 1.

$\mathcal{L}_{\mathrm{GAN}}$: The primary loss function to learn the mapping
from the latent space to the joint space.

  2. 2.

$\mathcal{L}_{\mathrm{rec}}$: The loss function constrains latent space and
joint space so that they can be reconstructed with each other.

  3. 3.

$\mathcal{L}_{\mathrm{map}}$: The loss function constrains the transformation
from the latent space to the joint space to be smooth.

  4. 4.

$\mathcal{L}_{\mathrm{col}}$: The loss function to learn various obstacle
situations even if the collision data is a small number on the whole,
including non-collision data.

The detail of four $\mathcal{L}$s will be explained in the following Section
3.1.1 through Section 3.1.4.

####  3.1.1 $\mathcal{L}_{\mathrm{GAN}}$: Construction of Latent Expression

cGANs are used to construct the mapping from the latent space to the joint
space. In GANs [1], latent expressions are constructed by training two models,
a generator $G$ and a discriminator $D$, alternately. The generator $G$
creates data variables $\bm{\theta}_{\mathrm{model}}$ from latent variables
$\bm{z}$. The discriminator $D$ estimates whether given variables are a sample
from the data set $\bm{\theta}_{\mathrm{non\mathchar 45\relax col}}$ or a
generated sample $\bm{\theta}_{\mathrm{model}}$ calculated from $\bm{z}$,
which is uniformly sampled from the latent space within $[0,1]$. That is, for
an N-dimensional vector as a latent variable, each of its elements is in the
range $[0,1]$. Since the latent space is a convex space and the boundaries of
the latent space can be arbitrarily determined in advance, any point of a line
segment connecting any point is in that latent space within a domain of
definition. Furthermore, it is possible to give conditions to the models by
introducing a $condition$ variable $\bm{c}$ [2]. In our case, $\bm{c}$ is a
depth image as the obstacle information.

Fig. 2 shows the concept of the proposed network model. Through the generator
$G$, the mapping from the latent space to collision-free joint space is
obtained. The discriminator $D$ identifies the joint angles, generated joint
angles $\bm{\theta}_{\mathrm{model}}$ by the generator $G$, and the actual
sampled joint angles $\bm{\theta}_{\mathrm{non\mathchar 45\relax col}}$. The
obstacle information is given as a depth image in condition $\bm{c}$. This
condition $\bm{c}$ is connected to the generator $G$ and the discriminator $D$
so that when the given obstacle information changes, the correspondence from
the latent space to joint space changes. In other words, our method does not
need to prepare a different network for each obstacle, and only one cGANs can
support multiple obstacle environments. The loss function,
$\mathcal{L}_{\mathrm{GAN}}$, for training cGANs is shown in equation (2).

$\begin{split}\mathcal{L}_{\mathrm{GAN}}(D,&G)=\mathbb{E}_{\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{\theta}\sim p_{\mathrm{non\mathchar
45\relax col}}\left(\bm{\theta}|\bm{c}\right)}\left[\log
D\left(\bm{\theta},\bm{c}\right)\right]\\\ &+\mathbb{E}_{\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{z}\sim
p_{\bm{z}}\left(\bm{z}\right)}\left[\log\left(1-D\left(G\left(\bm{z},\bm{c}\right),\bm{c}\right)\right)\right]\end{split}$
(2)

Where $p_{\mathrm{obs}}(\bm{c})$ is the distribution of obstacles positions
and $p_{\mathrm{non\mathchar 45\relax col}}(\bm{\theta}|\bm{c})$ is the
distribution of non-collision joint angles which the generator should aim to
generate. $p_{\bm{z}}(\bm{z})$ is the uniform distribution in the latent
space.

####  3.1.2 $\mathcal{L}_{\mathrm{rec}}$: Reconstruction of latent variables
and joint angles

This section describes an objective function that constrains the latent space
and joint space so that they can be reconstructed from each other. The
transformation from the latent space to joint space can be trained by
$\mathcal{L}_{\mathrm{GAN}}$, but the transformation from the joint space to
latent space requires the inverse transformation of the generator $G^{-1}$.
Therefore, encoder $E$ is trained as an inverse transformation of generator
$G^{-1}$ and simultaneously learns the reconstruction of the latent space and
the joint space so that each point in the joint space and latent space
corresponds one-to-one, which means constraining the generator $G$ to be a
single projection. This loss function, $\mathcal{L}_{\mathrm{rec}}$, is shown
in equation (3).

$\begin{split}\mathcal{L}_{\mathrm{rec}}&(G,E)=\mathbb{E}_{\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{\theta}\sim p_{\mathrm{non\mathchar
45\relax
col}}\left(\bm{\theta}|\bm{c}\right)}\left[\|G\left(E\left(\bm{\theta},\bm{c}\right),\bm{c}\right)-\bm{\theta}\|_{2}^{2}\right]\\\
&+\mathbb{E}_{\bm{c}\sim p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{z}\sim
p_{\bm{z}}\left(\bm{z}\right)}\left[\|E\left(G\left(\bm{z},\bm{c}\right),\bm{c}\right)-\bm{z}\|_{2}^{2}\right]\\\
\end{split}$ (3)

####  3.1.3 $\mathcal{L}_{\mathrm{map}}$: Specifying the Map from the Latent
Space to Joint Space

We will describe in this section how to map from the latent space to joint
space such that arbitrary planned paths in the latent space are smooth in
joint space for robot arms. For planning using a robot arm, the mapping from
the latent space to joint space has to be continuous without “twists”,
“distortions”, and rapid changes. In order to achieve this, the following two
things are performed: a) The number of dimensions for latent variables is
matched to the number of robot joints; each latent variable is mapped to
represent each joint, and the normalized ranges of latent variables and joint
angles are aligned. b) The generator $G$ is trained to output $\bm{\theta}$
when the latent variables $\bm{z}=\bm{\theta}$ are given as input of the
generator $G$. The procedure for calculating $\mathcal{L}_{\mathrm{map}}$ is
as follows three steps. 1) The obstacle condition is determined by sampling
from the distribution of obstacle positions: $\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right)$. 2) Non-collision joint angles is sampled
under this condition $\bm{c}$: $\bm{\theta}\sim p_{\mathrm{non\mathchar
45\relax col}}\left(\bm{\theta}|\bm{c}\right)$. 3) The output of the generator
is constrained to be $\bm{\theta}$ itself when this $\bm{\theta}$ is input as
$\bm{z}=\bm{\theta}$. The generator is trained to be close to an identity map
for non-collision joint angles under condition $\bm{c}$. Colliding joint
angles are not sampled in $\mathcal{L}_{\mathrm{map}}$. The constraint is not
added to the joint that collides with the obstacles so that the constructed
map is allowed to be distorted to avoid collisions. The loss function,
$\mathcal{L}_{\mathrm{map}}$, for training cGANs is shown in equation (4).

$\mathcal{L}_{\mathrm{map}}(G)=\mathbb{E}_{\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{\theta}\sim p_{\mathrm{non\mathchar
45\relax
col}}\left(\bm{\theta}|\bm{c}\right)}\left[\|G\left(\bm{z}=\bm{\theta},\bm{c}\right)-\bm{\theta}\|_{2}^{2}\right]$
(4)

####  3.1.4 $\mathcal{L}_{\mathrm{col}}$: Adaptability to Multiple Obstacle
Conditions

In this section, we describe how to adapt to various obstacle conditions. Even
though collision-free mapping from the latent space to joint space is trained
by equation (2), the network has a risk of mistaking collision points for non-
collision points, and vice versa since the number of non-collision data points
is much smaller than those with collisions. The collision joints must be
explicitly included in the equation during training. The loss function,
$\mathcal{L}_{\mathrm{col}}$, shown in equation (5) is introduced in order to
provide the data of the collision joints to the discriminator $D$.

$\mathcal{L}_{\mathrm{col}}(D)=\mathbb{E}_{\bm{c}\sim
p_{\mathrm{obs}}\left(\bm{c}\right),\,\bm{\theta}\sim
p_{\mathrm{col}}\left(\bm{\theta}|\bm{c}\right)}\left[\log\left(1-D\left(\bm{\theta},\bm{c}\right)\right)\right]$
(5)

Where $p_{\mathrm{col}}(\bm{\theta}|\bm{c})$ is the distribution of colliding
joint angles, including self-collision and collision with obstacles, which the
generator $G$ should thus refrain from generating. The discriminator $D$ is
trained to output $0$ for collision joints and $1$ for collision-free joints
for each obstacle. Furthermore, the generator $G$ is trained to acquire a
distribution to make the discriminator $D$ output $1$, as we are trying to
obtain a distribution for collision-free space.

###  3.2 Planning

In this section, we will describe the planning method. Section 3.2.1 describes
how to generate various optimal trajectories for different purposes, and
Section 3.2.2 explains how to guarantee collision avoidance with obstacles
since learning methods alone cannot wholly avoid them. Then, Section 3.2.3
explains computational complexity.

####  3.2.1 Optimal Path Trajectory Generation

In our method, the mapping and planning phases are separated, unlike
traditional path planners in joint space. Moreover, any path planner can be
used in the trained latent space (where any point is collision-free) without
considering obstacles since there are none in the latent space. Therefore, it
makes our method highly _customizable_. As optimization methods for any
optimization criterion, we can not only use any discrete optimization methods,
such as $A^{*}$ [10] on any graphs in the latent space but also continuous
optimization methods, such as Adam [31], thanks to the differentiable nature
of the generator $G$. The computational cost is also lower since collision
check calculations are no longer necessary, making our method also _scalable_.

As shown in Fig. 1, when the start joint angles $\bm{\theta}_{s}$ and the goal
joint angles $\bm{\theta}_{g}$ are given, the corresponding latent variables
are found by $\bm{z}_{s}=E(\bm{\theta}_{s},\bm{c})$,
$\bm{z}_{g}=E(\bm{\theta}_{g},\bm{c})$. Considering $\bm{z}_{s:g}$ as a path
connecting these in the latent space, the collision-free path is obtained as
$\bm{\theta}_{s:g}=G(\bm{z}_{s:g},\bm{c})$. $\bm{z}_{s:g}$ can be determined
arbitrarily within the latent space, and in the simplest case, it can be
connected by a straight line. Taking advantage of the differentiability of the
generator $G$, the path in the latent space can be calculated by optimizing
the cost function $\mathcal{L}_{\mathrm{opt}}$ to satisfy the objective using
the following equation:

$\begin{split}\mathcal{L}_{\mathrm{opt}}&\,=f(G,\bm{z}_{s:g},\bm{c})\\\
\hat{\bm{z}}_{s:g}&\,=\operatorname*{argmin}_{\bm{z}_{s:g}}\mathcal{L}_{\mathrm{opt}}\end{split}$
(6)

There are a variety of cost functions depending on the objectives. For
example, $\mathcal{L}_{\mathrm{opt}}$ can be minimization of the sum of
squares of velocities
$\mathcal{L}_{\mathrm{opt}}=\sum_{t}\|\bm{v}_{t}\|^{2}_{2}$, minimization of
acceleration $\mathcal{L}_{\mathrm{opt}}=\sum_{t}\|\bm{a}_{t}\|^{2}_{2}$, and
minimization of jerk
$\mathcal{L}_{\mathrm{opt}}=\sum_{t}\|\bm{j}_{t}\|^{2}_{2}$, where
$\bm{v}_{t}=\bm{\theta}_{t}-\bm{\theta}_{t-1}$,
$\bm{a}_{t}=\bm{v}_{t}-\bm{v}_{t-1}$, and
$\bm{j}_{t}=\bm{a}_{t}-\bm{a}_{t-1}$. Also, the path can be optimized by
combining them as the following equation:

$\begin{split}\mathcal{L}_{\mathrm{opt}}=\sum_{t}\|\bm{v}_{t}\|^{2}_{2}+\alpha\sum_{t}\|\bm{a}_{t}\|^{2}_{2}+\beta\sum_{t}\|\bm{j}_{t}\|^{2}_{2}\end{split}$
(7)

####  3.2.2 Collision Avoidance Guarantee (CAG)

The learning method described above does not guarantee 100% obstacle
avoidance. Therefore, the trajectory obtained by the generator $G$ is checked
to ensure that it does not collide with any obstacles. Moreover, the
trajectory is modified to guarantee collision avoidance if a colliding posture
is found. If the path includes a colliding posture, the non-colliding posture
before and after the collision trajectory is modified using existing planning
methods. In this study, we use RRT Connect [3].

####  3.2.3 Computational Complexity

Our proposed method (Ours w/o collision avoidance guarantee (CAG)) performs
forward calculations for the path’s number of iterations $I_{1}$, and the
computational complexity is $O(I_{1})$. Assume that the proposed method’s
number of iterations $I_{1}$ is sufficiently large so that the robot’s degrees
of freedom can be ignored. We do not consider path optimization here. Our
proposed method uses collision-free latent space and the computational
complexity does not depend on the complexity of obstacles. In the simplest
case, the path is a straight line connecting from the start to the goal in the
latent space, and $I_{1}$ is obtained by dividing the straight line distance
$L_{1}$ in latent space by the distance $D_{1}$ per step,
$O(I_{1}=L_{1}/D_{1})$.

The computational complexity of the RRT is $O({I_{2}}^{2})$ for the number of
iterations $I_{2}$ [32, 33]. Assume that the number of iterations $I_{2}$ is
sufficiently large so that the robot’s degrees of freedom can be ignored.
Since RRT’s calculation is performed in space with obstacles, unlike the
proposed method, the more obstacles there are, the more complex the trajectory
from the start to the goal will be, and the more trajectories will collide
with obstacles during the RRT calculation, the more iterations $I_{2}$ will be
required.

Figure 3: Experiment environment and depth image of obstacle. Figure 4:
Examples of a depth image with different $x$ and $y$ positions and the height
of the center plate. The shelf’s color in-depth image changes as the shelf’s
position changes in the $x$-coordinate. The test evaluation was performed with
untrained $x$, $y$, and the height of the center plate.

##  4 Experimental Setup

We performed experiments using a Gazebo simulation and an actual 6-DoF UR5e
robot arm. In learning-based methods using robotic arms in 3-D space, objects
like pillars or shelves are often used as obstacles. We use a shelve for
practical tasks. We set up an environment with a shelf in front of the robot
as an obstacle and evaluated the system’s performance under multiple
conditions by changing the obstacle position of the shelf and the height of
the center plate of the shelf (Fig. 3 and Fig. 4).

###  4.1 Data Collection

####  4.1.1 Obstacle Information

The size of the placed shelf as an obstacle is $120~{}\mathrm{cm}$ in height,
$80~{}\mathrm{cm}$ in width, and $40~{}\mathrm{cm}$ in depth. The origin of
the coordinate system is the floor just below the root of the robot, with the
forward direction in the positive $x$-axis, the left side in the positive
$y$-axis, and the upper side in the positive $z$-axis. The robot is placed on
a $10~{}\mathrm{cm}$ high pedestal. The shelf is placed in various locations,
which are divided into 5 positions by
$x\in[60~{}\mathrm{cm},70~{}\mathrm{cm}]$ and 9 positions by
$y\in[-10~{}\mathrm{cm},10~{}\mathrm{cm}]$, and the shelf is located so that
the training data and test data alternated (Fig. 4). Similarly, the height of
the board is divided into 11 parts in the range
$z\in[50~{}\mathrm{cm},60~{}\mathrm{cm}]$ so that the training data and test
data alternate. Therefore, there are $23\,(\mathrm{positions})\times
6\,(\mathrm{heights})=138$ different types of conditions for training data,
and $22\,(\mathrm{positions})\times 5\,(\mathrm{heights})=110$ different types
of condition for test data.

We used a depth image taken from behind the robot for the condition $\bm{c}$
that indicates the obstacle information. The depth image is taken in such a
way that the robot is not included in the image, and one depth image
corresponds to each obstacle condition. The depth image is a one-channel image
with a size of $60\times 80$ pixels. The shelf is placed in the
$[40~{}\mathrm{cm},90~{}\mathrm{cm}]$ range on the $x$-axis, and the depth
information is normalized by $[0,1]$ in that range.

####  4.1.2 Robot Information

The datasets of cGANs consists of 6 joint angles
$\bm{\theta}=(\theta_{1},\theta_{2},\theta_{3},\theta_{4},\theta_{5},\theta_{6})$
that represents the robot’s posture. The ranges of the joint angles are
$\theta_{1}\in[-90^{\circ},90^{\circ}]$,
$\theta_{2}\in[-120^{\circ},120^{\circ}]$, $\theta_{3\mathchar 45\relax
6}\in[-180^{\circ},180^{\circ}]$. The latent variables $\bm{z}$ are
6-dimensional because the robot has 6-DoF, and a uniform distribution in the
range $[0,1]$ is used.

We collected a total of 50,000 collision and non-collision data points of
robot postures for various obstacle conditions by random sampling and used
them for training. The percentage of collision data is about 43% of the total,
which includes self-collision and floor collision (about 37%) and shelf
collision (about 9%). In some cases, both collisions co-occur, so the total
exceeds 43%. Min-Max scaling was applied to each $\theta_{i}$, normalized to
the $[0,1]$ range.

[t]  Layer In Out Filter size Norma- -lization Activation function Conv1 1
conv. 1 4 (3,3) BN Leaky ReLU 2 conv. 4 4 (3,3) BN Leaky ReLU AvgPool1 4 4
(2,2) - - 3 conv. 4 8 (3,3) BN Leaky ReLU 4 conv. 8 8 (3,3) BN Leaky ReLU
AvgPool2 8 8 (2,2) - - 5 conv. 8 16 (3,3) BN Leaky ReLU AvgPool3 16 16 (2,2) -
- $\mathrm{FC_{obs\mathchar 45\relax 0}}$ 1120 1024 2 - BN Leaky ReLU
$\mathrm{FC_{obs\mathchar 45\relax 1}}$ 1024 2 1024 3 - BN Leaky ReLU
$\mathrm{FC_{obs\mathchar 45\relax 2}}$ 1024 2 1024 3 - BN Leaky ReLU
$\mathrm{FC_{obs\mathchar 45\relax 3}}$ 1024 2 1024 3 - BN Leaky ReLU $D$ 1 FC
6 256 - - Leaky ReLU 2 FC 256 512 - SN Leaky ReLU 3 FC 512 1024 3 - SN Leaky
ReLU 4 FC 1024 1024 3 - SN Leaky ReLU 5 FC 1024 1024 3 - SN Leaky ReLU 6 FC
1024 1024 - SN Leaky ReLU 7 FC 1024 1024 - SN Leaky ReLU 8 FC 1024 1 - -
Linear 1 FC 6 256 - - Leaky ReLU 2 FC 256 512 - SN Leaky ReLU 3 FC 512 1024 3
- SN Leaky ReLU $G$ 4 FC 1024 1024 3 - SN Leaky ReLU &4 5 FC 1024 1024 3 - SN
Leaky ReLU $E$ 6 FC 1024 1024 - SN Leaky ReLU 7 FC 1024 1024 - SN Leaky ReLU 8
FC 1024 6 - - Linear

Table 1: Network design

  * 1

$G$, $D$, and $E$ have independent feature extraction units.

  * 2

The output of $\mathrm{FC_{obs\mathchar 45\relax 0}}$ is the input to
$\mathrm{FC_{obs\mathchar 45\relax 1}}$, $\mathrm{FC_{obs\mathchar 45\relax
2}}$, and $\mathrm{FC_{obs\mathchar 45\relax 3}}$, respectively.

  * 3

The element-wise product of the output of $\mathrm{FC_{obs\mathchar 45\relax
1}}$ and the output of 3 FC is the input to the next layer. The same process
applies to the outputs of $\mathrm{FC_{obs\mathchar 45\relax 2}}$ and 4 FC,
and to the outputs of $\mathrm{FC_{obs\mathchar 45\relax 3}}$ and 5 FC.

  * 4

$G$ and $E$ have the same structure but independent parameters.

###  4.2 Network Design

Our network model is composed of $G$, $D$, and $E$ with fully connected
layers, and each network includes a two-dimensional convolutional layer
(conv.) as a feature extraction unit for conditions $\bm{c}$ (Fig. 2). The
network design details are shown in Table 1. For learning stabilization, batch
normalization [34] are spectral normalization [35] were used. Our network
model is implemented with PyTorch. Training is conducted on a machine equipped
with Intel Core<EMAIL_ADDRESS>CPU and GeForce RTX 3070, resulting in about
3 to 4 days of training time.

We describe $\lambda$s, which are the coefficients of each loss function in
the equation (1). They are set as $\lambda_{\mathrm{GAN}}=1$,
$\lambda_{\mathrm{rec}}=100$, $\lambda_{\mathrm{map}}=10$, and
$\lambda_{\mathrm{col}}=100$. When the distance between the robot and the
obstacle is less than $10~{}\mathrm{cm}$, $\lambda_{\mathrm{rec}}=0$ and
$\lambda_{\mathrm{map}}=0$ are used. If the robot collides with the shelf,
$\lambda_{\mathrm{col}}=1000$. This aims to increase safety by ensuring the
distance to collision, and generator, $G$, was trained to exclude postures
close to obstacles.

###  4.3 Settings for the Comparison Methods

The Python implementation in ROS was used for the RRT Connect [3] used in the
proposed method and for the RRT [7] and RRT Connect used for comparison with
the proposed method. We used the default parameters of the motion planning
framework MoveIt!.

##  5 Experiment Results

We will confirm the ii )  _Adaptability_ to various obstacles by evaluating
the constructed mapping in section 5.1. Next, i )  _Customizability_ will be
confirmed by planning on several optimization criteria in section 5.2, and iii
)  _Scalability_ will be confirmed by comparison with other planning methods
in section 5.3 and collision avoidance guarantee will be verified. Finally, we
show results with an actual robot, UR5e, in section 5.4. Also, pre-experiments
were conducted in a 2-D environment with various shapes and numbers of
obstacles. See Appendix for details.

Figure 5: Four regions of start and goal for trajectory generation: _left_ ,
_upper_ , _bottom_ , and _right_ relative to the shelf. Random postures were
set as the start and the goal from two regions. Table 2: The success rate of
planning. Planning was performed by connecting straight lines in latent space
for a given start and goal. The results of brackets include those that could
generate trajectories without collisions with obstacles but failed to
reconstruct.

| $\mathcal{L}_{\mathrm{map}}$ | $\mathcal{L}_{\mathrm{col}}$ | Dataset |  | Path Success
---
Rate [%]
Ours | w | w | Train | 72.7 (85.3)
Test | 70.9 (89.4)
w/o $\mathcal{L}_{\mathrm{map}}$ | w/o | w | Train | 22.9 (24.2)
Test | 15.8 (20.0)
w/o $\mathcal{L}_{\mathrm{col}}$ | w | w/o | Train | 17.6 (31.9)
Test | 17.0 (32.1)

###  5.1 Evaluation of Adaptability to Various Obstacles

We evaluated the accuracy of the mapping acquired as $G$ and $E$ by evaluating
the success rate of planning to verify ii )  _Adaptability_. For the
evaluation, random postures were set as the start and the goal from two
different regions among the regions whose end-effector positions were _left_ ,
_upper_ , _bottom_ , and _right_ relative to the shelf (Fig. 5). Then, for
various obstacle conditions as described in Section 4.1.1, we let the robot
arm’s end-effector plan a path to cross the shelf board using the above start
and goal posture. Note that the posture of the robot arm at the start and goal
is at least 5 $\mathrm{cm}$ away from the collision. For each obstacle
condition, three paths are generated; $138\,conditions\times
3\,paths=414\,paths$ are evaluated for train datasets, and
$110\,conditions\times 3\,paths=330\,paths$ are evaluated for test datasets.
Note that the obstacle conditions used in the test are not used in training.
Although there are several possible paths in the latent space, we use the path
that connects the two points by a straight line in the latent space after
calculating $\bm{z}_{s}=E(\bm{\theta}_{s},\bm{c})$ and
$\bm{z}_{g}=E(\bm{\theta}_{g},\bm{c})$ using the encoder $E$ for the start and
goal postures $\bm{\theta}_{s}$ and $\bm{\theta}_{g}$, respectively. The
length of $\bm{z}_{s:g}$ is fixed at 200 steps.

Success in planning means that the generated path does not contain any
collision postures and that the start and goal positions are reached. The
method for determining whether the start and goal positions have been reached
is to calculate the Euclidean distance difference of $(x,y,z)$ between the
end-effector position calculated from the given start and goal postures and
the reconstructed start and goal postures from the latent variables generated
by planning as follows:

$\begin{split}\|FK(\bm{\theta}_{\mathrm{rec}})-FK(\bm{\theta}_{\mathrm{target}})\|<\epsilon\end{split}$
(8)

where $FK()$ is forward kinematics to calculate end-effector position from
$\bm{\theta}$,
$\bm{\theta}_{\mathrm{rec}}=G\left(E(\bm{\theta}_{\mathrm{target}},\bm{c}),\bm{c}\right)$,
and $\epsilon=5.0~{}\mathrm{cm}$.

In this experiment, we verify the effectiveness of each loss function of the
proposed method, which consists of four loss functions $\mathcal{L}$s, as
shown in equation (1). Since $\mathcal{L}_{\mathrm{GAN}}$ and
$\mathcal{L}_{\mathrm{rec}}$ are the minimum required for the training of the
model, the following three conditions are used to examine the effectiveness of
the other $\mathcal{L}$s: (a) Our proposed method, (b) without
$\mathcal{L}_{\mathrm{map}}$ from equation (1), and (c) without
$\mathcal{L}_{\mathrm{col}}$ from equation (1). Note that the collision
avoidance guarantee (CAG) is not used in the planning of any method.

Table 2 shows the results of the experiment. The success rates shown in
brackets in Table 2 include those that could generate trajectories without
collisions with obstacles, but did not satisfy equation (8), i.e., failed to
reconstruct. The success rate drops significantly without either
$\mathcal{L}_{\mathrm{map}}$ or $\mathcal{L}_{\mathrm{col}}$. Since the path
success rate is low even when failure of reconstruction is considered, it can
be said that a significant rate of collisions with obstacles occurs in
trajectory.

We also confirmed that the success rate decreased by 18.5% when a depth image
different from actual obstacle information was provided to the condition. From
the results, we confirmed that the generated paths varied depending on the
conditions. The proposed method has a high success rate of more than 70% for
both the training and test dataset, indicating that it can generate
trajectories even under untrained obstacle conditions. In other words, the
_adaptability_ of the proposed method to various obstacle conditions is
verified.

Table 3: Evaluation of the optimized trajectory

Target to optimize | $\sum_{t}\|\bm{v}_{t}\|^{2}_{2}$ | $\sum_{t}\|\bm{a}_{t}\|^{2}_{2}$ | $\sum_{t}\|\bm{j}_{t}\|^{2}_{2}$
---|---|---|---
w/o Opt. | 1.16 $\pm$ 0.21 | 0.20 $\pm$ 0.12 | 0.38 $\pm$ 0.27
$\bm{v}$ | $\bm{0.75\pm 0.25}$ | 0.055 $\pm$ 0.012 | 0.11 $\pm$ 0.032
$\bm{a}$ | 1.06 $\pm$ 0.29 | $\bm{0.031\pm 0.014}$ | 0.033 $\pm$ 0.024
$\bm{j}$ | 1.17 $\pm$ 0.23 | 0.052 $\pm$ 0.017 | $\bm{0.026\pm 0.010}$
Mix of $\bm{v},\bm{a},\bm{j}$ | 0.92 $\pm$ 0.25 | 0.042 $\pm$ 0.011 | 0.049 $\pm$ 0.016

Figure 6: The values $\bm{v},\bm{a},\bm{j}$ of before and after optimization.

###  5.2 Planning on Arbitrary Optimization Criteria

In this section, we verify i )  _Customizability_. The proposed method can
generate multiple (in-finite) paths. The method for determining paths in the
latent space is not limited to just connecting the start and the goal linearly
but can be any path/trajectory planner. As examples of optimization, Table 3
shows the values of the trajectories before and after optimization when
velocity $\bm{v}$, acceleration $\bm{a}$, and jerk $\bm{j}$ are minimized, and
combinations are performed as described in Section 3.2.1. The hyper-parameters
in the combination of optimization were set to $\alpha=0.5$ and $\beta=0.5$ in
equation (7). The 330 trajectories (110 untrained conditions $\times$ three
pairs of start and goal) from the test dataset used in Section 5.1 were used
as the trajectories before optimization. Only those of these trajectories that
avoided obstacles before and after optimization were used to generate Table 3
results. The results of guaranteeing obstacle avoidance will be shown in
Section 5.3. From Table 3, it can be confirmed that the value subjected to
each optimization is the smallest. These optimizations took approximately
$2~{}\mathrm{s}$ to $10~{}\mathrm{s}$, and latent variables were updated from
500 to a maximum of 2,500 iterations.

As an example of trajectory optimization, Fig. 6 shows the values of $\bm{v}$,
$\bm{a}$, and $\bm{j}$ before optimization, which is just a straight line in
the latent space, and the trajectory after optimization by velocity
minimization and jerk minimization, respectively. The trajectory before
optimization has some parts where the velocity and jerk change suddenly. By
optimizing the trajectory with velocity minimization, the velocity of the
entire trajectory is suppressed. In the trajectory optimized by jerk
minimization, the sudden stops and starts at the start and goal are moderated.
Since the values of the graphs generated by each optimization are different,
we can say that different trajectories were generated by the optimization.

These results show that planning can be performed using arbitrary optimization
criteria, which demonstrates the _customizability_ of our proposed method.

[t]  Start-Goal Method Success Planning Path Rate [%] Time [$\mathrm{ms}$]
Length [$\mathrm{m}$] Left-Upper Ours w/o CAG 68.3 5.95 $\pm$ 0.16 1.48 $\pm$
0.21 Ours (only CAG) 100.0 114.35 $\pm$ 90.10 1.73 $\pm$ 0.50 Ours 100.0 40.32
$\pm$ 71.02 1.56 $\pm$ 0.34 RRT 100.0 419.16 $\pm$ 1305.27 2.03 $\pm$ 0.59 RRT
Connect 100.0 119.16 $\pm$ 2.54 2.26 $\pm$ 0.88 Left-Bottom Ours w/o CAG 70.2
5.95 $\pm$ 0.17 1.91 $\pm$ 0.36 Ours (only CAG) 100.0 6.02 $\pm$ 0.25 1.97
$\pm$ 0.50 Ours 100.0 5.97 $\pm$ 0.20 1.93 $\pm$ 0.40 RRT 100.0 473.35 $\pm$
1144.82 2.77 $\pm$ 1.47 RRT Connect 100.0 126.48 $\pm$ 7.71 2.43 $\pm$ 0.97
Left-Right Ours w/o CAG 81.8 5.94 $\pm$ 0.15 3.00 $\pm$ 0.39 Ours (only CAG)
100.0 35.53 $\pm$ 54.79 2.78 $\pm$ 0.43 Ours 100.0 11.32 $\pm$ 24.94 2.96
$\pm$ 0.40 RRT 100.0 201.96 $\pm$ 254.73 2.81 $\pm$ 0.80 RRT Connect 100.0
125.20 $\pm$ 2.41 3.14 $\pm$ 1.46 Upper-Bottom Ours w/o CAG 69.4 5.95 $\pm$
0.16 1.11 $\pm$ 0.27 Ours (only CAG) 100.0 89.17 $\pm$ 149.75 1.42 $\pm$ 0.78
Ours 100.0 31.45 $\pm$ 90.07 1.20 $\pm$ 0.50 RRT 98.4 1264.04 $\pm$ 3315.85
1.88 $\pm$ 1.05 RRT Connect 100.0 128.10 $\pm$ 30.49 2.03 $\pm$ 0.90 Upper-
Right Ours w/o CAG 73.2 5.94 $\pm$ 0.13 1.64 $\pm$ 0.32 Ours (only CAG) 100.0
76.23 $\pm$ 59.47 1.37 $\pm$ 0.56 Ours 100.0 24.77 $\pm$ 43.44 1.57 $\pm$ 0.41
RRT 100.0 377.65 $\pm$ 651.56 1.78 $\pm$ 0.69 RRT Connect 100.0 118.71 $\pm$
2.22 1.93 $\pm$ 0.71 Bottom-Right Ours w/o CAG 66.3 5.92 $\pm$ 0.12 1.91 $\pm$
0.48 Ours (only CAG) 100.0 112.90 $\pm$ 274.59 2.65 $\pm$ 1.19 Ours 100.0
42.02 $\pm$ 165.55 2.16 $\pm$ 0.86 RRT 97.5 854.78 $\pm$ 2119.77 2.44 $\pm$
1.51 RRT Connect 100.0 136.42 $\pm$ 78.67 2.16 $\pm$ 1.17

Table 4: Success rate and execution time for planning1

  * 1

The execution time is calculated only for planning and does not include the
GPU transfer time (about $10~{}\mathrm{ms}$)

###  5.3 Comparison of the Planning Times and Collision Avoidance Guarantee

Here, we investigate iii )  _Scalability_ and collision avoidance guarantee
(CAG). We evaluated the success rate and computation time of the proposed
method (which used CAG by RRT Connect if a collision occurred when planning
with cGANs). The 330 trajectories (110 untrained conditions $\times$ three
pairs of start and goal) described in Section 5.1 are used. These trajectories
include different start and goal pairs for each of the untrained obstacle
conditions with different shelf positions and different heights of the board
(Fig. 4). In addition to our method without CAG, the model-based planning
methods RRT and RRT Connect were used for comparison. Since the trajectories
generated by RRT and RRT Connect are different each time, three trials were
conducted with the same start, goal, and obstacle conditions. As well as the
proposed method used RRT Connect; therefore, three trials were conducted. To
investigate the computation time when RRT Connect is used for the CAG of the
proposed method, we denoted as Ours (only CAG) the success rate and
computation time when CAG was used for the data whose planning failed in Ours
w/o CAG. Note that this success rate and computation time do not include the
success rate and computation time when Ours w/o CAG succeeds. These results
are shown in Table 4. The success rate is the same as Section 5.1, which is
the percentage of trajectories that do not collide with any obstacles and
satisfy equation (8). Note that for the RRT and RRT Connect methods, if the
execution time exceeded 60 s, the method was counted as a failure to find a
path.

Ours w/o CAG is planning in the latent space where there is no collision with
obstacles, so the computation time is almost constant, independent of the
complexity of the environment. However, learning-based methods alone cannot
guarantee 100% collision avoidance with obstacles. In our results, the success
rate of learning a trajectory without contact with obstacles was more than
60%. Even though the RRT and RRT Connect methods achieve a high success rate,
these methods require more computation time as the complexity of the
environment increases because the collision check is required each time.
Furthermore, the trajectory generated by each trial is different each time due
to including randomness in RRT and RRT Connect. In particular, when the
complexity of the environment increases, the variance of generated
trajectories becomes larger. This experiment showed that the variance of the
computation time and the length of the generated path was significant for the
_Upper-Bottom_ and _Bottom-Right_ paths. This means a complex planning task is
included in the experiment. Our method uses RRT Connect to compute the
trajectory only before and after the collision. Thus, the computation time is
less than that of RRT or RRT Connect, which requires collision checks in all
trajectories, as shown in Ours (only CAG). The average computation time for
Ours is $27.67\pm 96.15~{}\mathrm{ms}$ in total, which is 21.8% of that for
RRT Connect of $126.79\pm 44.84~{}\mathrm{ms}$. While generating a single
trajectory may result in collisions with obstacles in our method, generating
multiple different trajectories, such as using different optimization criteria
or selection of different trajectories in the latent space, can also reduce
the likelihood of using RRT Connect. Therefore, our proposed method requires
even less computation than RRT and RRT Connect.

In summary, Ours and Ours w/o CAG require training time in advance, but the
planning time is almost constant, independent of the complexity of the
environment. RRT and RRT Connect do not require learning, but the computation
time increases with the complexity of the environment. Ours achieves both
advantages of low-calculation cost with the learning method and guaranteed
avoidance by the conventional planning method. These indicate that the
computational cost of our method is _scalable_ to the complexity of the
environment.

Figure 7: Demonstration on the actual robot. Different trajectories were
generated for (a) to (c). (a) Ours without trajectory optimization, (b) Ours
using trajectory optimization by speed minimization, (c) Generated
trajectories using RRT Connect.

###  5.4 Experiments Using UR5e

In this section, we show the results of experiments using the actual UR5e.
Using the model trained with the simulation dataset, UR5e performed planning.
The shelf was placed in the same position as in the simulator, and the depth
information was the same as in the simulator for the image. In the
experiments, we used (a) Ours without trajectory optimization, (b) Ours using
trajectory optimization by speed minimization, and (c) Generated trajectories
using RRT Connect. Fig. 7 shows the generated trajectories. In (a), we confirm
that our proposed method generates a collision-free path. In (b), a shorter
path is generated by optimizing for speed. In (c), using RRT Connect, the
trajectory generated by each trial is different each time. On the other hand,
in our method, if the same trajectory is selected in the latent space, the
same trajectory will be generated in the joint space.

##  6 Conclusion

In this research, the robot’s collision-free joint space is expressed as the
latent space of cGANs, and collision-free paths are obtained by mapping the
planning in the latent space to the joint space. We confirmed that i )
_Customizability_ ; any path can be planned in the latent space using any
optimization criteria, ii )  _Adaptability_ ; a single trained model could
handle multiple untrained obstacle conditions, and iii )  _Scalability_ ;
computational cost of planning does not depend on the obstacles. By modifying
the trajectory in case of a collision when planning is done by learning alone,
100% of collision avoidance can be guaranteed.

## Appendix

As a pre-experiment, we have published a paper in arXiv in which a 2-DoF robot
arm is tested on a 2-D plane 444The link of the article is the following:
https://arxiv.org/abs/2202.07203. Experiments are conducted in 2-D
environments with random numbers and shapes of obstacles. In the pre-
experiment paper, the focus is on analysis because of the small number of
DoFs. In this new paper, the model is extended to a 6-DoF robot in 3-D space,
but the concept of the model, which maps the non-collision posture to the
potential space, is the same.

## Acknowledgments

The authors would like to thank Avinash Ummadisingu for the proof check. H.
Mori would like to thank all colleagues in the ETIS lab at the Cergy-Pontoise
Univ., especially Prof. Mathias Quoy, Prof. Philippe Gaussier, and Assoc.
Prof. Alexandre Pitti for discussion about a preliminary result of the basic
idea of this article when he came up with the basic idea at the lab in 2016.

## References

  * [1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” _Advances in neural information processing systems_ , vol. 27, 2014.
  * [2] M. Mirza and S. Osindero, “Conditional generative adversarial nets,” _CoRR_ , vol. abs/1411.1784, 2014. [Online]. Available: http://arxiv.org/abs/1411.1784
  * [3] J. J. Kuffner and S. M. LaValle, “RRT-connect: An efficient approach to single-query path planning,” in _Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065)_ , vol. 2. IEEE, 2000, pp. 995–1001.
  * [4] C. W. Warren, “Multiple robot path coordination using artificial potential fields,” in _Proceedings., IEEE International Conference on Robotics and Automation_. IEEE, 1990, pp. 500–505.
  * [5] G. Li, A. Yamashita, H. Asama, and Y. Tamura, “An efficient improved artificial potential field based regression search method for robot path planning,” in _2012 IEEE International Conference on Mechatronics and Automation_ , 2012, pp. 1227–1232.
  * [6] N. D. Ratliff, J. Issac, D. Kappler, S. Birchfield, and D. Fox, “Riemannian motion policies,” _arXiv preprint arXiv:1801.02854_ , 2018.
  * [7] S. M. LaValle _et al._ , “Rapidly-exploring random trees: A new tool for path planning,” 1998.
  * [8] S. M. LaValle and J. J. Kuffner Jr, “Randomized kinodynamic planning,” _The international journal of robotics research_ , vol. 20, no. 5, pp. 378–400, 2001.
  * [9] W. Wang, X. Xu, Y. Li, J. Song, and H. He, “Triple rrts: an effective method for path planning in narrow passages,” _Advanced Robotics_ , vol. 24, no. 7, pp. 943–962, 2010.
  * [10] P. Hart, N. Nilsson, and B. Raphael, “A formal basis for the heuristic determination of minimum cost paths,” _IEEE Transactions on Systems Science and Cybernetics_ , vol. 4, no. 2, pp. 100–107, 1968. [Online]. Available: https://doi.org/10.1109/tssc.1968.300136
  * [11] M. Campana, F. Lamiraux, and J.-P. Laumond, “A gradient-based path optimization method for motion planning,” _Advanced Robotics_ , vol. 30, no. 17-18, pp. 1126–1144, 2016.
  * [12] S. Karaman and E. Frazzoli, “Incremental sampling-based algorithms for optimal motion planning,” in _Robotics: Science and Systems VI_. MIT Press, 2011, pp. 267–274.
  * [13] K. Naderi, J. Rajamäki, and P. Hämäläinen, “RT-RRT*: a real-time path planning algorithm based on rrt*,” 11 2015, pp. 113–118.
  * [14] A. H. Qureshi and Y. Ayaz, “Intelligent bidirectional rapidly-exploring random trees for optimal motion planning in complex cluttered environments,” _Robotics and Autonomous Systems_ , vol. 68, pp. 1–11, 2015.
  * [15] ——, “Potential functions based sampling heuristic for optimal path planning,” _CoRR_ , vol. abs/1704.00264, 2017. [Online]. Available: http://arxiv.org/abs/1704.00264
  * [16] Z. Tahir, A. H. Qureshi, Y. Ayaz, and R. Nawaz, “Potentially guided bidirectionalized RRT* for fast optimal path planning in cluttered environments,” _Robotics and Autonomous Systems_ , vol. 108, pp. 13–27, 2018\.
  * [17] T. Lalibertk and C. M. Gosselin, “Efficient algorithms for the trajectory planning of redundant manipulators with obstacle avoidance,” in _Proceedings of the 1994 IEEE International Conference on Robotics and Automation_ , 5 1994, pp. 2044–2049.
  * [18] T. Flash and N. Hogan, “The coordination of arm movements: an experimentally confirmed mathematical model.” _Journal of Neuroscience_ , vol. 5, pp. 1688–1703, 1985.
  * [19] A. Srinivas, A. Jabri, P. Abbeel, S. Levine, and C. Finn, “Universal planning networks: Learning generalizable representations for visuomotor control,” in _International Conference on Machine Learning_. PMLR, 2018, pp. 4732–4741.
  * [20] A. Tamar, S. Levine, and P. Abbeel, “Value iteration networks,” _CoRR_ , vol. abs/1602.02867, 2016. [Online]. Available: http://arxiv.org/abs/1602.02867
  * [21] B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” in _2018 IEEE International Conference on Robotics and Automation (ICRA)_. IEEE, 2018, pp. 7087–7094.
  * [22] A. Wang, T. Kurutach, K. Liu, P. Abbeel, and A. Tamar, “Learning robotic manipulation through visual planning and acting,” _CoRR_ , vol. abs/1905.04411, 2019. [Online]. Available: http://arxiv.org/abs/1905.04411
  * [23] R. Kumar, A. Mandalika, S. Choudhury, and S. S. Srinivasa, “Lego: Leveraging experience in roadmap generation for sampling-based planning,” _arXiv preprint arXiv:1907.09574_ , 2019.
  * [24] R. Terasawa, Y. Ariki, T. Narihira, T. Tsuboi, and N. Nagasaka, “3d-cnn based heuristic guided task-space planner for faster motion planning,” in _2020 International Conference on Robotics and Automation (ICRA)_. IEEE, 2020, pp. 9548–9554.
  * [25] K. Ota, Y. Sasaki, D. K. Jha, Y. Yoshiyasu, and A. Kanezaki, “Efficient exploration in constrained environments with goal-oriented reference path,” _arXiv preprint arXiv:2003.01641_ , 2020.
  * [26] S. Wen, J. Chen, S. Wang, H. Zhang, and X. Hu, “Path planning of humanoid arm based on deep deterministic policy gradient,” in _2018 IEEE International Conference on Robotics and Biomimetics (ROBIO)_ , 2018, pp. 1755–1760.
  * [27] A. Taniguchi, F. Sasaki, M. Muroi, and R. Yamashina, “Planning on topological map using omnidirectional images and spherical cnns,” _Advanced Robotics_ , vol. 36, no. 3, pp. 153–166, 2022.
  * [28] K. Kutsuzawa, H. Kusano, A. Kume, and S. Yamaguchi, “Motion generation considering situation with conditional generative adversarial networks for throwing robots,” _arXiv preprint arXiv:1910.03253_ , 2019.
  * [29] A. H. Qureshi, M. J. Bency, and M. C. Yip, “Motion planning networks,” _CoRR_ , vol. abs/1806.05767, 2018.
  * [30] B. Ichter and M. Pavone, “Robot motion planning in learned latent spaces,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 3, pp. 2407–2414, 2019\.
  * [31] D. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _International Conference on Learning Representations_ , 12 2014.
  * [32] C. Rodriguez, J. Denny, S. A. Jacobs, S. Thomas, and N. M. Amato, “Blind rrt: A probabilistically complete distributed rrt,” in _2013 IEEE/RSJ International Conference on Intelligent Robots and Systems_. IEEE, 2013, pp. 1758–1765.
  * [33] M. Svenstrup, T. Bak, and H. J. Andersen, “Minimising computational complexity of the rrt algorithm a practical approach,” in _2011 IEEE International Conference on Robotics and Automation_. IEEE, 2011, pp. 5602–5607.
  * [34] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” in _Proceedings of the 32nd International Conference on International Conference on Machine Learning \- Volume 37_ , ser. ICML’15. JMLR.org, 2015, p. 448–456.
  * [35] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” in _International Conference on Learning Representations_ , 2018.