# Connected Autonomous Vehicle Motion Planning with Video Predictions from
Smart, Self-Supervised Infrastructure

Jiankai Sun1, Shreyas Kousik2, David Fridovich-Keil3, and Mac Schwager1
1Stanford University, 2Georgia Institute of Technology, 3University of Texas
at Austin.

###### Abstract

Connected autonomous vehicles (CAVs) promise to enhance safety, efficiency,
and sustainability in urban transportation. However, this is contingent upon a
CAV correctly predicting the motion of surrounding agents and planning its own
motion safely. Doing so is challenging in complex urban environments due to
frequent occlusions and interactions among many agents. One solution is to
leverage smart infrastructure to augment a CAV’s situational awareness; the
present work leverages a recently-proposed “Self-Supervised Traffic Advisor”
(SSTA) framework of smart sensors that teach themselves to generate and
broadcast useful video predictions of road users. In this work, SSTA
predictions are modified to predict future occupancy instead of raw video,
which reduces the data footprint of broadcast predictions. The resulting
predictions are used within a planning framework, demonstrating that this
design can effectively aid CAV motion planning. A variety of numerical
experiments study the key factors that make SSTA outputs useful for practical
CAV planning in crowded urban environments.

Figure 1: Our proposed system is a distributed Neural Perception System
comprised of a network of static traffic cameras paired with their own neural
networks. The location of the ego car is marked by the orange box. The
networks predict future pixel-wise occupancy and departure times in each view,
which is our first contribution. A planning system then generates a path based
on collision checking in the pixel-wise predictions, which is our second
contribution. Finally, the path plan is handed to a CAV’s control system to
track.

##  I Introduction

Connected and autonomous vehicles (CAVs) have the potential to make urban
transportation safer, more efficient, and more sustainable. A key challenge
towards realizing this goal is that of motion planning; traditional methods
rely on static maps and real-time sensor data, and cannot always yield
accurate predictions of the future state of the environment due to occlusions
or other onboard sensing limitations. Hence, smart infrastructure has been
proposed as a way to augment CAV capabilities. However, collecting data to
train such smart infrastructure can often require prohibitive amounts of
manual labeling. Therefore, there is growing interest in using self-training
smart infrastructure to improve CAV safety and performance.

The present work considers a specific type of edge device proposed in our
prior work: the Self-Supervised Traffic Advisor (SSTA) [1]. This proposed
device automatically processes video data from traffic cameras to support CAV
motion planning. SSTAs are a scalable edge solution that avoid extensive hand-
labeling and can provide large-scale connected coverage of an urban
environment through self-training to share data. Our previous work studied the
challenges of self-training and networking, whereas this work assesses the
potential for SSTAs to benefit CAV motion planning and control.

Contributions: First, we develop a new output format for SSTAs as an
alternative to predicting raw video frames. Specifically, we propose to
predict Time to Next Occupancy (T2NO) and Time to Next Departure (T2ND), which
describe when each portion of an SSTA’s field of view will next be occupied
and subsequently unoccupied by a road user. We abbreviate these together as
“T2NO/D.” Second, we propose an approach to CAV motion planning that leverages
our novel T2NO/D outputs from SSTAs. Third, we numerically evaluate the
effectiveness of autonomously predicting T2NO/D in comparison to raw video
within a CAV motion planning framework. We find that our proposed method can
provide an effective smart infrastructure perception output that is directly
amenable to CAV planning and control.

##  II Related Work

SSTAs [1], and this paper, lie at the intersection of CAV infrastructure,
networked learning, and video prediction.

##### CAV Infrastructure

To maximize their utility, CAVs should communicate with self-driving and
human-driven vehicles, roadside infrastructure, and other road users; many
reviews of this literature are available [2, 3, 4, 5]. Given bandwidth and
connectivity limitations, it is critical to decide what, and when, to
communicate. For example, Gopalswamy et al. [6] propose to offload CAV
responsibilities for situational awareness and information sharing via
vehicle-to-infrastructure (V2I) communication. Alternatively, short-range
communications allow vehicle-to-vehicle (V2V) transmission of information such
as speed, heading, and brake status [7, 8]. Since V2V alone can cause network-
wide instabilities as the number of vehicles grows [9], it is critical to
explore alternative means of widespread, scalable communication that can
improve large-scale traffic metrics such as safety and efficiency [10]. In
this work, we propose a scalable infrastructure solution and study its
potential for aiding CAVs in an isolated V2I manner; we plan to explore mixing
V2V and V2I in future work.

##### Networked Learning

The concept of networked learning has been recently explored in the context of
connected and autonomous vehicles (CAVs) [11]. In this paradigm, multiple CAVs
can cooperate to improve their individual driving policies through message
passing and sharing of information [12, 13, 14]. We note that CAVs can
cooperate to perceive their surroundings [13] and to improve traffic
congestion and energy efficiency [15, 16, 17]. In this work, we instead
consider how networked learning in infrastructure can be used to improve
perception and planning for CAVs.

##### Video Prediction

Video prediction has the potential to impact traffic management, motion
planning for autonomous vehicles, and autonomous surveillance [18]. This task
is convenient for data collection because it can often be achieved without
manual data annotation. Deep learning methods can be especially well suited
for modeling spatiotemporal relationships in video and generating accurate
predictions of future events [19, 20, 21, 22]. However, most previous video
prediction is typically focused on a single camera, aiming to learn visual
dynamics [18] or interpolate video spatially and temporally [23]. Our prior
work on SSTAs [1] instead addressed the challenge of video prediction across a
network of _multiple_ static traffic cameras. The key remaining challenge
addressed in the present work is that, from a V2I perspective, predicting raw
video is not directly amenable to downstream CAV planning and control tasks.

##  III Method

In this section, we describe our system architecture (cf. Fig. 1) and
algorithms used for video prediction and motion planning. First, we provide an
overview of our proposed method. Next, we introduce the Time to Next Occupancy
(T2NO) and Time to Next Departure (T2ND) to predict future road user
trajectories. We then explain our training loss and neural network
architecture used at each SSTA. Finally, we design a collision-checking
technique for T2NO/D and utilize it for downstream planning and control.

######  Remark 1.

In this work, we consider planning only for a single CAV, which we henceforth
refer to as the _ego vehicle_. We defer multi-agent SSTA-enabled planning for
future work.

###  III-A Overview of Proposed Method

Our overall goal is CAV motion planning using video predictions generated by
SSTAs (Fig. 1). Motion planning requires collision checking, but performing
collision checking directly in raw video predictions is challenging. Instead,
we propose predicting future occupancy in each pixel of each SSTA view. Our
method has three steps: (1) calculating future occupancy using Alg. 1, (2)
collision checking pixels occupied by the ego vehicle against pixels with
predicted future occupancy, and (3) performing motion planning by modifying
the A∗ algorithm [24] to generate an optimal path that avoids intersecting
with obstacles in space and time.

###  III-B Time to Next Occupancy and Time to Next Departure

We leverage the fact that each SSTA is static to capture and use background
information. This enables us to compute two quantities that simplify collision
checking: Time to Next Occupancy and Time to Next Departure.

Before defining these quantities, we establish some notation, all within the
context of a single SSTA. Note, to ease notation, we drop the index $i$ for
considering the $i$th SSTA in this discussion. Suppose the current time is
$t=0$. Let $\mathbb{N}_{256}=\\{0,1,\cdots,255\\}$. Consider a sequence of
predicted images $(I_{t})_{t=0}^{T}$ with each image in
$\mathbb{N}_{256}^{H\times W\times 3}$ (i.e., height $H$, width $W$, and RGB
channel of dimension 3), and $T\in\mathbb{N}$ as the prediction horizon. Also
suppose we also have a background image $B\in\mathbb{N}_{256}^{H\times W\times
3}$; in practice we create $B$ for each SSTA by pixel-wise averaging of
approximately one minute of video data. Denote indexing into the $(i,j)$th
pixel of an image $I$ by $I[i,j]$. Let $\mathbb{N}_{n}$ denote the set
$\\{0,\cdots,n\\}$.

######  Definition 2.

The _Time to Next Occupancy_ (T2NO) is the first time at which a given pixel
in an SSTA’s image is predicted to be occupied, which we assess by comparison
against the background $B$. We represent T2NO as an array
$O\in\mathbb{R}^{H\times W}$.

Input: prediction $(I_{t})_{t=0}^{T}$, background $B$, occupancy threshold
$\tau_{O}$, departure threshold $\tau_{D}$

1

2$O,D\leftarrow+\infty_{H\times W}$ // preallocate T2NO and T2ND

3for  _$(i,j)\in\mathbb{N}_{h}\times\mathbb{N}_{w}$ (i.e., iterate over each
pixel)_ do

4 // first, compute time to next occupancy

5 for  _$t=0,1,\cdots,T$_ do

6 $\delta\leftarrow|I_{t}[i,j]-B[i,j]|$ // error between prediction and
background at pixel $(i,j)$

7 if  _$\delta\geq\tau_{O}$_ then

8 $O[i,j]\leftarrow t$ // get smallest $t$ for which prediction differs from
background

9 break

10

11

12 // second, compute time to next departure

13 if  _$O[i,j] <+\infty$ (i.e., if there is any future occupancy at the
current pixel)_ then

14 for  _$t=O[i,j],O[i,j]+1,\cdots,T$ (i.e., for all times after the time to
next occupancy)_ do

15 $\delta\leftarrow|I_{t}[i,j]-B[i,j]|$

16 if  _$\delta\leq\tau_{D}$_ then

17 $D[i,j]\leftarrow t$ // get smallest $t$ for which prediction is close to
background

18 break

19

20

21

Output: $O$ (T2NO) and $D$ (T2ND)

Algorithm 1 Time to Next Occupancy / Departure

On its own, T2NO does not capture all relevant dynamic information in a scene,
because, once a pixel is declared occupied at a time $t$, it is implied to be
occupied for all future time $t^{\prime}>t$, leading to the following
definition that alleviates this concern:

######  Definition 3.

The _Time to Next Departure_ (T2ND) measures the first time at which an
occupied pixel is no longer occupied. Similar to T2NO, we represent T2ND as an
array $D\in\mathbb{R}^{H\times W}$.

We compute both T2NO and T2ND using Alg. 1. By combining T2NO and T2ND, we can
assess when each pixel in an SSTA’s field of view will be occupied and then
free within the prediction horizon $T$.

As an example to clarify these concepts, consider a single car passing through
pixel $(i,j)$ at $10~{}$\mathrm{m}\mathrm{/}\mathrm{s}$$, and suppose every
pixel corresponds to a $1\times 1~{}$\mathrm{m}^{2}$$ patch of drivable area.
Suppose the T2NO is $2.0~{}$\mathrm{s}$$, indicating the earliest time at
which the car occupies that pixel. Suppose the car is $5~{}$\mathrm{m}$$ long.
Then the car will completely pass over the pixel in $0.5~{}$\mathrm{s}$$,
meaning that the T2ND at that pixel must be no less than $2.5~{}$\mathrm{s}$$.
An adjacent pixel in the direction of travel will have
$O[i,j]=2.1~{}$\mathrm{s}$$ and $D[i,j]\geq 2.6~{}$\mathrm{s}$$ (assuming the
car is predicted to maintain its speed and heading).

###  III-C Training to Predict T2NO/D

We follow the networked co-learning procedure outlined in [1, Sec. III.D].
However, we preprocess each $i$th SSTA’s recorded images to generate
$O_{t}^{i}$ and $D_{t}^{i}$ as output by Alg. 1 for each time step $t$ of the
training data (i.e. past video frames). Denote the T2NO/D predictions of the
$i$th SSTA at time ${\hat{t}}$ as $\hat{O}^{i}_{\hat{t}}$ and
$\hat{D}^{i}_{\hat{t}}$, starting from time $t$ up to a time $t+T$. We
minimize the mean-squared error loss between predicted and ground-truth
T2NO/D:

$\displaystyle\mathcal{L}_{t}=\sum_{i=1}^{N}\left(\sum_{{\hat{t}}=t}^{t+T}\left(\alpha\left\|O_{\hat{t}}^{i}-\hat{O}_{\hat{t}}^{i}\right\|^{2}+\beta\left\|D_{\hat{t}}^{i}-\hat{D}_{\hat{t}}^{i}\right\|^{2}\right)\right),$
(1)

where $\alpha,\beta>0$ are tunable hyperparameters, and we use Frobenius norms
(i.e., pixelwise error squared). Note, the predicted $\hat{O}^{i}_{\hat{t}}$
and $\hat{D}^{i}_{\hat{t}}$ are functions of the $i$th SSTA’s hidden state
$h_{\hat{t}}^{i}$, the set of received messages $Y_{\hat{t}}^{i}$ from its
connected SSTAs, and its current received video frame $I_{\hat{t}}^{i}$.

###  III-D Proposed Neural Network Architecture

We use the same architecture as in [1, Sec. III.F]. However, instead of
requiring a recursive rollout to generate T2NO/D predictions, we generate the
prediction with a single forward pass of the network at each SSTA. So, the
outputs of the $i$th network at timestep ${\hat{t}}$ are
$\hat{O}^{i}_{\hat{t}}$, $\hat{D}^{i}_{\hat{t}}$, and the message
$y_{\hat{t}}^{i}$.

######  Remark 4.

The method proposed in this work reduces the number of neural network
evaluations at each SSTA from a $T$-step recursive rollout to just a single
evaluation, since T2NO/D collapse temporal information into a single image.

###  III-E Collision Checking with T2NO/D

We seek to use SSTA predictions for collision checking within a CAV motion
planning framework. First, we explain how to isolate pixels associated with
the ego vehicle. Then, we consider collision checking against T2NO/D.

Ref Img $B$ | Cur Image $I$ |  $T=2$ |  $T=10$ |  $T=20$ |  $T=30$ | $T=60$
---|---|---|---|---|---|---
|  |  |  |  |  |  |  |  |  |  |  |  |
|  |  |  |  |  |  |  |  |  |  |  |  |

Figure 2: Visualization of T2NO for different time horizons (T2ND is omitted
for ease of understanding). The color scale runs from black at $t=0$ to yellow
as $t\to\infty$. The background image $B$ and the current image $I$ are also
shown on the left. We see that T2NO contains little time information when the
time horizon is short, whereas T2NO contains ambiguous information when the
time horizon is long, and different trajectories even overlap, impeding the
effectiveness of CAV motion planning. Even though there are multiple plausible
paths for cars to take in the future, depending on the road topology, this
ambiguity is not desired and hinders planning.

####  III-E1 Masking the Ego Vehicle’s Pixels

Recall that each SSTA outputs a sequence of images $(I_{t})_{t=0}^{T}$, with
each $I_{t}\in\mathbb{N}_{256}^{h\times w\times 3}$. We now introduce an
approach to extracting the pixels in an image corresponding to our ego
vehicle’s pose (2-D position and 1-D heading), which we denote
$(p_{t},\theta_{t})$ at time $t$. Suppose the vehicle has a rectangular body
of dimensions $l\times d$. Also suppose that each pixel, with indices $(i,j)$,
is associated with a corresponding spatial location $p_{i,j}$, which can be
found with the following map:

sp2px $\displaystyle:p_{i,j}\mapsto(i,j)$ (2)

Since each SSTA is a static camera with a known field of view, it is
straightforward to create this map via standard camera calibration techniques.
Also, note that we can represent the vehicle’s body at the state $(p,\theta)$
as an array of vertices arranged counterclockwise:

$\displaystyle
V_{t}\leftarrow\left(\tfrac{1}{2}\left[\begin{matrix}[r]\cos\theta_{t}&-\sin\theta_{t}\\\
\sin\theta_{t}&\cos\theta_{t}\end{matrix}\right]\left[\begin{matrix}[r]l&-l&-l&l\\\
d&d&-d&-d\end{matrix}\right]\right)+p_{t}$ (3)

Recall that our images $I_{t}$ are of dimension $h\times w\times 3$. We thus
create a mask for the given vehicle pose:

$\displaystyle P_{t}$ $\displaystyle\leftarrow{\texttt{sp2px}}(V_{t})$ (4)
$\displaystyle P_{t}$
$\displaystyle\leftarrow{\texttt{fillPoly}}(\mathbf{0}_{h\times w\times 3},\
P_{t}),$ (5) $\displaystyle M_{t}$ $\displaystyle\leftarrow
P_{t}==(255,255,255),$ (6)

where $P_{t}$ is used as an intermediate array. Equation (5) fills in a
polygon defined by the (pixel coordinate) vertices $V_{M}$, into a black image
($\mathbf{0}_{h\times w\times 3}$), so $P_{t}\in\mathbb{N}_{256}^{H\times
W\times 3}$. Note that fillPoly is available in OpenCV [25], and we assume the
default fill color is white, or $(255,255,255)$. Equation (6) creates the mask
$M_{t}\in\\{0,1\\}^{H\times W}$ (i.e., a Boolean array) which is true for
every entry (i.e., pixel) that is white, thereby identifying the pixels
corresponding to our vehicle.

####  III-E2 Collision Checking Against T2NO/D

Suppose we have the T2NO/D arrays $O\in\mathbb{R}^{H\times W}$ and
$D\in\mathbb{R}^{h\times w}$ output from Alg. 1. For each time step $t$, we
check:

$\displaystyle C_{t}$ $\displaystyle\leftarrow t\cdot M_{t}$ (7)
$\displaystyle{\texttt{chk}}_{t}$
$\displaystyle\leftarrow{\texttt{any}}\left(C_{t}\geq O\ \&\ M_{t}\leq
D\right)$ (8)

Equation (7) converts the Boolean mask $M_{t}$ to a real-valued array, where
entries for which $M_{t}$ is true are set to $t$, and all others are $0$.
Then, we directly check the entries of $M_{t}\in\mathbb{R}^{H\times W}$
against $O$ and $D$. If any of the vehicle’s occupied pixels (i.e., true
pixels) overlap with the time window created by $O$ and $D$, then
${\texttt{chk}}_{t}$ is true (i.e., the pose $(p_{t},\theta_{t})$ is in a
collision).

###  III-F Planning with T2NO/D

After obtaining T2NO/D, we plan a path for our ego vehicle. In this work, we
extend the classical A∗ algorithm [24] to leverage T2NO/D, but we note that
these representations can be readily applied to other planning algorithms.

To avoid collisions with dynamic obstacles, we modify A∗ to generate a time-
optimal path that avoids intersecting with the obstacles in both time and
space. Suppose the ego vehicle is at a time and position
$z_{0}=(0,p_{0})\in[0,T]\times\mathbb{R}^{2}$. Our implementation finds a list
of times and positions $z_{i}=(t_{i},p_{i})$, where $i\in\mathbb{N}$, which
define a trajectory for the ego vehicle to track. We seek to reach a user-
specified goal position $p_{\mathrm{\textnormal{goal}}}$ in the shortest time
while avoiding other vehicles. As per the classical A∗ notation, we use an
evaluation function to represent running cost:

$\displaystyle f(z_{i})=c(z_{i})+r_{t}(z_{i})$ (9)

where $c(z_{i})$ is the cost of the path from $z_{0}$ to $z_{i}$ and $r_{t}$
is a time-varying heuristic function that estimates the cost of the most
efficient path from $p_{i}$ to $p_{\mathrm{\textnormal{goal}}}$ that avoids
dynamic obstacles. In particular, we set

$\displaystyle
r_{t}(z_{i})=\left\|p_{i}-p_{\mathrm{\textnormal{goal}}}\right\|+K\cdot{\texttt{chk}}_{t_{i}},$

where $K>0$ is a scalar that penalizes being in collision at time $t_{i}$, and
${\texttt{chk}}_{t_{i}}$ is the collision check evaluated at time $t_{i}$ as
per (8) (and ${\texttt{true}}=1$). The final output of A∗ is a discrete
sequence of times and positions. We perform cubic spline interpolation to
smooth the sequence before handing it to the ego vehicle to track (in our
implementation, the ego vehicle uses a default PD controller from the CARLA
simulator [26]).

Next, we evaluate our proposed technical approach via numerical experiments.

##  IV Experiments

We now assess the utility of SSTAs and our proposed T2NO/D output format via a
series of numerical experiments. We find that SSTAs can generate predictions
which are effective for CAV motion planning.

(a) Histogram of Ego Vehicle Speed

(b) Speed Over Time

Figure 3: Ego Car Speed Statistics TABLE I: Performance in 4-view setting over 20 trials (mean $\pm$ standard deviation). Metrics |  Success Rate (%) $\uparrow$ |  Number of Collisions $\downarrow$ |  Total Timesteps to Reach Goals $\downarrow$ |  Total Control Effort $\downarrow$ |  Total Sudden Reversals $\downarrow$ |  Travel Distance (km) $\uparrow$
---|---|---|---|---|---|---
Monolith |  $\bm{65.0}$ |  $15.15\pm\bm{0.86}$ |  $\bm{484.15}\pm 6.72$ |  $\bm{245.49}\pm 5.91$ |  $66.76\pm 1.36$ |  $\bm{1.21\pm 0.34}$
AllComms |  60.0 |  $\bm{15.00}\pm 4.63$ |  $495.50\pm 6.76$ |  $251.47\pm\bm{4.01}$ |  $\bm{66.58\pm 1.25}$ |  $1.12\pm 0.35$
NoComms |  50.0 |  $17.50\pm 2.29$ |  $508.40\pm\bm{5.58}$ |  $260.28\pm 4.33$ |  $70.50\pm 2.94$ |  $1.04\pm 0.38$

###  IV-A Experiment Setup

####  IV-A1 Implementation Details

We evaluate our approach on a desktop computer with an NVIDIA GeForce RTX
2080Ti and an Intel Core i7-6800K CPU. We design the traffic scenario and
evaluate our pipeline with the CARLA 0.9.11 simulator. We train our networks
using the ADAM optimizer [27] and employ a mini-batch size of 10 sequences.
The learning rate is set to $10^{-3}$, and training is stopped after 800
epochs.

####  IV-A2 SSTAs

In each simulated traffic scenario, SSTAs are placed at each intersection and
record top-down images. Each SSTA communicates with its neighbors at adjacent
intersections, according to the topology of the road graph. Following prior
work [1], SSTAs are trained to predict traffic flow (see Fig. 2) using a
corpus of simulated traffic scenarios offline. Note that in contrast our prior
work explored the effects of online and lifelong learning.

####  IV-A3 Ego Vehicle

We perform planning for an ego vehicle as per Sec. III-F. The overall task is
to avoid other vehicles while attempting to reach sub-goals or a global goal
location. To ease identification of the ego vehicle, we set its color to grey,
and all other vehicles to red for visualization. The outcomes are presented in
Tab. I. Since T2NO/D is computed based on background subtraction, the actual
vehicle color does not impact our proposed method’s performance, which is an
improvement over our prior work.

The ego vehicle is initialized with the same start and goal locations in each
experiment, but the traffic pattern is randomly generated each time. An
experiment is terminated once the first vehicle reaches its goal location, and
we record this time $t^{*}$ as well as the corresponding distance between each
other vehicle and its goal as $\bar{d}$.

The ego vehicle utilizes a replanning frequency of
$20~{}$\mathrm{H}\mathrm{z}$$ (i.e., it plans in a receding-horizon manner).
This necessitates the SSTAs to update their prediction outputs at the same
frequency. The maximum speed of the ego vehicle is set to the CARLA default of
8.33 m/s. Please refer to Fig. 3 for a histogram plot displaying the actual
speeds achieved by the vehicle in an experiment.

####  IV-A4 Metrics

We evaluate our approach on safety, efficiency, and smoothness with the
following metrics:

  * –

Success Rate is the percentage of experiments in which the ego car is able to
reach the global goal position.

  * –

Number of Collisions is how many times the ego car collides with other
vehicles or obstacles before reaching its global goal. Note that in our
experiments, successful navigation may entail encountering multiple non-fatal
collisions to simulate a realistic and challenging environment, consistent
with the default setting in CARLA [26]. This approach aims to emulate real-
world conditions where occasional collisions may happen without compromising
the overall navigation capabilities of the autonomous vehicle.

  * –

Travel Distance is the total distance that the ego car is able to travel
before experiencing a collision or reaching its goal.

  * –

Total Timesteps to Reach Goal measures the duration taken to reach the goal,
if it was reached.

  * –

Total Control Effort is the sum of the norm of the ego vehicle’s 2-D
acceleration over the entire experiment. That is, if the vehicle experiences
an acceleration $\ddot{x}_{t}$ at time $t$, then this metric is
$\sum_{t=0}^{t_{\mathrm{\textnormal{final}}}}\|\ddot{x}_{t}\|$, where
$t_{\mathrm{\textnormal{final}}}$ is the final time in the given experiment
(i.e., when the vehicle reached the goal or crashed)

  * –

Total Sudden Reversals counts how often the vehicle’s longitudinal and lateral
acceleration changes sign over the course of an experiment.

Note, we only count successful trials (i.e., goal reached) for all metrics
except for Success Rate and Travel Distance. This is because failed trials
result in smaller Total Control Effort and Total Sudden Reversals due to
shorter distances, but this does not mean that failed trials are better than
successes. Also, there is no Total Timesteps to Reach Goal for unsuccessful
trials.

###  IV-B Independent Variables and Hypotheses

####  IV-B1 Influence of Communication

We aim to characterize the effects of SSTA communication upon traffic flow
using the metrics above. As such, we compare the performance of our method
using the following communication arrangements:

  * –

Monolith, in which a single model is trained on all images concatenated
together, with no need to communicate, as opposed to multiple models trained
at each location and requiring communication.

  * –

AllComms, in which each SSTA can communicate with _all_ others both during
training and at run time.

  * –

NoComms, in which SSTAs are trained completely independently and cannot
communicate with one another either at training time or run time.

Note, as a control on the experiment, we also evaluated a NoReplan baseline,
in which agents only receive the first SSTA prediction and hence never replan.
This baseline was never able to reach the goal, as expected, and is therefore
not included in the results below. Hypothesis: performance will increase as
communication increases, with the Monolith framework performing best.

###  IV-C Results and Discussion

####  IV-C1 Influence of Communication

From the results shown in Tab. I, our framework can effectively guide the
vehicles to reach the goal for 4-view scenarios. The NoComms and Monolith
baselines provide lower and upper limits for SSTA performance, respectively.
We see that, by communicating with adjacent units, SSTAs’ predictions still
capture most of the useful patterns in traffic flow; the performance of our
approach is only slightly worse than that of Monolith, and substantially
superior to that of NoComms.

####  IV-C2 Failure Cases

In Fig. 4, we show some examples of the predicted results and corresponding
planned path using our pipeline. We find that, for NoComms, the prediction is
not accurate enough, which causes two failure modes: (1) misjudging the
distance to the vehicle ahead of the ego car, resulting in an unnecessary
braking maneuver, and (2) failing to predict the presence of an obstacle and
causing a collision.

####  IV-C3 Practical Considerations

In addition, to assess the conservativeness of our method, we analyze the
vehicle speeds for all trajectories in a histogram and a velocity profile,
both shown in Fig. 3. The vehicle spends a substantial time traveling at a
variety of speeds, as expected in dense urban traffic, and is not excessively
conservative.

####  IV-C4 Main Takeaways

Overall, our experiments confirm the utility of SSTAs for aiding in CAV motion
planning. We note that these experiments relied entirely on the SSTA off-board
sensor for ego-motion planning. Hence, we see much more collisions than would
be expected for a safe, practical CAV. In a real-world deployment, we would
prescribe SSTA outputs as an augmentation of CAV sensor data; to isolate the
benefits of SSTAs, we ignore all onboard ego vehicle sensors and computation
for our experiments.

Figure 4: Failure Cases: two failure modes: (1) misjudging the distance to the
vehicle ahead of the ego car, resulting in an unnecessary braking maneuver,
and (2) failing to predict the presence of an obstacle and causing a
collision.

##  V Conclusion

This paper further develops SSTAs [1] by introducing a learning framework for
self-supervised traffic prediction paired with planning vehicle motion in a
smart city. Our experimental results indicate that the proposed framework
makes progress towards three goals: (1) by communicating with one another,
SSTAs can make more accurate forecasts of traffic flow, (2) CAVs can use SSTA
predictions outputs for downstream motion planning and control, and (3) this
pipeline can improve traffic metrics such as efficiency and safety. Of course,
limitations remain; future work will improve prediction quality, especially in
the transition region between different views, seek to detect anomalous data
(i.e., unpredictable traffic events), and explore notions of road user
privacy. Our work represents a step forward in the development of CAV motion
planning algorithms in partnership with smart infrastructure, with the long-
term goal of creating safer, more efficient, and more sustainable urban
transportation systems.

## Acknowledgment

Toyota Research Institute provided funds to support this work.

## References

  * [1] Jiankai Sun, Shreyas Kousik, David Fridovich-Keil and Mac Schwager  “Self-Supervised Traffic Advisors: Distributed, Multi-view Traffic Prediction for Smart Cities”  In _IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)_ , 2022
  * [2] Md Masud Rana and Kamal Hossain  “Connected and autonomous vehicles and infrastructures: A literature review”  In _International Journal of Pavement Research and Technology_ Springer, 2021, pp. 1–21
  * [3] Jacopo Guanetti, Yeojun Kim and Francesco Borrelli  “Control of connected and automated vehicles: State of the art and future challenges”  In _Annual reviews in control_ 45 Elsevier, 2018, pp. 18–40
  * [4] Junning Huang et al.  “Learning a Decision Module by Imitating Driver’s Control Behaviors”  In _CoRL_ 155, Proceedings of Machine Learning Research  PMLR, 2021, pp. 1–10
  * [5] Jiankai Sun, Hao Sun, Tian Han and Bolei Zhou  “Neuro-Symbolic Program Search for Autonomous Driving Decision Module Design”  In _CoRL_ 155, Proceedings of Machine Learning Research  PMLR, 2021, pp. 21–30
  * [6] Swaminathan Gopalswamy and Sivakumar Rathinam  “Infrastructure Enabled Autonomy: A Distributed Intelligence Architecture for Autonomous Vehicles”  In _2018 IEEE Intelligent Vehicles Symposium (IV)_ , 2018, pp. 986–992  DOI: 10.1109/IVS.2018.8500436
  * [7] Jianhua He, Hsiao-Hwa Chen, Thomas M Chen and Wenqing Cheng  “Adaptive congestion control for DSRC vehicle networks”  In _IEEE communications letters_ 14.2 IEEE, 2010, pp. 127–129
  * [8] Jianhua He et al.  “Cooperative Connected Autonomous Vehicles (CAV): Research, Applications and Challenges”  In _2019 IEEE 27th International Conference on Network Protocols (ICNP)_ , 2019, pp. 1–6  DOI: 10.1109/ICNP.2019.8888126
  * [9] Yang Zheng et al.  “Stability and scalability of homogeneous vehicular platoon: Study on the influence of information flow topologies”  In _IEEE Transactions on intelligent transportation systems_ 17.1 IEEE, 2015, pp. 14–26
  * [10] Jing Hou et al.  “Large-Scale Vehicle Platooning: Advances and Challenges in Scheduling and Planning Techniques”  In _Engineering_ Elsevier, 2023
  * [11] Vishnu Pandi Chellapandi, Liangqi Yuan, Stanislaw H Zak and Ziran Wang  “A Survey of Federated Learning for Connected and Automated Vehicles”  In _arXiv preprint arXiv:2303.10677_ , 2023
  * [12] Chaoyang He et al.  “Fedml: A research library and benchmark for federated machine learning”  In _preprint arXiv:2007.13518_ , 2020
  * [13] Javier Yu, Joseph Vincent and Mac Schwager  “DiNNO: Distributed Neural Network Optimization for Multi-Robot Collaborative Learning”  In _RA-L_ IEEE, 2022
  * [14] Ilai Bistritz, Ariana Mann and Nicholas Bambos  “Distributed distillation for on-device learning”  In _NeurIPS_ 33, 2020, pp. 22593–22604
  * [15] Junqi Yan, Manchu Xiong and Lihui Zhang  “A Cooperative Control Strategy of Connected and Autonomous Vehicles in Ramp Merging Areas with Mixed-Autonomy Traffic”  In _TOCS_ , 2021, pp. 214–218  DOI: 10.1109/TOCS53301.2021.9689004
  * [16] Haotian Shi et al.  “Connected automated vehicle cooperative control with a deep reinforcement learning approach in a mixed traffic environment”  In _Transportation Research Part C: Emerging Technologies_ 133, 2021, pp. 103421  DOI: https://doi.org/10.1016/j.trc.2021.103421
  * [17] Akira Kuchiki and Toru Namerikawa  “Cooperative Control of Connected and Automated Vehicles at Signal-free Intersections Considering Passenger Comfort”  In _SICE_ , 2022, pp. 909–914  DOI: 10.23919/SICE56594.2022.9905760
  * [18] Jacob Walker, Carl Doersch, Abhinav Gupta and Martial Hebert  “An uncertain future: Forecasting from static images using variational autoencoders”  In _ECCV_ , 2016, pp. 835–851  Springer
  * [19] Michael Mathieu, Camille Couprie and Yann LeCun  “Deep multi-scale video prediction beyond mean square error”  In _preprint arXiv:1511.05440_ , 2015
  * [20] Mohammad Babaeizadeh et al.  “Stochastic Variational Video Prediction”  In _ICLR_ , 2018
  * [21] Shruti Vyas, Yogesh S Rawat and Mubarak Shah  “Multi-view action recognition using cross-view video prediction”  In _ECCV_ , 2020, pp. 427–444  Springer
  * [22] Bowen Pan et al.  “Cross-view semantic segmentation for sensing surroundings”  In _RA-L_ 5.3 IEEE, 2020, pp. 4867–4873
  * [23] Chaochao Lu, Michael Hirsch and Bernhard Scholkopf  “Flexible spatio-temporal networks for video prediction”  In _CVPR_ , 2017, pp. 6523–6531
  * [24] Peter Hart, Nils Nilsson and Bertram Raphael  “A Formal Basis for the Heuristic Determination of Minimum Cost Paths”  In _IEEE Transactions on Systems Science and Cybernetics_ 4.2 Institute of ElectricalElectronics Engineers (IEEE), 1968, pp. 100–107  DOI: 10.1109/tssc.1968.300136
  * [25] G. Bradski  “The OpenCV Library”  In _Dr. Dobb’s Journal of Software Tools_ , 2000
  * [26] Alexey Dosovitskiy et al.  “CARLA: An open urban driving simulator”  In _CoRL_ , 2017, pp. 1–16  PMLR
  * [27] Diederik P. Kingma and Jimmy Ba  “Adam: A Method for Stochastic Optimization”  In _ICLR_ , 2015


# Utilizing Generative Adversarial Networks for Stable Structure Generation in
Angry Birds

Frederic Abraham 1, Matthew Stephenson 2

###### Abstract

This paper investigates the suitability of using Generative Adversarial
Networks (GANs) to generate stable structures for the physics-based puzzle
game Angry Birds. While previous applications of GANs for level generation
have been mostly limited to tile-based representations, this paper explores
their suitability for creating stable structures made from multiple smaller
blocks. This includes a detailed encoding/decoding process for converting
between Angry Birds level descriptions and a suitable grid-based
representation, as well as utilizing state-of-the-art GAN architectures and
training methods to produce new structure designs. Our results show that GANs
can be successfully applied to generate a varied range of complex and stable
Angry Birds structures.

## Introduction

Procedural Content Generation (PCG), which describes the creation of content
through algorithmic means, has become an increasingly prominent aspect of
video game development (Amato 2017). In the same timeframe as conventional PCG
techniques were being researched, the number of applications utilizing Machine
Learning (ML) approaches has also increased, with techniques such as Neural
Networks and Deep Learning receiving a large amount of attention (Goodfellow,
Bengio, and Courville 2016). Consequently, the use of ML approaches for
content generation, under the abbreviation PCGML, has become an area of
significant research interest (Summerville et al. 2017).

While many different types of ML algorithms have been used to generate
content, one of the most promising approaches in recent years has been the use
of Generative Ad- versarial Networks (GANs) (Gui et al. 2021; Jabbar, Li, and
Omar 2021; Saxena and Cao 2020). A GAN is the construct of two adversarial
networks, in which one network generates new content and the second network
discriminates the generated content to differentiate between real and
generated (Goodfellow et al. 2014). While the primary application of GANs has
largely been for image and video synthesis (Huang, Yu, and Wang 2018; Liu et
al. 2021), they have also been applied to several other domains including
video game content generation (Giacomello, Lanzi, and Loiacono 2018; Volz et
al. 2018). However, to the best of our knowledge, when it comes to generating
level-based content for video games, GANs have only been applied to discrete
domains with no physical constraints.

In this paper, we present an approach for generating stable structures made of
several rectangular blocks in a continuous physical environment. More
specifically, we generate structures for use in the 2D physics-based puzzle
game Angry Birds (Rovio 2023). This domain provides a unique and novel
challenge for applying GAN-based approaches, where considerations such as the
game’s continuous environment space and physical constraints must be taken
into account. Our results demonstrate that such an approach is viable for
generating complete Angry Birds structures.

The remainder of this paper is organized as follows. We first describe
background details and prior work related to GANs and their use in level
generation, as well as previous approaches to Angry Birds level generation.
Next, we describe the methodology we employed with regards to level encoding
and decoding, as well as our GAN model training process. We then describe our
experiments, providing details on how our specific GAN model was trained and
analyze the generated levels it was able to produce. We finish with a summary
conclusion of our approach and output, along with suggestions for future work.

## Background and Related Work

### Generative Adversarial Networks

A GAN is a framework for producing generative models via a process in which
two networks, the generator and the discriminator, are trained simultaneously
(Goodfellow et al. 2014). The generator aims to capture the data distribution
of the training data, while the discriminator, also called the critic, tries
to differentiate between samples drawn from the training data and samples
generated by the generator. The generator is trained to maximize the
probability that the discriminator mistakes its generated example as drawn
from the actual distribution. The central concept behind applying GANs is to
define the given task as a game between two opposing systems, which are then
trained in an adversarial manner to reach a zero-sum Nash equilibrium
(Moghadam et al. 2021). While GANs have been primarily used for image
synthesis (Goodfellow et al. 2014; Radford, Metz, and Chintala 2016; Liu and
Tuzel 2016; Karras et al. 2017; Huang, Yu, and Wang 2018), they have also been
successfully applied to many other applications including object detection
(Prakash and Karam 2021; Posilović et al. 2021), natural language processing
(Subramanian et al. 2017), audio enhancement (Torres-Reyes and Latifi 2019;
Biswas and Jia 2020), anomaly detection (Schlegl et al. 2017; Xia et al.
2020), and, most relevant for this paper, video game level generation.

In recent years, several variations and improvements to GANs have been
proposed for different aspects of the underlying framework. The architecture
of each network, the controllability, the training process, scaling,
adaptation and application in different domains are a few research directions
that have been investigated. Some notable advancements include the development
of Deep Convolutional GANs (DCGANs) (Radford, Metz, and Chintala 2016), which
introduced deeper architectures and improved training stability for image
synthesis tasks. Another significant improvement came with the introduction of
Wasserstein GANs (WGANs) (Arjovsky, Chintala, and Bottou 2017), which proposed
a new objective function to address training difficulties and mode collapse
issues (Salimans et al. 2016). The term mode collapse refers to the situation
where a GAN model repeatedly generates highly similar outputs that don’t
represent the variety of content present in the original training set.

### Level Generation with GANs

While GANs have been used to generate a variety of different types of game
content, such as the generation of NPC character sprites (Kim et al. 2023;
Coutinho and Chaimowicz 2022), we will focus primarily on the use of GANs to
generate game levels. One of the first to apply GANs in the context of level
generation was Giacomello et al., in their work to generate DOOM level images
based on human-designed examples (Giacomello, Lanzi, and Loiacono 2018). They
describe their preliminary results as a good starting point for researching
the viability of GANs compared to classical PCG. The generated level images
contained DOOM typical features and are reportedly interesting to explore,
although the generated data could not be decoded into playable levels.

Volz et al. also utilized GANs to generate complete Mario levels (Volz et al.
2018). They use a Covariance Matrix Adaptation Evolutionary Strategy (CMA-ES)
to search the latent space of the GANs generator to influence the outcome
based on different metrics over the generated levels. Their first approach is
to optimize different block distributions. For example, fewer stone blocks
could lead to an air level with greater difficulty. In their second approach,
they utilize a Mario AI (Togelius et al. 2013) that can produce playthrough
data of their generated levels. They focused on optimizing toward playable
levels with a scalable difficulty. The idea of using latent variable evolution
(LVE) to explore the generator’s latent space was first introduced by
Bontrager et al., in their works to match generated fingerprints to as many
real fingerprints as possible (Bontrager, Togelius, and Memon 2017). Evolving
the latent space to gain control over the output stands in contrast to
Conditional GANs (CGANs) (Mirza and Osindero 2014), which utilize a condition
vector combined with the noise vector as input to the generator to produce a
controllable output. They conclude that GANs can capture high-level structures
of the training level, although they may sometimes produce broken elements
such as incomplete pipes and structures.

Figure 1: An example Angry Birds level containing five distinct structures. Id |  Shape |  Name |  Dimensions
---|---|---|---
1 |  |  SquareHole |  (0.85, 0.85)
2 |  |  RectBig |  (2.06, 0.22)
3 |  |  RectMedium |  (1.68, 0.22)
4 |  |  RectSmall |  (0.85, 0.2)
5 |  |  RectFat |  (0.85, 0.43)
6 |  |  RectTiny |  (0.42, 0.22)
7 |  |  SquareTiny |  (0.22, 0.22)
8 |  |  SquareSmall |  (0.43, 0.43)
Table 1: Block types that are available in Science Birds.

### Angry Birds Level Generation

Angry Birds is a 2D physics-based simulation game, where players are tasked
with shooting birds at structures made of multiple smaller blocks. These
blocks are each made of a specific material (wood, ice or stone) which affects
their resistance to certain bird types. The players objective in each level is
to kill all of the pigs, which are often located within or on top of
structures, using a limited number of birds. An example Angry Birds level,
containing five structures and nine pigs, is shown in Figure 1. Due to the
fact that the original Angry Birds game is not open source, most research on
Angry Birds instead uses a Unity-based clone of the game called Science Birds,
created by Lucas Ferreira (Ferreira and Toledo 2014). This clone contains all
the same elements as the original Angry Birds game, and each level is
described in a single xml file containing the position of each object.

Table 1 shows the available block types that structures can be made from,
along with their names and dimensions (width, height). While there are several
additional “irregular” block shapes that are also available in Angry Birds,
specifically two triangular and two circular block types, the vast majority of
Angry Birds structures do not include these. Many prior structure generators
for Angry Birds either excluded irregular blocks entirely or used them in a
purely decorative manner, as the stability of a structure is much harder to
verify if irregular blocks are present. In addition to this, blocks are almost
always placed at 90-degree angles, once again to reduce the risk of structural
instability. As a result of this, our proposed GAN-based generator will be
solely trained on, and will therefore only output, structures made of the
block types specified in Table 1, placed in either a horizontal (angle = 0) or
vertical (angle = 90) orientation.

Over the past decade a large number of different level generators for Angry
Birds have been proposed, with many of them being entered into the AIBirds
Level generation competition (Stephenson et al. 2019). These generators have
experimented with a variety of different approaches, including Genetic
Algorithms (Ferreira and Toledo 2014), search-based techniques (Stephenson and
Renz 2016a, b, 2017), Monte Carlo Tree Search (Graves, Caramanis, and
Nagarajan 2016), and latent variable evolution in a variational auto encoder
(Tanabe et al. 2021), as well as focusing on a variety of different desirable
aspects, such as levels that contain structures which resemble quotes or
formulas (Jiang, Harada, and Thawonmas 2017), deceptive elements (Gamage et
al. 2021), or Rube Goldberg Machine mechanisms (Abdullah et al. 2019).

## Methodology

This section describes the different components that make up our proposed GAN
framework for Angry Birds levels generation. This includes level encoding,
level decoding and GAN Model Training.

### Level Encoding

This section describes the encoding process that was used to convert an XML
level description of a structure into a suitable grid-based data
representation that can be used to train our proposed GAN model.

#### Grid-Based Representation

One of the major challenges in this domain is the real-valued positioning and
dimensions of in-game objects. Continuous values such as these are impractical
for GAN models, which typically require the input to be in a discrete, grid-
based representation to function effectively (e.g., the pixels of an image, or
the individual tiles in a Mario level). As a result, our XML structure
representation first needs to be converted or encoded to a grid-based
representation with a specified degree of precision (i.e., a raster size).
Figure 2 shows an example of a simple Angry Birds structure with continuous
block positions and dimensions, represented visually as a wireframe diagram
with different colors for each block material and pigs, that will be used as a
reference throughout this section.

Figure 2: An example Angry Birds structure represented as a colored wireframe
diagram.

In order to discretize a structure to a specified level of precision, a
suitable raster size has to be chosen. A smaller raster size better represents
the blocks true dimensions, and is able to capture small gaps and imperfect
positionings that a larger raster size may miss. The downside of a smaller
raster size is that it can dramatically increase the dimensionality of the
encoded structure representation, requiring a larger GAN model that results in
significantly longer training and generation times. Conversely, a bigger
raster size results in a smaller encoded output representation where only a
few grid cells represent any given block. This comes with a loss of finer
details, that may potentially result in incorrectly represented block sizes.

As a suitable compromise between these two considerations, we selected the
largest raster size that resulted in a near-integer value when dividing by any
block’s true dimension. This was done to reduce the risk of the same block
type being encoded at different sizes, depending on its relative position to
the grid lines. For the block sizes present in ScienceBirds a raster size 0.07
resulted in quotients with only a small remainder, and was subsequently chosen
as our raster size. This raster size of 0.07 in-game distance units, can be
equated to 1 unit of our proposed grid dimension encoding.

Using this chosen raster size, we can now convert any given XML level
description into a matching grid-based encoding. For each block, the
horizontal and vertical start and end positions are calculated by taking the
given center position of the block and adding/subtracting half of the
width/height respectively. To transform these positions into the grid indices,
each value is divided by the raster size (0.07) and rounded to the nearest
integer. Doing this for each object has the effect of converting its position
and size to exactly fit our defined grid dimensions. Figure 3 shows how the
example structure shown in Figure 2 can be converted using this proposed
encoding approach into a rasterized image.

Figure 3: An encoded (rasterized) 2D image of an Angry Birds structure to a
precision (resolution) of 0.07.

One limitation with this structure encoding approach is that number and size
of the grid cells is fixed, which places a bound on the maximum structure size
that could be encoded. For our GAN model, we fixed our grid dimension at
128x128 cells or approximately 9x9 in-game distance units. This can be
increased to support larger structures if desired, although this will
naturally result in longer training and generation times.

#### Multilayer Representation

While it is possible to represent all four object types (wood/ice/stone blocks
and pigs) as different values (wood = 1, ice = 2, etc.) within a single 2D
grid representation, this makes training our GAN model more difficult. With
this representation approach, our GAN model must simultaneously decide about
the positioning of the elements and their material within a single layer.
Looking at prior work, GAN data representations are usually not only two-
dimensional. For example, face synthesis requires three channels for the RGB-
Color space, and Volz et al. (Volz et al. 2018) used ten channels in a one-hot
encoding, one for each Mario block type. By moving each object type to
separate layers, the decision for each layer is simplified to only predicting
if an element of the associated type is present. Figure 4 shows a visual
representation of this multilayer encoding approach for the same 2D image
shown in Figure 3. Using this multilayer representation splits the blocks of
each material type into separate layers, with pigs and air (i.e., empty space)
each getting their own layer as well, making for a total of five layers.

### Level Decoding

This section describes the inverse of the previous process, that of taking a
structure that is output by our trained GAN model using our specified grid-
based representation and decoding it into a valid XML level description for
Science Birds.

#### Confidence Decoding

Most GAN-based approaches for procedural content generation use an encoding
method that allows for a one-to-one decoding, meaning that no extra steps are
required to create the level. This can potentially lead to broken elements,
such as a pipe that may be missing several tiles (Volz et al. 2018) and can be
solved by encoding the whole group of tiles together. The decoding process can
also include a healing section, where a post-processing algorithm fixes
errors, tests for playability and repairs the level accordingly.

When using the multilayered encoding defined in the previous section our GAN
outputs a real-valued matrix for each object type, with each matrix having the
same dimensions as our defined grid representation. A value closer to one for
a given matrix entry represents a higher confidence that an object of that
type should be present in the corresponding grid cell, with a value close to
one for the air layer representing that there should be no object present. In
other words, our output matrices represent the predicted confidence that each
GAN layer has for an object of its type being present at any given grid cell.
This multilayer representation can be subsequently converted back into a flat
2D image using the argmax operator, where the layer with the highest
confidence value output is used. However, simply determining the object type
for each grid cell will not be sufficient, as our output XML representation
can only include blocks of pre-defined sizes. In other words, each block is
encoded into, and must also therefore be decoded from, multiple grid cells.

Figure 4: A multilayer representation of an Angry Birds structure, with
separate layers for each object type visualized along the z-axis

(a) Selection-Ranking matrix with no clipping

(b) Selection-Ranking matrix with hit probabilites clipped at 0.98

Figure 5:  Heatmap representations of the Selection-Ranking matrix for each
block type, calculated based on the layer-wise multiplication of the Hit-
Probability matrix and Size-Ranking matrix. Subfigure (a) represents the
original Selection-Ranking matrix; while subfigure (b) represents the same
Selection-Ranking matrix but with the Hit-Probabilities matrix clipped at
0.98. Each image represents a specific block type and corresponds to an
individual layer of the Selection-Ranking matrix. The value above each image
represents the maximum value present in this layer.

#### Matrix Creation

To decode a generated structure image back into a valid xml level description,
we need to identify suitable block positions for each material that maintain
the structure’s overall design. The first step of our proposed process
involves the creation of a 3-dimensional Selection-Ranking matrix, with
dimensions 128x128x13. Each layer of this matrix represents a specific block
type, including separate layers for horizontal and vertical orientations of
rectangular block shapes, giving a total of 13 layers. Each of these layers
contains 128x128 values representing each position of our encoded grid space,
with the value at each position indicating its suitability for placing a block
of the associated type. This Selection-Ranking matrix is a combination of two
sub-matrices, called the Hit-Probabilities matrix and the Size-Ranking matrix.

The Hit-Probabilities matrix for each layer is created by applying a Gaussian
kernel in the shape of the associated block type across the encoded structure
representation, see Table 1. Each layer of this matrix represents how well the
associated block type would match the encoded structure representation if
placed at each possible position (i.e., what is the percentage of overlap
between the block and the structure). However, using this matrix alone leads
to an issue, namely that smaller blocks (shown in the lower rows) will have a
much higher hit probability than larger blocks, simply due to their smaller
size. If only the highest hit probabilities were used in the selection
process, any small imperfections in the representation would dismiss any
larger block types that may have been intended to be there, and a group of
smaller blocks would likely be used instead. To address this, we introduced an
additional Size-Ranking matrix that applies a sum kernel to add up the values
of all covered pixels for each block type at each location.

Using the encoded Angry Birds structure representation from Figure 3 as our
input, Figure 5a shows the layer-wise multiplication of these two sub-matrices
into the combined Selection-Ranking matrix, where yellow areas represent
locations where placing the associated block type would most match the encoded
structure representation. It can be seen in Figure 5a that the horizontal
RectBig layer has the highest selection value (113.54), even though it crosses
gaps in the encoded structure image. By clipping our Hit-Probabilities matrix
at a high value, in this case 0.98, any block types that cross gaps such as
these are removed. In practice, this has the effect of making our Selection-
Ranking matrix more distinct with harder edges, see Figure 5b, and ensures
that any small gaps in the original structure encoding are more likely to be
maintained in the decoded output. This final clipped version of the Selection-
Ranking matrix is then used as input for the subsequent block selection
algorithm.

(a) Iteration 1

(b) Iteration 2

(c) Iteration 8

(d) Iteration 17

Figure 6:  Heatmap representations of the clipped Selection-Ranking matrix for
iterations 1, 2, 8 and 17 of the block selection process. The block type
(matrix layer) and location with the highest Selection-Ranking matrix value is
chosen at the start of each iteration (represented by the blue block
outlines). Any values/positions in the Selection-Ranking matrix that are no
longer valid, due to overlapping this selected block, are set to zero for
future iterations (represented by the red outlines). This process repeats
until all values in the Selection-Ranking matrix are equal to zero.  Figure 7:
The result of the decoding process.

#### Block Selection

Our block selection algorithm starts by identifying the highest value in our
clipped Selection-Ranking matrix, which corresponds to the “best fit” location
for a specific block type, see blue block outline for the horizontal
RectMedium layer in Figure 6a. We then place a block of this type at this
location and set any values in our matrix that would overlap this block to
zero, see red outlines for the other block types/layers in Figure 6a. This
process repeats until all values in our matrix are set to zero. The clipped
Selection-Ranking matrix shown in Figure 5b takes a total of 18 iterations for
this process to finish, where in each iteration a new block is selected and
added to the final structure output. Figure 6 shows several iterations of the
Selection-Ranking matrix throughout this block selection process. We then use
a circular kernel to identify possible pig positions, followed by the same
location selection process.

The last step of the decoding process is to address the issue of blocks
slightly overlapping each other, which can occur when a block has an uneven
width/height. We therefore apply a small structure adjustment that moves each
block up and to the right, until it no longer overlaps any blocks that are
below it or on its left side.

The finished result of this decoding process can be seen in Figure 7. By
comparing this to the original structure representation in Figure 2 we can see
that there are some minor differences between them, but the overall shape and
design of the structure is sufficiently similar. This example demonstrates
that our proposed encoding and decoding process is able to successfully
convert between playable XML level descriptions, and grid-based structure
representations that are appropriate for GAN training.

### GAN Training

With the encoding and decoding process now defined, this section describes how
our encoded structure representation is processed and used for training our
GAN model. In order to train our GAN model, two aspects need to be selected.
Firstly, the specific architecture that will be used for the generator and
discriminator networks. Secondly, the objective function that will be used to
update the weights in these networks towards a desirable output.

Figure 8: Generator network architecture. Figure 9: Discriminator network
architecture.

#### GAN Architecture

Our proposed GAN model is based on the Deep Convolutional Generative
Adversarial Network (DCGAN) architecture (Radford, Metz, and Chintala 2016).
This is the first Convolutional Neural Network based GAN architecture and has
been shown to perform well in image generation tasks (Jabbar, Li, and Omar
2021). Compared to a more standard GAN architecture, which only used fully
connected layers and pooling layers, the proposed DCGAN architecture instead
uses transposed convolutions to create the required image size.

The generator network portion of our proposed GAN is shown in Figure 8. This
network exclusively uses transposed convolutional layers to enlarge the image,
with a stride value of two effectively doubling the resolution size with each
layer block, as this has been shown to significantly improve the stability of
GAN training (Jabbar, Li, and Omar 2021). The standard batch normalization
layers were instead replaced by layer normalization (Ba, Kiros, and Hinton
2016), to reduce the risk of mode collapse during training. The generator uses
the ReLU activation function (Nair and Hinton 2010) between the layers and a
Tanh function at the output layer, similar to the original implementation by
Goodfellow et al. (Goodfellow et al. 2014).

The discriminator network is visualized in Figure 9. While the generator uses
transposed convolutional layers, the discriminator uses convolutional layers
to reduce the image size and arrive at a decision. The discriminator uses a
Leaky ReLU activation function (Xu et al. 2015), based on the recommendation
by Radford et al. in their architecture guidelines for stable Deep
Convolutional GANs (Radford, Metz, and Chintala 2016).

#### Objective Function

For our objective function, we decided to use the Wasserstein objective
function that is based on the Wasserstein or Earth Mover’s distance metric
(Arjovsky, Chintala, and Bottou 2017). This approach provides several
benefits, including increased training stability and reduced risk of mode
collapse. In addition, we use an updated version of this objective function
that applies a gradient penalty to the discriminator network (Gulrajani et al.
2017), rather than the regular weight clipping approach. This change has been
shown to further increase GAN training stability and convergence likelihood.

Figure 10: Example GAN outputs and corresponding decoded Angry Birds
structures.

## Experiments

Our experiments consisted of training the proposed GAN model on an example set
of encoded levels descriptions, as well as evaluating the quality and variety
of output structures that it was able to produce. The complete set of all
generated levels, as well as our fully trained GAN model, is open access and
can be downloaded from a public GitHub
repository111https://github.com/Blaxzter/Utilizing-Generative-Adversarial-
Networks-for-Stable-Structure-Generation-in-Angry-Birds.

### Model Training

Due to the limited number of human-made levels that are currently available
for Science Birds, we decided to use the open-source level generator Iratus
Aves (Stephenson and Renz 2017) to provide structures for our training
dataset. Using this generator, we created an initial training dataset of 5000
XML level descriptions that each define a single structure (including pigs).

One important consideration when training GANs is to ensure that the training
dataset is sufficiently balanced in terms of its content diversity, as this
can reduce the risk of mode collapse occurring. To achieve this, we applied a
filter to remove any overly similar structures from our training dataset,
which in turn would increase the average structure diversity. We first applied
a metadata filter to remove any structures with the same number of blocks for
each material, as well as the same width and height with a 0.1 unit margin. We
next applied a shape filter to remove any structures with the same encoding
outline, regardless of block shape or material type. After applying both these
filters, our training dataset size was reduced from 5000 to 3566.

Our GAN model was trained for 15000 epochs on our filtered training dataset of
3566 generated structures. This was performed using the RWTH High-Performance
Computing cluster (Aachen 2023) and took approximately 48 hours to complete.

### Results

Our trained GAN model was used to generate 8000 different structure
representations, which were then decoded into playable XML level descriptions.
This process took around 15 minutes on a desktop computer with an Intel
i7-5820k CPU, with over 95% of that time being used for structure decoding.
Figure 10 displays several of these GAN generated structure representation
outputs, along with their decoded XML level descriptions loaded into Science
Birds. Based on these examples, we can see that many of the GAN outputs
contained small distortions or noise that makes the type or shape of certain
blocks ambiguous. Despite this, our proposed decoding approach can typically
handle these minor imperfections and is able to produce reasonably accurate
and complete structures for Angry Birds.

#### Structure Stability

The first test we conducted was to investigate how many of our 8000 generated
structures were stable when loaded into the Science Birds game engine. The
stability of any given structure can be determined using one of two
approaches. The “Block Velocity” measure determines that a structure is stable
if all blocks are stationary when the level is loaded. The “Block Destruction”
measure determines that a structure is stable if no blocks are destroyed after
the level is loaded. Blocks in Angry Birds will typically be destroyed if they
fall from a sufficient height or collide with other blocks, meaning that this
measure of stability is a good test for if a structure has collapsed.

Using the Block Velocity measure, 945 structures would be classified as stable
and 7055 as unstable. Using the Block Destruction measure, 3487 structures
would be classified as stable and 4533 as unstable. Please note, that the
Block Destruction measure is a strictly weaker version of the Block Velocity
measure (i.e., any structure which is classified as stable by the Block
Velocity measure is also always classified as stable by the Block Destruction
measure). This large disparity would indicate that, while our generated
structures often contain blocks that move slightly after loading, they are
much less likely to collapse completely.

#### Structure Diversity

With regards to structure diversity, our generated structures varied
significantly in terms of their width, height, density, shape and block
frequency. Across all 8000 generated structures, the average width was 4.87
(±0.87) and the average height was 3.67 (±1.36). The average density of each
structure, calculated as the percentage of the available level space that is
occupied by an object, was 37.05% (±10.56%). The average number of blocks was
24.01 (±10.18), with an exact break down by block type and orientation
provided in Table 2. For reference, please refer to Table 1 for the dimensions
of each block type. The average number of pigs in each structure was 2.76
(±1.76), with 9.1% of generated structures containing zero pigs. While this
may initially seem like a serious problem, given that a level with no pigs is
already solved, it is important to remember that our proposed approach is
intended to create single structures rather than complete levels. Structures
without pigs can therefore still be included within a level, as long as they
are placed alongside one or more structures that do contain pigs. There was
also very little difference between any of these values when comparing stable
and unstable structures separately.

Id |  Name |  Frequency
---|---|---
1 |  SquareHole |  8.75% (±5.89%)
2h |  RectBig |  5.89% (±3.20%)
2v |  RectBig (Vert) |  13.46% (±10.21%)
3h |  RectMedium |  4.14% (±2.13%)
3v |  RectMedium (Vert) |  5.91% (±4.26%)
4h |  RectSmall |  4.18% (±2.05%)
4v |  RectSmall (Vert) |  7.33% (±5.24%)
5h |  RectFat |  5.11% (±2.81%)
5v |  RectFat (Vert) |  15.81% (±16.84%)
6h |  RectTiny |  6.84% (±4.12%)
6v |  RectTiny (Vert) |  6.99% (±4.90%)
7 |  SquareTiny |  9.04% (±5.95%)
8 |  SquareSmall |  6.55% (±4.56%)
Table 2: Average frequency of each block type across all 8000 generated
structures (±SD).

In terms of creating new and novel structure designs, while our generated
structures appear to have several similar design elements to those present in
the original training dataset, which is to be expected of a GAN based
approach, they also have many differences. For example, the structures
produced by the Iratus Aves generator are created using rows of blocks with
the same height, leading to highly symmetrical designs (Stephenson and Renz
2017). However, this was not the case for many of our GAN generated
structures, such as those shown in the bottom left and top right examples for
Figure 10.

## Conclusion

In this paper we have presented, implemented, trained, and evaluated a
framework for using GANs to generate new structures for Angry Birds. One of
the main contributions of this paper is the proposed encoding and decoding
process, that can accurately convert between a playable XML level description
and a grid-based structure representation more suited for GANs. Using this in
conjunction with state-of-the-art GAN architectures, we were able to
successfully train a GAN model to produce complete and highly varied structure
designs. While many of these generated structures were initially stable, some
of them unfortunately collapsed when loaded into our simulation engine.
However, the number of unstable structures was not overwhelmingly large, and
such structures could easily be discarded after generation via a simple
generate-and-test approach. As such, we believe that the use of GAN models to
generate Angry Birds structures may be used to provide an abundance of both
new training content for AI agents and gameplay experiences for human players.

We would also like to highlight that this paper is a condensed version of an
original Masters thesis (Abraham 2022). This paper describes the most
successful approach from this thesis, but several alternative
encoding/decoding approaches were also investigated. We would encourage
interested readers to take a closer look at this thesis for more details.

In terms of future work, one of the first extensions we might make is to train
our proposed model on a larger range of content. Our experimental training set
contained 3566 structures from a single generator, but there are over a dozen
different level generators for Angry Birds that have been proposed during the
past decade. Utilizing several of these generators could provide a much more
diverse set of training levels, and potentially a more varied range of output
structures. Beyond this, we could also experiment with different GAN
architectures, stable diffusion models, data representations, or improvements
to our encoding/decoding processes. Lastly, this approach could be applied to
other physics-based domains beyond Angry Birds.

## References

  * Aachen (2023) Aachen, R. 2023.  RWTH High Performance Computing.  https://help.itc.rwth-aachen.de/en/service/t4ibytwt1n3p/.  Accessed: 2023-08-16.
  * Abdullah et al. (2019) Abdullah, F.; Paliyawan, P.; Thawonmas, R.; Harada, T.; and Bachtiar, F. A. 2019\.  An Angry Birds Level Generator with Rube Goldberg Machine Mechanisms.  In _2019 IEEE Conference on Games (CoG)_ , 1–8.
  * Abraham (2022) Abraham, F. M. 2022.  _Utilizing generative adversarial networks for stable structure generation in a physics-based simulation_.  Master’s thesis, Maastricht University.
  * Amato (2017) Amato, A. 2017.  _Procedural Content Generation in the Game Industry_ , 15–25.  Cham: Springer International Publishing.  ISBN 978-3-319-53088-8.
  * Arjovsky, Chintala, and Bottou (2017) Arjovsky, M.; Chintala, S.; and Bottou, L. 2017.  Wasserstein Generative Adversarial Networks.  In Precup, D.; and Teh, Y. W., eds., _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , 214–223. PMLR.
  * Ba, Kiros, and Hinton (2016) Ba, L. J.; Kiros, J. R.; and Hinton, G. E. 2016.  Layer Normalization.  _CoRR_ , abs/1607.06450.
  * Biswas and Jia (2020) Biswas, A.; and Jia, D. 2020.  Audio Codec Enhancement with Generative Adversarial Networks.  _ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 356–360.
  * Bontrager, Togelius, and Memon (2017) Bontrager, P.; Togelius, J.; and Memon, N. D. 2017.  DeepMasterPrint: Generating Fingerprints for Presentation Attacks.  _CoRR_ , abs/1705.07386.
  * Coutinho and Chaimowicz (2022) Coutinho, F.; and Chaimowicz, L. 2022.  On the Challenges of Generating Pixel Art Character Sprites Using GANs.  _Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment_ , 18(1): 87–94.
  * Ferreira and Toledo (2014) Ferreira, L.; and Toledo, C. 2014.  A Search-based Approach for Generating Angry Birds Levels.  In _Proceedings of the 9th IEEE International Conference on Computational Intelligence in Games_ , CIG’14.
  * Gamage et al. (2021) Gamage, C.; Pinto, V.; Renz, J.; and Stephenson, M. 2021.  Deceptive Level Generation for Angry Birds.  In _2021 IEEE Conference on Games (CoG)_ , 1–8.
  * Giacomello, Lanzi, and Loiacono (2018) Giacomello, E.; Lanzi, P. L.; and Loiacono, D. 2018.  DOOM Level Generation Using Generative Adversarial Networks.  In _2018 IEEE Games, Entertainment, Media Conference (GEM)_ , 316–323.
  * Goodfellow, Bengio, and Courville (2016) Goodfellow, I.; Bengio, Y.; and Courville, A. 2016.  _Deep Learning_.  MIT Press.  http://www.deeplearningbook.org.
  * Goodfellow et al. (2014) Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; and Bengio, Y. 2014.  Generative Adversarial Nets.  In _Advances in Neural Information Processing Systems_ , volume 27.
  * Graves, Caramanis, and Nagarajan (2016) Graves, M.; Caramanis, C.; and Nagarajan, S. 2016.  _Procedural content generation of Angry Birds levels using Monte Carlo Tree Search_.  Master’s thesis, The University of Texas at Austin.
  * Gui et al. (2021) Gui, J.; Sun, Z.; Wen, Y.; Tao, D.; and Ye, J. 2021.  A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications.  _IEEE Transactions on Knowledge and Data Engineering_.
  * Gulrajani et al. (2017) Gulrajani, I.; Ahmed, F.; Arjovsky, M.; Dumoulin, V.; and Courville, A. C. 2017\.  Improved Training of Wasserstein GANs.  _CoRR_ , abs/1704.00028.
  * Huang, Yu, and Wang (2018) Huang, H.; Yu, P. S.; and Wang, C. 2018.  An Introduction to Image Synthesis with Generative Adversarial Nets.  _CoRR_ , abs/1803.04469.
  * Jabbar, Li, and Omar (2021) Jabbar, A.; Li, X.; and Omar, B. 2021.  A Survey on Generative Adversarial Networks: Variants, Applications, and Training.  _ACM Comput. Surv._ , 54(8).
  * Jiang, Harada, and Thawonmas (2017) Jiang, Y.; Harada, T.; and Thawonmas, R. 2017.  Procedural generation of angry birds fun levels using pattern-struct and preset-model.  In _2017 IEEE Conference on Computational Intelligence and Games (CIG)_ , 154–161.
  * Karras et al. (2017) Karras, T.; Aila, T.; Laine, S.; and Lehtinen, J. 2017.  Progressive Growing of GANs for Improved Quality, Stability, and Variation.  _CoRR_ , abs/1710.10196.
  * Kim et al. (2023) Kim, J.; Jin, K.; Jang, S.; Kang, S.; and Kim, Y. 2023.  Game effect sprite generation with minimal data via conditional GAN.  _Expert Systems with Applications_ , 211: 118491.
  * Liu et al. (2021) Liu, M.-Y.; Huang, X.; Yu, J.; Wang, T.-C.; and Mallya, A. 2021.  Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications.  _Proceedings of the IEEE_ , PP: 1–24.
  * Liu and Tuzel (2016) Liu, M.-Y.; and Tuzel, O. 2016.  Coupled Generative Adversarial Networks.  In _Advances in Neural Information Processing Systems_ , volume 29.
  * Mirza and Osindero (2014) Mirza, M.; and Osindero, S. 2014.  Conditional Generative Adversarial Nets.  _CoRR_ , abs/1411.1784.
  * Moghadam et al. (2021) Moghadam, M. M.; Boroumand, B.; Jalali, M.; Zareian, A.; Javad, A. D.; and Manshaei, M. H. 2021.  Game of GANs: Game Theoretical Models for Generative Adversarial Networks.  _CoRR_ , abs/2106.06976.
  * Nair and Hinton (2010) Nair, V.; and Hinton, G. E. 2010.  Rectified Linear Units Improve Restricted Boltzmann Machines.  In _Proceedings of the 27th International Conference on International Conference on Machine Learning_ , ICML’10, 807–814. Madison, WI, USA: Omnipress.  ISBN 9781605589077.
  * Posilović et al. (2021) Posilović, L.; Medak, D.; Subašić, M.; Budimir, M.; and Lončarić, S. 2021.  Generative adversarial network with object detector discriminator for enhanced defect detection on ultrasonic B-scans.  _Neurocomputing_ , 459: 361–369.
  * Prakash and Karam (2021) Prakash, C.; and Karam, L. 2021.  It GAN DO Better: GAN-based Detection of Objects on Images with Varying Quality.  _IEEE Transactions on Image Processing_.
  * Radford, Metz, and Chintala (2016) Radford, A.; Metz, L.; and Chintala, S. 2016.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks.  In Bengio, Y.; and LeCun, Y., eds., _4th International Conference on Learning Representations, ICLR_.
  * Rovio (2023) Rovio. 2023.  Angry Birds.  https://www.angrybirds.com/games/rovio-classics-angry-birds/.  Accessed: 2023-08-16.
  * Salimans et al. (2016) Salimans, T.; Goodfellow, I. J.; Zaremba, W.; Cheung, V.; Radford, A.; and Chen, X. 2016.  Improved Techniques for Training GANs.  _CoRR_ , abs/1606.03498.
  * Saxena and Cao (2020) Saxena, D.; and Cao, J. 2020.  Generative Adversarial Networks (GANs): Challenges, Solutions, and Future Directions.  _CoRR_ , abs/2005.00065.
  * Schlegl et al. (2017) Schlegl, T.; Seeböck, P.; Waldstein, S. M.; Schmidt-Erfurth, U.; and Langs, G. 2017.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery.  In _Information Processing in Medical Imaging_ , 146–157.
  * Stephenson and Renz (2016a) Stephenson, M.; and Renz, J. 2016a.  Procedural generation of complex stable structures for angry birds levels.  In _2016 IEEE Conference on Computational Intelligence and Games (CIG)_ , 1–8.
  * Stephenson and Renz (2016b) Stephenson, M.; and Renz, J. 2016b.  Procedural Generation of Levels for Angry Birds Style Physics Games.  In _Proceedings of the Twelfth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment_ , AIIDE’16. AAAI Press.  ISBN 978-1-57735-772-8.
  * Stephenson and Renz (2017) Stephenson, M.; and Renz, J. 2017.  Generating varied, stable and solvable levels for angry birds style physics games.  In _2017 IEEE Conference on Computational Intelligence and Games (CIG)_ , 288–295.
  * Stephenson et al. (2019) Stephenson, M.; Renz, J.; Ge, X.; Ferreira, L.; Togelius, J.; and Zhang, P. 2019\.  The 2017 AIBIRDS Level Generation Competition.  _IEEE Transactions on Games_ , 11(3): 275–284.
  * Subramanian et al. (2017) Subramanian, S.; Rajeswar, S.; Dutil, F.; Pal, C.; and Courville, A. 2017.  Adversarial Generation of Natural Language.  In _Proceedings of the 2nd Workshop on Representation Learning for NLP_ , 241–251. Vancouver, Canada: Association for Computational Linguistics.
  * Summerville et al. (2017) Summerville, A.; Snodgrass, S.; Guzdial, M.; Holmgård, C.; Hoover, A. K.; Isaksen, A.; Nealen, A.; and Togelius, J. 2017.  Procedural Content Generation via Machine Learning (PCGML).  _CoRR_ , abs/1702.00539.
  * Tanabe et al. (2021) Tanabe, T.; Fukuchi, K.; Sakuma, J.; and Akimoto, Y. 2021.  Level Generation for Angry Birds with Sequential VAE and Latent Variable Evolution.  In _Proceedings of the Genetic and Evolutionary Computation Conference_ , GECCO ’21, 1052–1060. New York, NY, USA: Association for Computing Machinery.  ISBN 9781450383509.
  * Togelius et al. (2013) Togelius, J.; Shaker, N.; Karakovskiy, S.; and Yannakakis, G. N. 2013.  The Mario AI Championship 2009-2012.  _AI Magazine_ , 34(3): 89–92.
  * Torres-Reyes and Latifi (2019) Torres-Reyes, N.; and Latifi, S. 2019.  Audio Enhancement and Synthesis using Generative Adversarial Networks: A Survey.  _International Journal of Computer Applications_ , 182: 27–31.
  * Volz et al. (2018) Volz, V.; Schrum, J.; Liu, J.; Lucas, S. M.; Smith, A. M.; and Risi, S. 2018.  Evolving Mario Levels in the Latent Space of a Deep Convolutional Generative Adversarial Network.  In _Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2018)_. New York, NY, USA: ACM.
  * Xia et al. (2020) Xia, B.; Bai, Y.; Yin, J.; Li, Y.; and Xu, J. 2020.  LogGAN: a Log-level Generative Adversarial Network for Anomaly Detection using Permutation Event Modeling.  _Information Systems Frontiers_ , 23(2): 285–298.
  * Xu et al. (2015) Xu, B.; Wang, N.; Chen, T.; and Li, M. 2015.  Empirical Evaluation of Rectified Activations in Convolutional Network.  _CoRR_ , abs/1505.00853.