# Communication Topology Co-Design in Graph Recurrent Neural Network Based
Distributed Control

Fengjun Yang† and Nikolai Matni∗ †F. Yang is with the Dept. of Computer and
Information Sciences, University of Pennsylvania, Philadelphia, PA. ∗N. Matni
is with the Dept. of Electrical and Systems Engineering, University of
Pennsylvania, Philadelphia, PA. N. Matni is generously supported by NSF award
CPS-2038873 and NSF CAREER award ECCS-2045834.

###### Abstract

When designing large-scale distributed controllers, the information sharing
constraints between sub-controllers, as defined by a communication topology
interconnecting them, are as important as the controller itself. Controllers
implemented using dense topologies typically outperform those implemented
using sparse topologies, but it is also desirable to minimize the cost of
controller deployment. Motivated by the above, we introduce a compact but
expressive graph recurrent neural network (GRNN) parameterization of
distributed controllers that is well suited for distributed controller and
communication topology co-design. Our proposed parameterization enjoys a local
and distributed architecture, similar to previous Graph Neural Network
(GNN)-based parameterizations, while further naturally allowing for joint
optimization of the distributed controller and communication topology needed
to implement it. We show that the distributed controller/communication
topology co-design task can be posed as an $\ell_{1}$-regularized empirical
risk minimization problem that can be efficiently solved using stochastic
gradient methods. We run extensive simulations to study the performance of
GRNN-based distributed controllers and show that (a) they achieve performance
comparable to GNN-based controllers while having fewer free parameters, and
(b) our method allows for performance/communication density tradeoff curves to
be efficiently approximated.

##  1 Introduction

Future large-scale autonomous systems, such as intelligent transportation,
power, and communication networks, will be composed of dynamically
interconnected subsystems and data-driven decision makers. When designing the
distributed control policies for such systems, the controller architectures,
as defined by the actuation, sensing, and communication topologies of the
controller, can no longer be taken for granted, as controllers with denser
architectures will typically outperform controllers implemented using less
resources. However, it is also desirable to minimize the cost of deploying
controllers, leading to a non-trivial tradeoff between controller performance
and architectural complexity.

A rich body of work exists addressing the controller architecture co-design
problem in the context of (distributed) linear optimal control. When no
communication constraints are imposed on the controller, i.e., when the
control problem is centralized, it has been shown that several metrics defined
in terms of the system controllability/observability Gramians are
(sub)modular. This allows for efficient algorithms to be deployed for
actuator/sensor selection that enjoy approximation guarantees, see for example
Pequito et al. (2015); Tzoumas et al. (2016); Summers and Lygeros (2014);
Summers et al. (2015) and references therein. When communication constraints
are present, augmenting the distributed control problem with a suitable
regularizer to encourage simplicity in the controller architecture has proven
to be successful. For example, $\ell_{1}$ regularization is used to co-design
sparse static state feedback gains in Lin et al. (2013) and synchronization
topologies in Fardad et al. (2014); group norm penalties are used to co-design
actuation/sensing schemes in Matni and Chandrasekaran (2016); Dhingra et al.
(2014); and a specialized atomic norm (Chandrasekaran et al., 2012) is used to
design communication delay constraints that are well-suited to
$\mathcal{H}_{2}$ distributed optimal control in Matni (2015).

In this paper, we assume that the actuation and sensing architecture is fixed,
and focus on co-designing the communication topology of the distributed
controller. In particular, we model a distributed controller as a collection
of sub-controllers, each equipped with a fixed set of actuators and sensors,
that exchange their respective measurements with each other subject to
communication delays imposed by the to-be-designed underlying communication
topology. In general, one can select communication topologies that range in
density from completely decentralized, i.e., that allow no communication
between sub-controllers, to “virtually centralized,” i.e, that allow
instantaneous communication between all sub-controllers.

However, there are now well understood limits on the simplicity of the
designed communication topology (Rotkowitz and Lall, 2005; Rotkowitz et al.,
2010; Lessard and Lall, 2011) if we wish for the resulting distributed control
synthesis task to be convex. In particular a sufficient, and under mild
assumptions necessary, condition for a distributed linear optimal controller
to be specified by the solution to a convex optimization problem111For a more
detailed overview of the relationship between information exchange constraints
and the convexity of distributed optimal control problems, we refer the reader
to Bamieh and Voulgaris (2005); Mahajan et al. (2012) and the references
therein. is that the communication delays between sub-controllers allow for
information to be exchanged between them at least as quickly as their control
actions propagate through the plant (Rotkowitz et al., 2010). This has limited
the subset of the Pareto frontier that could be explored in previous work to
communication topologies that are at least as dense as the topology of the
underlying plant (see for example, the base QI communication graph in Matni
(2015, §4.2)).

Motivated by these limitations on tractability and topology sparsity, this
paper proposes an alternative approach. We draw upon the recent successes of
_graph neural network_ (GNN)-based controllers in the context of distributed
control (Tolstaya et al., 2020; Gama et al., 2020a; Khan et al., 2020; Gama
and Sojoudi, 2021), and propose a novel distributed controller and
communication topology co-design procedure. We propose a compact _graph
recurrent neural network_ (GRNN) parameterization for distributed controllers,
and show that the control parameters and communication topology, as encoded by
the _graph shift operator_ defining the GRNN-based distributed controller, can
be jointly optimized via an $\ell_{1}$-regularized empirical risk minimization
problem. Finally, through extensive empirical evaluations, we show that (a)
distributed controllers based on our GRNN parameterization can achieve
performance comparable to previously proposed GNN-based parameterizations
while having fewer free parameters, and (b) our method allows for
performance/communication density tradeoff curves to be efficiently
approximated, even in the extremely sparse communication topology regime.

##  2 Preliminaries

###  2.1 Distributed Optimal Control

Consider a system composed of $N$ subsystems, with the $i$-th subsystem
described by a state vector $x_{i}(t)\in\mathbb{R}^{p}$ and a control input
$u_{i}(t)\in\mathbb{R}^{q}$. We model inter-subsystem interactions through a
time-invariant unweighted digraph
$\mathcal{G}_{dyn}=(\mathcal{V},\mathcal{E}_{dyn})$. Each element in the
vertex set $\mathcal{V}=\\{v_{1},v_{2},...,v_{N}\\}$ corresponds to a
subsystem. The edge set $\mathcal{E}_{dyn}\subset\mathcal{V}\times\mathcal{V}$
contains an element $(i,j)\in\mathcal{E}_{dyn}$ only when subsystem $i$
directly influences the states of subsystem $j$ through its states or control
inputs. We assume linear time-invariant dynamics, allowing us to write the
dynamics at subsystem $i$ as

$x_{i}(t+1)=\sum_{j:(j,i)\in\mathcal{E}_{dyn}}A_{ij}x_{j}(t)+B_{ij}u_{j}(t),$
(2.1)

for suitable matrices describing local interactions between subsystems.
Equivalently, the dynamics of the full system can be expressed in terms of the
joint state $x(t):=(x_{i}(t))_{i=1}^{N}$ and control
$u(t):=(u_{i}(t))_{i=1}^{N}$ vectors as

$x(t+1)=Ax(t)+Bu(t),$ (2.2)

where matrices $A$ and $B$ are constructed such that the full dynamics (2.2)
are consistent with the subsystem dynamics in (2.1). Note that the full system
matrices $(A,B)$ are sparse when the subsystems are sparsely interconnected,
i.e., when the graph $\mathcal{G}_{dyn}$ is sparse.

Our goal is to efficiently explore the space of distributed controllers
composed of sub-controllers that have access to a combination of instantaneous
local information and delayed global information. Formally, the sub-controller
for subsystem $i$ at time $t$ can directly observe a local information set
$\mathcal{J}_{i,t}$, which might fail to capture the full state of system
(2.2). For example, $\mathcal{J}_{i,t}$ might only include the state of
subsystem $i$, i.e. $x_{i}(t)$, but not those of other subsystems. We assume
that local information can be exchanged between subsystems, subject to a delay
defined by a communication topology, modeled as a time-invariant unweighted
digraph $\mathcal{G}_{c}=(\mathcal{V},\mathcal{E}_{c})$ (note that the
communication topology $\mathcal{G}_{c}$ need not equal the interaction
topology $\mathcal{G}_{dyn}$). If $(i,j)\in\mathcal{E}_{c}$, subsystem $i$ can
share its local information set $\mathcal{J}_{i,t}$ with subsystem $j$, with a
$1$-step delay, i.e., subsystem $j$ gains access to $\mathcal{J}_{i,t}$ at
time $t+1$. Subsystem $j$ can then relay this information to its neighbors in
$\mathcal{G}_{c}$ with a further time delay.

Denoting the directed distance from vertex $i$ to vertex $j$ in a digraph
$\mathcal{G}$ as $\mathbf{dist}^{\mathcal{G}}(i\rightarrow j)$, and vertex
$i$’s $d$-hop incoming neighbors as
$\mathbf{in}^{\mathcal{G}}_{i}(d):=\left\\{v_{j}\;|\;\mathbf{dist}^{\mathcal{G}}(v_{j}\rightarrow
v_{i})\leqslant d\in\mathbb{N}\right\\}$, the information sharing constraints
imposed by the communication topology $\mathcal{G}_{c}$ mean that at time $t$,
the total information available to subsystem $i$ is given by

$\mathcal{I}_{i,t}(\mathcal{G}_{c}):=\underbrace{\mathcal{J}_{i,t}}_{\textrm{local
information}}\cup\underbrace{\left(\bigcup_{d=1}^{t}\mathcal{J}_{j\in\mathbf{in}^{\mathcal{G}_{c}}_{i}(d),t-d}\right)}_{\textrm{delayed
global information}}.$ (2.3)

This then restricts the action taken by subsystem $i$ at time $t$ to be of the
form

$u_{i}(t)=\gamma_{i,t}\left(\mathcal{I}_{i,t}(\mathcal{G}_{c})\right),$ (2.4)

for $\gamma_{i,t}$ a suitable map from the total information set
$\mathcal{I}_{i,t}(\mathcal{G}_{c})$ to control actions. This formulation also
makes clear that by varying the density of the communication topology defined
by $\mathcal{G}_{c}$, we can vary the total amount of information available to
each subsystem $i$.

In the following, we summarize recent results showing that GNNs offer a
natural means of parameterizing such controllers, and introduce our proposed
parameterization for control and communication delay co-design.

###  2.2 Graph Neural Networks for Distributed Control

GNNs are a class of neural networks, built around the notion of a graph
filter, that are well-suited for processing and representing data with graph
structure. They enjoy many additional desirable properties, such as being
permutation equivariant and Lipschitz continuous to changes in the network
(Gama et al., 2020b; Gama and Sojoudi, 2021), making them natural candidates
for distributed controllers. Following conventions in the GNN literature, we
rewrite the joint state $x(t)\in\mathbb{R}^{Np}$ and control vectors
$u(t)\in\mathbb{R}^{Nq}$ as

${\mathbf{X}}(t)=\begin{bmatrix}x_{1}^{T}(t)\\\ \vdots\\\
x_{N}^{T}(t)\end{bmatrix}\in\mathbb{R}^{N\times
p},{\mathbf{U}}(t)=\begin{bmatrix}u_{1}^{T}(t)\\\ \vdots\\\
u_{N}^{T}(t)\end{bmatrix}\in\mathbb{R}^{N\times q},$ (2.5)

as this representation more naturally allows for the required graph
convolutions to be defined. In what follows, we tailor our discussion to using
GNNs in the context of distributed control.

####  2.2.1 Graph Convolutions

The notation in (2.5) allows one to express linear information exchanges as a
matrix multiplication while enforcing the communication topology as a
constraint on the multiplicand. Specifically, consider a matrix
$S\in\mathbb{R}^{N\times N}$ which satisfies the property that $S_{ij}\neq 0$
only if $(j,i)\in\mathcal{E}_{c}$, and define the matrix
$U\in\mathbb{R}^{N\times p}$ as $U=S{\mathbf{X}}$. Then,

$U_{ij}=\sum_{k=1}^{N}S_{ik}{\mathbf{X}}_{kj}=\sum_{k:(k,i)\in\mathcal{E}_{c}}S_{ik}{\mathbf{X}}_{kj}=\sum_{k:v_{k}\in\mathbf{in}^{\mathcal{G}_{c}}_{i}(1)}S_{ik}{\mathbf{X}}_{kj},$
(2.6)

where the first equality follows from the definition of matrix multiplication,
the second equality follows the definition of nonzero elements of $S$, and the
third from the definition of the vertex $i$’s $1$-hop incoming neighbors. This
latter definition emphasizes that the $i$-th row of $U$ can be computed by
subsystem $i$ using information received from $1$-hop incoming neighbors in
$\mathcal{G}_{c}$, i.e., only the ${\mathbf{X}}_{kj}$ in (2.6) such that
$v_{k}\in\mathbf{in}^{\mathcal{G}_{c}}_{i}(1)$ are needed.

In general, left multiplying a joint state or input vector by the matrix $S$
corresponds to communication and aggregation of data between $1$-hop
neighbors, as defined by the communication topology $\mathcal{G}_{c}$. By
associating a unit delay with such a communication and aggregation step, we
can model the _spatiotemporal_ propagation of information across a graph,
making this operator a natural tool for encoding information sharing
constraints in a distributed control setting. We note that such an operation
is analogous to the time-shift operator in classical signal processing, and is
thus referred to as the graph shift operator. Examples of valid graph shift
operators on a graph $\mathcal{G}$ include its adjacency matrix and its
Laplacian. For the rest of the paper, we denote the set of valid graph shift
operators on graph $\mathcal{G}$ as
$\mathcal{S}_{\mathcal{G}}\subset\mathbb{R}^{N\times N}$.

Graph convolution extends the graph shift operation by aggregating information
over multiple time steps. Consider a graph $\mathcal{G}$ and a corresponding
graph shift operator $S\in\mathcal{S}_{\mathcal{G}}$. We can define a _linear
distributed controller_ ${\mathbf{U}}(t)$ in terms of the graph convolution of
the joint state ${\mathbf{X}}(t)$ over $K$ steps specified by the set of
weights $H=\\{H_{k}\\}_{k=0}^{K-1},H_{k}\in\mathbb{R}^{p\times q}$ as

${\mathbf{U}}(t)=H_{\mathcal{G}}\circledast{\mathbf{X}}(t)=\sum_{k=0}^{K-1}S^{k}{\mathbf{X}}(t-k)H_{k},$
(2.7)

where each row of resulting control signal
${\mathbf{U}}(t)\in\mathbb{R}^{N\times q}$ can be computed at subsystem $i$
using $k$-step delayed information received from subsystems $j$ such that
$\mathbf{dist}_{\mathcal{G}}(j\rightarrow i)=k\leqslant K$. To see this, we
first note that, by the properties of graph shift operators, $S^{k}_{ij}\neq
0$ only if $v_{j}\in\mathbf{in}_{i}^{\mathcal{G}}(k)$, i.e., node $j$ is a
$k$-hop incoming neighbor of node $i$ in $\mathcal{G}$. Thus, for each
$k\in\\{0,...,K-1\\}$ in the sum, the $i$-th row of ${\mathbf{U}}(t)$
aggregates $k$-step delayed information from $k$-hop incoming neighbors of
subsystem $i$, with information set given by
$\mathcal{J}_{i,t}=\\{x_{i}(t)\\}$. We therefore have that the total
information set needed to compute the $i$-th row of ${\mathbf{U}}(t)$ is given
by

$\displaystyle\mathcal{I}_{i,t}(\mathcal{G}_{c})$
$\displaystyle=\\{x_{i}(t)\\}\cup\left(\bigcup_{k=1}^{K-1}\\{x_{j}(t-k)\,|\,j\in\mathbf{in}^{\mathcal{G}_{c}}_{i}(k)\\}\right),$

i.e., the collection of $k$-step delayed states from $k$-hop neighbors, for
$k=0,\dots,K-1$. We highlight that the graph convolution encodes both spatial
and temporal structure, as captured by the propagation of signals by repeated
application of the graph shift and time delay operator within the convolution,
making it well-suited for parameterizing controllers subject to information
sharing constraints as in (2.4). Finally, we note that a filter $H_{k}$ is
applied uniformly across all rows of its multiplicand. Thus, the result from
right multiplying $H_{k}$ to the aggregated signal $S^{k}{\mathbf{X}}(t-k)$
can be computed locally for any matrix $H_{k}$.

####  2.2.2 Graph Neural Networks

A graph convolution as defined above can be viewed as a two-step process:
first, a set of delayed information is aggregated at each node using the graph
shift operator $S$; then, the aggregated information is used to synthesize an
output by using the filter weights $\\{H_{k}\\}$. GNNs (Bruna et al., 2013;
Kipf and Welling, 2016; Atwood and Towsley, 2015; Gama et al., 2018) seek to
expand the expressiveness of graph convolutions by repeatedly applying one or
both steps in combination with nonlinearities.

One architecture that achieves this is delayed aggregation GNNs (AGNNs) (Gama
et al., 2018; Tolstaya et al., 2020). Informally, an AGNN first aggregates
information through a graph convolution over the states. The output is then
fed through a standard convolution neural network (CNN). As the CNN is acting
on signals synthesized from locally available information, such an
architecture naturally encodes the information sharing specified by the graph
shift operator of the graph convolution layer. In the interest of space, we
omit a detailed description of AGNNs, and instead refer the reader to Gama et
al. (2020a) for a more detailed overview of their use in the context of
distributed control.

When information exchange constraints are defined only in terms of spatial
constraints, and not delays, one can repeated apply both the aggregation and
the convolution steps to parameterize the corresponding distributed controller
using a graph convolution neural network (GCNN). A GCNN controller with $L$
layers can be written recursively as

$\displaystyle{\mathbf{U}}(t)$ $\displaystyle=X_{L}$ (2.8) $\displaystyle
X_{l}$
$\displaystyle=\sigma\left(\sum_{k=0}^{K_{l}}S^{k}X_{l-1}H_{l,k}\right),\quad
l=1,...,L$ $\displaystyle X_{0}$ $\displaystyle={\mathbf{X}}(t),$

where $\sigma$ is applied element-wise and $H_{l,K}$ is a set of trainable
parameters. Note that while the application of the graph shift operator
maintains locality of information exchange, the sub-controller at subsystem
$i$ will need to instantaneously collect the local information
$\mathcal{J}_{j,t}$ from all neighboring subystems $j$ satisfying
$\mathbf{dist}^{\mathcal{G}_{c}}(i,j)\leqslant\sum_{l=0}^{L}K_{l}$ – thus one
must be careful to balance the depth $L$ of the network, the density of the
graph shift operator $S$, and the aggregation horizon $K_{l}$, to ensure that
locality is indeed preserved. The size of parameter matrices
$H_{l,k}\in\mathbb{R}^{r_{l}\times c_{l}}$ can be arbitrary as long as
internal state dimensions are consistent, i.e., $r_{l}=c_{l-1}$ and
$r_{0}=p,c_{L}=q$. The total number of parameters of a GNN is thus a tunable
hyperparameter totaling $\sum_{l=0}^{L}(K_{l}+1)r_{l}c_{l}$. We end by noting
that distributed controllers parameterized as GCNNs will outperform
comparable222By comparable, we mean with comparably chosen hyper-parameters
such that the overall number of free parameters of the function classes are
approximately the same. delayed AGNNs defined in terms of the same graph shift
operator $S$, because the former does not enforce delay constraints on
information sharing, whereas the latter does. As such, we use GCNN-based
controllers as a baseline for comparison in our experiments.

####  2.2.3 Graph Recurrent Neural Networks

For the problem of communication topology co-design, we propose parameterizing
distributed controllers using GRNNs – GRNNs have been successfully used for
decentralized control in the context of imitation learning in Gama et al.
(2020a). GRNNs extend the aforementioned GNN-based controller
parameterizations by introducing a local hidden state
$z_{i}(t)\in\mathbb{R}^{h}$ at each sub-controller, resulting in a _dynamic_
distributed controller. We further allow sub-controllers to communicate both
their local state and internal state with neighboring subsystems, i.e., in
this parameterization, the local information set of subsystem $i$ is given by
$\mathcal{J}_{i,t}:=\\{x_{i}(t),z_{i}(t)\\}$. Let
${\mathbf{Z}}(t):=(z_{i}(t)^{\top})_{i=1}^{N}\in\mathbb{R}^{N\times h}$ denote
the full internal state obtained by stacking the subsystem internal states as
in equation (2.5). While more general forms of GRNNs exist and can be used, we
propose using the following compact GRNN parameterization, where an update
step can be computed as

$\displaystyle{\mathbf{Z}}(t)$
$\displaystyle=\sigma(S\;{\mathbf{Z}}(t-1)\;W+{\mathbf{X}}(t)\;F),$ (2.9)
$\displaystyle{\mathbf{U}}(t)$ $\displaystyle={\mathbf{Z}}(t)\;G,$

where $F\in\mathbb{R}^{p\times h},G\in\mathbb{R}^{h\times
q},W\in\mathbb{R}^{h\times h}$ are trainable parameters.

We selected this architecture because it enjoys certain properties that make
it naturally suited for encoding and optimizing communication delay structure.
At each time step, the update step (2.9) applies the graph shift operator $S$
once on the internal states, which ensures that information within the
controller is consistent with information flow across the communication
network. Thus a controller parameterized by equation (2.9) satisfies the
information sharing constraints defined in (2.3) and (2.4) with the
information set $\mathcal{J}_{i,t}=\\{x_{i}(t),z_{i}(t)\\}$.333We note that
$x_{i}(t)$ is in fact only ever needed at subsystem $i$, but we include it in
$\mathcal{J}_{i,t}$ to be consistent with equations (2.3) and (2.4). Further,
the parameterization (2.9) is an affine mapping of the graph shift operator
$S$ composed with a point-wise nonlinearity $\sigma$. The simplicity with
which $S$ appears in parameterization (2.9) makes it well suited for treating
$S$ as an optimization variable. In particular, whereas existing GNN-based
distributed controllers (Tolstaya et al., 2020; Gama et al., 2020a; Khan et
al., 2020; Gama and Sojoudi, 2021) assume a _fixed_ graph shift operator $S$,
we take a slight departure and _jointly optimize_ the GRNN-based distributed
controller parameter values and graph shift operator sparsity pattern and
parameter values. Doing so allows us to co-design a distributed controller and
the communication graph topology $\mathcal{G}_{c}$ needed to implement it.

##  3 Co-Design via Regularized Empirical Risk Minimization

Given a distributed linear system (2.1), our goal is to explore the design
space of distributed controllers satisfying communication constraints of the
form (2.4) while minimizing a cost function over a finite horizon
$T\in\mathbb{Z}^{+}$. We consider two problem formulations. First, we show how
to solve the distributed controller design problem on a _given_ communication
network $\mathcal{G}_{c}$ using GRNN with an empirical risk minimization (ERM)
approach akin to that proposed in Gama and Sojoudi (2021). Next, we build on
this formulation and show how to jointly design a distributed controller and
the communication topology $\mathcal{G}_{c}$ needed to implement it through
the use of $\ell_{1}$-regularized ERM.

###  3.1 Distributed GRNN Control Design: Given Topology

We consider the linear system (2.2), which can be equivalently rewritten as

$\textrm{vec}({\mathbf{X}}(t+1))=A\;\textrm{vec}({\mathbf{X}}(t))+B\;\textrm{vec}({\mathbf{U}}(t)).$
(3.1)

Let $\mathcal{X}\subseteq\mathbb{R}^{N\times p}$ be a set, $\mathcal{D}$ be a
distribution over $\mathcal{X}$, and the initial conditions of our system be
distributed as ${\mathbf{X}}(0)\sim{}\mathcal{D}$. Suppose the quality of a
system trajectory is characterized by the cost function
$J\left(\\{{\mathbf{X}}(t)\\}_{t=0}^{T},\\{{\mathbf{U}}(t)\\}_{t=0}^{T}\right)$.
We can then pose our optimal control problem as

$\begin{array}[]{rl}\mathrm{minimize}_{\\{\gamma_{i,t}\\}}&\mathbb{E}_{{\mathbf{X}}(0)\sim\mathcal{D}}\left[J\left(\\{{\mathbf{X}}(t)\\}_{t=0}^{T},\\{{\mathbf{U}}(t)\\}_{t=0}^{T}\right)\right]\\\
\textrm{s.t.}&\text{dynamics \eqref{eq: stacked linear dynamics}},\\\
&u_{i}(t)=\gamma_{i,t}(\mathcal{I}_{i,t}(\mathcal{G}_{c})),\;i=1,...,N,\,t=0,...,T\end{array}$
(3.2)

In general, this problem is non-convex even if the cost function is quadratic
and the distribution $\mathcal{D}$ is Gaussian (corresponding to LQG control)
due to the communication constraints (2.3) and (2.4) (encoded as
$u_{i}(t)=\gamma_{i,t}(\mathcal{I}_{i,t}(\mathcal{G}_{c}))$). We thus aim to
approximate the optimal solution to problem (3.2) by using the GRNN
parameterization (2.9) for distributed controllers, and leveraging ERM.

As presented in Section 2.2.3, GRNN models can encode the communication
constraints (2.3) and (2.4) induced by topology $\mathcal{G}_{c}$ by
restricting the graph shift operator to satisfy
$S\in\mathcal{S}_{\mathcal{G}_{c}},$ i.e., by enforcing that $S$ be a valid
graph shift operator for the communication topology $\mathcal{G}_{c}$. In
practice, we enforce this constraint by restricting the support of $S$ to be
consistent with that of the adjacency matrix of the graph $\mathcal{G}_{c}$.
Thus, we draw initial conditions
$\\{{\mathbf{X}}_{p}(0)\\}_{p=1}^{k}\sim{}\mathcal{D}^{k}$ and pose the
distributed GRNN control design problem as the following ERM problem:

$\begin{array}[]{rl}\underset{S,F,G,W}{\mathrm{minimize}}&\frac{1}{k}\sum_{p=0}^{k}\left[J\left(\\{{\mathbf{X}}_{p}(t)\\}_{t=0}^{T},\\{{\mathbf{U}}_{p}(t)\\}_{t=0}^{T}\right)\right]\\\
\textrm{s.t.}&\text{dynamics (\ref{eq: stacked linear dynamics}),}\\\
&{\mathbf{Z}}_{p}(t)=\sigma(S\;{\mathbf{Z}}_{p}(t-1)\;W+{\mathbf{X}}_{p}(t)\;F),\\\
&{\mathbf{U}}_{p}(t)={\mathbf{Z}}_{p}(t)\;G,\,p=0,...,k,\\\
&S\in\mathcal{S}_{\mathcal{G}_{c}},\,(S,F,G,W)\in\Theta,\end{array}$ (3.3)

where the set $\Theta$ is a user-specified set constraining the overall
expressivity of the model to prevent over-fitting. A typical example would to
impose Frobenius norm bounds on the parameters, i.e.,
$\Theta=\\{(S,F,G,W)\,:\,\|S\|_{F}^{2}\leqslant B_{S},\|F\|_{F}^{2}\leqslant
B_{F},\|G\|_{F}^{2}\leqslant B_{G},\|W\|_{F}^{2}\leqslant B_{W}\\}.$

Following Gama and Sojoudi (2021), this problem can be approximately solved in
a self-supervised manner using stochastic gradient descent. During training,
we simulate the initial conditions forward with the controllers parameterized
with the current parameter iterates $\\{\hat{S},\hat{F},\hat{G},\hat{W}\\}$,
and compute their cost. The gradient of the cost with respect to the
parameters can then be found efficiently via back propagation through time
(Goodfellow et al., 2016, §10.2.2). We note that in contrast to prior work
(Tolstaya et al., 2020; Gama et al., 2020a; Khan et al., 2020; Gama and
Sojoudi, 2021), we propose optimizing the parameters of the graph shift
operator $S$ in addition to the parameters $\\{W,F,G\\}$.

###  3.2 Communication Topology Co-Design

Unlike the previous setting, here we assume that the topology
$\mathcal{G}_{c}$ is also to be designed. In particular, we aim to develop a
methodology that allows us to efficiently explore the tradeoff space between
communication complexity, as measured by the cardinality of the edge set
$|\mathcal{E}_{c}|$, and controller performance. To do so, we propose a two
step process.

First, we co-design the communication topology $\mathcal{G}_{c}$ by solving
the $\ell_{1}$-regularized ERM problem

$\begin{array}[]{rl}\underset{S,F,G,W}{\mathrm{minimize}}&\frac{1}{k}\sum\limits_{p=0}^{k}\left[J\left(\\{{\mathbf{X}}_{p}(t)\\}_{t=0}^{T},\\{{\mathbf{U}}_{p}(t)\\}_{t=0}^{T}\right)\right]+\lambda\|S\|_{1}\\\
\textrm{s.t.}&\text{dynamics (\ref{eq: stacked linear dynamics}),}\\\
&{\mathbf{Z}}_{p}(t)=\sigma(S\;{\mathbf{Z}}_{p}(t-1)\;W+{\mathbf{X}}_{p}(t)\;F),\\\
&{\mathbf{U}}_{p}(t)={\mathbf{Z}}_{p}(t)\;G,\,p=0,...,k,\\\
&(S,F,G,W)\in\Theta.\end{array}$ (3.4)

Note that in contrast to optimization problem (3.3), we have dropped the
constraint that the graph shift operator satisfy
$S\in\mathcal{S}_{\mathcal{G}_{c}}$, and instead added an $\ell_{1}$
regularizer to the objective function, which is known to promote sparsity
(Tibshirani, 1996; Donoho, 2006). Thus, by varying the regularization weight
$\lambda$, we can trade off sparsity of the network against controller
performance.

Denote by $\hat{S}(\lambda)$ the solution to optimization problem (3.4) for a
given value of $\lambda$, and let
$\hat{A}(\lambda)=\mathrm{supp}(\hat{S}(\lambda))$ be the corresponding
adjacency matrix defined by its support.444Note that in practice, we apply a
thresholding procedure to determine adjacency matrix, i.e., $\hat{A}_{ij}=1$
if an only if $|\hat{S}_{ij}|\geqslant\varepsilon$, for some small numerical
threshold $\varepsilon$. The designed graph $\hat{\mathcal{G}}(\lambda)$ is
then taken to be the graph induced by the adjacency matrix $\hat{A}(\lambda)$.
We then proceed with a standard _refinement step_ , and solve optimization
problem (3.3) subject to information sharing constraints imposed by the
designed communication topology $\hat{\mathcal{G}}(\lambda)$.

##  4 Numerical Experiments

All code needed to reproduce the examples found in this section will be made
at available at the following repository: https://github.com/unstable-
zeros/grnn-comms-codesign.

We consider the distributed linear quadratic regulator (LQR) problem over
$N=20$ subsystems and a finite time horizon $T=50$. Specifically, we consider
the case where the state and control actions of the subsystems are scalars,
i.e., $p=q=1$. Under this assumption,
${\mathbf{X}}(t)=x(t),{\mathbf{U}}(t)=u(t)$, and the problem can be written as

$\begin{array}[]{rl}\mathrm{minimize}&\mathbb{E}_{x(0)\sim{}\mathbb{N}(0,I)}\left[\sum_{t=0}^{T-1}x(t)^{\intercal}Qx(t)+u(t)^{\intercal}Ru(t)+x^{\intercal}(T)Px(T)\right],\\\
\textrm{s.t.}&x(t+1)=Ax(t)+Bu(t),\;t=0,1,...,T-1,\\\
&u_{i}(t)=\gamma_{i,t}(\mathcal{I}_{i,t}(\mathcal{G}_{c}))\\\
&\quad\quad\text{for $i=1,\dots,N,\,t=0,\dots,T$},\end{array}$ (4.1)

for dynamics matrices $A,B\in\mathbb{R}^{N\times N}$, $N\times N$ symmetric
cost matrices $Q\succcurlyeq 0,R\succ 0,P\succcurlyeq 0$, and communication
topology $\mathcal{G}_{c}$.

Generating Problem Instances: An instance of the LQR problem (4.1) is fully
defined by the tuple $(\mathcal{G}_{c},A,B,Q,R,P)$. We generate problem
instances using the same process for all experiments. For each instance, we
start by creating a communication topology $\mathcal{G}_{c}$ by randomly
sampling $N$ numbers $\\{u_{i}\\}_{i=1}^{N}\sim{}U[0,1]$, and creating a bi-
directional link between $v_{i}$ and each of its $5$ nearest points as defined
by the topology on the interval $[0,1]$ under the metric
$d(v_{i},v_{j})=|u_{i}-u_{j}|$. Following Gama and Sojoudi (2021), we generate
the dynamics matrices $A$ and $B$ to share the same eigenvectors as the
normalized adjacency matrix of $\mathcal{G}_{c}$, and sample their eigenvalues
i.i.d. from the standard normal distribution – hence both $A$ and $B$ are
symmetric matrices. Then, to induce further sparsity in the dynamics, we set
entries $A_{ij}=0$ and $B_{ij}=0$ whenever
$\mathbf{dist}^{\mathcal{G}_{c}}(i\rightarrow j)>3$. Finally, we normalize the
matrices $A$ and $B$ to have the prescribed norms for the experiments being
run. We take the cost matrices $Q=R=I_{N}$. Finally, the terminal state cost
matrix $P$ is taken to be the solution to the Discrete Algebraic Riccati
Equation (DARE)

$A^{\intercal}PA-P-A^{\intercal}PB(B^{\intercal}PB+R)^{-1}B^{\intercal}PA=-Q.$
(4.2)

This ensures that the centralized solution is stabilizing. We end by noting
that communication topologies $\mathcal{G}_{c}$ generated this way have, on
average, $141$ directed edges (out of a possible 400).

In all of the following experiments, we use the ADAM algorithm (Kingma and Ba,
2014) to approximately solve the resulting ERM problems with a batch size of
$20$, and forgetting factors of $0.9$ and $0.999$. For the GRNN models, we
choose the learning rate to be $0.02$, and apply cosine annealing on top of
ADAM for learning rate scheduling. For the GCNN models, we use a learning rate
of $0.01$, and decay the learning rate by a factor of $0.9$ every $10$
batches, as specified in Gama et al. (2020a). The initial conditions in each
batch are sampled i.i.d. from $\mathbb{N}(0,I_{N})$. We train the models with
$750$ batches of initial conditions.

###  4.1 Performance Benchmarks

In this set of experiments, we compare the performance of different variants
of the GRNN parameterization (2.9) with a GCNN benchmark from Gama and Sojoudi
(2021) on the LQR problem. We note that we focus exclusively on GNN based
controllers in our study, as Gama and Sojoudi (2021) has already conducted
extensive studies comparing GNN-based distributed controllers against more
traditional (fully-connected) neural network parameterizations of distributed
controllers.

Architectures: We consider four different variants of the GRNN-based
controller (2.9). All the GRNN variants considered share the same architecture
with a hidden state dimension of $5$. However, they differ in their training
objective functions and constraints.

  * •

(GRNN) approximately solves the LQR problem (4.1) using the ERM problem (3.3).

  * •

(GRNN-Dense) does not place constraints on the graph shift operator $S$, and
can thus design a centralized controller.

  * •

(GRNN-Sparse) co-designs a communication topology and distributed controller
using (3.4) to identify a topology $\hat{\mathcal{G}}$ and (3.3) restricted to
the identified topology for the subsequent refinement step555We take the
threshold value for determining the adjacency matrix support to be
$\varepsilon=0.004$..

  * •

(GRNN-Fixed) restricts $S$ to be constant and only optimizes over the
parameters $F,G$, and $W$.

  * •

GCNN is the GCNN architecture in Gama and Sojoudi (2021), which we take as a
benchmark. We take the graph shift operator $S$ to be the normalized adjacency
matrix of $\mathcal{G}_{c}$. The GCNN model has $2$ layers with $5$ and $1$
filter taps, respectively and uses a total of $192$ parameters.

Training and evaluation: We run our experiments across $50$ random LQR
instances. We report the costs achieved by each controller normalized with
respect to that achieved by the centralized LQR optimal controller
$K_{LQR}=-(B^{\intercal}PB)^{-1}B^{\intercal}PA$ computed via the solution to
the DARE (4.2). We note that the centralized cost represents a fundamental
lower limit on the achievable performance, and thus a normalized cost of $1$
is optimal.

We run this experiment on $50$ randomly generated pairs of communication
networks $\mathcal{G}_{c}$ and dynamics matrices $(A,B)$. Every $10$ batches,
we evaluate the controllers over a validation set of $100$ initial conditions
and report the average performance. To test the performance of the GRNN method
on both stable and unstable environments, we repeat the process on two
scenarios, one with the norm of the dynamics matrix $\|A\|_{2}=0.995$, the
other with $\|A\|_{2}=1.05$. When training the GRNNs, we also apply weigh-
decay (WD) (Frobenius-norm regularization) on the parameters $(S,F,W,G)$, when
applicable. When training the GCNNs, we experimented with no WD on the filter-
taps, as specified in Gama and Sojoudi (2021), as well as with weight-decay:
we found the results with and without WD to be qualitatively and
quantitatively similar, and present here only those without WD. When solving
the co-design problem (3.4), we set the regularization penalty $\lambda$ to
$1$ and $2$ for $\|A\|_{2}=0.995$ and $\|A\|_{2}=1.05$, respectively: these
values were selected based off of the co-design experiments we describe next.
We report the median relative cost for all methods in Table 1, and plot the
learning curves of the controllers during the training in Figure 1.

Discussion: First, we note that the GRNN parametrization outperforms the the
GCNN benchmark on both the stable and unstable dynamics despite the latter
parameterization _not enforcing communication delay constraints_. This
suggests that the GRNN model can leverage the recurrent structure to aggregate
information without needing a large number of filter taps. We further observe
that all GRNN architectures enjoy reduced variance as compared to the GCNN
architecture666The GCNN architecture with WD has only slightly less variance
at the expense of even worse performance.. Interestingly, we also observe that
the GRNN-Sparse controllers, for which the communication topology is co-
designed, are able to outperform the GRNN models with a prescribed
communication topology that is designed to be well aligned with the system
dynamics (via their eigenspaces). Furthermore, the GRNN-Sparse controllers use
on average, $53.4$ and $56.1$ directed communication links for the stable and
unstable dynamics, respectively, far fewer than the average number of links in
the given communication topology. This indicates that our co-design algorithm
was able to identify key links in the communication topology needed for
control, and further suggests that by restricting the controller
parameterization to a simpler (sparser) model class, less data is required to
learn a high-performing controller. We leave formalizing these observations in
the context of the ERM framework to future work.

Architecture | $\|A\|_{2}=.995$ | $\|A\|_{2}=1.05$
---|---|---
Autonomous | 2.534 | 69.294
GRNN | 1.112 | 1.153
GRNN-Dense | 1.091 | 1.110
GRNN-Sparse | 1.093 | 1.124
GRNN-Fixed | 1.267 | 2.210
GCNN | 1.212 | 1.715
Table 1: Median normalized cost for the perfomrance benchmarking experiments
described in §4-A over $50$ randomly generated problem instances. The GCNN
baseline is that described in Gama and Sojoudi (2021).

Figure 1: Training curve for the respective distributed controller
parameterizations for problem instances with $\|A\|_{2}=0.995$ (Left) and
$\|A\|_{2}=1.05$ (Right). The solid line denotes the median of normalized
costs, and the shaded regions denotes quartiles, over $50$ randomly generated
problem instances.

###  4.2 Controller-Topology Co-design

We show how our approach allows us to efficiently explore the trade-off
between the sparsity of the communication topology of a distributed controller
and the performance that it achieves via our co-design problem (3.4). We run
our GRNN co-design algorithm for four families of LQR instances, each
characterized by the operator norm of the $A$ matrix defining the system
dynamics, with $\|A\|_{2}$ ranging from stable (less than 1) to unstable
(greater than 1). For each value of $\|A\|_{2}$, we solve the co-design
problem (3.4) with regularization weight $\lambda$ varying across several
orders of magnitude, sweeping out a curve $\hat{\mathcal{G}}(\lambda)$ in
communication topology space: a representative example of such a curve is
illustrated in Fig. 3. We record the cost achieved by the co-designed
distributed controller, as well as number of directed links needed to
implemented it as captured by the cardinality of the designed edge set
$|\mathcal{E}_{c}|$ of the communication topology over $30$ realizations for
each value of $\|A\|_{2}$. We plot the resulting trade-off curves in Figure 2.

Discussion: As the results show, there is a clear trade-off between the
sparsity of the communication topology and the performance of the system. We
observe that the trade-off becomes more pronounced as the norm of $A$ grows,
i.e., as the open-loop system becomes more unstable and consequently,
difficult to control. Interestingly, we observe that for all values of
$\|A\|_{2}$, our co-design algorithm is able to explore the entire design
space from dense (virtually centralized) to (nearly) completely decentralized.
We emphasize that such (nearly) completely decentralized communication
topologies do not satisfy the conditions needed for convexity/tractability of
linear distributed optimal control such as quadratic invariance (Rotkowitz and
Lall, 2005; Rotkowitz et al., 2010) or funnel causality (Bamieh and Voulgaris,
2005), and yet reasonably well-performing distributed optimal controllers are
identified. While prior work (Matni and Chandrasekaran, 2016) identifies
conditions under which such co-designed architectures are optimal, we are not
aware of such conditions in the context of data-driven distributed control. We
view our results as strong empirical evidence that such an investigation
should be undertaken.

Figure 2: Trade-off curves of the normalized cost versus communication
topology complexity (characterized via the number of directed edges
$|\hat{\mathcal{E}}|$) for varying values of $\|A\|_{2}$. For each value of
$\|A\|_{2}$, we plot the median values across $30$ randomly generate problem
instances, with horizontal and vertical error bars denoting the quartiles in
number of edges and normalized cost, respectively.

---

(a)

---

(b)

Figure 3: A visualization of the evolution of the designed topology
$\hat{\mathcal{G}}(\lambda)$ as the regularization parameter $\lambda$ is
varied. Under each identified topology, we indicate the regularization
parameter value $\lambda$, the identified edge set cardinality
$|\hat{\mathcal{E}}|$, and the normalized cost achieved by the co-designed
controller $J$. For this example, $\|A\|_{2}=1.1$. The color of the nodes
represent the eigenvector of $A$ associated with the leading eigenvalue
$\lambda=-1.1$, with nodes with large values colored more prominently. As can
be seen, as the graph becomes more sparse, links between these nodes remain.

##  5 Conclusion & Future Work

Jointly designing communication topologies with distributed controllers is
difficult in general. In this work, we proposed a data-driven method to solve
the co-design problem by parametrizing the distributed controllers as a GRNN
and solving an $\ell_{1}$-regularized ERM problem that allows for a principled
tradeoff between control cost and communication topology complexity. We
demonstrated empirically that our proposed GRNN parametrization can generate
distributed controllers with good performance and our method allows us to
efficiently explore the trade-off space between the sparsity of the designed
communication network and the performance of the distributed controllers.
Future work will look to provide generalization bounds on the learned
controllers by leveraging recent results that allow for uniform convergence
guarantees to be applied to dynamical systems under suitable stability
assumptions (Tu et al., 2021) along with quadratic constraint based conditions
for the stability of dynamical systems modelled as recurrent neural networks
(Revay et al., 2020).

## References

  * Pequito et al. (2015) Sérgio Pequito, Soummya Kar, and George J Pappas.  Minimum cost constrained input-output and control configuration co-design problem: A structural systems approach.  In _2015 American control conference (ACC)_ , pages 4099–4105. IEEE, 2015.
  * Tzoumas et al. (2016) Vasileios Tzoumas, Ali Jadbabaie, and George J Pappas.  Sensor placement for optimal kalman filtering: Fundamental limits, submodularity, and algorithms.  In _2016 American Control Conference (ACC)_ , pages 191–196. IEEE, 2016.
  * Summers and Lygeros (2014) Tyler H Summers and John Lygeros.  Optimal sensor and actuator placement in complex dynamical networks.  _IFAC Proceedings Volumes_ , 47(3):3784–3789, 2014.
  * Summers et al. (2015) Tyler H Summers, Fabrizio L Cortesi, and John Lygeros.  On submodularity and controllability in complex dynamical networks.  _IEEE Transactions on Control of Network Systems_ , 3(1):91–101, 2015.
  * Lin et al. (2013) Fu Lin, Makan Fardad, and Mihailo R Jovanović.  Design of optimal sparse feedback gains via the alternating direction method of multipliers.  _IEEE Transactions on Automatic Control_ , 58(9):2426–2431, 2013.
  * Fardad et al. (2014) Makan Fardad, Fu Lin, and Mihailo R Jovanović.  Design of optimal sparse interconnection graphs for synchronization of oscillator networks.  _IEEE Transactions on Automatic Control_ , 59(9):2457–2462, 2014.
  * Matni and Chandrasekaran (2016) Nikolai Matni and Venkat Chandrasekaran.  Regularization for design.  _IEEE Transactions on Automatic Control_ , 61(12):3991–4006, 2016.
  * Dhingra et al. (2014) Neil K Dhingra, Mihailo R Jovanović, and Zhi-Quan Luo.  An admm algorithm for optimal sensor and actuator selection.  In _53rd IEEE Conference on Decision and Control_ , pages 4039–4044. IEEE, 2014.
  * Chandrasekaran et al. (2012) Venkat Chandrasekaran, Benjamin Recht, Pablo A Parrilo, and Alan S Willsky.  The convex geometry of linear inverse problems.  _Foundations of Computational mathematics_ , 12(6):805–849, 2012.
  * Matni (2015) Nikolai Matni.  Communication delay co-design in $h_{2}$-distributed control using atomic norm minimization.  _IEEE Transactions on Control of Network Systems_ , 4(2):267–278, 2015.
  * Rotkowitz and Lall (2005) Michael Rotkowitz and Sanjay Lall.  A characterization of convex problems in decentralized control.  _IEEE transactions on Automatic Control_ , 50(12):1984–1996, 2005.
  * Rotkowitz et al. (2010) Michael Rotkowitz, Randy Cogill, and Sanjay Lall.  Convexity of optimal control over networks with delays and arbitrary topology.  _International Journal of Systems, Control and Communications_ , 2(1-3):30–54, 2010.
  * Lessard and Lall (2011) Laurent Lessard and Sanjay Lall.  Quadratic invariance is necessary and sufficient for convexity.  In _Proceedings of the 2011 American Control Conference_ , pages 5360–5362. IEEE, 2011.
  * Bamieh and Voulgaris (2005) Bassam Bamieh and Petros G Voulgaris.  A convex characterization of distributed control problems in spatially invariant systems with communication constraints.  _Systems & control letters_, 54(6):575–583, 2005.
  * Mahajan et al. (2012) Aditya Mahajan, Nuno C Martins, Michael C Rotkowitz, and Serdar Yüksel.  Information structures in optimal decentralized control.  In _2012 IEEE 51st IEEE Conference on Decision and Control (CDC)_ , pages 1291–1306. IEEE, 2012.
  * Tolstaya et al. (2020) Ekaterina Tolstaya, Fernando Gama, James Paulos, George Pappas, Vijay Kumar, and Alejandro Ribeiro.  Learning decentralized controllers for robot swarms with graph neural networks.  In _Conference on Robot Learning_ , pages 671–682. PMLR, 2020.
  * Gama et al. (2020a) Fernando Gama, Ekaterina Tolstaya, and Alejandro Ribeiro.  Graph neural networks for decentralized controllers.  _arXiv preprint arXiv:2003.10280_ , 2020a.
  * Khan et al. (2020) Arbaaz Khan, Ekaterina Tolstaya, Alejandro Ribeiro, and Vijay Kumar.  Graph policy gradients for large scale robot control.  In _Conference on Robot Learning_ , pages 823–834. PMLR, 2020.
  * Gama and Sojoudi (2021) Fernando Gama and Somayeh Sojoudi.  Distributed linear-quadratic control with graph neural networks.  _arXiv preprint arXiv:2103.08417_ , 2021.
  * Gama et al. (2020b) Fernando Gama, Joan Bruna, and Alejandro Ribeiro.  Stability properties of graph neural networks.  _IEEE Transactions on Signal Processing_ , 68:5680–5695, 2020b.
  * Bruna et al. (2013) Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun.  Spectral networks and locally connected networks on graphs.  _arXiv preprint arXiv:1312.6203_ , 2013.
  * Kipf and Welling (2016) Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  _arXiv preprint arXiv:1609.02907_ , 2016.
  * Atwood and Towsley (2015) James Atwood and Don Towsley.  Diffusion-convolutional neural networks.  _arXiv preprint arXiv:1511.02136_ , 2015.
  * Gama et al. (2018) Fernando Gama, Antonio G Marques, Geert Leus, and Alejandro Ribeiro.  Convolutional neural network architectures for signals supported on graphs.  _IEEE Transactions on Signal Processing_ , 67(4):1034–1049, 2018.
  * Goodfellow et al. (2016) Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio.  _Deep learning_ , volume 1.  MIT press Cambridge, 2016.
  * Tibshirani (1996) Robert Tibshirani.  Regression shrinkage and selection via the lasso.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 58(1):267–288, 1996.
  * Donoho (2006) David L Donoho.  Compressed sensing.  _IEEE Transactions on information theory_ , 52(4):1289–1306, 2006.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ , 2014.
  * Tu et al. (2021) Stephen Tu, Alexander Robey, and Nikolai Matni.  Closing the closed-loop distribution shift in safe imitation learning.  _arXiv preprint arXiv:2102.09161_ , 2021.
  * Revay et al. (2020) Max Revay, Ruigang Wang, and Ian R Manchester.  Convex sets of robust recurrent neural networks.  _arXiv preprint arXiv:2004.05290_ , 2020.