Approximate non-linear model predictive control with safety-augmented
  neural networks
Model predictive control (MPC) achieves stability and constraint satisfaction
for general nonlinear systems, but requires computationally expensive online
optimization. This paper studies approximations of such MPC controllers via
neural networks (NNs) to achieve fast online evaluation. We propose safety
augmentation that yields deterministic guarantees for convergence and
constraint satisfaction despite approximation inaccuracies. We approximate the
entire input sequence of the MPC with NNs, which allows us to verify online if
it is a feasible solution to the MPC problem. We replace the NN solution by a
safe candidate based on standard MPC techniques whenever it is infeasible or
has worse cost. Our method requires a single evaluation of the NN and forward
integration of the input sequence online, which is fast to compute on
resource-constrained systems. The proposed control framework is illustrated
using two numerical non-linear MPC benchmarks of different complexity,
demonstrating computational speedups that are orders of magnitude higher than
online optimization. In the examples, we achieve deterministic safety through
the safety-augmented NNs, where a naive NN implementation fails.
Henrik Hose, Johannes Köhler, Melanie N. Zeilinger, Sebastian Trimpe
# Approximate non-linear model predictive control

###### Abstract

Model predictive control (MPC) achieves stability and constraint satisfaction for general nonlinear systems, but requires computationally expensive online optimization. This paper studies approximations of such MPC controllers via neural networks (NNs) to achieve fast online evaluation. We propose safety augmentation that yields deterministic guarantees for convergence and constraint satisfaction despite approximation inaccuracies. We approximate the entire input sequence of the MPC with NNs, which allows us to verify online if it is a feasible solution to the MPC problem. We replace the NN solution by a safe candidate based on standard MPC techniques whenever it is infeasible or has worse cost. Our method requires a single evaluation of the NN and forward integration of the input sequence online, which is fast to compute on resource-constrained systems. The proposed control framework is illustrated on three non-linear MPC benchmarks of different complexity, demonstrating computational speedups orders of magnitudes higher than online optimization. In the examples, we achieve deterministic safety through the safety-augmented NNs, where naive NN implementation fails.

 Non-linear model predictive control; approximate MPC; machine learning; constrained control

## 1 Introduction

Model predictive control (MPC) is an advanced control strategy for non-linear systems that provides theoretical guarantees for constraint satisfaction and stability [1]. Convenient software packages are available to implement the required constrained optimization problems and parse them for dedicated numerical solvers, e.g., [2]. However, practical applications remain limited by long computing times in resource-constrained systems and, for general non-linear systems, it is difficult to acquire a priori certification of computation time bounds. Neural networks (NNs) have been successfully trained to imitate (approximate) the behavior of MPC schemes [3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20], resulting in fast online sampling suitable for embedded applications. Yet, providing hard guarantees for constraint satisfaction and convergence for general, non-linear systems remains challenging for such approximate MPCs (cf. discussion in Sec. II).

### _Contribution_

We propose approximate MPC with safety augmented NNs (illustrated in Fig. 1), a novel algorithm that provides deterministic guarantees for feasibility and convergence. In particular, we approximate the optimal open-loop input sequence of MPC with a function approximator that is simple to evaluate (here, a NN). This allows us to check online whether the approximator provides a feasible and stabilizing solution to the MPC problem, and is thus safe. This approach requires a single forward simulation of the system dynamics, a small additional computational cost, but significantly faster than MPC with full online optimiziation. We use standard MPC techniques online to generate a safe candidate input sequence based on a previous safe input sequence with appended terminal control law. If the NN solution is unsafe or does not improve the cost compared to the candidate, the candidate is applied. This approach ensures constraint satisfaction (safety) and convergence without any assumptions on the approximation quality.

If the NN approximation is poor, the safety augmentation would often _not_ use the NN. In an additional result, we therefore show that the naive NN would be safe if a robust MPC is sufficiently well approximated. We use this in a constructive way by approximating robust MPC schemes.

Numerical examples on three non-linear MPC benchmark systems illustrate that our approach: (i) reduces the overall computational time by several orders of magnitude compared to online optimization; and (ii) provides deterministic safety. The examples also demonstrate the importance of the proposed safety augmentation, as naively (always) applying the NN solution leads to non-negligible constraint violations in all three examples.

### _Outline_

This article continues by presenting related work on approximate MPC with guarantees in Section 2. After formalizing the control problem in Section 3, we develop our safety augmentation method in Section 4, the main contribution of this paper. In Section 5, we present an additional result: a sufficient condition for the NN solution to be safe by approximating a robust MPC. We provide numerical examples with three non-linear benchmark systems in Section 6 and the article concludes in Section 7.

### _Notation_

The set of integers is denoted by \(\mathbb{I}\), the set of integers from \(0\) to \(N\) by \(\mathbb{I}_{0:N}\), and the set of non-negative integers by \(\mathbb{N}\). By \(K_{\infty}\), we

Fig. 1: Approximate MPC with safety-augmented NN. Input sequences by a NN controller are checked during the proposed safe online evaluation (see Alg. 1). If they yield constraint satisfaction and stability, they are applied to the system. Otherwise, a safe candidate obtained from the last time step – shifted with appended terminal controller – is applied as fallback.
denote the class of functions \(\alpha:\mathbb{R}_{\geq 0}\rightarrow\mathbb{R}_{\geq 0}\) which are continuous, strictly increasing, satisfy \(\alpha(0)=0\), and are unbound. For two sets, \(\mathcal{A}\) and \(\mathcal{B}\), \(\mathcal{A}\oplus\mathcal{B}\) is the Minkowski sum and \(\mathcal{A}\ominus\mathcal{B}\) is the Pontryagin difference. We denote the square diagonal matrix with elements \(v\in\mathbb{R}^{n}\) on the diagonal as \(\text{diag}(v)\in\mathbb{R}^{n\times n}\). \(I_{N}\) is the identity matrix of size \(N\).

## 2 Related work

Explicit solutions to MPC problems can be obtained for linear systems [21], but this is not straightforward for non-linear systems. While the idea of approximating non-linear MPCs to speed up online computation is not in itself new (cf. approaches using NNs [3, 4] or multi-parametric non-linear programming [22]), the resulting approximate MPC generally does not provide guarantees for constraint satisfaction (safety). In [23, 24], qualitative results for constraint satisfaction are derived, i.e., there exist a sufficiently small approximation error and set of initial conditions such that constraint satisfaction is achieved. However, with this result it is difficult to verify safety. In the following, we discuss existing methods to ensure constraint satisfaction with approximate MPC, including combination of robust MPC and probabilistic validation, warm starting of online optimization with the NN, additional safety filters, or constrained learning. Furthermore, we discuss classical results on suboptimal MPC, which have conceptual similarities to the proposed method although the problem setting is quite different.

**Robust with respect to approximation:** One way to account for inevitable approximation errors is to explicitly add robustness to the underlying MPC scheme. In [25], constraint satisfaction and stability are shown with Lipschitz-based constraint tightening (cf. also [26, 27]), but an admissible approximation error is typically not achievable in practice. In [5, 6], more advanced robustification (cf. [28, 29]) is used to handle realistic approximation errors by treating them as (hypothetical) input disturbance. To obtain bounds on the approximation error, statistical validation is proposed in [5] and applied in hardware experiments in [6]. The probabilistic guarantees are given by sampling independent identically distributed (i.i.d.) initial conditions, performing simulation in closed loop with the NN controller, and using Hoeffding's inequality to guarantee, with high probability, that the approximation error is smaller than the hypothetical disturbance. A similar statistical approach is used in [7] to validate safety. Due to the probabilistic nature of these approaches, a non-zero probability of faulty inputs remains, and safety is not deterministically guaranteed in closed loop. Furthermore, validating by sampling closed-loop trajectories is computationally expensive and the validation can be conservative. The method in [5, 6] also requires a robust MPC scheme which introduces conservatism, and the small, uniform bound on the approximation error can be hard to achieve for large systems in practice. By contrast, we provide deterministic guarantees without assumptions on the approximation error and our method is applicable to high dimensional systems.

**NN as warm start:** Using NN predictions as warm starting solutions to numerical solvers potentially reduces computation time, while the solver guarantees constraint satisfaction, meaning that standard MPC guarantees apply. A speedup by a factor of two is indeed observed for linear systems with active set solvers in [8] and for large-scale systems in [30]. However, this method requires online optimization, for which worst case computation time bounds can be long. Furthermore, in [9], warm starting with NN predictions was observed to increase the average computation time in non-linear numerical examples. Our approach differs from these methods in that we do not require online optimization and therefore achieve a considerably smaller computation time.

**Safety filters:** Safety enforcing methods aim to modify the outputs of the NN by finding the closest safe inputs. For linear systems, one can consider an _explicit_ safety filter, which simply use a projection based on control invariant sets. This is how the authors of [12, 13, 31] ensure NN controller safety. However, like computing control barrier functions, computing control invariant sets for general non-linear systems is challenging. This method is therefore limited to small dimensional or linear systems or is conservative (yields a small feasible set). For linear systems, it is also possible to explicitly encode polytopic constraints in the NN approximation to ensure constraint satisfaction [18]. However, this approach does not apply to non-linear systems. These issues can be addressed by "predictive" safety filters, which formulate an optimal control problem that minimizes the difference between a safe and an unsafe input [32]. An MPC formulation with a terminal set that is invariant under a terminal controller serves as a safety certificate that the filtered control inputs are safe. Similar to safety filters, [11] proposes approximating a complete sequence of inputs of a non-linear MPC with a NN and solving an optimization problem online that finds the closest safe trajectory. Unfortunately, solving online optimization problems can be computationally expensive and does not admit a worst-case computation time bound. While such predictive safety filters can also be approximated with NNs [33], safety guarantees again require uniform approximation error bounds, similar to [5, 6]. Like predictive safety filters, our method uses a sequence of inputs and terminal ingredients to guarantee safety. However, we do not require online optimization.

**Constrained learning:** Instead of simply fitting a function approximator to imitate the behavior of an MPC (e.g., by supervised learning), constrained learning methods aim to improve performance and constraint satisfaction. To enforce constraint satisfaction of the NN (at least on the states visited during training), optimization of the MPC and training can be performed simultaneously, e.g., by considering MPC cost and constraints in the training loss via slack variables as in [14, 15], or with a loss on the dual of the MPC problem as in [11]. The authors of [16] propose another method for jointly optimizing the parameters of the NN approximator and solving the optimal control problem for non-linear systems, both with off-the-shelf non-linear solvers. In [17], the optimization problem is reformulated as a recurrent NN which can be efficiently trained. All of these constrained learning methods aim for data efficiency and constraint satisfaction on the states visited during training. However, no guarantees are given for states not seen during training. Furthermore, the methods are only computationally tractable for a few training points and the system size is limited. In [10], constrained learning is used to approximate primal and dual MPC problems for linear systems by choosing the number of samples based on the scenario method. At evaluation time, primal feasibility (safety) is checked and suboptimality is quantified with the additional dual approximation. In case the approximate sequence is unsafe or "too" suboptimal, a separate, safe backup controller is invoked. Checking for primal feasibility online is also part of our method. However, results in [10] are presented for linear systems and extension to non-linear is not straightforward, e.g., for finding a backup controller. By contrast, our method is applicable to medium and large scale systems and provides deterministic guarantees without the need for a separate backup controller.

**Suboptimal MPC:** In non-linear MPC, it is not always necessary to solve optimization problems to optimality. As long as the optimized input sequence is feasible and reduces the value function compared to the candidate, the closed-loop system will be persistently feasible and stable, cf. a classic result [34]. In [35], stability for robust suboptimal MPC is directly enforced by a constraint on cost reduction
over the previous time step in the optimization problem. In [36], inherent robustness of a suboptimal MPC with warm starting is shown. Even though the setting of suboptimal MPC is quite different from the setting we consider here, we use a similar idea, i.e., we ensure persistent feasibility and convergence by providing a feasible, suboptimal control sequence from an NN that performs at least as well as the candidate sequence.

## 3 System Description

We consider general, non-linear, discrete time systems of the form

\[x(t+1)=f(x(t),u(t)) \tag{1}\]

with state \(x(t)\in\mathbb{R}^{n_{\text{x}}}\), input \(u(t)\in\mathbb{R}^{n_{\text{u}}}\), and \(t\in\mathbb{N}\). We assume \(f\) is continuous and consider compact, polytopic constraints of the form

\[\mathcal{X}=\{x\in\mathbb{R}^{n_{\text{x}}}\mid L_{\text{x}}x\leq 1\}, \quad\mathcal{U}=\{u\in\mathbb{R}^{n_{\text{u}}}\mid L_{\text{u}}u\leq 1\}. \tag{2}\]

The objective is to drive the system to \(x=0\) with \(f(0,0)=0\) and ensure constraint satisfaction \((x(t),u(t))\in\mathcal{X}\times\mathcal{U}\)\(\forall\)\(t\in\mathbb{N}\) for a large set of initial conditions. This objective can be achieved by MPC [1].

### MPC problem

In MPC, a sequence of inputs \(\textbf{u}\ =\ [u_{0}^{\top},\ \ldots,\ u_{N-1}^{\top}]^{\top}\in\mathcal{U}^{N}\) is computed. The predicted state is then \(\phi(k;x;\textbf{u})\) when applying (1) for \(k\) inputs from the sequence **u**, starting at \(x\). In MPC, we consider a finite-horizon cost of the form

\[V(x,\textbf{u})=\sum_{k=0}^{N-1}(\ell(\phi(k;x;\textbf{u}),\textbf{u}_{k}))+V _{\text{f}}(\phi(N;x;\textbf{u})), \tag{3}\]

with horizon \(N\), stage cost \(\ell(x,u)\), and terminal cost \(V_{\text{f}}(x)\). We assume \(V\) is continuous. We denote the set of feasible input trajectories from a given state \(x\) that are valid (suboptimal) solutions to the MPC optimization problem (5) by

\[\mathcal{U}^{N}(x)=\{\textbf{u}\in\mathcal{U}^{N}\mid \phi(k;x;\textbf{u})\in\mathcal{X},k\in\mathbb{I}_{0:N-1}, \tag{4}\] \[\phi(N;x;\textbf{u})\in\mathcal{X}_{\text{f}}\},\]

where \(\mathcal{X}_{\text{f}}\subseteq\mathcal{X}\) is the terminal set. The corresponding nominal MPC formulation is

\[V^{*}(x)=\min_{\textbf{u}\in\mathcal{U}^{N}(x)}V(x,\textbf{u}). \tag{5}\]

The feasible set of (5) is denoted by \(\mathcal{X}_{\text{feas}}\). Solving this optimization problem for the current value \(x\) gives a corresponding trajectory of optimal inputs **u**, which we denote by \(\Pi_{\text{MPC}}:\mathbb{R}^{n_{\text{x}}}\rightarrow\mathcal{U}^{N}\). Applying the first input of this sequence, \(\textbf{u}_{0}\), to the system yields a control policy of the form \(u=\kappa_{\text{MPC}}(x)\), which can be applied in closed loop to the system. We consider the following standard conditions for MPC (cf. [1]):

**Assumption 1**: _There exist a function \(\alpha_{\ell}\in\mathcal{K}_{\infty}\) and a feedback \(K_{\text{f}}:\mathcal{X}_{\text{f}}\rightarrow\mathcal{U}\) such that for any \(x\in\mathcal{X}\), \(u\in\mathcal{U}\):_

* \(\ell(x,u)\geq\alpha_{\ell}(\|x\|)\)_,_
* _the terminal set is positively invariant under the terminal controller_ \[x\in\mathcal{X}_{\text{f}}\implies f(x,K_{\text{f}}(x))\in\mathcal{X}_{\text {f}},\] (6)
* _the terminal cost is a control Lyapunov function for_ \(x\in\mathcal{X}_{\text{f}}\)__ \[V_{\text{f}}(f(x,K_{\text{f}}(x)))-V_{\text{f}}(x)\leq-\ell(x,u).\] (7)

These conditions ensure stability of the closed loop system and persistent feasibility, i.e., if a solution to (5) exists at \(t=0\), then it exists for all times \(t\in\mathbb{N}\). Corresponding proofs alongside constructive methods to determine \(\mathcal{X}_{\text{f}}\), \(K_{\text{f}}\), and \(V_{\text{f}}\) can be found in [1]. Implementing MPC requires online optimization to solve (5), which can be computationally expensive. We circumvent this issue by approximating the MPC with NNs.

### Problem formulation: approximate MPC with guarantees

We approximate the MPC via an NN offline by first generating a dataset of initial conditions and corresponding, optimal input sequences, i.e., solutions to (5), and then training the NN in a standard supervised learning fashion [4, 5, 6, 7, 8, 9]. The approximation \(\Pi_{\text{NN}}\approx\Pi_{\text{RMPC}}\) maps a state \(x\) to a sequence of inputs **u** with horizon length \(N\) as

\[\Pi_{\text{NN}}:\mathbb{R}^{n_{\text{x}}}\rightarrow\mathcal{U}^{N}. \tag{8}\]

The approximation is not exact, \(\Pi_{\text{NN}}\neq\Pi_{\text{MPC}}\). Therefore, naively (always) applying the first input \(\textbf{u}_{0}\) of \(\textbf{u}(t)=\Pi_{\text{NN}}(x(t))\) to the system does not guarantee constraint satisfaction; that is, \(x(t)\notin\mathcal{X}\) for some \(t\in\mathbb{N}\) when applying \(\textbf{u}(t)=\Pi_{\text{NN}}(x(t))\) and \(x(0)\in\mathcal{X}_{\text{feas}}\) (see Sec. 6 for examples where naively applying \(\Pi_{\text{NN}}\) fails).

_Problem statement:_ We seek to develop a safety augmentation such that for a given NN approximation, \(\Pi_{\text{NN}}\), of an MPC (5), the system is guaranteed to satisfy constraints and converge; that is, \(x(t)\in\mathcal{X}\) for all \(t\in\mathbb{N}\) and \(x(t)\to 0\) for \(t\rightarrow\infty\).

## 4 Safe Online Evaluation

In this section, we address the problem stated above by augmenting the NN in a safe online evaluation. We check online if \(\Pi_{\text{NN}}(x(t))\) is a feasible solution to (5) and provide a deterministically safe feedback policy that chooses between the NN sequence, if it is safe, and a safe fallback candidate. The safety augmentation further ensures that the closed-loop system converges. This idea is implemented in Algorithm 1, which we now elaborate.

```
1:\(\Pi_{\text{NN}}(\cdot)\), \(\mathcal{U}^{N}(\cdot)\), \(K_{\text{f}}(\cdot)\), \(V(\cdot,\cdot)\), \(\tilde{\textbf{u}}(0)=\textbf{u}_{\text{init}}\)
2:for all\(t\in\textbf{u}\)do
3:\(x\gets x(t)\) {measure state at time \(t\)}
4:\(\tilde{\textbf{u}}(t)\leftarrow\Pi_{\text{NN}}(x)\) {evaluate approximation}
5:if\(\tilde{\textbf{u}}(t)\in\mathcal{U}^{N}(x)\) {check if\(\tilde{\textbf{u}}(t)\) is feasible}then
6:\(\textbf{u}(t)\leftarrow\operatorname*{argmin}_{\textbf{u}(t)\in\{\tilde{ \textbf{u}}(t),\tilde{\textbf{u}}(t)\}}V(x,\textbf{u}(t))\) {choose \(\text{min cost}\)}
7:else
8:\(\textbf{u}(t)\leftarrow\tilde{\textbf{u}}(t)\) {keep candidate}
9:endif
10:\(u(t)\leftarrow\textbf{u}_{0}(t)\) {apply input to system}
11:\(\tilde{\textbf{u}}(t+1)\leftarrow\{\textbf{u}(t)_{1:N-1},K_{\text{f}}(\phi(N;x; \textbf{u}(t))\}\) {create candidate sequence}
12:endfor
```

**Algorithm 1** Approximate MPC with safety-augmented NN

Algorithm 1 validates predicted input sequences by an approximate MPC (8) and provides a safe fallback solution should the approximate solution be rejected. For each newly measured state \(x(t)\), the NN evaluation gives a predicted input sequence \(\hat{\textbf{u}}(t)\), which is checked for feasibility in line 4, i.e., \(\hat{\textbf{u}}(t)\in\mathcal{U}^{N}(x)\). This requires the system dynamics to be forward integrated along the prediction horizon (4). If \(\hat{\textbf{u}}(t)\) is safe, we select the sequence of minimum cost \(\textbf{u}(t)\) from \(\hat{\textbf{u}}(t)\), and the safe candidate \(\hat{\textbf{u}}(t)\) in line 5. Finally, the first input \(\textbf{u}_{0}(t)\) is applied to the system (1) and a new safe candidate is created by shifting \(\textbf{u}(t)\) and appending the terminal controller.

For initialization of Algorithm 1, we state the following assumption:
**Assumption 2**: _The system is initialized in state \(x(0)\) with a safe candidate solution \(\mathbf{u}_{\text{init}}\in\ \mathcal{U}^{N}(x(0))\) in Algorithm 1._

**Remark 1**: _Possible means of initialization could be: (i) start in a steady state, where \(\mathbf{u}_{\text{init}}\) is easy to construct; (ii) initialize with \(\mathbf{u}_{\text{init}}=\Pi_{\text{MPC}}(x(0))\), which can be computed offline if the system is started in a known state \(x(0)\); (iii) initialize with \(\mathbf{u}_{\text{init}}=\Pi_{\text{NN}}(x(0))\), which satisfies Assumption 2 if \(\Pi_{\text{NN}}(x(0))\in\mathcal{U}^{N}(x(0))\)._

The following theorem summarizes the closed-loop properties of Algorithm 1.

**Theorem 1**: _Let Assumptions 1 and 2 hold. Then, the closed-loop system resulting from Algorithm 1 satisfies \((x(t),u(t))\in\mathcal{X}\times\mathcal{U}\) for all \(t\in\mathbb{N}\) and converges to the state \(x=0\)._

In the following, we first prove constraint satisfaction and then convergence for all \(t\in\mathbb{N}\).

**Part I:** Using induction, we show that \(\tilde{\mathbf{u}}(t)\in\mathcal{U}^{N}(x(t))\) for all \(t\in\mathbb{N}\). Induction start: at \(t=0\) by Assumption 2. Induction step: we know that \(\mathbf{u}(t)\in\mathcal{U}^{N}(x(t))\), since \(\mathbf{u}(t)\) is selected only from feasible sequences in lines 4 to 8. Whenever \(\mathbf{u}(t)\in\mathcal{U}^{N}(x(t))\), Assumption 1 ensures that the safe candidate \(\tilde{\mathbf{u}}(t+1)\) constructed in line 10 is feasible, i.e., \(\tilde{\mathbf{u}}(t+1)\in\mathcal{U}^{N}(x(t+1))\) (cf. standard candidate solution in nominal MPC proof [1]). It follows by induction that \(\mathbf{u}\in\mathcal{U}^{N}(x(t))\ \forall\ t\in\mathbb{N}\) and therefore \((x(t),u(t))\in\mathcal{X}\times\mathcal{U}\).

**Part II:** For the shifted candidate sequence in Alg. 1 line 10 it holds that

\[\begin{split}& V(x(t+1),\tilde{\mathbf{u}}(t+1))\\ &\quad=\sum_{k=1}^{N-1}\ell(\phi(k,x(t),\mathbf{u}(t)),\mathbf{u}(t)\\ &\quad\quad+\ell(\phi(N;x(t);\mathbf{u}(t)),K_{\mathcal{I}}(\phi(N;x( t);\mathbf{u}(t)))\\ &\quad\quad+V_{\mathcal{I}}(\phi(N;x(t+1);\tilde{\mathbf{u}}(t+1))). \end{split} \tag{9}\]

By Assumption 1, it also holds that

\[\begin{split}& V(x(t+1),\tilde{\mathbf{u}}(t+1))\\ =& V(x(t),\mathbf{u}(t))-\ell(x(t),\mathbf{u}_{0}(t))\\ &\quad+\ell(\phi(N;x(t);\mathbf{u}(t)),K_{\mathcal{I}}(\phi(N;x(t); \mathbf{u}(t))))\\ &\quad-V_{\mathcal{I}}(\phi(N;x(t);\mathbf{u}(t))+V_{\mathcal{I}}( \phi(N;x(t+1);\tilde{\mathbf{u}}(t+1)))\\ &\quad\stackrel{{\text{\eqref{eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eqeq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eq:eqeq
\(f(x,u)\|_{\infty}\) and \(\mathcal{B}_{\mathcal{E}}:=\{\tilde{d}_{\mathsf{w}}\in\mathbb{R}^{n_{\mathsf{x}}} \mid\|\tilde{d}_{\mathsf{w}}\|_{\infty}\leq\tilde{\epsilon}\}\). The tightened constraints are

\[\bar{\mathcal{U}}=\mathcal{U}\ominus\mathcal{B}_{\epsilon},\quad\bar{ \mathcal{X}}_{k}=\mathcal{X}\ominus c_{k}\mathcal{B}_{\mathcal{E}},\quad c_{k} =\sum_{i=0}^{k-1}L_{\mathsf{f}}^{i}, \tag{15}\]

where \(L_{\mathsf{f}}\) is the Lipschitz constant of \(f\) w.r.t. \(x\), i.e., \(\|f(x_{1},u)-f(x_{2},u)\|_{\infty}\leq L_{\mathsf{f}}\|x_{1}-x_{2}\|_{\infty}, \;\forall\;x_{1},x_{2}\in\mathcal{X},u\in\mathcal{U}\). The feasible set in (13) is

\[\begin{split}\bar{\mathcal{U}}^{N}(x)=\{\bar{\mathsf{u}}\in\bar{ \mathcal{U}}^{N}\mid&\phi(k;x;\bar{\mathsf{u}})\in\bar{ \mathcal{X}}_{k},k\in\mathbb{I}_{0:N-1},\\ &\phi(N;x;\bar{\mathsf{u}})\in\bar{\mathcal{X}}_{\mathsf{f}}\}, \end{split} \tag{16}\]

with an appropriate terminal set \(\bar{\mathcal{X}}_{\mathsf{f}}\subseteq\mathcal{X}_{\mathsf{f}}\), cf. [27, Thm. 1]. This Lipschitz-based method can be improved, e.g., using a more general feedback and tube parametrization [28], which we also use in the examples in Sec. VI. Many other robust MPC methods to satisfy (14) also exist, e.g., [29].

### Guarantees for approximations of robust MPC

In this section, we use a robust MPC (see Sec. V-A) to train an NN \(\Pi_{\text{NN}}\) using supervised learning as before. Then, we provide conditions for the safety augmentation (Alg. 1) to not reject \(\Pi_{\text{NN}}\). For an approximation with a bound on the approximation error, we can state the following Lemma.

**Lemma 2**: _Let \(\Pi_{\text{IMMC}}\) satisfy (14) and suppose the approximation error of the NN satisfies \(\|\Pi_{\text{NN}}(x)-\Pi_{\text{IMMC}}(x)\|_{\infty}\leq\epsilon\;\forall\;x \in\bar{\mathcal{X}}_{\text{feas}}\). Then, for all \(x\in\bar{\mathcal{X}}_{\text{feas}}\): \(\Pi_{\text{NN}}(x)\in\mathcal{U}^{N}(x)\)._

Let \(\boldsymbol{d}(x)=\Pi_{\text{NN}}(x)-\Pi_{\text{IMMPC}}(x)\), then \(\boldsymbol{d}(x)\in\mathcal{B}_{\epsilon}^{N}\forall x\in\bar{\mathcal{X}}_{ \text{feas}}\). As \(\Pi_{\text{IMMC}}\) satisfies (14), it follows that \(\Pi_{\text{IMMC}}(x)+\boldsymbol{d}(x)=\Pi_{\text{NN}}(x)\in\mathcal{U}^{N}(x)\), for which Algorithm 1 line 4 is always satisfied. \(\blacksquare\)

Lemma 2 allows us to design an approximate MPC that - under Algorithm 1 - will often be applied to the system and therefore mitigate potentially long periods of simply replaying the safe candidate. We emphasize that deterministic safety holds by Theorem 1 even if the conditions in Lemma 2 do not hold. The \(\epsilon\) approximation of \(\Pi_{\text{IMMC}}\) is similar to prior work [5, 6], where the \(\epsilon\) approximation is validated offline by performing simulations in closed loop with an NN controller, giving probabilistic safety guarantees directly. In our setting, we do not require the uniform \(\epsilon\) approximation error bound everywhere, because safety is already guaranteed by Algorithm 1, making it considerably simpler to find an approximator for large systems. In comparison to [5, 6], Lemma 2 serves as a sufficient condition, e.g., on a (subset of sampled states. We can use it in a constructive way to find "good" approximations \(\Pi_{\text{NN}}\) in our numerical examples.

**Remark 3**: _Lemma 2 does not guarantee that the proposed Algorithm 1 will always apply the NN solution, because the candidate in line 5 could have preferable cost. However, it is safe to always apply the NN prediction in line 5._

## VI Numerical examples

In this section, we demonstrate our method on three common numerical example systems for non-linear MPC: a continuous stir tank reactor [5, 37], a quadcopter model [28, 38], and a chain of hanging masses connected by springs [39]. First, the system dynamics and MPC formulations are summarized for all three systems. The NNs are trained using supervised learning by generating optimal input sequences of a robust MPC design (Sec. V) over a large set of random initial states. Finally, we compare our proposed approach, approximate MPC via safety-augmented NNs (Sec. IV), with online optimization and always (naively) applying the NN approximation. We benchmark the safety augmented NN with respect to computation time of online optimization and constraint satisfaction of the naive NN. The code for all examples is available at [https://github.com/hshose/soeampc.git](https://github.com/hshose/soeampc.git).

### Implementation details

We now describe some details of the MPC formulations (terminal ingredients and robustification), dataset generation, and the learning of approximations with NNs. For all three examples, we use a terminal set \(\mathcal{X}_{\mathsf{f}}=\{x\mid\|P_{\mathsf{f}}^{0.5}x\|\leq\alpha\}\), terminal cost \(V_{\mathsf{f}}=x^{\top}\mathbb{P}x\), and terminal controller \(u=K_{\mathsf{f}}x\), with parameters computed as in [40]. For simple robustification with respect to (hypothetical) input disturbances \(d_{\mathsf{w}}\), we use a tube defined by \(\dot{s}=-\rho s+\bar{w}\) and \(s(0)=0\), where \(\rho\) is a contraction rate and \(\bar{w}\) is a disturbance bound. The tightened constraints are of the form \(L_{\mathsf{x},j}x+c_{j}s\leq 1\), \(j\in\mathbb{I}_{n_{\text{com}}}\), where \(n_{\text{com}}\) is the number of constraints. The terminal set is tightened such that at time \(N\) the tube is inside the nominal terminal set. The tube parameters \(\rho\) and \(\bar{w}\), stabilizing feedback \(K_{\mathsf{f}}\), and constraint tightening factors \(c_{j}\) are computed following [28, 41]. The stabilizing feedback \(u=K_{\mathsf{f}}x+v\), where \(v\) is the optimization variable of the RMPC, is substituted directly into the MPC formulation. Additional details regarding the robust MPC design including specific linear matrix inequalities (LMIs) for the terminal ingredients and robustification parameters can be retrieved from the code.

In the next step, the offline dataset generation is performed using the acados toolbox [2] - a fast sequential quadratic programming (SQP) solver for non-linear programs (NLPs) - with HPIPM as quadratic program (QP) solver for the stir tank and quadcopter, and DAQP for the chain mass example. Maximum SQP iterations are set to \(400\) in the case of the quadcopter and \(1\times 10^{3}\) for the stir tank and chain mass system. We initialize the solver with a forward simulation of the clipped terminal controller. We only consider samples for NN training for which acados produced a feasible solution. A full list of solver settings is available in the code.

After the dataset generation, NN approximations are trained using Tensorflow with the Adam optimizer. We use fully connected feedforward NNs with tangent hyperbolic activations. The input layer has size \(n_{\mathsf{x}}\), while the output layer has \(n_{\mathsf{u}}\cdot N\) neurons with linear activations. Number and sizes of hidden layers were empirically determined by increasing the depth and width until a reasonable approximator was found. The additive disturbance \(d_{\mathsf{w}}\) is a design parameter in the robust MPC design that we use to facilitate learning a feasible solution (Sec. V). The \(d_{\mathsf{w}}\) in our examples was empirically determined as a tradeoff between simplification of the approximation problem and conservativeness, starting with small assumed disturbances and slowly increasing them until the NN gives a feasible solution to (5) for a reasonable percentage of testpoints.

### Benchmark systems

In the following, we provide details of the dynamics and parameters of the three benchmark systems.

#### Vi-B1 Stir tank reactor

The non-linear stir tank reactor with \(x\in\mathbb{R}^{2}\) and \(u\in\mathbb{R}\) is used as a benchmark in previous work on MPC [5, 37]. The continuous-time system dynamics are given by

\[\dot{x}=\begin{bmatrix}\frac{1-x_{1}}{\theta}-kx_{1}\exp\left(-\frac{M}{x_{2}} \right)\\ \frac{x_{\mathsf{f}}-x_{2}}{\theta}+kx_{1}\exp\left(-\frac{M}{x_{2}}\right)-\gamma u (x_{2}-x_{\mathsf{c}}))\end{bmatrix} \tag{17}\]

with parameters \(\theta=20\), \(k=300\), \(M=5\), \(x_{\mathsf{f}}=0.3947\), \(x_{\mathsf{c}}=0.3816\), and \(\gamma=0.117\). The goal is to drive the system to the unstable stationary point \(x_{\mathsf{c}}=[0.2632,0.6519]^{\top}\) and \(\mathrm{t}=0.7853\). The constraints are \(\|x-x_{\mathsf{c}}\|_{\infty}\leq 0.2\) and \(0\leq u\leq 2\). Initial states are
sampled from the constraint set. The sampling time is \(T_{\text{s}}=2\), and we use a horizon of \(N=10\) steps. For the MPC, we choose quadratic cost with weights \(Q=I_{2}\) and \(R=10^{-5}\). The terminal ingredients are computed as in [5]. We use \(|d_{w,2}|\leq 1\cdot 10^{-4}\) on state \(x_{2}\) and fit an NN with \(2\) hidden layers, \(50\) neurons per layer.

#### 3.2.2 Quadcopter

We use a ten-state quadcopter model with an RMPC formulation as in [28]. We use the original system dynamics and parameters from [38]. These dynamics are

\[\dot{x}_{i} =v_{i},\,i\in\mathbb{I}_{1:3} \dot{v}_{i} =g\tan{(\phi_{i})},\,i\in\mathbb{I}_{1:2} \tag{18a}\] \[\dot{v}_{3} =-g+\frac{k_{T}}{m}u_{3} \dot{\phi}_{i} =-d_{1}\phi_{i}+\omega_{i},\,i\in\mathbb{I}_{1:2}\] (18b) \[\dot{\omega}_{i} =-d_{0}\phi_{i}+n_{0}u_{i},\,i\in\mathbb{I}_{1:2}, \tag{18c}\]

with state \(x=[x_{1},x_{2},x_{3},v_{1},v_{2},v_{3},\phi_{1},\omega_{1},\phi_{2},\omega_{2} ]^{\top}\in\mathbb{R}^{10}\) and input \(u=[u_{1},u_{2},u_{3}]^{\top}\in\mathbb{R}^{3}\). The steady-state input is \(u_{\text{e},3}=\frac{g_{m}}{k_{0}}\). The constants are \(d_{0}=80\), \(d_{1}=8\), \(n_{0}=40\), \(k_{\text{P}}=0.91\), \(m=1\), \(3\), and \(g=9.81\). Constraints are \(x_{1}\leq 0.145\) (a wall), \(|\phi_{1,2}|\leq\frac{g}{3}\), \(|u_{1,2}|\leq\frac{g}{3}\), and \(u_{3}\in[0,2g]\). The quadratic state and input cost terms have weights \(Q=\text{diag}([200\cdot 1_{3},1_{3},0.01\cdot 1_{4}])\) and \(R=\text{diag}([8,8,0.8])\). Initial conditions are randomly sampled from \(|x_{1,2}|\leq 2.5\), \(x_{1}\leq 0.145\), \(|x_{3}|\leq 3\), \(|v_{1,2}|\leq 3\), \(|v_{3}|\leq 5\), \(|\phi_{1,2}|\leq\frac{g}{3}\), and \(|\omega_{1,2}|\leq 3\pi\). The sampling time is \(T_{\text{s}}=0.1\)s with a horizon \(N=10\). The terminal set and controller are computed by solving LMIs as described in detail in [28, 40]. We choose \(\|d_{w}\|_{\infty}\leq 5\cdot 10^{-3}\) and fit an NN with 6 hidden layers and \([200,400,600,600,400,200]\) neurons.

#### 3.2.3 Chain mass

The chain mass system is a standard benchmark problem for non-linear MPC introduced in [39]: a chain of \(M\) point masses in three dimensional space, connected by springs, where the first is fixed and the last can be freely moved by the input. The dynamics of the system are given by:

\[\dot{p}_{i} =v_{i},\,i\in\mathbb{I}_{2:M-1}\qquad\dot{p}_{M}=u \tag{19a}\] \[\dot{v}_{i} =\frac{1}{m}(F_{i,i+1}-F_{i-1,i})+g,\,i\in\mathbb{I}_{2:M-1}\] (19b) \[F_{i,i+1} =D(1-L/\left\|p_{i+1}-p_{i}\right\|)(p_{i+1}-p_{i}), \tag{19c}\]

with \(D=1\), \(L=0.033\), \(m=0.033\) and \(q=[0,0,-9.81]^{\top}\). The state is \(x=[p_{2}^{\top},\ldots,p_{M}^{\top},v_{2}^{\top},\ldots,v_{M-1}^{\top}]^{\top} \in\mathbb{R}^{5M-9}\) and the input \(u=v_{M}\ \in\ \mathbb{R}^{3}\). The objective is to move the masses into an equilibrium position, where the last free mass is at position \([1,0,0]\). There is a wall constraint for all masses such that \(p_{y,i}\geq-0.1\). The input is constrained to \(\|u\|_{\infty}\leq 1\). Initial states are sampled from \(\|p_{i}\|_{\infty}\leq 0.25\) and \(\|v_{i}\|_{\infty}\leq 0.5\). The sampling time is \(T_{\text{s}}=0.133\)s with a horizon of \(N=15\) for \(M=3\) masses. The quadratic state and input cost terms are \(Q=\text{diag}([1_{3(M-2)},25\cdot 1_{3},1_{3(M-2)}])\) and \(R=I_{3}\). We choose \(\|d_{w}\|_{\infty}\leq 5\cdot 10^{-3}\). Terminal set and constraint tightening is performed as for the previous examples. We fit a NN with 6 hidden layers and \([200,400,600,600,400,200]\) neurons.

### _Results_

In the following, we present the results for the three benchmark systems. First, we show the training effort and approximation quality achieved. Second, performance in terms of computation time is compared with online optimization. Finally, we show that our method is safe in all three examples, while naively (always) applying the NN leads to constraint violations in closed loop.

#### 3.3.1 Dataset generation and NN training

We now describe the size and computation times of generated datasets, give an indication of NN sizes, and provide NN training times for all three systems. To gain some intuition into how well each approximator generalizes, it is tested on \(10^{5}\) points not used during training by forward simulating and checking whether the NN provides a feasible solution to (5). The results are summarized in Table I.

In the stir tank example, a relatively small NN approximates the dataset well (more than \(99\,\mathrm{\char 37}\) of predictions are valid solutions to (5) on test data), which is in line with numerical results in [5]. For the ten-state quadcopter example, a dataset corresponding to approximately \(5\) gridpoints per state dimension suffices to fit a relatively large approximator, which yields a feasible solution to the MPC problem in over \(70\,\mathrm{\char 37}\) of initial conditions. Contributing factors to this could be that the quadcopter system is almost linear (only non-linearity is the \(\tan(\phi)\) in the velocity derivative), only loosely coupled in three Cartesian directions by the MPC cost function, and has few constraints that become active. The chain mass system, on the other hand, is "very" non-linear and chaotic, which makes the approximation problem inherently harder. This is reflected in a large dataset and approximator, which yield a feasible sequence for less than \(10\,\mathrm{\char 37}\) of initial conditions during testing.

The total time required to evaluate samples of the MPC on random initial conditions is the dominant time factor offline. However, this was executed in parallel. In our paper, we used the RWTH high-performance compute cluster with about \(100\) nodes in parallel and \(24\) cores per node to generate the datasets within a few hours. In our examples, NN training is performed on a single GPU. While it is fast for the stir tank and quadcopter, the chain mass example requires over 5 days of training, which is significant. However, this could be improved by finding better hyperparameters and distributing the training across multiple GPUs.

#### 3.3.2 Online computation times

We next evaluate the online computation times for the proposed method and compare them with solving the NLP online. The results of this comparison are summarized in Table II.

Computation times for our method include NN inference in Tensorflow and forward simulation of the system dynamics using acados implicit fourth order integration. Both of these have very low variance and we therefore only report the mean value. The combined online evaluation time is in the order of \(1\,\mathrm{ms}\). This is about three orders of magnitude faster than solving the NLP online with acados and about as fast as a single SQP iteration. We note that the high number of required iterations are partially due to the terminal set constraint required for persistent feasibility and the lack of warm starting. Yet, our method provides deterministic safety while a single SQP iteration might return an infeasible solution to (5). The NLP solver running to convergence or maximum iterations has a much higher variance

\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{dataset} & \multicolumn{2}{c}{neural network} \\ \cline{2-5}  & points & computation & parameters & training & solution \\  & \([10^{6}]\) & time1  & time2  & feasible \\  & & \([10^{3}\) core-h] & & [h] & [\%] \\ \hline \hline stir tank & 0.9 & 0.5 & \(3\times 10^{3}\) & 4 & 99 \\ quadcopter & 9.6 & 4.1 & \(1\times 10^{6}\) & 37 & 72 \\ chain mass & 19.1 & 17.5 & \(1\times 10^{6}\) & 127 & 9.5 \\ \hline \hline \end{tabular}
\end{table} TABLE I: Dataset and NN approximator overview. Number of points and cumulative offline computation time for the dataset generation, trainable parameters and training time for the NN approximator. “Solutions feasible” is the percentage of test points, for which the NN gives a feasible solution to (5).
in computation time due to variable number of SQP and underlying QP iterations.

#### 3.2.3 Constraint satisfaction (safety)

To evaluate the effectiveness of our safety augmentation (Alg. 1), we compare the closed-loop system behavior with naively (always) applying the NN solution. We randomly sample initial states and use the first \(10^{5}\) points where the MPC has a feasible solution, i.e., initial conditions where we know that the (5) is initially feasible. We simulate the system dynamics and check constraint satisfaction in closed loop for a (sufficiently large) finite time. The results are summarized in Table 3.

With the proposed safety augmented NNs, the systems adhere to constraints in closed loop - they are _deterministically_ safe. By contrast, naively applying the NN solution results in constraint violations for all three systems, underscoring the importance of safety augmentation. Even when the NN provides an initially feasible solution to (5), i.e., \(\Pi_{\text{NN}}(x(0))\in\mathcal{U}^{N}(x(0))\), it might not be safe to naively evaluate for all times in closed loop. An illustrative example for catastrophic failure (crash into the wall) when naively applying the NN to the quadcopter system is depicted in Fig. 2: the NN controller is feasible for the first \(0.6\,\mathrm{s}\) of the closed loop experiment. Afterwards, the proposed safety augmentation detects some unsafe NN input sequences and applies the safe candidate solution. The naive implementation that always applies the new NN input crashes into the wall \(0.3\,\mathrm{s}\) later. The safety augmentation continues to apply NN predictions when safe and successfully evades the wall constraint.

Table 3 also summarizes how often our method rejects the NN prediction and chooses the safe candidate solution in closed loop. It further states the reasons for rejection, which could be state or terminal constraint violation (safety) in Algorithm 1 line 4, or failure to decrease the cost (convergence) in line 5. A trajectory could be rejected for multiple reasons. The results in Table 3 show that all three reasons for rejection might be relevant, depending on the specific system and constraints. For example, in the stir tank system, constraint violation is rarely the reason for applying the safe candidate; in the quadcopter system, state constraint violation is a major reason for rejecting NN predictions; and in the chain mass system, the terminal constraint is almost always the reason for applying the safe candidate.

The numerical examples given in this section demonstrate that the proposed method is fast to evaluate and provides deterministic guarantees for safety, while naively always applying the NN solution can be unsafe and performing online optimization is much slower.

## 4 Conclusion

We propose approximate non-linear MPC with safety augmented NNs, which provides deterministic safety and convergence in closed loop despite approximation errors. We approximate entire input sequences and check online whether an input sequence by

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline  & \multicolumn{2}{c}{**safety augmented NN (ours)**} & \multicolumn{2}{c}{NLP solver} \\ \cline{2-7}  & NN & forward & & & & \\  & inference & simulation & **total** & & & \\  & [ms] & [ms] & **[ms]** & & mean & max & iter \\ \hline \hline stir tank & 0.5 & 0.5 & 1.0 & 430 & 1060 & 0.7 \\ quadcopter & 0.7 & 0.6 & 1.3 & 1592 & 9561 & 2.4 \\ chain mass & 0.8 & 0.8 & 1.6 & 2209 & 3208 & 2.1 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Online computation times4, comparing the proposed method with online NLP solver acados [2]. Our method is three orders of magnitude faster than solving the MPC problem (5) and as fast as a single iteration of the solver (that does not guarantee constraint satisfaction).

\begin{table}
\begin{tabular}{l c c|c c c} \hline \hline  & \multicolumn{2}{c}{**[\%] of safe**} & \multicolumn{2}{c}{reason for applying the safe} \\  & \multicolumn{2}{c}{**simulations**} & \multicolumn{2}{c}{candidate} & \multicolumn{2}{c}{candidate} & \multicolumn{2}{c}{terminal} & \multicolumn{1}{c}{cost} \\  & naive & **safe** & & & & \\  & (NN & **ours**) & & applied & & constr. & constr. & [\%] \\  & always) & & & [\%] & [\%] & [\%] \\ \hline \hline stir tank & 99.9 & **100** & 48.8 & 0.2 & 0.0 & **99.9** \\ quadcopter & 84.8 & **100** & 8.16 & **72.5** & 2.6 & 40.6 \\ chain mass & 98.9 & **100** & 98.9 & 0.1 & **99.8** & 41.6 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Safety augmented vs. naive NN in closed loop. Starting in random initial conditions from the feasible set of the MPC, the naive NN results in constraint violations, while our proposed method is always safe. The percentage of closed loop time steps at which safety augmentation applies the safe candidate sequence and associated reason are given in gray.

Figure 2: Exemplary closed-loop simulation of naive and safety augmented NN. Naively (always) applying \(\Pi_{\text{NN}}\) (dotted line) for \(1.1\,\mathrm{s}\), the quadcopter crashes (\(\times\)) into the wall at \(x_{1}=0.145\,\mathrm{m}\). With the safety-augmented NN presented in Algorithm 1 (solid), a safe candidate is chosen over unsafe \(\Pi_{\text{NN}}\) predictions for three time steps (\(\bullet\)) and the system is guaranteed to adhere to constraints.
the NN is feasible and improves cost compared to a safe candidate sequence. This candidate sequence is constructed from the shifted previous trajectory with appended terminal controller. This approach combines the benefits of online computations (simulating system dynamics to check for safety) and offline learning (avoiding online optimization). In three numerical examples, we illustrate how our method provides safety and improves overall computation time by three orders of magnitude compared to solving NLPs online with state of the art solvers. The proposed approach enables fast control on embedded platforms, which is relevant for robotics, for example. Future work will investigate such applications. Future improvements to the proposed method could consider additional disturbances and tailored training methods to improve sample efficiency.
