$u_k$ is a torque applied to the steering wheel by the controller. The parameters of the corresponding continuous-time model can be found in [@Kopf.2018]. The system state ${\boldsymbol{x}}_k$ is given by $$\begin{aligned}
 \label{eq:Zustaende Fzg}
{\boldsymbol{x}}_k = \begin{bmatrix}
\beta_k & {\psi}_{r,k} & \psi_k & y_k & {\delta}_{v,k} & \delta_k
\end{bmatrix}^\intercal,\end{aligned}$$ with sideslip angle $\beta_k$, yaw angle $\psi_k$, yaw rate ${\psi}_{r,k}$, lateral deviation from the origin $y_k$, steering wheel angle $\delta_k$ and angular velocity ${\delta}_{v,k}$. Geometric relations are depicted in Fig. \[fig:einspurmodell\], where $v$ is the constant velocity ($\SI{20}{\meter\per \second}$ in our example) and $\delta_s = 0.0625\delta$ denotes the steering angle in contrast to the steering wheel angle $\delta$.

As we desire to track the lateral position $x_{4,\text{ref}} = y_{\text{ref}}$, we choose $$\begin{aligned}
{\boldsymbol{Q}}=\text{diag}(0, 0, 0, 100, 0, 0) \text{ and } {\boldsymbol{R}}=1.\end{aligned}$$

Evaluation Method
-----------------

In order to compare our method with the class of RL tracking algorithms where the reference trajectory is assumed to be generated by a time-invariant exo-system ${\boldsymbol{f}}_{\text{ref}}({\boldsymbol{r}}_k)={\boldsymbol{F}}_{\text{ref}}{\boldsymbol{r}}_k$, let both the reference angle $\alpha_{\text{ref}}$ (system 1) and the reference lateral position $y_{\text{ref}}$ (system 2) be generated by $$\begin{aligned}
\label{eq:refe}
{\boldsymbol{r}}_{k+1} &= \underbrace{{\begin{bmatrix*}[r]0.9801&0.1987\\-0.1987&0.9801
\end{bmatrix*}}}_{{\boldsymbol{F}}_{\text{ref}}}{\boldsymbol{r}}_k, \\
\alpha_{\text{ref}} &= y_{\text{ref}} = {\begin{bmatrix*}[r]1&0
\end{bmatrix*}}{\boldsymbol{r}}_k\label{eq:refe1}\end{aligned}$$ during the training procedure. For comparison reasons, 