of the system state ${\boldsymbol{x}}_i$ from the reference ${\boldsymbol{r}}_i$ at time step $i$. ${\boldsymbol{Q}}\in\mathbb{R}^{n\times n}$, ${\boldsymbol{Q}}={\boldsymbol{Q}}^\intercal\succeq{\boldsymbol{0}}$ is a symmetric, positive semidefinite matrix penalizing deviations of the state ${\boldsymbol{x}}_i$ from the reference ${\boldsymbol{r}}_i$, ${\boldsymbol{R}}\in\mathbb{R}^{m\times m}$ and ${\boldsymbol{R}}={\boldsymbol{R}}^\intercal\succ{\boldsymbol{0}}$ is a symmetric, positive definite matrix penalizing the control effort. Furthermore, [${\gamma \in\left[0,1\right)}$]{} a discount factor and $c_i \in \mathbb{R}$ denotes the one-step cost.

The choice of our cost function is motivated as follows: On one hand, the reference to be tracked is usually known only on a finite horizon $N$ (e.g. a road course). On the other hand, the Q-learning method that we use in Section \[section\_learning\] relies on an infinite horizon cost function which allows the efficient formulation of a Bellman-like equation. Thus, this cost function accounts for a given reference of finite horizon $N$ and enables stability and convergence of the associated RL algorithm due to the infinite horizon.

Therefore, our problem definition can be summarized in Problem \[problem1\].

\[problem1\] At each time step $k$, find the optimal control input ${\boldsymbol{u}}_k^*$ and apply it to system , where ${\boldsymbol{u}}_k^*, {\boldsymbol{u}}_{k+1}^*, \dots$ is the control sequence minimizing the discounted cost subject to the system dynamics , where ${\boldsymbol{A}}$ and ${\boldsymbol{B}}$ are unknown, 