sequence is upper bounded by ${\boldsymbol{Y}}$ according to Lemma \[lemma:upper\_bounded\] and ${\boldsymbol{0}}\preceq{\boldsymbol{H}}^{(i)}\preceq{\boldsymbol{H}}^{(i+1)}$, the limit ${\boldsymbol{H}}^{(\infty)}$ exists, i.e. the value iteration converges to $$\begin{aligned}
    {\boldsymbol{H}}^{(\infty)}={\boldsymbol{G}}+\gamma{\boldsymbol{M}}\left({\boldsymbol{L}}\left({\boldsymbol{H}}^{(\infty)}\right)\right)^\intercal{\boldsymbol{H}}^{(\infty)}{\boldsymbol{M}}\left({\boldsymbol{L}}\left({\boldsymbol{H}}^{(\infty)}\right)\right).
    \end{aligned}$$ Furthermore, ${\boldsymbol{L}}\left({\boldsymbol{H}}^{(\infty)}\right)$ minimizes $\hat{Q}_{0}^{(\infty)}$ according to Lemma \[lemma:hilfslemma\]. Thus, $$\begin{aligned}
    \lim_{i\rightarrow \infty}\epsilon_k^{(i)}&=c_k+\gamma{{\boldsymbol{z}}_{k+1}^{*}}^\intercal{\boldsymbol{H}}^{(\infty)}{\boldsymbol{z}}_{k+1}^{*}\nonumber \\ &-{\boldsymbol{z}}_k^\intercal \, F\left({\boldsymbol{H}}^{(\infty)}, {\boldsymbol{L}}\left({\boldsymbol{H}}^{(\infty)}\right)\right){\boldsymbol{z}}_k = 0.
    \end{aligned}$$ Therefore, ${\boldsymbol{H}}^{(i)}\rightarrow{\boldsymbol{H}}^{(\infty)}={\boldsymbol{H}}$, i.e. ${\boldsymbol{w}}^{(i)}\rightarrow{\boldsymbol{w}}^*$ and ${\boldsymbol{L}}^{(i)}\rightarrow{\boldsymbol{L}}^*$ which completes the proof.

Results {#simulationResults}
=======

\[sectionsimulation\] In order to validate our proposed method, we show results for two LQ-tracking problems with unknown system dynamics. We furthermore compare our results with an RL tracking method which assumes that the reference can be described by a time-invariant exo-system ${\boldsymbol{f}}_{\text{ref}}$.

For excitation purposes (cf. ), the variance of the input excitation ${\boldsymbol{\xi}}$ in is set to ${\boldsymbol{\Sigma}}=0.1$ and Gaussian noise $\xi_{\text{ref}}\sim \mathcal{N}\left({0},0.1\right)$ is added to the reference trajectory which we intend to track while learning is active. We furthermore choose the number of data tuples to $M=1.2L$, with $L$ according to , the stopping criterion to $e_{{\boldsymbol{w}}}=\SI{1e-6}{}$, the discount factor to $\gamma = 0.9$, the horizon on which the reference is known to $N=10$ and finally ${\boldsymbol{x}}_0= {\boldsymbol{0}}$ for all experiments.

Simulation Examples
-------------------

The first system is a second-order rotatory mass-spring-damper system, whereas the second system is a sixth-order 