learned.

In order to extend learning-based controllers to the tracking case, various RL-based tracking controllers belonging to the second group of approaches have been proposed recently. For unknown internal system dynamics but a known control coefficient matrix, adaptive tracking controllers approximating the value function are proposed for discrete-time [@Dierks.2009] and continuous-time [@Modares.2014; @Zhang.2017] systems. Notable critic-only Q-learning methods for completely unknown dynamics are proposed in [@Luo.2016] and [@Kiumarsi.2014], where [@Kiumarsi.2014] focuses on the linear-quadratic setting and [@Luo.2016] allows nonlinear system dynamics. Other works combine system identification and adaptive schemes including past information [@Lv.2017], use filter-based goal representation Heuristic Dynamic Programming for tracking [@Mu.2017b], focus on learning a tracking controller from input-output data rather than assuming full state information [@Kiumarsi.2015b], consider the nonzero-sum game case [@Kopf.2018], utilize the exo-system of a prescribed robot impedance model in order to learn a model-following behavior for assistive human-robot interaction [@Modares.2016], focus on systems with matched uncertainties [@Mu.2017] or consider tracking on an infinite horizon with unbounded cost [@Bernhard.2018].

All of these methods [@Dierks.2009; @Kiumarsi.2014; @Kiumarsi.2015b; @Kopf.2018; @Luo.2016; @Lv.2017; @Modares.2014; @Modares.2016; @Mu.2017; @Mu.2017b; @Zhang.2017; @Bernhard.2018] consider the case where the reference trajectory is generated by a time-invariant exo-system ${\boldsymbol{f}}_{\text{ref}}$, i.e. rely on the assumption that 