Robotics, Karlsruhe Institute of Technology, Karlsruhe, Germany'
author:
- Florian Köpf
- Johannes Westermann
- Michael Flad
- Sören Hohmann
bibliography:
- 'mybib.bib'
title: 'Adaptive Optimal Control for Reference Tracking Independent of Exo-System Dynamics'
---

Adaptive Dynamic Programming ,Optimal Tracking ,Reinforcement Learning ,Learning Systems ,Machine Learning ,Artificial Intelligence ,Optimal Control

Introduction
============

Reinforcement Learning (RL) has recently gained extensive attention as a model-free adaptive optimal control method which learns optimal behavior from interaction with the environment and observations of resulting states and rewards (see e.g. [@Sutton.2018; @Wang.2017; @Luo.2017] and references therein).

The existing control-oriented approaches can be classified in two groups: Some methods focus on the regulation problem, i.e. steering the system towards an equilibrium point (typically ${\boldsymbol{0}}$), e.g. [@Kamalapurkar.2016; @Luo.2017; @Vamvoudakis.2017; @Wang.2016; @Wei.2017]. The second group is devoted to a specific tracking case, where the reference ${\boldsymbol{r}}_k$ to be tracked is generated by an exo-system, i.e. it is assumed that there exists ${\boldsymbol{r}}_{k+1}={\boldsymbol{f}}_{\text{ref}}({\boldsymbol{r}}_k)$.

It is noteworthy that a controller that has learned to solve the regulation problem is not directly applicable to the tracking case as the reference trajectory influences the associated rewards that the RL agent is facing and therefore affects the value function which has to be 