dynamical systems span a behavioral space that provides a rich substrate for adaptive behavior.

Fluid Adaptivity as a Challenge for DRL
=======================================

We have considered finding stepping stones in a behavioral space in an EA approach. Again, these ideas are finding now their way into DRL and it shows that they can be successfully leveraged to address more complex problems. Maintaining diverse and multiple solutions concomitantly active appears beneficial with respect to changes in the environment. This has already been successfully incorporated into reinforcement learning. Hierarchical approaches are applied in meta-learning and transfer learning settings. These aim for generalization between different, but related tasks and try to reuse existing learned structure between tasks. Importantly, the structure of a hierarchy allows to not only converge towards one single solution, but to divide the problem and maintain different solutions that can be activated when appropriate (similar to population-based approaches [@DBLP:journals/corr/abs-1807-01281] in which a whole population of different behavioral strategies is evolved). But still, these approaches typically require quite a lot of training and are therefore mostly applied in dynamic simulations of the robotic agent. The problem of simply dealing with a variable real-world setting in a sample-efficient way still remains a challenge. 