by suitably designed flexibly interacting modules.'
author:
- 'Malte Schilling$^{1}$, Helge Ritter$^{1}$, Frank W. Ohl$^{2}$[^1][^2][^3]'
bibliography:
- 'References.bib'
title: '**From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement Learning — Insights from Biological Systems on Adaptive Flexibility** '
---

Introduction
============

Adaptive behavior in many animals and humans is still unparalleled by current artificial intelligence (AI) control approaches. While deep learning (DL) or evolutionary algorithm (EA) approaches produce viable solutions in strictly controlled settings, they are still struggling in several characteristic scenarios. A first such scenario is when existing DL and EA approaches face variability arising from complex situations in which numerous degrees of freedom have to be considered [@neftci_reinforcement_2019; @sutton2019]. A second typical scenario is the occurrence of unpredicted changes in the task or environments that require higher levels of behavioral flexibility and adaptivity — in natural contexts often on quite fast time scales as compared to the learning dynamics itself. While current AI approaches often excel at finding solutions in narrowly defined problem spaces, and here can outperform biological systems (for example [@Silver_2016]), these do typically not transfer to more natural, open and dynamic scenarios; this also typically includes the step from simulation to real physics. Generally, research 