the maze changes to a condition where they would be again relevant. In contrast, biological agents can handle dynamic environments that in themselves are constantly acting and reacting. For example, in Huh et al. [@huh_model-based_2009] rats were trained in a free-choice task in a maze with multiple targets that probabilistically provided a reward. In one condition, the probability for a reward increased for the unchosen target. The animals learned this change of probabilities. Importantly, this could not be accounted for by a simple reinforcement learning approach that learns a sequence of decisions. But it was found better accounted for using a model-based reinforcement learning approach. Similar results have been obtained in research on reversal learning, where a learning agent first learns a set of appropriate behaviors for a given set of situations and then, suddenly, has to show different associations of learned behaviors to the set of situations, in order to reach behavioral goals. While standard RL agents have to be re-trained entirely in such reversal scenarios, biological systems are able to remap hierarchically organized structures of representations apparently created during initial training, and solve subsequent reversal problems much more effectively [@jarvers_reversal_2016]. There is evidence that biological brains solve such 