problems by an efficiently structured communication between separable brain modules that implement RL strategies and those that implement knowledge representation [@schulz2016]. These neuronal strategies in biological systems are reminiscent of model-based RL approaches. The general idea of model-based reinforcement learning is to “decompose the RL problem into two subproblems: learning a model of the environment, and then planning with this model” [@silver_predictron:_2016]. Model-based reinforcement learning has become a prominent approach. We will not consider this approach in detail here, but want to remark, as a side note, that the view of dividing the problem into two subproblems will suffer from similar shortcomings we discussed before. When trying to learn the interaction with the environment as one holistic model, this will require a long time as there are many dimensions to this problem. Such a model would probably try to capture too much variability.

Alternatively, we propose to follow a more modular approach in which models are learned in the environment and in which the focus is on the compositional semantics of how these modules can interact. Note that such an approach takes a different perspective on reinforcement learning that is breaking up the standard and simple notion of an agent 