Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map. The precise form of this representation is often considered to be a metric representation of space. An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks. Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment. To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output. We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state. These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities. Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods. Furthermore, we demonstrate that non-metric representations are useful for navigation tasks.   Spatial navigation is an important task that requires a correct internal representation of the world, and thus its mechanistic underpinnings have attracted the attention of scientists for a long time (O'Keefe & Nadel, 1978) . A standard tool for navigation is a euclidean map, and this naturally leads to the hypothesis that our internal model is such a map. Artificial navigation also relies on SLAM (Simultaneous localization and mapping) which is based on maps (Kanitscheider & Fiete, 2017a) . On the other hand, both from an ecological view and from a pure machine learning perspective, navigation is firstly about reward acquisition, while exploiting the statistical regularities of the environment. Different tasks and environments lead to different statistical regularities. Thus it is unclear which internal representations are optimal for reward acquisition. We take a functional approach to this question by training recurrent neural networks for navigation tasks with various types of statistical regularities. Because we are interested in internal representations, we opt for a two-phase learning scheme instead of end-to-end learning. Inspired by the biological phenomena of evolution and development, we first pre-train the networks to emphasize several aspects of their internal representation. Following pre-training, we use Q-learning to modify the network's readout weights for specific tasks while maintaining its internal connectivity. We evaluate the performance for different networks on a battery of simple navigation tasks with different statistical regularities and show that the internal representations of the networks manifest in differential performance according to the nature of tasks. The link between task performance and network structure is understood by probing networks' dynamics, exposing a low-dimensional manifold of slow dynamics in phase space, which is clustered into three major categories: continuous attractor, discrete attractors, and unstructured chaotic dynamics. The different network attractors encode different priors, or inductive bias, for specific tasks which corresponds to metric or topology invariances in the tasks. By combining networks with different inductive biases we could build a modular system with improved multiple-task learning. Overall we offer a paradigm which shows how dynamics of recurrent networks implement different priors for environments. Pre-training, which is agnostic to specific tasks, could lead to dramatic difference in the network's dynamical landscape and affect reinforcement learning of different navigation tasks. Our work explores how internal representations for navigation tasks are implemented by the dynamics of recurrent neural networks. We show that pre-training networks in a task-agnostic manner can shape their dynamics into discrete fixed points or into a low-D manifold of slow points. These distinct dynamical objects correspond to landmark memory and spatial memory respectively. When performing Q learning for specific tasks, these dynamical objects serve as priors for the network's representations and shift its performance on the various navigation tasks. Here we show that both plane attractors and discrete attractors are useful. It would be interesting to see whether and how other dynamical objects can serve as inductive biases for other domains. In tasks outside of reinforcement learning, for instance, line attractors were shown to underlie network computations (Mante et al., 2013; Maheswaranathan et al., 2019 ). An agent that has to perform several navigation tasks will require both types of representations. A single recurrent network, however, has a trade-off between adapting to one type of task or to another. The attractor landscape picture provides a possible dynamical reason for the tradeoff. Position requires a continuous attractor, whereas stimulus memory requires discrete attractors. While it is possible to have four separated plane attractors, it is perhaps easier for learning to converge to one or the other. A different solution to learn multiple tasks is by considering multiple modules, each optimized for a different dynamical regime. We showed that such a modular system is able to learn multiple tasks, in a manner that is more flexible than any single-module network we could train. Pre-training alters network connectivity. The resulting connectivity is expected to be between random networks (Lukoševičius & Jaeger, 2009) and designed ones (Burak & Fiete, 2009) . It is perhaps surprising that even the untrained RandNet can perform some of the navigation tasks using only Qlearning of the readout (with appropriate hyperparameters, see Tables 2,3 and section 4 "Linking dynamics to connectivity" in Appendix). This is consistent with recent work showing that some architectures can perform various tasks without learning (Gaier & Ha, 2019) . Studying the connectivity changes due to pre-training may help understand the statistics from which to draw better random networks (Appendix section 4). Apart from improving the understanding of representation and dynamics, it is interesting to consider the efficiency of our two-stage learning compared to standard approaches. We found that end-toend training is much slower, cannot learn topological tasks and has weaker transfer between tasks (See Appendix section 5.2). Thus it is interesting to explore whether this approach could be used to accelerate learning in other domains, similar to curriculum learning (Bengio et al., 2009
Task agnostic pre-training can shape RNN's attractor landscape, and form diverse inductive bias for different navigation tasks    This paper studies the internal representations of recurrent neural networks trained on navigation tasks, and finds that RNNs pre-trained to use path integration contain 2D continuous attractors while RNNs pre-trained for landmark memory contain discrete attractors. This paper explores how pre-training recurrent networks on different navigational objectives confers different benefits for solving downstream tasks, and shows how different pretraining manifests as different dynamical structures in the networks after pre-training.