weakness of this paper in its current form is clarity, which hopefully can be improved, as although successor representation is an increasingly popular area in RL, it's also fairly complicated, hence it needs to be well explained (as e.g. is done in Gershman, J. Neurosci 2018). I also have a few more technical comments: - It's not exactly clear where is reward in section 2. Tabular case and rewards being linear in the state representation are mentioned; however, how exactly this is done should be explained more explicitly (or at least referred to where it's explained in the supplementary information - currently SI has information about parameters, algorithm and task settings details, but not methodological explanations) - It is mentioned that the mixture model is learned by gradient descent - it would be nice to see further discussion about how exactly this is done and why that is biologically realistic (as gradient descent is not something typically performed in the brain). - It would be nice to see not only summary statistics, but also typical trajectories performed by the model (and other candidate models) at different stages of learning - It is mentioned that epsilon = 0 works best for BSR, but in section 4.2 it's stated that for the puddle world epsilon = 0.2 was used for all models - why is that? Normally when comparing different models/algorithms, effort should be taken to find the best performing parameters (or more generally most suitable formalisations) for each model. - What exactly is the correlation coefficient in section 5.1 (0.90 or 0.95) between? - In Fig. 4, is it possible that GPI with noise added could reproduce the data similarly well or are there other measures to show that GPI cannot have as good fit with behavioural data (e.g. behavioural trajectories? time to goal?) - Finally this approach seems to be suitable for modelling pattern separation tasks, for which there is also behavioural data available - it would be nice to have some discussion on this. - There are a number of typos throughout the paper, which although don't obscure meaning should be corrected in the final version.
- In Fig. 4, is it possible that GPI with noise added could reproduce the data similarly well or are there other measures to show that GPI cannot have as good fit with behavioural data (e.g. behavioural trajectories? time to goal?) - Finally this approach seems to be suitable for modelling pattern separation tasks, for which there is also behavioural data available - it would be nice to have some discussion on this.