- It is unclear to me what scientific insight we get from this model and formalism over the prior task-optimized approaches. For instance, this model (as formulated in Section 2.3) is not shown to be a prototype approximation to these non-linear RNN models that exhibit emergent behavior. So it is not clear that your work provides any further “explanation” as to how these nonlinear models attain such solutions purely through optimization on a task. - Furthermore, I am not really sure how “emergent” the hexagonal grid patterns really are in this model. Given partitioning of the generator matrices into blocks in Section 2.5, it almost seems by construction we would get hexagonal grid patterns and it would be very hard for the model to learn anything different.
While the ideas of this paper are mathematically elegant, I do not see the added utility these models provide over prior approaches nor how they provide a deeper explanation of the surprising emergent grid firing patterns observed in task-optimized nonlinear RNNs. For these reasons, I recommend rejection.
- It is unclear to me what scientific insight we get from this model and formalism over the prior task-optimized approaches. For instance, this model (as formulated in Section 2.3) is not shown to be a prototype approximation to these non-linear RNN models that exhibit emergent behavior. So it is not clear that your work provides any further “explanation” as to how these nonlinear models attain such solutions purely through optimization on a task.