Characterising representation dynamics in recurrent neural networks for
  object recognition
Recurrent neural networks (RNNs) have yielded promising results for both
recognizing objects in challenging conditions and modeling aspects of primate
vision. However, the representational dynamics of recurrent computations remain
poorly understood, especially in large-scale visual models. Here, we studied
such dynamics in RNNs trained for object classification on MiniEcoset, a novel
subset of ecoset. We report two main insights. First, upon inference,
representations continued to evolve after correct classification, suggesting a
lack of the notion of being ``done with classification''. Second, focusing on
``readout zones'' as a way to characterize the activation trajectories, we
observe that misclassified representations exhibit activation patterns with
lower L2 norm, and are positioned more peripherally in the readout zones. Such
arrangements help the misclassified representations move into the correct zones
as time progresses. Our findings generalize to networks with lateral and
top-down connections, and include both additive and multiplicative interactions
with the bottom-up sweep. The results therefore contribute to a general
understanding of RNN dynamics in naturalistic tasks. We hope that the analysis
framework will aid future investigations of other types of RNNs, including
understanding of representational dynamics in primate vision.
Sushrut Thorat, Adrien Doerig, Tim C. Kietzmann
# Characterising representation dynamics in

###### Abstract

**Recurrent neural networks (RNNs) have yielded promising results for both recognizing objects in challenging conditions and modeling aspects of primate vision. However, the representational dynamics of recurrent computations remain poorly understood, especially in large-scale visual models. Here, we studied such dynamics in RNNs trained for object classification on MiniFcoset, a novel subset of ecoset. We report two main insights. First, upon inference, representations continued to evolve after correct classification, suggesting a lack of the notion of being "done with classification". Second, focusing on "readout zones" as a way to characterize the activation trajectories, we observe that misclassified representations exhibit activation patterns with lower L2 norm, and are positioned more peripherally in the readout zones. Such arrangements help the misclassified representations move into the correct zones as time progresses. Our findings generalize to networks with lateral and top-down connections, and include both additive and multiplicative interactions with the bottom-up sweep. The results therefore contribute to a general understanding of RNN dynamics in naturalistic tasks. We hope that the analysis framework will aid future investigations of other types of RNNs, including understanding of representational dynamics in primate vision1.**

Footnote 1: This article is a revision of the 2023 Conference on Cognitive Computational Neuroscience (CCN) paper, in which we present a new analysis in the Appendix, and include suggestions made by the CCN reviewers.

**Keywords: recurrent neural networks, object recognition, neural representations, dynamics, naturalistic tasks, readout zones**

## 1 Introduction

Feedback connections are ubiquitous in brains (Felleman and Van Essen, 1991). The resulting recurrent computations are advantageous in challenging conditions such as recognizing objects in clutter (Wyatte, Jilk, O'Reilly, 2014; Kreiman and Serre, 2020) and natural scenes (Spoerer, Kietzmann, Mehrer, Charest, and Kriegeskorte, 2020). Research into the representation dynamics underlying recurrent computations is nascent but accelerating (Mante, Sussillo, Shenoy, & Newsome, 2013; Zamir et al., 2017; Quax and van Gerven, 2018; Mastrogiuseppe and Ostojic, 2018; van Bergen and Kriegeskorte, 2020; Thorat, Aldegheri, and Kietzmann, 2021; Lindsay, Mrsic-Flogel, & Sahani, 2022; Driscoll, Shenoy, & Sussillo, 2022). Moving to a more naturalistic setting, this work investigates representations and their dynamics in a deep recurrent convolutional neural network (RNN), as they contribute to improving classification responses to natural images. While we provide novel insights into temporal trajectories of the RNNs, the developed framework applies more broadly and can be applied to both, artificial and biological neural network dynamics, and hence contributes to the toolbox available to researchers interested in modelling vision with deep neural networks (Doerig et al., 2023).
## 2 Model system and dataset

In our RNN models2, lateral or local top-down connections are included (Fig. 1A). Such RNNs have been used as models of human neural dynamics and behavior (Kietzmann et al., 2019; Spoerer et al., 2020; Doerig et al., 2022). The lateral and top-down connections interacted with the bottom-up sweep through either additive or multiplicative interactions. The RNNs were unrolled for 10 timesteps. The RNNs were trained to classify the input images at each timestep (their readouts had no bias terms; see Appendix 6.1.1). The \(64\times 64\,\mathrm{px}\) RGB images were taken from MiniEcoset3, which is a novel subset of ecoset (Mehrer, Spoerer, Jones, Kriegeskorte, & Kietzmann, 2021) containing 100 object classes that follow a hierarchical object structure.

Footnote 2: The training and evaluation scripts can be found at: github.com/KietzmannLab/BLT-Pytorch-CCN23

Footnote 3: MiniEcoset can be found at: osf.io/msna2/

## 3 Analysis

We start our analyses by focusing on an RNN with lateral connections which interact with the feedforward sweep additively. Please note that these results generalize across RNN configuration (Fig. 3A).

### Learned categorical structure

We start our analysis by asking whether the RNN successfully learns the hierarchical structure encoded in the dataset statistics. To do so, we computed the similarities between the readout vectors (rows of the readout weight matrix, corresponding to connections from the final AvgPool layer to each of the readout neurons), as they can give us insight into which classes are considered similar by the RNN. Cosine similarity (\(\bar{A}\cdot\bar{B}/|\bar{A}||\bar{B}|\)) was computed between each pair of the readout vectors.

Hierarchical clustering on the pairwise similarities revealed meaningful clusters (Fig. 1B) resembling the dataset structure and the animacy organization observed in primate brains (Grill-Spector & Weiner, 2014). This suggests that our choice of architecture and dataset leads to an interpretable feature extractor.

Figure 1: (A) Architecture of the recurrent neural network. The lateral or top-down connections interact with bottom-up processing additively or multiplicatively. (B) Readout vectors capture referential semantic features of the data.
### Convergent representation dynamics

Next, we moved into analysing the representational dynamics of the RNNs, asking whether they exhibit a signature of being "done with classification", as expected in a stable RNN with attractor dynamics (Linsley, Karkada Ashok, Govindarajan, Liu, & Serre, 2020). Additionally, we asked if the changes in pre-readout representations (i.e., final AvgPool layer activations) are smaller for images that are already correctly classified as opposed to images that are not yet correctly classified. For this analysis, we focused on images that were classified correctly and consistently starting from a given timestep \(t\) (termed stable classification with \(t_{stable}=t\); we only consider these images for subsequent analyses). To define representational changes, we analysed the \(l^{2}\)-norms of the change in representations across time, as a function of \(t_{stable}\).

As seen in Fig. 2A (left), the amount of representational change did not depend on \(t_{stable}\): the changes in representations were not smaller for images that were classified correctly at earlier timesteps. However, the change in all representations did decrease with timesteps. These results indicate that although all representations "settle" across time, the rate of settling is independent of the correctness of classification. Interestingly, as seen in Fig. 2A (right), this reduction in the rate of change was also observed pre-training, suggesting these dynamics are a property of the network architecture. Finally, note that in contrast to previous findings (Linsley et al., 2020), these RNNs exhibits stable state dynamics despite them being trained with backpropagation through time (BPTT), as discussed in Appendix 6.3.

### Signatures of stable classification

Originating from the observation that, on average, representations move the same distance regardless of correct classification, we hypothesized that representations that are able to transition into another class may initially be closer to the decision boundary, whereas the ones that do not transition are initially far from the boundary (and are therefore unable to leave the current class). As we show in the Appendix 6.1, in networks with linear readouts and argmax decisions, the "readout zones", in which representations are assigned to a given class, resemble conical structures (a 2D schematic is shown in Fig. 3B). Given this structure, being closer to the decision boundary either entails having a lower L2 norm or having a lower cosine similarity with the readout vector (see Appendix 6.1.1 for further explanation). To explore this hypothesis, we assessed whether currently incorrectly-classified representations (that will eventually become correct) indeed have lower norms and/or lower cosine similarities with the readout vector of the current class.

At each timestep \(t\), we compared both properties of the representations with \(t_{stable}\leq t\) (i.e., currently

Figure 2: (A) The amount of change in representations does not depend on the correctness of classification, in both trained and random RNNs. (B) Signatures of stable correct classification: the norm of the representation and its cosine similarity to the readout vector of the current class are higher. (C) Signature of the future correct class: for currently misclassified representations, the cosine similarity to the correct class readout vector is higher.
correct) and the representations with \(t_{stable}>t\) (i.e., currently incorrect): their norms, and their cosine similarities to the current readout. As seen in Fig. 2B, both properties were smaller for \(t_{stable}<t\) than for \(t_{stable}\leq t\): the norms and cosine similarities were lower for representations that were incorrectly classified at a given timestep. As seen in Fig. 3A, these patterns (averaged across timesteps) are independent of the kind of feedback used or how it interacts with the bottom-up sweep. This confirms the hypothesis that currently incorrect representations are closer to the decision boundary.

What constrains incorrect images to be closer to the decision boundary? There are two main possibilities: either any feedforward sweep, including in a purely feedforward network, automatically projects them to this position, or the feedforward sweep is shaped by the fact that recurrent computations move representations the same distance regardless of correct classification. To answer this question, we tested if the norms of two feedforward networks could predict the how fast images are correctly classified by the RNN. They do, as can be seen in Appendix 6.2. This suggests that the requirement of the recurrent computations (representations moving out of a class should be closer to the decision boundary) are satisfied by the representations instantiated by the feedforward sweep. The reason why incorrect images are closer to the decision boundary is independent of recurrence, and the effect of recurrent computations is to move these representations from the incorrect to the correct class. What properties of the images lead to their representations being initialized closer to the decision boundary remains to be explored.

### Signatures of the correct class

We have now established that currently misclassified objects reside closer to the decision boundary (in the incorrect readout zone). Do these currently incorrectly-classified representations exhibit any signatures of their correct classes? Evidence for this would be provided if the cosine similarity of an incorrectly-classified representation to its correct class readout vector was higher than its cosine similarity to the readout vectors corresponding to the correct class of other incorrectly-classified representations in the same readout zone (see Fig. 3B for a schematic). As seen in Fig. 2C, the cosine similarity of the incorrectly-classified representations to the corresponding correct class readout vector is indeed higher than the cosine similarity to other correct classes' readout vectors. Hence, there are signatures of the correct classes in the incorrectly classified representations. This pattern (averaged across timesteps) is independent of the kind of feedback (lateral vs. top-down) and how it interacts with the bottom-up sweep (additive vs. multiplicative; Fig. 3A).

An intriguing question that arises from this is whether and how recurrent computations utilize these nascent features to correct the classification. Future work in understanding these dynamics shall consider: How do the incorrectly classified representations move through other classes to arrive at their correct classes? How do the feedback connections hierarchically (given Fig. 1B) constrain the feedforward sweep to lead to those trajectories? Are similar dynamics/representations found in biological visual systems?

## 4 Conclusions

In the RNNs studied here, the magnitude of changes in network activations are surprisingly similar across images and decrease with model timesteps. This shows that the extent of recurrent dynamics experienced by image representations does not depend on the correctness of classification. In addition, we highlight an interesting representation arrangement, presented schematically in Fig. 3B: image representations that are currently incorrectly classified (red and blue squares) have lower norms, and are closer to the current readout zone's decision boundary. The initial norm of the representation depends on the alignment of the image features with the feedforward weights, and can be seen as indicating the certainty of the network's inference after the feedforward sweep. For representations where certainty is low, recurrence can more easily move them towards the correct readout zone.

This work reported our first advances in deriving a framework for understanding representational dynamics in RNNs trained on naturalistic images, which we hope will further clarify how recurrent systems, both artificial and biological, reach their decisions. Future work should investigate the representation trajectories in other recurrent systems, including spatiotemporal data from the primate visual system.
## 5 Acknowledgments

The project was partially funded by the European Union (ERC, TIME, Project 101039524). Compute resources were funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation, Project number 456666331).
