Aligned and oblique dynamics in recurrent neural networks
The relation between neural activity and behaviorally relevant variables is
at the heart of neuroscience research. When strong, this relation is termed a
neural representation. There is increasing evidence, however, for partial
dissociations between activity in an area and relevant external variables.
While many explanations have been proposed, a theoretical framework for the
relationship between external and internal variables is lacking. Here, we
utilize recurrent neural networks (RNNs) to explore the question of when and
how neural dynamics and the network's output are related from a geometrical
point of view. We find that training RNNs can lead to two dynamical regimes:
dynamics can either be aligned with the directions that generate output
variables, or oblique to them. We show that the choice of readout weight
magnitude before training can serve as a control knob between the regimes,
similar to recent findings in feedforward networks. These regimes are
functionally distinct. Oblique networks are more heterogeneous and suppress
noise in their output directions. They are furthermore more robust to
perturbations along the output directions. Crucially, the oblique regime is
specific to recurrent (but not feedforward) networks, arising from dynamical
stability considerations. Finally, we show that tendencies towards the aligned
or the oblique regime can be dissociated in neural recordings. Altogether, our
results open a new perspective for interpreting neural activity by relating
network dynamics and their output.
Friedrich Schuessler, Francesca Mastrogiuseppe, Srdjan Ostojic, Omri Barak
# Aligned and oblique dynamics in recurrent neural networks

###### Abstract

The relation between neural activity and behaviorally relevant variables is at the heart of neuroscience research. When strong, this relation is termed a neural representation. There is increasing evidence, however, for partial dissociations between activity in an area and relevant external variables. While many explanations have been proposed, a theoretical framework for the relationship between external and internal variables is lacking. Here, we utilize recurrent neural networks (RNNs) to explore the question of when and how neural dynamics and the network's output are related from a geometrical point of view. We find that RNNs can operate in two regimes: dynamics can either be aligned with the directions that generate output variables, or oblique to them. We show that the magnitude of the readout weights can serve as a control knob between the regimes. Importantly, these regimes are functionally distinct. Oblique networks are more heterogeneous and suppress noise in their output directions. They are furthermore more robust to perturbations along the output directions. Finally, we show that the two regimes can be dissociated in neural recordings. Altogether, our results open a new perspective for interpreting neural activity by relating network dynamics and their output.
## 1 Introduction

The relation between neural activity and behavioral variables is often expressed in terms of neural representations. Sensory input and motor output have been related to the tuning curves of single neurons [21, 24, 41] and, since the advent of large-scale recordings, to population activity [6, 57, 73]. Both input and output can be decoded from population activity, [10, 37], even in real time, closed-loop settings [56, 74]. However, neural activity is often not fully explained by observable behavioral variables. Some components of the unexplained neural activity have been interpreted as random trial-to-trial fluctuations [15], potentially linked to unobserved behavior [40, 65]. Activity may further be due to other ongoing computations not immediately related to behavior, such as preparatory motor activity in a null-space of the motor readout [22, 29]. Finally, neural activity may partially be due to other constraints, for example related to the underlying connectivity [2, 44], the process of learning [56], or stability, i.e., the robustness of the neural dynamics to perturbations [55].

Here we aim for a theoretical understanding of neural representations: What determines how strongly activity and behavioral output variables are related? To this end, we use trained recurrent neural networks (RNNs). In this setting, output variables are determined by the task at hand, and neural activity can be described by its projection onto the principal components (PCs). We show that these networks can operate between two extremes: an "aligned" regime in which the output weights and the largest PCs are strongly correlated. In the second, "oblique" regime, the output weights and the largest PCs are poorly correlated.

What determines the regime a network operates in? We show that quite general considerations lead to a link between the magnitude of output weights and the regime of the network. As a consequence, we can use output magnitude as a control knob for trained RNNs. Indeed, when we train RNN models on different neuroscience tasks, large output weights led to oblique dynamics, and small output weights to aligned dynamics.

We then considered the functional consequences of the two regimes. Building on the concept of feedback loops driving the network dynamics [50, 68], we show that in the aligned regime, the largest PCs and the output are qualitatively similar. In the oblique regimes, in contrast, the two may be qualitatively different. This functional decoupling in oblique networks leads to large freedom for the neural dynamics. Different networks with oblique dynamics thus tend to employ different dynamics for the same tasks. Aligned dynamics, in contrast, are much more stereotypical. Furthermore, as a result of how neural dynamics and output are coupled, oblique and aligned networks react differently to perturbations of the neural activity along the output direction. Further, oblique (but not aligned) networks develop an additional negative feedback loop that suppresses output noise. We finally link our theoretical results to experimental data by showing that these different regimes can be identified in neural recordings from several experiments.

Altogether, our work opens a new perspective relating network dynamics and their output, yielding both important insights for modeling brain dynamics as well as experimentally accessible questions about learning and dynamics in the brain.
## 2 Results

### Aligned and oblique population dynamics

We consider an animal performing a task while both behavior and neural activity are recorded. For example, the task might be to produce a periodic motion, described by the output \(z(t)\) of Fig. 1A. For simplicity, we assume that the behavioral output can be decoded linearly from the neural activity [18, 37, 54, 56, 74]. We can thus write

\[z(t)=\sum_{i=1}^{N}w_{\text{out},i}\,x_{i}(t)=\mathbf{w}_{\text{out}}^{T} \mathbf{x}(t)\,, \tag{1}\]

with readout weights \(\mathbf{w}_{\text{out}}\). The activity of neuron \(i\in\{1,\,\dots,\,N\}\) is given by \(x_{i}(t)\), and we refer to the vector \(\mathbf{x}\) as the state of the network.

Neural activity has to generate the output in some subspace of the state space, where each axis represents the activity of one neuron. In the simplest case (Fig. 1B, top), the output is produced along the largest PCs of activity, as shown by the fact that projecting the neural activity \(\mathbf{x}(t)\) onto the largest PCs returns the target oscillation (Fig. 1D, top). We call such dynamics "aligned" because of the alignment between the subspace spanned by the largest PCs and the output vector (red).

There is, however, another possibility. Neural activity may have many other components not directly related to the output, and these other components may even dominate the overall activity. In this case (Fig. 1B-D, bottom), the two largest PCs are not enough to read out the output, and smaller PCs are needed. We call such dynamics "oblique", because the subspace spanned by the largest PCs and the output vector are poorly aligned.

We consider these two possibilities as distinct regimes, noting that intermediate solutions are also possible. The actual regime of neural dynamics has important consequences for how one interprets

Figure 1: Schematic of aligned and oblique dynamics in recurrent neural networks. **A** Output generated by both networks. **B** Neural activity of aligned (top) and oblique (bottom) dynamics, visualized in the space spanned by three neurons. Here, the activity (green) is three-dimensional, but most of the variance is concentrated along the two largest PCs (blue). For aligned dynamics, the output weights (red) are small and lie in the subspace spanned by the largest PCs; they are hence correlated to the activity. For oblique dynamics, the output weights are large and lie outside of the subspace spanned by the largest PCs; they are hence poorly correlated to the activity. **C** Projection of activity onto the two largest PCs. For oblique dynamics, the output weights are orthogonal to the leading PCs. **D** Evolution of PC projections over time. For aligned dynamics, the projection on the PCs resembles the output \(z(t)\), and reconstructing the output from the largest two components is possible. For the oblique dynamics, such reconstruction is not possible, because the projections oscillate much more slowly than the output.
neural recordings. For aligned dynamics, analyzing the dynamics within the largest PCs may lead to insights about the computations generating the output [73]. For oblique dynamics, such an analysis is hampered by the dissociation between the large PCs and the components generating the output [54].

### Magnitude of output weights controls regime

What determines which regime the network operates in? To probe this, we directly use the notion of alignment between output weights and states. For this, we assume that the output weights are known, and compute their correlation \(\rho\), to the states:

\[\rho(t)=\mathbf{w}_{\text{out}}^{T}\mathbf{x}(t)\,/\left(\left\|\mathbf{w}_{ \text{out}}\right\|\left\|\mathbf{x}(t)\right\|\right). \tag{2}\]

For aligned dynamics, the correlation is large in magnitude, corresponding to the alignment between the large PCs of the neural activity and the output weights (Fig. 1B top). In contrast, for oblique dynamics, this correlation is small (Fig. 1B bottom). Note that the concept of correlation can be generalized to accommodate multiple time points and multidimensional output (see Section 4.3).

This allows us to express the output in terms of this correlation:

\[z(t)=\rho(t)\left\|\mathbf{w}_{\text{out}}\right\|\left\|\mathbf{x}(t)\right\|. \tag{3}\]

This factorization into correlation and magnitudes (as quantified by vector norms) has one immediate consequence. We consider a situation in which a network learns to generate output by adapting its dynamics for given, fixed output weights. The magnitude of the output \(\left\|z\right\|\) is set by the definition of the task. We moreover assume that the magnitude of the activity \(\left\|\mathbf{x}\right\|\) is constrained - which we show

Figure 2: Aligned and oblique dynamics for a cycling task [54]. **A** network with two outputs needed to generate either clockwise or anticlockwise rotations, depending on the context (top). Our model RNN (bottom) received a context input pulse, generated dynamics \(\mathbf{x}(t)\) via recurrent weights \(W\), and yielded output as linear projections of the states. We trained the recurrent weights \(W\) with gradient descent. **B-C** Resulting internal dynamics for two networks with small (top) and large (bottom) output weights, corresponding to aligned and oblique dynamics, respectively. **B** Dynamics projected on the first 2 PCs and the remaining direction \(\mathbf{w}_{\text{out},\perp}\) of the first output vector (for \(z_{1}\)). The output weights are amplified to be visible. Arrowheads indicate the direction of the dynamics. Note that for the large output weights, the dynamics in the first two PCs co-rotated, despite the counter-rotating output. **C** Output reconstructed from the largest PCs, with dimension \(D=2\) (full lines) or \(8\) (dotted). Two dimensions already yield a fit with \(R^{2}=0.99\) for aligned dynamics (top), but almost no output for oblique (bottom, \(R^{2}=0.005\), no arrows shown). For the latter, a good fit with \(R^{2}>90\%\) is only reached with \(D=8\).
below is the case for robust RNNs in presence of noise. In this case, correlation and output weight magnitude must compensate each other. For aligned dynamics, we expect small output weights, because these suffice to generate the output via strongly correlated activity. For oblique dynamics, we need large output weights to amplify small correlation (Fig. 1B bottom).

We tested whether output weights can serve as a control knob to select dynamical regimes using an RNN model trained on an abstract version of the cycling task introduced in Ref. [54]. The networks were trained to generate a 2D signal that rotated in the plane spanned by two outputs \(z_{1}(t)\) and \(z_{2}(t)\) (Fig. 2A). An input pulse at the beginning of each trial indicated the desired direction of rotation. We set up two models with either small or large output weights, and trained the recurrent weights of each with gradient descent.

After both models learned the task (Section 4.1), we projected the network activity into a three-dimensional space spanned by the two largest PCs of the dynamics \(\mathbf{x}(t)\). A third direction, \(\mathbf{w}_{\text{out},\perp}\), spanned the remaining part of the first output vector \(\mathbf{w}_{\text{out},1}\). The resulting plots, Fig. 2B, corroborate our hypothesis: Small output weights led to aligned dynamics with large correlation between the largest PCs and the output weights. In contrast, the large output weights of the second network were almost orthogonal, or oblique, to the two leading PCs. Further qualitative differences between the two solutions in terms of the direction of trajectories will be discussed below.

Another way to quantify these regimes is by the ability to reconstruct the output from the large PCs of neural activity, as quantified by the coefficient of determination \(R^{2}\). For the aligned network, the projection on the two largest PCs (Fig. 2C, solid) already led to a good reconstruction. For the oblique

Figure 3: Testing how the magnitude of output weights determines regimes across multiple neuroscience tasks [54, 52, 37, 69]. **A** Correlation and norms of output weights and neural activity. For each task, we initialized networks with small or large output weights (dark vs light orange). The initial norms \(\|\mathbf{w}_{\text{out}}\|\) are indicated by the dashed lines. Learning does not change the norm dramatically. Note that all y-axes are logarithmically scaled. **B** Variance of \(\mathbf{x}\) explained and \(R^{2}\) of reconstructed output for projections of \(\mathbf{x}\) on increasing number of PCs. Results from one example network trained on the cycling task for each condition are shown. **C** Number of PCs necessary to reach 90% of the variance of \(\mathbf{x}(t)\) or of the \(R^{2}\) of the output reconstruction (top/bottom; dotted lines in **B**). In **A, C**, violin plots show the distribution over 5 sample networks, with vertical bars indicating the mean and the extreme values (where visible).
networks, the two largest PCs were not sufficient, and we needed eight dimensions (Fig. 2C, dashed) to obtain a good reconstruction (\(R^{2}>0.9\)). In contrast to the differences in these fits, the neural dynamics themselves were much more similar between the networks. Specifically, \(90\%\) of the variance was explained by 4 and 5 dimensions for the aligned and oblique networks, respectively.

Can we use the output weights to induce aligned or oblique dynamics in more general settings? We trained RNN models with small or large initial output weights on five different neuroscience tasks. All weights (input, recurrent, and output) were trained using the Adam algorithm (Section 4.1). After training, we measured the three quantities of Eq. (3): magnitudes of neural activity and output weights, and the correlation between the two. The results in Fig. 3A show that across tasks, initialization with large output weights led to oblique solutions (small correlation), and with small output weights to aligned solutions (large correlation). The output weights magnitude generally increased during training if it was initially small, but the initial difference in scale largely remained.

In Fig. 3B-C, we adopted the perspective of Fig. 2C and quantified how well we can reconstruct the output from a projection of \(\mathbf{x}\) onto its largest \(D\) PCs. As expected, for an increasing number \(D\), both the variance of \(\mathbf{x}\) explained and the quality of output reconstruction increased (Fig. 3B). The manner in which both quantities increased, however, differs between the two regimes. While the variance explained increased similarly in both cases, the quality of the reconstruction increased much more slowly for the model with large output weights. We quantified this phenomenon by comparing the dimensions at which either the variance of \(\mathbf{x}\) explained or \(R^{2}\) reaches \(90\%\), denoted by \(D_{x,90}\) and \(D_{\mathrm{fit},90}\), respectively.

In Fig. 3C, we compare \(D_{x,90}\) and \(D_{\mathrm{fit},90}\) across multiple networks and tasks. Generally, larger output weights led to larger numbers for both. However, the number of PCs necessary to obtain a good reconstruction increased much more drastically for large output weights than the dimension of the data. Thus, the output was less well represented by the large PCs of the dynamics for networks with large output weights, in accordance with our notion of oblique dynamics.

Importantly, reaching the aligned and oblique regimes relies on ensuring robust and stable solutions, which we achieve by adding noise to the dynamics during training. This yields a similar magnitude of neural activity \(\|\mathbf{x}\|\) across networks and tasks (Fig. 3A). We show in Methods, Section 4.6, that learning in simple, noise-free conditions with large output weights can lead to solutions not captured by either aligned or oblique dynamics; those solutions, however, are unstable. Furthermore, we observed that some of the qualitative differences between aligned and oblique dynamics are less pronounced if we initialized networks with small recurrent weights and initially decaying dynamics (Fig. 18).

Figure 4: Variability between learners for the two regimes. **A-B** Examples of networks trained on the cycling task with small (aligned) or large (oblique) output weights. The top left and central networks, respectively, are the same as those plotted in Fig. 2. **C** Dissimilarity between solutions across different tasks. Aligned solutions (dark) were less dissimilar to each other than oblique ones (light). The violin plots show the distribution over all possible different pairs for five samples (mean and extrema as bars).
### Neural dynamics decouple from output for the oblique regime

What are the functional consequences of the two regimes? A hint might be seen in an intriguing qualitative difference between the aligned and oblique solutions for the cycling task in Fig. 2. For the aligned network, the two trajectories for the two different contexts (green and purple) are counter-rotating (Fig. 2B, top). This agrees with the output, which also counter-rotates as demanded by the task (Fig. 2A). In contrast, the neural activity of the oblique network _co-rotates_ in the leading two PCs (Fig. 2B, bottom). This is despite the counter-rotating output, since this network also solves the task (not shown). This also indicates why reconstructing the output from the leading two PCs is not possible (Fig. 2C). Taken together, aligned and oblique dynamics differ in the coupling between leading neural dynamics and output. For aligned dynamics, the two are strongly coupled. For oblique dynamics, the two decouple qualitatively.

Such a decoupling for oblique, but not aligned, dynamics leads to a prediction regarding the universality of solutions [35, 45, 72]. For aligned dynamics, the coupling implies that the internal dynamics are strongly constrained by the task. We thus expect different learners to converge to similar solutions, even if their initial connectivity is random and unstructured. In Fig. 4A, we show the dynamics of three randomly initialized aligned networks trained on the cycling task, projected onto the three leading PCs. Apart from global rotations, the dynamics in the three networks are very similar.

For oblique dynamics, the task-defined output exerts weaker constraints on the internal dynamics. Any variability experienced during learning can potentially build up, and eventually create qualitatively different solutions. Three examples of oblique networks solving the cycling tasks indeed show visibly different dynamics (Fig. 4B). Further analysis shows that the models also differ in the frequency components in the leading dynamics (Fig. 17).

The degree of variability between learners depends on the task. The differences observable in the PC projections were most striking for the cycling task. For the flipflop task, for example, solutions were generally noisier in the oblique regime than in the aligned, but did not have observable qualitative differences in either regime (Fig. 22). We quantified the difference between models for the different neuroscience tasks considered before. To compare different neural dynamics, we used a dissimilarity measure invariant under rotation (Section 4.4) [75]. The results are shown in Fig. 4C. Two observations stand out: First, across tasks, the dissimilarity was higher for networks in the oblique regime than for those in the aligned. Second, both overall dissimilarity and the discrepancy between regimes differed strongly between tasks. The largest dissimilarity (for oblique solutions) and the largest discrepancy between regimes was found for the cycling. The smallest discrepancy between regimes was found for the flipflop task. Such difference between tasks is consistent with the differences in the range of possible solutions for different tasks, as reported in Refs. [35, 72].

What are the underlying mechanisms for the qualitative decoupling in oblique networks, but not aligned? For aligned dynamics, we saw that the small output weights demand large activity to generate the output. In other words, the activity along the largest PCs must be coupled to the output. For oblique dynamics, this constraint is not present, which opens the possibility for small components outside the largest PCs to generate the output. If this is the case, we have a decoupling, such as the observed co-rotation in the cycling task, and the possible variability between solutions. We discuss this point in more detail in Section 4.9.

In the following two sections, we will explore how the decoupling between neural dynamics and output for oblique, but not aligned, dynamics influences the response to perturbations and the effects of noise during learning.

### Differences in response to perturbations

To understand how networks respond to external perturbations and internal noise requires some understanding how dynamics are generated. Dynamics of trained networks are mostly generated internally, through recurrent interactions. In robust networks, these internally generated dynamics are a prominent part of the largest PCs (among input-driven components; Sections 4.6 and 4.8). Internally-generated dynamics are sustained by positive feedback loops, through which neurons excite each other. Those loops are low-dimensional, with activity along a few directions of the dynamics being amplified and fed back along the same directions. This results in dynamics being driven by effective feedback loops along the
largest PCs (Fig. 5A). As shown above, the largest PCs can either be aligned, or not aligned, with the output weights. This leads to predictions about how aligned and oblique networks differentiate in their responses to perturbations along different directions.

We apply perturbations to the neural activity at a single point in time: \(\mathbf{x}(t)\) evolves undisturbed until time \(t_{p}\). At that point, it is shifted to \(\mathbf{x}(t_{p})+\Delta\mathbf{x}\). After the perturbation, we let the network evolve freely and compare this evolution to that of an unperturbed copy. Such a perturbation mimics a very short optogenetic perturbation applied to a selected neural population [13, 42].

Our intuition about feedback loops suggests that networks respond strongly to a perturbation that is aligned with the directions contributing to the feedback loop, but weakly to a perturbation that is orthogonal to them. In particular, if a perturbation is applied along the output weights, aligned and oblique dynamics should dissociate, with a strong disruption of dynamics for aligned, but not for oblique dynamics (Fig. 5A).

To test this, we compare the response to perturbations along the output direction and along the largest PCs. In Fig. 5B, we show the output after such perturbations for an aligned (top) and an oblique network (bottom) trained on the cycling task. The time point and amplitude are the same for both directions and networks. For each network and type of perturbation, there is an immediate deflection and a long-term response. For both networks, perturbing along the PCs (blue) leads to a long-term phase shift. Only in the aligned network, however, perturbation along the output direction (red) leads to a visible long-term response. In the oblique network, the amplitude of the immediate response is larger, but the long-term response is _smaller_. Our results for the oblique network, but not for the aligned, agree with simulations of networks generating EMG data from the cycling experiments [58].

To quantify the relative long-term susceptibility of networks to perturbations along output weights or PCs, we sampled from different times \(t_{p}\) and different directions in the 2D subspaces spanned either by the two output vectors or the two largest PCs. For each perturbation, we measured the loss of the

Figure 5: Perturbations differentially affect dynamics in the aligned and oblique regimes. **A** Cartoon illustrating the relation between perturbations along output weights or PCs and the feedback loops driving autonomous dynamics. **B** Output after perturbation for aligned (top) and oblique (bottom) networks trained on the cycling task. The unperturbed network (light red line) yields a sine wave along the first output direction \(z_{1}\). At \(t_{p}=9\), a perturbation with amplitude \(\|\Delta\mathbf{x}\|=34\) is applied along the output weights (dashed red) or the first PC (dashed-dotted blue). The perturbations only differ in the directions applied. While the immediate response for the oblique network to a perturbation along the output weights is much larger, \(z_{1}(t_{p})\approx 80\), the long-term dynamics yield the same output as the unperturbed network. See also Fig. 19 for more details. **C** Loss for perturbations of different amplitudes for the two networks in **B**. Lines and shades are means and std devs over different perturbation times \(t_{p}\in[5,15]\) and random directions spanned by the output weights (red) or the two largest PCs (blue). The loss is the mean squared error between output and target for \(t>20\). Gray dot indicates example in **B**. **D** Relative susceptibility of networks to perturbation directions for different tasks and dynamical regimes. We measured the area under the curve (AUC) of loss over perturbation amplitude for perturbations along the output weights or two largest PCs. The relative susceptibility is the ratio between the two AUCs. The example in **C** is indicated by gray triangles.
perturbed networks on the original task (excluding the five time points after \(t_{p}\) that correspond to the immediate deflection after the perturbation). Fig. 5C shows that the aligned network is almost equally susceptible to perturbations along the PCs and the output weights. In contrast, the oblique network is much more susceptible to perturbations along the PCs.

We repeated this analysis for oblique and aligned networks trained on five different tasks. We computed the area under the curve (AUC) for the two loss profiles in Fig. 5C. We then defined the "relative susceptibility" as the ratio \(\text{AUC}_{\mathbf{w}_{\text{out}}}/\text{AUC}_{\text{PC}}\), Fig. 5D. For aligned networks (red), the relative susceptibility was close to 1 indicating similarly strong responses to both types of perturbations. For oblique networks (yellow), it was much smaller than 1, indicating that long-term responses to perturbations along the output direction were weaker than those to perturbations along the PCs.

### Noise suppression for oblique dynamics

In the oblique regime, the output weights are large. To produce the correct output (and not a too large one), the large PCs of the dynamics are almost orthogonal to the output weights. The large output weights, however, pose a robustness problem: Small noise in the direction of the output weights is also amplified at the level of the readout. We show that learning leads to a slow process of sculpting noise statistics to avoid this effect (Fig. 11). Specifically, a negative feedback loop is generated that suppresses fluctuations along the output direction (Fig. 6A, Fig. 10). Because the positive feedback loop that gives rise to the large PCs is mostly orthogonal to the output direction, it remains unaffected by this additional negative feedback loop. A detailed analysis of how learning is affected by noise shows that, for large output weights, the network first learns a solution that is not robust to noise. This solution is then transformed to increasingly stable and oblique dynamics over longer time scales (Sections 4.7 and 4.8).

To illustrate the effect of the negative feedback loop, we consider the fluctuations around trial averages. We take a collection of states \(\mathbf{x}(t)\) and then subtract the task-conditioned averages \(\bar{\mathbf{x}}(t)\) to compute \(\delta\mathbf{x}(t)=\mathbf{x}(t)-\bar{\mathbf{x}}(t)\). We then project \(\delta\mathbf{x}(t)\) onto three different direction categories: the largest PCs of

Figure 6: Noise suppression along the output direction in the oblique regime. **A** A cartoon of the feedback loop structure for aligned (top) and oblique (bottom) dynamics. The latter develops a negative feedback loop which suppresses fluctuations along the output direction. **B** Comparing the distribution of variance of mean-subtracted activity along different directions for network trained on cycling task (see Fig. 20): PCs of trial-averaged activity (blue), readout (red), and random (grey) directions. For the PCs and output weights, we sampled 100 normalized combinations of either the first two PCs or the two output vectors. For the random directions, we drew 1000 random vectors in the full, \(N\)-dimensional space. **C** Noise compression across tasks as measured by the ratio between variance along output and random directions. The dashed line indicates neither compression nor expansion. Black markers indicate the values for the two examples in **B-C**. Note the log-scales in **B-C**.
the averaged data \(\bar{\mathbf{x}}(t)\), the output directions, or randomly drawn directions.

How strongly the activity fluctuates along each direction is quantified by the variance of the projections (Fig. 6B). For both aligned and oblique dynamics, the variance is much larger along the PCs than along random directions. This is not necessarily expected, because the PCA was performed on the _averaged_ activity, hence without the fluctuations. Instead, it is a dynamical effect: the same positive feedback that generates the autonomous dynamics also amplifies the noise (Section 4.8).

The two network regimes, however, dissociate when considering the variance along the output direction. For aligned dynamics, there is no negative feedback loop, and \(\mathbf{w}_{\text{out}}\) is correlated with the PCs. The variance along the output direction is hence similar to that along the PCs, and larger than along random directions. For oblique dynamics, the negative feedback loop suppresses the fluctuations along the output direction, so that they become weaker than along random directions.

In Fig. 6C, we quantify this dissociation across different tasks. We measured the ratio between variance along output and random directions. Aligned networks have a ratio much large than one, indicating that the fluctuations along the output direction are increased due to the autonomous dynamics along the PCs. In contrast, oblique networks have a ratio smaller than 1 for all tasks, which indicates noise compression along the output.

### Aligned and oblique dynamics in experimental settings

For the cycling task, we observed that solutions were qualitatively different for the two regimes, with trajectories either counter- or co-rotating (Fig. 2B). Interestingly, the experimental results of Russo et al. [54] matched the oblique, but not the aligned dynamics. The authors observed co-rotating dynamics in the leading PCs of motor cortex activity despite counter-rotating activity of simultaneously recorded muscle activity. Here we test whether our theory can help to more clearly and quantitatively distinguish between the two regimes in experimental settings.

In typical experimental settings, we do not have direct access to output weights. We can, however,

Figure 7: Quantifying aligned and oblique dynamics in experimental data [11, 20, 23, 47, 54]. **A** Cartoon of a two types of experimental data considered. In motor control experiments (top), we first needed to obtain the output weights \(\mathbf{w}_{\text{out}}\) via linear regression. We then computed the correlation \(\rho\) and the reconstruction dimension \(D_{\text{fit},90}\), i.e. the number of PCs of \(\mathbf{x}\) necessary to obtain a coefficient of determination \(R^{2}>90\%\). In BCI experiments (bottom), the output (cursor velocity) is generated from neural activity \(\mathbf{x}(t)\) via output weights \(\mathbf{w}_{\text{out}}\) defined by the experimenter. This allowed us to directly compute correlation and fitting dimension. **B** Correlation \(\rho\) (top) and relative fitting dimension \(D_{\text{fit},90}/D_{x,90}\) (bottom) for a number of publicly available data sets. The cycling task data (purple) were trial-conditioned averages, the BCI experiments (red) and NLB tasks (yellow) single-trial data. Results for the full data sets are shown as dots, violin plots indicate results for 20 random subsets of 25% of the data points in each data set (see small text below x-axis).
approximate these by fitting neural data to simultaneously recorded behavioral output, such as hand velocity in a motor control experiment (Fig. 7A top). Following the model above, where the output is a weighted average of the states, we reconstruct the output from the neural activity with linear regression. To quantify the dynamical regime, we then compute the correlation \(\rho\) between the weights from fitting and the neural data. Additionally, we can also quantify the regime by the "relative fitting dimension" \(D_{\text{fit},90}/D_{x,90}\), where \(D_{\text{fit},90}\) is the number of PCs necessary to recover the output and \(D_{x,90}\) number to the number of PCs necessary to represent 90% of the variance of the neural data. We computed both the correlation and the relative fitting dimension for different publicly available data sets (Fig. 7B). For details, see Section 4.5.

We started with data sets from two monkeys performing the cycling task [54]. The data contained motor cortex activity, hand movement, and EMG from the arms, all averaged over multiple trials of the same condition. In Fig. 7B, we show results for reconstructing the hand velocity. The correlation was small \(\rho\in\{0.10,0.12\}\). To obtain a good reconstruction, we needed a substantial fraction of the dimension of the neural data: The relative fitting dimension was \(D_{\text{fit},90}/D_{x,90}\in\{0.57,0.48\}\). Both quantities are consistent with oblique dynamics, as suggested by the qualitative agreement between oblique RNN solutions and experimental data. Fitting other behavioral outputs yielded similar results (Fig. 23). Our results are in agreement with previous studies, showing that the best decoding directions for EMG are only weakly correlated with the leading PCs of motor cortex activity [59].

We also analyzed data made available through the Neural Latents Benchmark (NLB) [47]. In two different tasks, monkeys needed to perform movements along a screen. In a maze task, the monkeys were trained to follow a trajectory through a maze with their hand [9]. In a random target task (RTT), the monkeys had to reach for a randomly generated target position on a screen, with a successive target point generated once the previous one was reached [36]. In both cases, we reconstructed the output from neural activity on single trials. The correlation was slightly smaller than for the cycling task, \(\rho\in\{0.07,0.08\}\), and the relative fitting dimension was higher, \(D_{\text{fit},90}/D_{x,90}\in\{0.65,0.62\}\). These values indicate that the neural dynamics for these two tasks are also oblique.

Do we always observe oblique dynamics in experimental data? One setting in which this may not be the case are brain computer interface (BCI) experiments [56]. In these experiments, monkeys were trained to control a cursor on a screen via activity read out from their motor cortex (Fig. 7A bottom). The output weights were set by the experimenter (so we don't need to fit). Importantly, the output weights were typically chosen to be spanned by the largest PCs (the "neural manifold"), suggesting aligned dynamics. We tested this hypothesis on three example data sets [11, 20, 23]. We obtained higher correlation values, \(\rho=0.2\) (Fig. 7B). The relative fitting dimension was much smaller than for the non-BCI data sets, especially for the two largest data sets, where \(D_{\text{fit},90}/D_{x,90}\in\{0.03,0.04\}\). Both measures are thus consistent with the hypothesis that the neural dynamics arising in BCI experiments is more aligned than those in non-BCI settings.

The results above show that our measures can indeed be used to distinguish dynamical regimes in neural data. It would be interesting to test the differences between BCI and non-BCI data on larger data sets, and different experiments with different dimensions of neural data [19, 64]. Finally, we hypothesize that larger and out-of-manifold weights in the BCI setting, if learnable [43], would lead to qualitatively different solutions. One interesting scenario would be to train a monkey on the cycling task [54] but using a BCI either with small, inside-manifold or large, outside-manifold weights.
## 3 Discussion

We analyzed the relationship between neural dynamics and behavior, asking to which extent a network's output is represented in its dynamics. We identified two different limiting regimes: aligned dynamics, in which most of the neural activity in a network is related to its output, and oblique dynamics, where the output is only a small modulation on top of the dominating dynamics. We demonstrated that these two regimes have different functional implications. We also showed how they might arise through learning, and how they relate to experimental findings.

Linking neural activity to external variables is one of the core challenges of neuroscience [24]. In most cases, however, such links are far from perfect. The activity of single neurons can be related in a nonlinear, mixed manner, to task variables [49]. Even when considering populations of neurons, a large fraction of neural activity is not easily explained by external variables [1]. Various explanations have been proposed for this disconnect. In the visual cortex, activity has been shown to be related to "irrelevant" external variables, such as body movements [65]. Follow-up work showed that, in primates, some of these effects can be explained by the induced changes on retinal images [71], but this study still explained only half of the neural variability. An alternative explanation hinges on the redundancy of the neural code, which allows "null spaces" in which activity can visit without affecting behavior [28, 29, 51]. While such accounts can explain why irrelevant activity is _allowed_, they do not explain whether and why it is _necessary_. Here, we showed that merely ensuring stability of neural dynamics can lead to the oblique regime with a decoupling between output and dynamics.

We showed theoretically and in simulations that, when training recurrent neural networks, the magnitude of output weights is a central parameter that controls which regime is reached. This finding is vital for the use of RNNs as hypothesis generators [3, 67, 73], where it is often implicitly assumed that training results in universal solutions [35] (even though biases in the distribution of solutions have been discussed [70]). Here, we show that a specific control knob allows to move between qualitatively different solutions of the same task, thereby expanding the control over the hypothesis space [45, 72]. Note in particular that the default initialization in standard learning frameworks has large output weights, which results in oblique dynamics (or unstable solutions if training without noise, see Methods, Section 4.6).

The role of the magnitude of output weights is also discussed in machine learning settings, where different learning regimes have been found [8, 25, 26, 39]. In particular, "lazy" solutions were observed for large output weights in feed-forward networks. We show in Methods, Section 4.6, that these are unstable for recurrent networks, and are replaced in a second phase of learning by oblique solutions. It would be interesting to see whether an analog to the oblique regime also exists in feed-forward settings. On a broader scale, which learning regime is relevant when modeling biological learning is an open question that is just started to be explored [14].

The particular control knob we studied has an analog in the biological circuit - the synaptic weights. We can thus use experimental data to study whether the brain might rely on oblique or aligned dynamics. Existing experimental work has partially addressed this question. In particular, the work by Russo et al. [54] has been a major inspiration for our study. Our results share some of the key findings from that paper - the importance of stability leading to "untangled" dynamics [66] and a dissociation between hidden dynamics and output. In addition, we suggest a specific mechanism to reach oblique dynamics - via large output weights. Furthermore, we characterize the aligned and oblique regimes along experimentally accessible axes.

We see three avenues for exploring our results experimentally. First, it would be interesting to test whether simultaneous measurements of neural dynamics and muscle activity, such as in Ref. [54], support the idea of noise compression along the output direction. We make a suggestion on how to test this in Fig. 6C. Second, we show how the dynamical regimes dissociate under perturbations along specific directions. Experiments along these lines have recently become possible [7, 13, 53]. Future work is left to combine our model with biological constraints that induce additional effects during perturbations, e.g., through non-normal synaptic connectivity [5, 30, 34, 42]. Third, our work connects to the setting of brain-computer interfaces [20, 56, 74]. In this case, the output weights are directly controlled by the experimenter, hence allowing to test the effect of output weights on network dynamics.

Overall, our results provide an explanation for the plethora of relationships between neural activity and external variables.
## Acknowledgements

FS was supported by the Deutsche Forschungsgemeinschaft (German Research Foundation) under Germany's Excellence Strategy - EXC 2002/1 "Science of Intelligence" - project no. 390523135. OB was supported by the Israeli Science Foundation (grant 1442/21) and an HFSP research grant (RGP0017/2021). SO was supported by the program "Ecoles Universitaires de Recherche" ANR-17-EURE-0017

## Author contributions

Conceptualization, F.S., F.M., S.O., and O.B.; Methodology, F.S.; Formal Analysis, F.S.; Investigation, F.S.; Writing - Original Draft, F.S., F.M., S.O., and O.B.; Writing - Review & Editing, F.S., F.M., S.O., and O.B.; Supervision, F.M., S.O., O.B..

## Declaration of Interests

The authors declare no competing interests.

## Data availability

The code for all simulations and generation of figures is available on github, [https://github.com/frschu/aligned_oblique_in_rnns/](https://github.com/frschu/aligned_oblique_in_rnns/).