A simple connection from loss flatness to compressed representations in
  neural networks
The generalization capacity of deep neural networks has been studied in a
variety of ways, including at least two distinct categories of approaches: one
based on the shape of the loss landscape in parameter space, and the other
based on the structure of the representation manifold in feature space (that
is, in the space of unit activities). Although these two approaches are
related, they are rarely studied together explicitly. Here, we present an
analysis that bridges this gap. We show that in the final phase of learning in
deep neural networks, the compression of the manifold of neural representations
correlates with the flatness of the loss around the minima explored by SGD.
This correlation is predicted by a relatively simple mathematical relationship:
a flatter loss corresponds to a lower upper bound on the compression metrics of
neural representations. Our work builds upon the linear stability insight by Ma
and Ying, deriving inequalities between various compression metrics and
quantities involving sharpness. Empirically, our derived inequality predicts a
consistently positive correlation between representation compression and loss
sharpness in multiple experimental settings. Overall, we advance a dual
perspective on generalization in neural networks in both parameter and feature
space.
Shirui Chen, Stefano Recanatesi, Eric Shea-Brown
# A simple connection from loss flatness to compressed representations in neural networks

###### Abstract

Deep neural networks' generalization capacity has been studied in a variety of ways, including at least two distinct categories of approach: one based on the shape of the loss landscape in parameter space, and the other based on the structure of the representation manifold in feature space (that is, in the space of unit activities). These two approaches are related, but they are rarely studied together and explicitly connected. Here, we present a simple analysis that makes such a connection. We show that, in the last phase of learning of deep neural networks, compression of the volume of the manifold of neural representations correlates with the flatness of the loss around the minima explored by ongoing parameter optimization. We show that this is predicted by a relatively simple mathematical relationship: loss flatness implies compression of neural representations. Our results build closely on prior work of Ma and Ying (1), which shows how flatness (i.e., small eigenvalues of the loss Hessian) develops in late phases of learning and lead to robustness to perturbations in network inputs. Moreover, we show there is no similarly direct connection between local dimensionality and sharpness, suggesting that this property may be controlled by different mechanisms than volume and hence may play a complementary role in neural representations. Overall, we advance a dual perspective on generalization in neural networks in both parameter and feature space.

## Introduction

The remarkable capacity of deep neural networks to generalize has been studied in many ways. Generalization is a complex phenomenon influenced by myriad factors, including model architecture, dataset size and diversity, and the specific task used to train a network - and researchers continue to develop new techniques to enhance the generalization. This is a vast field; however, from a theoretical point of view, we can identify two distinct categories of approach. These are studies that show that neural network generalization is linked to (a) properties of minima of the loss function that learning algorithms find in parameter space, and (b) to properties of the representations that optimized networks find in feature space - that is, in the space of their neural activations.

In parameter space, both empirical studies and theoretical analyses have shown that deep neural networks often converge to flat and wide minima and that this can underlie their good generalization performance (e.g. [1, 2, 3, 4, 5, 6, 7, 8]). Flat minima refer to regions in the loss landscape where the loss function has a relatively large basin: put simply, the loss doesn't change much in different directions around the minimum. El-geant arguments show how models that converge to flat minima are more likely to generalize well [9]. The intuition behind this is that in a flat minimum, small perturbations or noise in the input data are less likely to cause significant changes in the model's output, resulting in improved robustness to variations in data from training to test.

In feature space, the phenomenon of compression (or neural collapse) refers to the observation that, as neural representations develop through the course of training, the activity patterns in feedforward and recurrent neural networks can become more compact and reside in lower-dimensional spaces [10, 11, 12, 13, 14, 15]. This is closely related to the earlier information bottleneck ideas [16], which quantified allied effects via entropy. This phenomenon, which in many (but not all [17, 18]) cases develops through training, can be enhanced by the architectural design of a network: for example, higher layers of a deep neural network usually have fewer neurons compared to the input layer, and autoencoders include internal "bottleneck layers" with significantly fewer neurons [19]. Together compression and low-dimensionality in feature space can result in networks isolating the most important and discriminative features of input data, thus enhancing their ability to generalize. Throughout, we focus on the second, or final, stage of learning, which proceeds after the learning algorithm has already found parameters that give optimal performance (i.e., minimum loss) on the training data [19, 20, 1]. Here, additional learning still occurs, which changes the properties of the solutions in both feature and parameter space in very interesting ways.

Our paper proceeds as follows. First, we review arguments of Ma and Ying [1] that flatter minima are induced during the final phase of learning under gradient descent, and that this flatness can constrain the gradient of the loss with respect to network inputs. Next, we prove that lower sharpness implies a lower upper bound of the local volume (compression) of the representation manifold in feature space. We conclude our findings with simulations that confirm our central theoretical results and show how they can be applied in practice.

## Background and setup

Consider a feedforward neural network \(f\) with input data \(\mathbf{x}\in\mathbb{R}^{M}\) and parameters \(\boldsymbol{\theta}\). The output of the network is:

\[\mathbf{y}=f(\mathbf{x};\boldsymbol{\theta})\;, \tag{1}\]
with and \(\mathbf{y}\in\mathbb{R}^{N}\) (\(N<M\)). Consider a quadratic loss \(L(\mathbf{y},\mathbf{y}_{\text{true}})=||\mathbf{y}-\mathbf{y}_{\text{true}}||^{2}\) function of the outputs and ground true data \(\mathbf{y}_{\text{true}}\). In the following, we'll simply write \(L(\mathbf{y})\), \(L(f(\mathbf{x},\mathbf{\theta}))\) or simply \(L(\mathbf{\theta})\) to highlight the dependence of the loss on the output, the network or its parameters.

During the last phase of learning, Ma and colleagues have recently argued that SGD appears to regularize the sharpness of the loss (4) (see also (5-8)). This is to say that the dynamics of SGD lead network parameters to minima where the local loss landscape is flatter or wider. This is best captured by the sharpness, measured by the sum of the eigenvalues of the Hessian:

\[S(\mathbf{\theta})=\text{Tr}(H)\;, \tag{2}\]

with \(H=\nabla^{2}L(\mathbf{\theta})\) being the Hessian, and \(\lVert\cdot\rVert_{F}\) is the Frobenius norm. A solution with low sharpness is a flatter solution. Following (1, 20), we define \(\mathbf{\theta}^{*}\) to be an "interpolation solution" on the zero loss manifold in the parameter space (the zero loss manifold in what follows), where \(f(\mathbf{x}_{i},\mathbf{\theta}^{*})=\mathbf{y}_{i}\) for all \(i\)'s (with \(i\in\{1..n\}\) indexing the training set) and \(L(\mathbf{\theta}^{*})=0\). On the zero loss manifold, in particular, we have

\[S(\mathbf{\theta}^{*})=\frac{1}{n}\sum_{i=1}^{n}\lVert\nabla_{\mathbf{\theta}}f( \mathbf{x}_{i},\mathbf{\theta}^{*})\rVert_{F}^{2} \tag{3}\]

Next, we briefly review the arguments of (2, 4, 20) for why sharpness is minimized once loss has attained its minimum. On such a manifold, we have \(\nabla_{\mathbf{\theta}}L(f_{\text{net}}(\mathbf{x};\mathbf{\theta}^{*}))=0\).

Consider a perturbation \(\delta\mathbf{\theta}^{*}\) of such a point. Then the second-order expansion of the loss around \(\mathbf{\theta}^{*}+\delta\mathbf{\theta}^{*}\) is:

\[L(\mathbf{\theta}^{*}+\delta\mathbf{\theta}^{*})\approx L(\mathbf{\theta}^{*})+\nabla^{T} L(\mathbf{\theta}^{*})\delta\mathbf{\theta}^{*}+\frac{1}{2}\delta\mathbf{\theta}^{*T}H \delta\mathbf{\theta}^{*}\;, \tag{4}\]

where \(H\) is again the Hessian matrix. Taking the gradient with respect to \(\mathbf{\theta}\) we obtain:

\[\begin{split}\nabla_{\mathbf{\theta}}L(\mathbf{\theta}^{*}+\delta\mathbf{ \theta}^{*})&=\nabla_{\mathbf{\theta}}L(\mathbf{\theta}^{*})+\nabla_{\bm {\theta}}\nabla^{T}L(\mathbf{\theta}^{*})\delta\mathbf{\theta}^{*}\\ &\qquad\qquad+\nabla_{\mathbf{\theta}}(\frac{1}{2}\delta\mathbf{\theta}^ {*T}H\delta\mathbf{\theta}^{*})\\ &=0+H\delta\mathbf{\theta}^{*}+\nabla_{\mathbf{\theta}}(\frac{1}{2}\delta \mathbf{\theta}^{*T}H\delta\mathbf{\theta}^{*})\;.\end{split} \tag{5}\]

The leading term in this expansion is \(H\delta\mathbf{\theta}^{*}\). As the loss does not vary along the zero-loss manifold, the Hessian along this manifold has zero eigenvalues. Therefore, \(H\delta\mathbf{\theta}^{*}\) can be viewed as a projection of the perturbation \(\delta\mathbf{\theta}^{*}\) along directions that are orthogonal to the zero-loss manifold. The third term can instead be viewed as an Hessian-based loss. This indicates that minimizing the total loss also requires minimizing the Hessian along the zero-loss manifold.

In order to see why minimizing the sharpness of the solution leads to more compressed representations, we need to move from parameter space to input space. To do so we review the argument of Ma and colleagues (1) that relates variations in input data \(\mathbf{x}\) and input weights. Thus, at least for a subset of parameters \(\mathbf{\theta}\) (the input weights \(\mathbf{W}\)), there exists a direct relationship between network effects of changes in parameter space and changes in input space.

Let \(\mathbf{W}\) be the input weights to the network, and \(\bar{\mathbf{\theta}}\) the corresponding set of parameters. Following (1), as the weights \(\mathbf{W}\) multiply the inputs \(\mathbf{x}\) we have the following identities:

\[\begin{split}\lVert\nabla_{\mathbf{W}}f(\mathbf{W}\mathbf{x}; \bar{\mathbf{\theta}})\rVert_{F}&=\sqrt{\sum_{i,j,k}J_{jk}^{2}x_{i}^{2 }}=\lVert J\rVert_{F}\lVert\mathbf{x}\rVert_{2}\\ \lVert\nabla_{\mathbf{x}}f(\mathbf{W}\mathbf{x};\bar{\mathbf{\theta }})\rVert_{F}&=\lVert\mathbf{W}^{T}J\rVert_{F}\;,\end{split} \tag{6}\]

where \(J=\frac{\partial f(\mathbf{W}\mathbf{x};\bar{\mathbf{\theta}})}{\partial(\mathbf{ W}\mathbf{x})}\) is a complex expression as computed in, e.g., backpropagation. From Eq. (6) and the sub

Figure 1: Trends in key variables across SGD training of the vgg10 network with fixed batch size (equal to 20) and varying learning rates (0.05, 0.1 and 0.2). After the loss is minimized (so that an interpolation solution is found) sharpness and volumes decrease together. Moreover, higher learning rates lead to lower sharpness and hence stronger compression. From left to right in row-wise order: train loss, test accuracy, sharpness (Eq. (3)), log volumetric ratio (Eq. (11)), left-hand side of Eq. (8) (axes titled \(G\)), and local dimensionality of the network output (Eq. (14)).
multiplicative property of the Frobenius norm, we have:

\[\|\nabla_{\mathbf{x}}f(\mathbf{W}\mathbf{x};\tilde{\mathbf{\theta}})\|_{F}\leq\frac{ \|\mathbf{W}\|_{F}}{\|\mathbf{x}\|_{2}}\|\nabla_{\mathbf{W}}f(\mathbf{W} \mathbf{x};\tilde{\mathbf{\theta}})\|_{F}\;, \tag{7}\]

which is an upper bound on \(\|\nabla_{\mathbf{x}}f(\mathbf{W}\mathbf{x};\tilde{\mathbf{\theta}})\|_{F}\). If the norms \(\|\mathbf{W}\|_{F}\) and \(\|\mathbf{x}\|_{2}\) are not excessively large or small respectively, these bounds control the gradient with respect to inputs via the gradient with respect to weights. Following (1), this in turn reveals the impact of flatness in the loss function:

\[\frac{1}{n}\sum_{i=1}^{n}\lVert\nabla_{\mathbf{x}}f(\mathbf{x}_{i },\mathbf{\theta}^{*})\rVert_{F}^{k} \leq\frac{\|\mathbf{W}\|_{F}^{k}}{\text{min}_{i}\|\mathbf{x}_{i} \|_{2}^{k}}\frac{1}{n}\sum_{i=1}^{n}\lVert\nabla_{\mathbf{w}}f(\mathbf{x}_{i}, \mathbf{\theta}^{*})\rVert_{F}^{k}\] \[\leq\frac{\|\mathbf{W}\|_{F}^{k}}{\text{min}_{i}\|\mathbf{x}_{i} \|_{2}^{k}}\frac{1}{n}\sum_{i=1}^{n}\lVert\nabla_{\mathbf{\theta}}f(\mathbf{x}_{i},\mathbf{\theta}^{*})\rVert_{F}^{k}\] \[\leq\frac{\|\mathbf{W}\|_{F}^{k}}{\text{min}_{i}\|\mathbf{x}_{i} \|_{2}^{k}}S(\mathbf{\theta}^{*})^{k/2} \tag{8}\]

for all \(k\geq 2\). Thus, as in (1), the effect of input perturbations is constrained by the sharpness of the loss function. The flatter the minimum of the loss, the lower the effect of input space perturbations on the network function. \(f(\mathbf{x},\mathbf{\theta}^{*})\) as determined by gradients.

## Appendix F From robustness to inputs to compression of representations

We now further analyze variations in the input and how they propagate through the network to shape representations of sets of inputs. Consider an input data point \(\bar{\mathbf{x}}\) drawn from the training set: \(\bar{\mathbf{x}}=\mathbf{x}_{i}\) for a specific \(i\in\{1..n\}\). The set of all possible perturbations around \(\bar{\mathbf{x}}\) in input space is the ball \(\mathcal{B}(\bar{\mathbf{x}})_{r}\): \(\mathbf{x}\in\mathcal{B}(\bar{\mathbf{x}})_{r}\) if \(\|\mathbf{x}-\bar{\mathbf{x}}\|_{2}<r\). To determine how the network represents these inputs, we will quantify how the ball \(\mathcal{B}(\bar{\mathbf{x}})_{r}\) expands or contracts as it propagates through the network's layers and reaches the network's output. This amounts to asking whether the transformation of the differential volume around \(\bar{\mathbf{x}}\) is an expansion or a contraction.

To do so we propagate the ball through the network transforming each point \(\mathbf{x}\) into its image \(f(\mathbf{x})\). Following a Taylor expansion for points within \(\mathcal{B}(\bar{\mathbf{x}})_{r}\) as \(r\to 0\) we have:

\[f(\mathbf{x})=f(\bar{\mathbf{x}})+\nabla_{\mathbf{x}}(f(\bar{ \mathbf{x}},\mathbf{\theta}^{*}))(\mathbf{x}-\bar{\mathbf{x}}). \tag{9}\]

We can express the limit of the covariance matrix \(C_{f(\mathcal{B}(\mathbf{x}))}\) of the output \(f(\mathbf{x})\) as

\[C_{f}^{\text{lim}}=\lim_{\alpha\to 0}C_{f(\mathcal{B}(\mathcal{A}))}= \alpha\nabla_{\mathbf{x}}f(\bar{\mathbf{x}},\mathbf{\theta}^{*})\nabla_{\mathbf{x }}^{T}f(\bar{\mathbf{x}},\mathbf{\theta}^{*})\;, \tag{10}\]

where \(\alpha\) depends on the input's covariance, given as \(C_{\mathcal{B}(\mathbf{x})}=\alpha\mathcal{I}\), with \(\mathcal{I}\) as the identity matrix. Our covariance expressions capture the distribution of points in \(\mathcal{B}(\bar{\mathbf{x}})_{r}\) as they go through the network \(f(\bar{\mathbf{x}},\mathbf{\theta}^{*})\). We note that our analysis of the covariance in feature space is an approximation at second order, akin to approximating the representation manifold around \(f(\bar{\mathbf{x}})\) with a Gaussian distribution.

We quantify how a network compresses its input volumes via the local volumetric ratio, between a cube of side length \(h\) at \(\bar{\mathbf{x}}\) and its image under transformation \(f\) is:

\[d\operatorname{Vol}^{ratio}\lvert_{f(\bar{\mathbf{x}},\mathbf{ \theta}^{*})} =\lim_{h\to 0}\frac{\operatorname{Vol}(f(\bar{\mathbf{x}},\mathbf{ \theta}^{*}))}{\operatorname{Vol}(\bar{\mathbf{x}})} \tag{11}\] \[=\sqrt{\det\left(\nabla_{\mathbf{x}}f^{T}\nabla_{\mathbf{x}}f \right)}\]

which is equal to the square root of the product of all positive eigenvalues of \(C_{f}^{\text{lim}}\). Exploiting the bound on the gradients derived earlier in Eq. (7) we derive a similar bound for the

Figure 2: Trends in key variables across SGD training of the vqg10 network with fixed learning rate size (equal to 0.1) and varying batch size (8, 20 and 32). After the loss is minimized (so that an interpolation solution is found) sharpness and volumes decrease together. Moreover, lower batch sizes lead to lower sharpness and hence stronger compression. From left to right in row-wise order: train loss, test accuracy, sharpness (Eq. (3)), log volumetric ratio (Eq. (11)), left-hand side of Eq. (8) (axes titled \(G\)), and local dimensionality of the network output (Eq. (14)).
volumetric ratio:

\[\begin{split} d\operatorname{Vol}^{ratio}\lvert_{f(\bar{\mathbf{x}}, \boldsymbol{\theta}^{*})}&\leq\left(\frac{\operatorname{Tr}\nabla_{ \mathbf{x}}f^{T}\nabla_{\mathbf{x}}f}{N}\right)^{N/2}\\ &=N^{-N/2}\|\nabla_{\mathbf{x}}f(\mathbf{x}_{i},\boldsymbol{ \theta}^{*})\|_{F}^{N}\end{split} \tag{12}\]

where the first line uses the inequality between arithmetic and geometric means and the second the definition of the Frobenius norm. Introducing the averaged volumetric ratio across all input points \(dV^{ratio}(\boldsymbol{\theta}^{*})=\frac{1}{n}\sum_{i=1}^{n}d\operatorname{ Vol}^{ratio}\lvert_{f(\mathbf{x}_{i},\boldsymbol{\theta}^{*})}\), and comparing this with Eq. (8) we obtain:

\[dV^{ratio}(\boldsymbol{\theta}^{*})\leq\frac{\|\mathbf{W}\|_{F}^{N}}{\text{ min}_{i}\|\mathbf{x}_{i}\|_{2}^{N}}\left(\frac{S(\boldsymbol{\theta}^{*})}{N} \right)^{N/2}\;. \tag{13}\]

This equation implies that, as SGD dynamics drives the minimization of flattened minima of the loss function in parameter space, the manifold of data representation through the network becomes increasingly compressed. Our analysis demonstrates that these two phenomena are linked by the generalization properties of the network to input perturbations.

We note that an alternative way to analyze the same effect would be by means of the entropy of the representations in feature space. The entropy of a multivariate Gaussian distribution is defined as \(H=\frac{1}{2}\ln\det\left(2\pi eC_{f}^{r}\right)\) and other works have focused on entropic quantities (e.g. [21, 22]).

## Experiments

### Sharpness and compression: verifying the theory.

The theoretical results derived above show that during the later phase of training - the interpolation phase - the volume of the network's representation is upper bounded by a function of the sharpness of the loss function in parameter space. This links sharpness and representation's volume: the flatter is the loss landscape, the more compressed is the upper bound on the representation volume.

It remains to test in practice, however, whether these bounds are sufficiently tight so that a clear relationship between sharpness and representation volume appears. As one such test, we ran the following experiment. We trained a VGG10 network [23] to classify images from the CIFAR-10 dataset, and calculated the sharpness (Eq. (2)), the log volumetric ratio (Eq. (11)) and the left-hand side of Eq. (8) (the gradient with respect to the inputs, a quantity we term G in the figures below) during the training phase of a (Fig 1 and 2).

We trained the network (VGG10) using SGD on images from 2 classes (out of 10) so that convergence to the interpolation regime, i.e. zero loss, was faster. We explored the influence of two specific parameters that have a substantial effect on the network's training: learning rate and batch size. For each pair of learning rate and batch size parameters, we computed all quantities at hand across 100 input samples and five different random initializations for network weights.

In the first set of experiments, we studied the link between a decrease in sharpness during the latter phases of training and volume compression Fig. 1. We noticed that when the network reaches the interpolation regime, and the sharpness decreases, so does the volume. The quantity G similarly decreases. All these results were consistent across multiple learning rates for a fixed batch size (of 20): specifically, for learning rates that gave lower values of sharpness, volume was lower as well.

We then repeated the experiments while keeping the learning rate fixed (lr=0.1) and varying the batch size. The same broadly consistent trends emerged linking a decrease in the sharpness to a compression in the representation volume Fig. 2. However, we also find that while sharpness stops decreasing after about iteration \(50\cdot 10^{3}\) for batch size 32, the volume keeps decreasing as learning proceeds. This suggests that there may be other mechanisms at play, beyond sharpness, in driving the compression of volumes.

### Sharpness and compression on test set data.

Despite the fact that Eq. (3) is exact for interpolation solutions only (i.e., those with zero loss), we found that the test loss is small enough (Fig. 3) so that it should be a good approximation for test data as well. Therefore we analyzed our simulations to study trends in sharpness and volume for these held-out test data as well (Fig. 3). We discovered that this sharpness increased rather than diminished as a result of training. We hypothesized that sharpness could correlate with the difficulty of classifying testing points. This was supported by the fact that the sharpness of misclassified test data was even greater than that of all test data. Despite this increase in sharpness, the volume followed the same pattern as the training set. This suggests that compression in representation space is a robust phenomenon that can be driven by additional phenomena beyond sharpness. Nevertheless, the compression still is weaker for misclassified test samples that have higher sharpness than other test samples. Overall, these results emphasize an interesting distinction between how sharpness evolves for training vs. test data.

### Sharpness and local dimensionality.

Lastly, we analyze the representation's local dimensionality in a manner analogous to the volume analysis. Apriori, it is ambiguous whether the dimensionality of the data representation should increase or decrease as the volume is compressed. For instance, the volume could decrease while maintaining its overall form and symmetry, thus preserving its dimensionality. Alternatively, one or more of the directions in the relevant tangent space could be selectively compressed, leading to an overall reduction in dimensionality. To study this quantitatively, we use a local measure of dimensionality, the local Participation Ratio, given by:

\[D_{\text{PR}}(f(\bar{\mathbf{x}}))=\lim_{\alpha\to 0}\frac{\operatorname{Tr}[C_{f( \mathcal{B}(\mathbf{x}))}]^{2}}{\operatorname{Tr}[(C_{f(\mathcal{B}(\mathbf{x }))})^{2}]}=\frac{\operatorname{Tr}[C_{f}^{\lim}]^{2}}{\operatorname{Tr}[(C_{f }^{\lim})^{2}]} \tag{14}\]

(cf. [24, 25, 26]). This quantity can be averaged across a set of points within a given radius, giving it its local property: \(D_{\text{PR}}(\theta^{*})=\frac{1}{n}\sum_{i}^{n}D_{\text{PR}}(f(\mathbf{x}_{i} ))\).
Figures 1 and 2 show our experiments computing the Participation Ratio over the course of learning. Here, we find that the local dimensionality of the representation decreases as the loss decreases to 0, which is consistent with the viewpoint that the network compresses representations in feature space as much as possible, retaining only the directions that code for task-relevant features [27, 28]. However, the local dimensionality exhibits unpredictable behavior that cannot be explained by the sharpness (or the volume) once the network is on the zero-loss manifold and training continues. This discrepancy is consistent with the bounds established by our theory, which only bound the numerator of Eq. (14). It is also consistent with volume compression overall: for example, if we let \(\mathbf{\lambda}\) be all the eigenvalues of \(C_{f}^{\text{lim}}\), then the local dimensionality can be written as \(D_{\text{PR}}=(\|\mathbf{\lambda}\|_{1}/\|\mathbf{\lambda}\|_{2})^{2}\), which retains the same value when \(\mathbf{\lambda}\) is arbitrarily scaled. This shows how local dimensionality is a distinct quality of network representations compared with volume, and is driven by mechanisms that differ from sharpness alone. We emphasize that the dimensionality we study here is a local measure, on the finest scale around a point on the "global" manifold of unit activities; dimension on larger scales (i.e., across categories or large sets of task inputs [10, 24]) may show different trends.

## Conclusion

In the final phase of neural network training, it is a widely observed phenomenon that SGD training effectively reduces the (eigenvalues of the) Hessian of the loss in parameter space. In other words, training finds increasingly flat minima, and this refinement occurs in the last, and sloweds, phase of training. Here, we have shown that this same phenomenon leads to compression of a network's representation of its inputs. Our results unite often separate perspectives on generalization in neural networks and add to mounting evidence for why the final phase of learning crucially shapes network representations.
