considered tasks: the larger the layer rotations, the better the
generalization performance. The best performance is consistently obtained when
nearly all layers reach a cosine distance of $1$, which corresponds to final
weights that are orthogonal to their initialization (cfr. fifth column of
Figure 4.1). This observation would have limited value if many configurations
(amongst which the best one) lead to cosine distances of $1$. However, we
notice that most configurations do not. In particular, rotating the layers
weights very slightly is sufficient for the network to achieve $100\%$
training accuracy (cfr. third column of Figure 4.1).

We also observe that layer rotation rates (rotation with respect to the
previous iterate) translate remarkably well into layer rotations (rotation
with respect to the initialization). For example, the $\alpha=0$ setting used
in the fifth column indeed leads all layers to rotate quasi synchronously. As
discussed in Section 4.1.2, this is not self-evident. Understanding why this
happens (and why the first and last layers seem to be less tameable) is an
interesting direction of research resulting from our work.

Figure 4.1: Analysis of the layer rotation curves (cfr. Section 4.1.1) and
test accuracies ($\eta$) induced by different layer rotation rate
configurations (using Layca for training) on the five tasks of Table 3. The
configurations are parametrized by $\alpha$, that controls which layers have
the highest rotation rates (first layers for $\alpha<0$, last layers for
$\alpha>0$, or no prioritization for $\alpha=0$), and $\rho(0)$, the initial
global rotation rate value shared by all layers. $\Delta\eta$ is computed with
respect to the best configuration (last column), which corresponds to
$\alpha=0$ and $\rho(0)=3^{-3}$ for the five tasks. This visualization unveils
large differences in generalization ability across configurations which seem
to follow a simple yet consistent rule of thumb: the larger the layer rotation
for each layer, the better the generalization performance. Training accuracies
are provided in Section 4.2 ($\approx 100\%$ in all configurations).

####  4.3.3 Layer rotation’s relationship with training speed

While generalization is the main focus of our work, we observed through our
experiments that layer rotation rates also influenced the training speed of
our models in a remarkable way. Figure 4.2 depicts the loss curves obtained
for different values of $\alpha$ and $\rho(0)$ on the first three tasks of
Table 3. It appears that the larger or the more uniform the layer rotation
rates are, the higher the plateaus in which loss curves get stuck into. The
plateaus might be due to a form of interference between the different neurons’
training processes. The precision with which the height of plateaus can be
controlled through the $\alpha$ and $\rho(0)$ parameters is striking and
further supports the idea that layer rotation controls a fundamental yet
implicit mechanism in deep learning. Following our rule of thumb, this result
also suggests that high plateaus are additional indicators of good
generalization performance. This is consistent with the systematic occurrence
of high plateaus in the loss curves of state of the art networks (e.g., He,
Zhang, Ren et al. (2016); Zagoruyko and Komodakis (2016)).

Figure 4.2: Loss curves obtained for different $\alpha$ and $\rho(0)$ values
on the first three tasks of Table 3, using Layca for training. The $\alpha$
and $\rho(0)$ configurations are specified for each column in the associated
legend. The visualizations unveil a remarkable phenomenon: the more uniform or
the larger the layer rotation rates, the higher the plateaus in which the loss
gets stuck into. The sudden drops correspond to a reduction of the global
learning rate as specified in 4.2.

###  4.4 A study of layer rotation in standard training settings

Section 4.3 uses Layca to study the relation between layer rotations and
generalization or training speed in a controlled setting. This section
investigates the layer rotation configurations that naturally emerge when
using SGD, weight decay or adaptive gradient methods for training. First of
all, these experiments will provide supplementary evidence for the rule of
thumb proposed in Section 4.3. Second, we’ll see that studying training
methods from the perspective of layer rotation can provide useful insights to
explain their behaviour.

The experiments are performed on the five tasks of Table 3. The learning rate
parameter is tuned independently for each training setting through grid search
over 10 logarithmically spaced values ($3^{-7},3^{-6},...,3^{2}$), except for
C10-CNN2 and C100-WRN where learning rates are taken from their original
implementations when using SGD + weight decay, and from (Wilson, Roelofs,
Stern et al., 2017) when using adaptive gradient methods for training. The
test accuracies obtained in standard settings will often be compared to the
best results obtained with Layca, which are provided in the 5th column of
Figure 4.1.

####  4.4.1 Analysis of SGD’s learning rate

The influence of SGD’s learning rate on generalization has been highlighted by
several works (Jastrzebski, Kenton, Arpit et al., 2017; Smith and Le, 2017;
Smith and Topin, 2017; Hoffer, Hubara, and Soudry, 2017; Masters and Luschi,
2018). The learning rate parameter directly affects layer rotation rates,
since it changes the size of the updates. In this section, we verify if the
learning rate’s impact on generalization is coherent with our rules of thumb.

Figure 4.3 displays the layer rotation curves and test accuracies generated by
different learning rate configurations during vanilla SGD training on the five
tasks of table 3. We observe that test accuracy increases for larger layer
rotations (consistent with our rule of thumb) until a tipping point where it
starts to decrease (inconsistent with our rule of thumb). We show in Figure
4.4 that these problematic cases involve extremely abrupt layer rotations that
do not translate in improvements of the training loss. These observations thus
highlight an important condition for our rules of thumb to hold true: the
monitored layer rotations should coincide with actual training (i.e.
improvements on the loss function).

A second interesting observation is that the layer rotation curves obtained
with vanilla SGD are far from the ideal scenario disclosed in Section 4.3,
where the majority of the layers’ weights reached a cosine distance of 1 from
their initialization. In accordance with our rules of thumb, SGD also reaches
considerably lower test performances than Layca. A more extensive tuning of
the learning rate (over 10 logarithmically spaced values) did not help SGD to
solve its two systematic problems: 1) layer rotations are not uniform and 2)
the layers’ weights stop rotating before reaching a cosine distance of 1.

Figure 4.3: Layer rotation curves and the corresponding test accuracies
generated by vanilla SGD with different learning rates. Colour code, axes and
$\Delta\eta$ computation are the same as in Figure 4.1. The influence of the
learning rate parameter on generalization is consistent with our rule of thumb
(larger layer rotations $\rightarrow$ better generalization), except for cases
with abrupt layer rotations. We further show in Figure 4.4 that these abrupt
layer rotations do not translate in improvements of the loss. Moreover,
despite extensive learning rate tuning, SGD induces test performances that are
significantly below Layca’s optimal configuration (cfr. $5^{th}$ column of
Figure 4.1). This is also in accordance with our rules of thumb, since SGD
does not seem to be able to generate layer rotations that reach a cosine
distance of 1. Figure 4.4: Layer rotation and training curves during the first
epoch of SGD training with high learning rates (cfr. Figure 4.3). The
visualization reveals large layer rotations that are sometimes performed in a
single iteration. Importantly, these iterations do not induce improvements in
training accuracy, which probably explains why these configurations escape the
scope of our rule of thumb.

####  4.4.2 Analysis of SGD and weight decay

Several papers have recently shown that, in batch normalized networks, the
regularization effect of weight decay was caused by an increase of the
effective learning rate (van Laarhoven, 2017; Hoffer, Banner, Golan et al.,
2018; Zhang, Wang, Xu et al., 2019). More generally, reducing the norm of
weights increases the amount of rotation induced by a given training step. It
is thus interesting to see how weight decay affects layer rotations, and if
its impact on generalization is coherent with our rule of thumb. Figure 4.5
displays, for the 5 tasks, the layer rotation curves generated by SGD when
combined with weight decay (in this case, equivalent to
$L_{2}$-regularization). We observe that weight decay solves SGD’s problems (
cfr. Section 4.4.1): all layers’ weights reach a cosine distance of 1 from
their initialization, and the resulting test performances are on par with the
ones obtained with Layca.

This experiment not only provides important supplementary evidence for our
rules of thumb, but also novel insights around weight decay’s regularization
ability in deep nets: weight decay seems to be key for enabling large layer
rotations (weights reaching a cosine distance of 1 from their initialization)
during SGD training. Since the same behaviour can be achieved with tools that
control layer rotation rates (cfr. Layca), without an extra parameter to tune,
our results could potentially lead weight decay to disappear from the standard
deep learning toolkit.

Figure 4.5: Layer rotation curves and the corresponding test accuracies
generated by SGD with weight decay. Colour code, axes and $\Delta\eta$
computation are the same as in Figure 4.1. The application of weight decay
during SGD training enables layer rotations that reach a cosine distance of 1
and leads to test performances comparable to Layca’s optimal configuration
(cfr. $5^{th}$ column of Figure 4.1). These results thus provide supplementary
evidence for our rule of thumb and a new perspective on weight decay
regularization.

####  4.4.3 Analysis of learning rate warmups

We’ve seen in Section 4.4.1 that during SGD training, high learning rates
could generate abrupt layer rotations at the very beginning of training that
do not improve the training loss. In this section, we investigate if these
unstable layer rotations could be the reason why learning rate warmups are
sometimes necessary when using high learning rates He, Zhang, Ren et al.
(2016); Goyal, Dollar, Girshick et al. (2018). For this experiment, we use a
deeper network that notoriously requires warmups for training: ResNet-110 He,
Zhang, Ren et al. (2016). The network is trained on the CIFAR-10 dataset with
standard data augmentation techniques. We use a warmup strategy that starts at
a 10 times smaller learning rate and linearly increases to reach the final
learning rate in a specified number of epochs.

Figure 4.6 displays the layer rotation and training curves when training with
a high learning rate ($3^{-1}$) and different warmup durations (0,5,10 or 15
epochs of warmup). We observe that without warmup, SGD generates unstable
layer rotations and training accuracy doesn’t improve before the 25th epoch.
Using warmups brings significant improvements: a $75\%$ training accuracy is
reached after 25 epochs, with only some instabilities in the training curves
-that again are synchronized with abrupt layer rotations. Finally, we also use
Layca for training (with a $3^{-3}$ learning rate). We observe that it doesn’t
suffer from SGD’s instabilities in terms of layer rotation. Hence, Layca
achieves large layer rotations and good generalization performance without the
need for warmups.

Figure 4.6: Layer rotation and training curves obtained when using different
warmup durations (0,5,10 or 15 epochs) during the training of ResNet-110 on
CIFAR-10 with high learning rates ($3^{-1}$). The curves are shown for the 25
first epochs only -out of 200. $\eta$ is the final test accuracy. We observe
that SGD generates unstable layer rotations that translate into a stagnation
or a decrease of the training accuracy. Using warmups drastically reduces
these instabilities. Layca doesn’t generate instabilities and reaches high
generalization performance (and large layer rotations) without the need for
warmups.

####  4.4.4 Analysis of adaptive gradient methods

The recent years have seen the rise of adaptive gradient methods in the
context of machine learning (e.g. RMSProp (Tieleman and Hinton, 2012), Adagrad
(Duchi, Hazan, and Singer, 2011), Adam (Kingma and Ba, 2015)). The key element
distinguishing adaptive gradient methods from their non-adaptative equivalents
is a parameter-level tuning of the learning rate based on the statistics of
each parameter’s partial derivative. Initially introduced for improving
training speed, (Wilson, Roelofs, Stern et al., 2017) observed that these
methods also had a considerable impact on generalization. Since these methods
affect the rate at which individual parameters change, they might also
influence the rate at which layers change (and thus layer rotations).

We first observe to what extent the parameter-level learning rates of Adam
vary across layers. We monitor Adam’s estimate of the second raw moment, which
is used for determining the parameter-level learning rates, when training on
the C10-CNN1 task. The estimate is computed by:

$v_{t}=\beta_{2}\cdot v_{t-1}+(1-\beta_{2})\cdot g_{t}^{2}$

where $g_{t}$ and $v_{t}$ are vectors containing respectively the gradient and
the estimates of the second raw moment at training step $t$, and $\beta_{2}$
is a parameter specifying the decay rate of the moment estimate. While our
experiment focuses on Adam, the other adaptive methods (RMSprop, Adagrad) also
use statistics of the squared gradients to compute parameter-level learning
rates. Figure 4.7 displays the $10^{th}$, $50^{th}$ and $90^{th}$ percentiles
of the moment estimations, for each layer separately, as measured at the end
of epochs 1, 10 and 50. The conclusion is clear: the estimates vary much more
across layers than inside layers. This suggests that adaptive gradient methods
might have a drastic impact on layer rotations.

Figure 4.7: Adam’s parameter-wise estimates of the second raw moment
(uncentered variance) of the gradient during training on C10-CNN1, represented
for each layer separately through their $10^{th}$, $50^{th}$ and $90^{th}$
percentiles. The results provide evidence that the parameter-level statistics
used by adaptive gradient methods vary mostly between layers and negligibly
inside layers.

##### Adaptive gradient methods can reach SGD’s generalization ability with
Layca

Since adaptive gradient methods probably affect layer rotations, we will
verify if their influence on generalization is coherent with our rule of
thumb. Figure 4.8 ($1^{st}$ line) provides the layer rotation curves and test
accuracies obtained when using adaptive gradient methods to train the 5 tasks
described in Table 3. We observe an overall worse generalization ability
compared to Layca’s optimal configuration and small and/or non-uniform layer
rotations. We also observe that the layer rotations of adaptive gradient
methods are considerably different from the ones induced by SGD (cfr. Figure
4.3). For example, adaptive gradient methods seem to induce larger rotations
of the last layers’ weights, while SGD usually favors rotation of the first
layers’ weights. Could these differences explain the impact of parameter-level
adaptivity on generalization? In Figure 4.8 ($2^{nd}$ line), we show that when
Layca is used on top of adaptive methods (to control layer rotation), adaptive
methods can reach test accuracies on par with SGD + weight decay. Our
observations thus suggest that adaptive gradient methods’ poor generalization
properties are due to their impact on layer rotations. Moreover, the results
again provide supplementary evidence for our rule of thumb.

Figure 4.8: Layer rotation curves and the corresponding test accuracies
generated by adaptive gradient methods (RMSProp, Adam, Adagrad,
RMSProp+$L_{2}$ and Adam+$L_{2}$ respectively for each task/column) without
($1^{st}$ line) and with ($2^{nd}$ line) control of layer rotation with Layca.
Colour code, axes and $\Delta\eta$ computation are the same as in Figure 4.1.
In the first line, we observe an overall worse generalization ability compared
to Layca’s optimal configuration (cfr. $5^{th}$ column of Figure 4.1) -despite
extensive learning rate tuning, together with small and/or non-uniform layer
rotations (in accordance with our rule of thumb). When Layca is used on top of
adaptive methods to control layer rotation (second line), adaptive methods can
reach test accuracies on par with SGD + weight decay.

##### SGD can achieve adaptive gradient methods’ training speed with Layca

We’ve seen that the negative impact of adaptive gradient methods on
generalization was largely due to their influence on layer rotations. Could
layer rotations also explain their positive impact on training speed? To test
this hypothesis, we recorded the layer rotation rates emerging from training
with adaptive gradient methods, and reproduced them during SGD training with
the help of Layca. We then observe if this SGD-Layca optimization procedure
(that doesn’t perform parameter-level adaptivity) could achieve the improved
training speed of adaptive gradient methods. Figure 4 shows the training
curves during training of the 5 tasks of Table 3 with adaptive gradient
methods, SGD+weight decay and SGD-Layca-AdaptCopy (which copies the layer
rotation rates of adaptive gradient methods). While adaptive gradient methods
train significantly faster than SGD+weight decay, we observe that their
training curves are nearly indistinguishable from SGD-Layca-AdaptCopy. Our
study thus suggests that adaptive gradient methods impact on both
generalization and training speed is due to their influence on layer
rotations.

Figure 4.9: Training curves for the 5 tasks of Table 3 with adaptive gradient
methods (RMSProp, Adam, Adagrad, RMSProp+$L_{2}$ and Adam+$L_{2}$ respectively
for each task/column), SGD+weight decay and SGD-Layca-AdaptCopy. During
training with SGD-Layca-AdaptCopy, Layca is used to reproduce the layer
rotations generated by an adaptive gradient method on the same task444When
copying the layer rotations of Adam with SGD-Layca-AdaptCopy we also integrate
Adam’s momentum scheme.. We observe that this training procedure (which
doesn’t perform parameter-level adaptivity) achieves the same improvements in
training speed as adaptive gradient methods.

###  4.5 Related work

The idea that neurons from different layers potentially train at different
rates was motivated by the works on vanishing and exploding gradients (Bengio,
Simard, and Frasconi, 1994; Hochreiter, 1998; Glorot and Bengio, 2010). These
pioneering works revealed that the norm of gradients is affected by its
propagation through the layers, potentially leading to training difficulties.
Based on this observation, several other works also designed and studied tools
for controlling deep neural network training on a per-layer basis (Yu, Lin,
Salakhutdinov et al., 2017; Ginsburg, Gitman, and You, 2018; Bernstein,
Vahdat, Yue et al., 2020). While these don’t conduct a systematic study of
layer-level training’s relationship with generalization or training speed,
they show that layer-level control leads to more stable training, reduced
hyperparameter tuning and ultimately better generalization performance.
Interestingly, (Liu, Bernstein, Meister et al., 2021) recently showed that
controlling rotation of weight vectors at the neuron-level could improve a
network’s performance even further. This indicates that training differences
can also occur amongst the neurons of a layer, and further supports our
initial hypothesis which states that emergent neuron-level training processes
play a crucial role in deep neural networks.

###  4.6 On the interpretation of layer rotations

The previous sections of this chapter demonstrate the remarkable consistency
and explanatory power of layer rotation’s relationship with generalization.
This suggests that layer rotation controls fundamental aspects of deep neural
network training. Whether these aspects relate to the clustering abilities and
mechanisms studied in Chapters 2 and 3 relies on the hypothesis that layer
rotation captures the extent by which hidden neurons have been able to “train”
during the network’s training process. In this section, we provide an
additional experiment to support the link between all these concepts.

We use a toy experiment to visualize how layer rotation affects the features
learned by hidden neurons. We train a single hidden layer MLP (with 784 hidden
neurons) on a reduced MNIST dataset (1000 samples per class, to increase
overparameterization). This toy network has the advantage of having
intermediate features that are easily visualized: the weights associated to
hidden neurons live in the same space as the input images. Starting from an
identical initialization, we train the network with four different learning
rates using Layca, leading to four different layer rotation configurations
that all reach $100\%$ training accuracy but different generalization
abilities (in accordance with our rule of thumb).

Figure 4.10 displays the features obtained by the different layer rotation
configurations (for 5 randomly selected hidden neurons). This visualization
unveils an important phenomenon: layer rotation does not seem to affect which
features are learned, but rather to what extent they are learned during the
training process. The larger the layer rotation, the more prominent the
features -and the less retrievable the initialization. Ultimately, for a layer
rotation close to 1, the final weights of the network got rid of all remnants
of the initialization. This experiment thus supports our initial hypothesis:
fully accomplishing emergent neuron-level training processes is not necessary
for reaching $100\%$ training accuracy, but doing so anyway leads to better
clustering and generalization abilities.

Figure 4.10: A single hidden layer MLP is trained on a reduced MNIST dataset
(1000 examples per class). Starting from an identical initialization, the
network is trained with four different learning rates using Layca, leading to
four different layer rotation configurations that all reach $100\%$ training
accuracy but different generalization abilities (in accordance with our rule
of thumb). The learned intermediate features (associated to 5 randomly
selected neurons) are visualized for the different layer rotation
configurations. The results suggest that layer rotation does not affect which
features are learned, but rather to what extent they are learned during the
training process.

##  Chapter 5 Discussion and perspectives

Our work argues that the natural clustering prior plays a key role in deep
learning and generalization. Empirical evidence supporting this hypothesis has
been presented in Chapters 2,3 and 4. This chapter takes a step back and
attempts to see the bigger picture behind our work. More precisely, we (i)
discuss to what extent this work effectively validates our hypothesis (Section
5.1), (ii) predict a rebirth of clustering algorithms for training deep neural
networks (Section 5.2) and (iii) discuss the societal impact of deep learning
research (Section 5.3).

###  5.1 Towards validating our hypothesis

In order to validate our hypothesis, this work provides a collection of
intuitions and experiments. In this section, we detail how future work could
improve the generality of our experiments, provide complementary empirical
evidence for our hypothesis and incorporate mathematical formalisms.

##### Generalizing our experiments

We ask machine learning models to be able to generalize to a large variety of
contexts. So should the empirical claims that support a theory. The
experiments presented in this work involve a restricted set of deep learning
setups. Using other datasets, architectures, training algorithms and
hyperparameters could lead to different or contradicting conclusions. This
limitation is particularly strong in Chapter 3, which studies implicit
clustering mechanisms in a setup involving synthetic data and simple neural
network architectures. Additionally, the study of large-scale datasets (e.g.
ImageNet (Deng, Dong, Socher et al., 2009)) and different modalities (natural
language and sounds) would also greatly improve our work.

##### Providing complementary empirical evidence

Identifying and studying implicit phenomena is a difficult endeavour.
Conducting many complementary experimental studies are key to (i) offer
different perspectives that will help better characterize the phenomenon and
(ii) mitigate the risk of misinterpretation due to unrelated confounding
factors.

In order to complement our work, a first path worth investigating is the study
of densely annotated datasets such as Broden introduced by Bau, Zhou, Khosla
et al. (2017). Our work makes already use of datasets with two levels of class
labels for studying implicit clustering. We interpret the subclasses of such
datasets as single clusters, but these could in fact be composed of multiple
clusters themselves. Moreover, class labels are attributed to the whole image,
while the associated objects/concepts are relevant to only a part of it.
Datasets like Broden mitigate these two drawbacks by providing many levels of
class labels as well as pixel-wise annotations. This holds the potential for
improved measures of clustering abilities and more precise studies of
clustering mechanisms in natural image datasets.

Another path of investigation consists in further evaluating the explanatory
power of our hypothesis. How well does it explain a variety of phenomena? Our
work demonstrates the potential of implicit clustering to explain the benefits
of pre-training, the coherent gradients hypothesis, neuron interpretability,
the benefits of large learning rates, weight decay and others. However, all
these explanations remain partial, and many other phenomena are not addressed
(e.g. “double descent” curves (Belkin, Hsu, Ma et al., 2019), the lottery
ticket hypothesis (Frankle and Carbin, 2018) or the benefits of skip
connections (Balduzzi, Frean, Leary et al., 2017)). Conducting in depth
studies of all these phenomena under the light of implicit clustering could
further evaluate our hypothesis.

##### Formalizing intuitions

The amount of equations and mathematical symbols is remarkably low in this
thesis. Our work is indeed mainly composed of informal arguments and
intuitions. Being able to translate our claims into mathematical language
would help increase their precision and falsifiability. Moreover, mathematics
greatly facilitate the exploration of ideas through deductive reasoning
(Wigner, 1960). Formalizing the intuitions and results presented in Chapter 3
might be a good place to start, given the simplicity of the associated
experimental setup. Additionally, Berner, Grohs, Kutyniok et al. (2021)
provides a nice overview of several mathematical studies of deep learning,
which could provide inspiration for this difficult endeavour.

###  5.2 A rebirth of clustering algorithms

Backpropagation, the algorithm behind SGD, has been the most popular algorithm
for training deep neural networks for at least two decades. And for a good
reason: it outperforms all other alternatives by a large margin on standard
datasets. And yet, there is still a long way to go to reach human-level
generalization abilities (cfr. Section 1.2.2). This begs the question: should
we continue building on backpropagation and SGD, or explore novel algorithms?
If natural data exhibit a clustered structure, as the natural clustering prior
states, aren’t clustering algorithms a more natural choice for training deep
neural networks? Coates and Ng (2012) trains deep neural networks with K-means
clustering, but their method did not match SGD’s performance. Isn’t this
contradictory with our hypothesis?

While they clearly lack SGD’s capabilities on standard classification
datasets, clustering algorithms have achieved some successes on natural image-
related tasks in the past (Zoran and Weiss, 2011; Coates, Ng, and Lee, 2011).
Moreover, identifying the right priors appears to be especially crucial for
clustering algorithms (Estivill-Castro, 2002). We believe that a better
characterization of the natural clustering prior in terms of the shape of
clusters, their relative density, the distance between them or their
relationship with class labels could lead to improved clustering algorithms.
For example, many clustering algorithms assume that all clusters contain
approximately the same amount of training examples. However, recent works
suggest that the ability of deep neural networks to “memorize” atypical and
poorly represented sub-populations is key for their performance on natural
image classification tasks (Feldman, 2020; Jiang, Zhang, Talwar et al.,
2021b). Additionally, many clustering algorithms assume spherical cluster
shapes while many aspects of natural images vary in specific, anisotropic ways
(e.g. scaling, rotation, translation of objects and object parts).

Hence, the current supremacy of SGD over clustering algorithms might not be
the final picture. Because natural clustering-related priors would be more
easily integrated into their design, our work predicts a rebirth of clustering
algorithms in the coming years, bringing us one step closer to human-level
generalization abilities.

###  5.3 The societal impact of deep learning research

Human societies are complex systems. Understanding how they will react to new
technologies is a daunting task. Yet, the well-being of billions of
individuals can be at stake. The growing integration of deep learning
technologies in the industry raises these difficult questions. Because these
were an integral part of my own research experience, and encouraged by the
NeurIPS conference’s call for a “Broader impact” section111NeurIPS’ call for a
“Broader impact” section: In order to provide a balanced perspective, authors
are required to include a statement of the potential broader impact of their
work, including its ethical aspects and future societal consequences. Authors
should take care to discuss both positive and negative outcomes., I shortly
discuss my personal take on the societal impact of deep learning research.

The ethical concerns around deep learning are numerous. Deep learning
technology can be used for autonomous military drones (Shane, Metz, and
Wakabayashi, 2018), generating misinformation automatically (Radford, Wu,
Amodei et al., 2019), racial discrimination (Kickuchiyo, 2019), mass
manipulation on social networks (Coombe, Curtis, and Orlowski, 2020) and many
others. Moreover, deep learning confers power to the data owners. The
centralization of data in big tech companies leads to rising social, economic
and political inequalities (Harari, 2018). Of course, many positive
opportunities also emerge from deep learning such as healthcare applications
(Panesar, 2019), scientific progress (Jumper, Evans, Pritzel et al., 2021b)
and tackling climate change (Rolnick, Donti, Kaack et al., 2019)). But from my
very limited perspective, I feel that the negative outcomes largely outnumber
the positive ones in terms of practical impact.

As researchers and engineers, it might be appealing to leave these ethical
considerations to political institutions. They possess the power to regulate
technologies, and are expected to preserve common good. But reality doesn’t
necessarily match expectations. The technological progress could be too fast
for ankylosed political systems. Big tech companies could heavily limit a
government’s room for manoeuvre when profit is at stake. As Professor Harari
claims, many political decisions could in fact be in the hands of the
scientists and engineers that develop today’s technologies (Harari, 2016).

What are scientists and engineers expected to do then? This remains an open
question. Several brave individuals decide to quit the field entirely (e.g.,
Amadeo (2018); Yuan (2020)). Conferences organize workshops around the topic
(e.g., Chowdhury (2021); Li, Isupova, Haghtalab et al. (2021)). Partnerships
with civil society and media organizations are built (e.g. Partnership On
AI222https://partnershiponai.org/). In my humble opinion, throwing deep
learning technology into a competitive ecosystem is bound to raise
inequalities and harm common good. Hence, I believe that designing tools to
facilitate cooperation of citizens around shared goals, at small and large
scale, has a big potential for positive change. These tools could for example
take the form of web applications that facilitate knowledge sharing and
collective decision making or that connect people with similar goals. I
believe there is a lot of room for improvement around collaborative software
design, and that this endeavour constitutes a promising path towards a society
where deep learning researchers can pursue their quest with a peace of mind.

## Conclusion

Deep learning gained a lot of popularity for the plethora of applications it
enables. But the open questions and mysteries behind these successes are at
least as fascinating. They challenge our understanding of generalization,
which constitutes the most fundamental aspect of machine learning. Moreover,
they exhibit connections with the human brain, one of the universe’s greatest
mysteries. In this thesis, we propose a novel path towards a better
understanding of deep learning.

Our work builds on the idea that generalization is strongly influenced by the
priors integrated into a learning system’s design. We propose a natural
clustering prior for supervised image classification problems, and study (i)
to what extent this prior is integrated into deep learning systems and (ii) if
its integration influences generalization.

We provide a collection of experiments supporting the occurrence of an
implicit clustering ability, mechanism and hyperparameter in deep learning.
Moreover, we demonstrate empirically that these components consistently
influence the generalization abilities of deep neural networks. We further
highlight many connections between our observations and previous work on
neuron interpretability, the early phase of training, pre-training, the
coherent gradients hypothesis and others.

Overall, our work reveals that the natural clustering prior offers a promising
path towards understanding the generalization abilities of deep learning
systems. Additionally, it unveils a path towards new clustering-based training
algorithms that could push the generalization abilities of such learning
systems even further. With these exciting perspectives in mind, we look
forward to the future developments of the fascinating field of deep learning.

## References

  * Absil, Mahony, and Sepulchre (2010) Absil, PA, Mahony, R, and Sepulchre, R (2010).  Optimization On Manifolds : Methods And Applications.  In _Recent Advances in Optimization and its Applications in Engineering_ , pages 125—-144. Springer.  ISBN 9783642125973.
  * Achille, Rovere, and Soatto (2019) Achille, A, Rovere, M, and Soatto, S (2019).  Critical Learning Periods in Deep Neural Networks.  In _ICLR_ , pages 1–14.  arXiv:arXiv:1711.08856v3.
  * Agarwal, Barham, Brevdo et al. (2016) Agarwal, A, Barham, P, Brevdo, E, Chen, Z, Citro, C, et al. (2016).  TensorFlow : Large-Scale Machine Learning on Heterogeneous Distributed Systems.  _arXiv preprint arXiv:1603.04467_.
  * Chollet et al. (2015) Chollet et al., F (2015).  Keras.
  * Alcorn, Li, Gong et al. (2019) Alcorn, MA, Li, Q, Gong, Z, Wang, C, Mai, L, et al. (2019).  Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects.  In _CVPR_ , pages 4845–4854.
  * Amadeo (2018) Amadeo, R (2018).  A dozen Google employees quit over military drone project.  _Ars Technica_.
  * Arjovsky (2021) Arjovsky, M (2021).  _Out of Distribution Generalization in Machine Learning_.  Ph.D. thesis.  arXiv:2103.02667.
  * Arora (2018) Arora, S (2018).  Toward theoretical understanding of deep learning.  In _ICML tutorial_.
  * Arpit, Jastrzebski, Ballas et al. (2017) Arpit, D, Jastrzebski, S, Ballas, N, Krueger, D, Bengio, E, et al. (2017).  A Closer Look at Memorization in Deep Networks.  In _ICML_.  arXiv:arXiv:1706.05394v1.
  * Balduzzi, Frean, Leary et al. (2017) Balduzzi, D, Frean, M, Leary, L, Lewis, J, and Ma, Kurt Wan-Duo McWilliams, B (2017).  The Shattered Gradients Problem: If resnets are the answer, then what is the question?  In _ICML_.  arXiv:arXiv:1702.08591v1.
  * Barrett and Dherin (2021) Barrett, D and Dherin, B (2021).  Implicit Gradient Regularization.  In _ICLR_.
  * Bau, Zhou, Khosla et al. (2017) Bau, D, Zhou, B, Khosla, A, Oliva, A, and Torralba, A (2017).  Network Dissection: Quantifying Interpretability of Deep Visual Representations.  In _CVPR_.
  * Beery, van Horn, and Perona (2018) Beery, S, van Horn, G, and Perona, P (2018).  Recognition in Terra Incognita.  In _ECCV_. Springer Verlag.  arXiv:1807.04975.
  * Belkin, Hsu, Ma et al. (2019) Belkin, M, Hsu, D, Ma, S, and Mandal, S (2019).  Reconciling modern machine-learning practice and the classical bias-variance trade-off.  In _Proceedings of the National Academy of Sciences_.  10.1073/pnas.1903070116.
  * Bengio (2009) Bengio, Y (2009).  Learning Deep Architectures for AI.  _Foundations and trends in Machine Learning_ , 2(1):1–127.
  * Bengio, Courville, and Vincent (2012) Bengio, Y, Courville, A, and Vincent, P (2012).  Representation Learning: A Review and New Perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828.  10.1109/TPAMI.2013.50.  arXiv:1206.5538.
  * Bengio, Lee, Bornschein et al. (2015) Bengio, Y, Lee, DH, Bornschein, J, Mesnard, T, and Lin, Z (2015).  Towards Biologically Plausible Deep Learning.  _arXiv:1502.04156_.  arXiv:1502.04156.
  * Bengio, Simard, and Frasconi (1994) Bengio, Y, Simard, P, and Frasconi, P (1994).  Learning long-term dependencies with gradient descent is difficult.  _IEEE transactions on neural networks_ , 5(2):157–166.
  * Berner, Grohs, Kutyniok et al. (2021) Berner, J, Grohs, P, Kutyniok, G, and Petersen, P (2021).  The Modern Mathematics of Deep Learning.  _arXiv:2105.04026_.  arXiv:arXiv:2105.04026v1.
  * Bernstein, Vahdat, Yue et al. (2020) Bernstein, J, Vahdat, A, Yue, Y, and Liu, MY (2020).  On the distance between two neural networks and the stability of learning.  In _NeurIPS_ , volume 2020-Decem. Neural information processing systems foundation.  arXiv:2002.03432.
  * Bishop (2006) Bishop, CM (2006).  _Pattern recognition and machine learning_.  Springer.
  * Cammarata, Carter, Goh et al. (2020) Cammarata, N, Carter, S, Goh, G, Olah, C, Petrov, M, et al. (2020).  Thread: Circuits.  _Distill_.
  * Chapelle and Zien (2005) Chapelle, O and Zien, A (2005).  Semi-supervised classification by low density separation.  In _International workshop on artificial intelligence and statistics_ , pages 57–64.
  * Chatterjee (2020) Chatterjee, S (2020).  Coherent gradients: an approach to understanding generalization in gradient-based optimization.  In _ICLR_.  arXiv:arXiv:2002.10657v1.
  * Chatterjee and Zielinski (2020) Chatterjee, S and Zielinski, P (2020).  Making coherence out of nothing at all: measuring the evolution of gradient alignment.  _arXiv:2008.01217_.  arXiv:arXiv:2008.01217v1.
  * Chowdhury (2021) Chowdhury, T (2021).  The moral (and morale) compass: A search for meaning in AI research.
  * Coates, Ng, and Lee (2011) Coates, A, Ng, A, and Lee, H (2011).  An Analysis of Single-Layer Networks in Unsupervised Feature Learning.  In _Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics_ , volume 15, pages 215–223.
  * Coates and Ng (2012) Coates, A and Ng, AY (2012).  Learning Feature Representations with K-means.  In _Neural Networks: Tricks of the Trade, 2nd edn_. Springer.
  * Coombe, Curtis, and Orlowski (2020) Coombe, D, Curtis, V, and Orlowski, J (2020).  The Social Dilemma.
  * CS231N (2016) CS231N, S (2016).  Tiny ImageNet Visual Recognition Challenge.  _https://tiny-imagenet.herokuapp.com/_.
  * Cubuk, Zoph, Mane et al. (2019) Cubuk, ED, Zoph, B, Mane, D, Vasudevan, V, and Le, QV (2019).  AutoAugment: Learning Augmentation Policies from Data.  In _CVPR_ , Section 3, pages 113–123.  arXiv:1805.09501.
  * Dalal and Triggs (2005) Dalal, N and Triggs, B (2005).  Histograms of oriented gradients for human detection.  In _CVPR_ , pages 886–893.  10.1109/CVPR.2005.177ï.
  * Dauber, Feder, Koren et al. (2020) Dauber, A, Feder, M, Koren, T, and Livni, R (2020).  Can implicit bias explain generalization? Stochastic convex optimization as a case study.  In _NeurIPS_. Neural information processing systems foundation.  arXiv:2003.06152.
  * Deng, Dong, Socher et al. (2009) Deng, J, Dong, W, Socher, R, Li, LJ, Li, K, et al. (2009).  ImageNet: A Large-Scale Hierarchical Image Database.  In _CVPR_ , pages 248–255.
  * DeVries and Taylor (2017) DeVries, T and Taylor, GW (2017).  Improved Regularization of Convolutional Neural Networks with Cutout.  _arXiv:1708.04552_.  arXiv:1708.04552.
  * Dodge and Karam (2017) Dodge, S and Karam, L (2017).  A Study and Comparison of Human and Deep Learning Recognition Performance Under Visual Distortions.  _2017 26th International Conference on Computer Communications and Networks, ICCCN 2017_.  arXiv:1705.02498.
  * Donahue, Jia, Vinyals et al. (2014) Donahue, J, Jia, Y, Vinyals, O, Hoffman, J, Zhang, N, et al. (2014).  DeCAF : A Deep Convolutional Activation Feature for Generic Visual Recognition.  In _ICML_ , pages 647–655.
  * Du, Lee, Li et al. (2019) Du, S, Lee, J, Li, H, Wang, L, and Zhai, X (2019).  Gradient Descent Finds Global Minima of Deep Neural Networks.  In _ICML_ , pages 1675–1685. PMLR.
  * Duchi, Hazan, and Singer (2011) Duchi, J, Hazan, E, and Singer, Y (2011).  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization.  _Journal of Machine Learning Research_ , 12(Jul):2121–2159.
  * Eldan and Shamir (2016) Eldan, R and Shamir, O (2016).  The Power of Depth for Feedforward Neural Networks.  In _Annual Conference on Learning Theory_ , volume 49, pages 907–940. PMLR.
  * Engstrom, Tran, Tsipras et al. (2019) Engstrom, L, Tran, B, Tsipras, D, Schmidt, L, and Madry, A (2019).  Exploring the Landscape of Spatial Robustness.  _ICML_.  arXiv:1712.02779.
  * Estivill-Castro (2002) Estivill-Castro, V (2002).  Why so many clustering algorithms: a position paper.  _ACM SIGKDD Explorations Newsletter_ , 4(1):65–75.  10.1145/568574.568575.
  * Feldman (2020) Feldman, V (2020).  Does Learning Require Memorization? A Short Tale about a Long Tail.  In _Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing_. ACM, New York, NY, USA.  10.1145/3357713.
  * Fort, Dziugaite, Paul et al. (2020) Fort, S, Dziugaite, GK, Paul, M, Kharaghani, S, Roy, DM, et al. (2020).  Deep learning versus kernel learning : an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel.  In _NeurIPS_.
  * Frankle and Carbin (2018) Frankle, J and Carbin, M (2018).  The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.  In _ICLR_. International Conference on Learning Representations, ICLR.  arXiv:1803.03635.
  * Frankle, Dziugaite, Roy et al. (2020) Frankle, J, Dziugaite, GK, Roy, DM, and Carbin, M (2020).  Linear Mode Connectivity and the Lottery Ticket Hypothesis.  In _ICML_.  arXiv:arXiv:1912.05671v4.
  * Frankle, Schwab, and Morcos (2020) Frankle, J, Schwab, DJ, and Morcos, AS (2020).  The Early Phase of Neural Network Training.  In _ICLR_.  arXiv:arXiv:2002.10365v1.
  * Fritzke (1997) Fritzke, B (1997).  Some competitive learning methods.
  * Fukushima (1980) Fukushima, K (1980).  Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position.  _Biological cybernetics_ , 202(36):193–202.
  * Geiping, Goldblum, Pope et al. (2021) Geiping, J, Goldblum, M, Pope, PE, Moeller, M, and Goldstein, T (2021).  Stochastic Training is Not Necessary for Generalization.  _arXiv:2109.14119_.  arXiv:2109.14119.
  * Geirhos, Medina Temme, Rauber et al. (2018) Geirhos, R, Medina Temme, CR, Rauber, J, Schütt, HH, Bethge, M, et al. (2018).  Generalisation in humans and deep neural networks.  In _NeurIPS_.  arXiv:1808.08750v3.
  * Geirhos, Rubisch, Michaelis et al. (2018) Geirhos, R, Rubisch, P, Michaelis, C, Bethge, M, Wichmann, FA, et al. (2018).  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness.  In _ICLR 2019_. International Conference on Learning Representations, ICLR.  arXiv:1811.12231.
  * Geman, Bienenstock, and Doursat (1992) Geman, S, Bienenstock, E, and Doursat, R (1992).  Neural networks and the bias/variance dilemma.  _Neural computation_ , 4(1):1–58.
  * Ginsburg, Gitman, and You (2018) Ginsburg, B, Gitman, I, and You, Y (2018).  Large Batch Training of Convolutional Networks with Layer-wise Adaptive Rate Scaling.
  * Glorot and Bengio (2010) Glorot, X and Bengio, Y (2010).  Understanding the difficulty of training deep feedforward neural networks.  In _AISTATS_ , pages 249–256.
  * Golatkar, Achille, and Soatto (2019) Golatkar, A, Achille, A, and Soatto, S (2019).  Time Matters in Regularizing Deep Networks : Weight Decay and Data Augmentation Affect Early Learning.  In _NeurIPS_.
  * Goodfellow, Bengio, and Courville (2016) Goodfellow, I, Bengio, Y, and Courville, A (2016).  _Deep Learning_.  Book in preparation for MIT Press.
  * Goodfellow, Shlens, and Szegedy (2015) Goodfellow, IJ, Shlens, J, and Szegedy, C (2015).  Explaining and Harnessing Adverarial Examples.  In _ICLR_.  arXiv:arXiv:1412.6572v3.
  * Goyal, Dollar, Girshick et al. (2017) Goyal, P, Dollar, P, Girshick, R, Noordhuis, P, Wesolowski, L, et al. (2017).  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour.  _arXiv:1706.02677_.  arXiv:1706.02677.
  * Goyal, Dollar, Girshick et al. (2018) Goyal, P, Dollar, P, Girshick, R, Noordhuis, P, Wesolowski, L, et al. (2018).  Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour.  _arXiv:1706.02677_.  arXiv:arXiv:1706.02677v2.
  * Graves, Mohamed, and Hinton (2013) Graves, A, Mohamed, Ar, and Hinton, G (2013).  Speech recognition with deep recurrent neural networks.  _IEEE international conference on acoustics, speech and signal processing_ , pages 6645—-6649.  arXiv:1303.5778v1.
  * Gulrajani and Lopez-Paz (2021) Gulrajani, I and Lopez-Paz, D (2021).  In search of lost domain generalization.  In _ICLR_.
  * Gur-Ari, Roberts, and Dyer (2018) Gur-Ari, G, Roberts, DA, and Dyer, E (2018).  Gradient Descent Happens in a Tiny Subspace.  _arXiv:1812.04754_.  arXiv:arXiv:1812.04754v1.
  * Harari (2016) Harari, YN (2016).  _Homo Deus: A brief history of tomorrow_.  Harvill Secker.  ISBN 978-5-906837-92-9.
  * Harari (2018) Harari, YN (2018).  Why Technology Favors Tyranny.
  * Hassabis, Kumaran, and Summerfield, Christopher Botvinick (2017) Hassabis, D, Kumaran, D, and Summerfield, Christopher Botvinick, M (2017).  Neuroscience-inspired artificial intelligence.  _Neuron_ , 95(2):245—-258.
  * He, Fan, Wu et al. (2020) He, K, Fan, H, Wu, Y, Xie, S, and Girshick, R (2020).  Momentum Contrast for Unsupervised Visual Representation Learning.  In _CVPR_ , pages 9729–9738.
  * He, Zhang, Ren et al. (2015) He, K, Zhang, X, Ren, S, and Sun, J (2015).  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification.  In _ICCV_.  arXiv:1502.01852v1.
  * He, Zhang, Ren et al. (2016) He, K, Zhang, X, Ren, S, and Sun, J (2016).  Deep Residual Learning for Image Recognition.  In _CVPR_ , pages 770–778.  arXiv:arXiv:1512.03385v1.
  * Hebb (1949) Hebb, DO (1949).  _The organization of behavior_.
  * Hendrycks and Dietterich (2019) Hendrycks, D and Dietterich, T (2019).  Benchmarking Neural Network Robustness to Common Corruptions and Perturbations.  In _ICLR_. International Conference on Learning Representations, ICLR.  arXiv:1903.12261.
  * Hinton, Mcclelland, and Rumelhart (1987) Hinton, GE, Mcclelland, JL, and Rumelhart, DE (1987).  Distributed Representations.  In _Parallel Distributed Processing: Explorations in the micro-structure of cognition_.
  * Hinton, Osindero, and Teh (2006) Hinton, GE, Osindero, S, and Teh, YW (2006).  A Fast Learning Algorithm for Deep Belief Nets.  _Neural Computation_ , 18(7):1527–1554.  10.1162/NECO.2006.18.7.1527.
  * Hoai and Zisserman (2013) Hoai, M and Zisserman, A (2013).  Discriminative sub-categorization.  In _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , pages 1666–1673.  10.1109/CVPR.2013.218.
  * Hochreiter (1998) Hochreiter, S (1998).  The vanishing gradient problem during learning recurrent neural nets and problem solutions.  _IJUFKS_ , 6(2):1–10.
  * Hoffer, Banner, Golan et al. (2018) Hoffer, E, Banner, R, Golan, I, and Soudry, D (2018).  Norm matters: efficient and accurate normalization schemes in deep networks.  _arXiv:1803.01814_.  arXiv:1803.01814.
  * Hoffer, Hubara, and Soudry (2017) Hoffer, E, Hubara, I, and Soudry, D (2017).  Train longer , generalize better : closing the generalization gap in large batch training of neural networks.  In _NIPS_ , pages 1729—-1739.  arXiv:arXiv:1705.08741v1.
  * Hubel and Wiesel (1962) Hubel, DH and Wiesel, TN (1962).  Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex.  _The Journal of Physiology_ , 160(1):106–154.  10.1113/JPHYSIOL.1962.SP006837.
  * Huh, Agrawal, and Efros (2016) Huh, M, Agrawal, P, and Efros, AA (2016).  What makes ImageNet good for transfer learning?  _arXiv:1608.08614_.  arXiv:arXiv:1608.08614v2.
  * Ioffe and Szegedy (2015) Ioffe, S and Szegedy, C (2015).  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.  _ICML_ , pages 448—-456.  10.1007/s13398-014-0173-7.2.  arXiv:1502.03167.
  * Jastrzebski, Kenton, Arpit et al. (2017) Jastrzebski, S, Kenton, Z, Arpit, D, Ballas, N, Fischer, A, et al. (2017).  Three Factors Influencing Minima in SGD.  _arXiv:1711.04623_.  arXiv:arXiv:1711.04623v1.
  * Jastrzebski, Szymczak, Fort et al. (2020) Jastrzebski, S, Szymczak, M, Fort, S, Arpit, D, Tabor, J, et al. (2020).  The Break-Even Point on Optimization Trajectories of Deep Neural Networks.  In _ICLR_.
  * Jiang, Neyshabur, Mobahi et al. (2020) Jiang, Y, Neyshabur, B, Mobahi, H, Krishnan, D, and Bengio, S (2020).  Fantastic Generalization Measures and Where to Find Them.  In _ICLR_.  arXiv:arXiv:1912.02178v1.
  * Jiang, Zhang, Talwar et al. (2021a) Jiang, Z, Zhang, C, Talwar, K, and Mozer, MC (2021a).  Characterizing Structural Regularities of Labeled Data in Overparameterized Models.  In _ICML_ , pages 5034–5044. PMLR.
  * Jiang, Zhang, Talwar et al. (2021b) Jiang, Z, Zhang, C, Talwar, K, and Mozer, MC (2021b).  Characterizing Structural Regularities of Labeled Data in Overparameterized Models.  In _ICML_ , pages 5034–5044. PMLR.
  * Jumper, Evans, Pritzel et al. (2021a) Jumper, J, Evans, R, Pritzel, A, Green, T, Figurnov, M, et al. (2021a).  Highly accurate protein structure prediction with AlphaFold.  _Nature_.
  * Jumper, Evans, Pritzel et al. (2021b) Jumper, J, Evans, R, Pritzel, A, Green, T, Figurnov, M, et al. (2021b).  Highly accurate protein structure prediction with AlphaFold.  _Nature_ , 596(7873):583–589.  10.1038/s41586-021-03819-2.
  * Kamnitsas, Castro, Le Folgoc et al. (2018) Kamnitsas, K, Castro, DC, Le Folgoc, L, Ian, W, Ryutaro, T, et al. (2018).  Semi-Supervised Learning via Compact Latent Space Clustering.  In _ICML_.  arXiv:arXiv:1806.02679v2.
  * Kaufman and Rousseeuw (2009) Kaufman, L and Rousseeuw, PJ (2009).  _Finding groups in data: an introduction to cluster analysis_.  John Wiley & Sons.
  * Kawaguchi, Kaelbling, and Bengio (2017) Kawaguchi, K, Kaelbling, LP, and Bengio, Y (2017).  Generalization in Deep Learning.  _arXiv:1710.05468_.  arXiv:arXiv:1710.05468v6.
  * Keskar, Mudigere, Nocedal et al. (2017) Keskar, NS, Mudigere, D, Nocedal, J, Smelyanskiy, M, and Tang, PTP (2017).  On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima.  In _ICLR_.
  * Kickuchiyo (2019) Kickuchiyo (2019).  [D] Has anyone noticed a lot of ML research into facial recognition of Uyghur people lately?
  * Kingma and Ba (2015) Kingma, DP and Ba, JL (2015).  Adam: A method for stochastic optimization.  In _ICLR_.  arXiv:arXiv:1412.6980v9.
  * Krizhevsky and Hinton (2009) Krizhevsky, A and Hinton, G (2009).  Learning Multiple Layers of Features from Tiny Images.  Technical report, University of Toronto.
  * Krizhevsky, Sutskever, and Hinton (2012) Krizhevsky, A, Sutskever, I, and Hinton, GE (2012).  ImageNet Classification with Deep Convolutional Neural Networks.  In _NIPS_.
  * Krueger, Caballero, Jacobsen et al. (2021) Krueger, D, Caballero, E, Jacobsen, JH, Zhang, A, Binas, J, et al. (2021).  Out-of-Distribution Generalization via Risk Extrapolation.  In _ICML_.
  * van Laarhoven (2017) van Laarhoven, T (2017).  L2 Regularization versus Batch and Weight Normalization.  _arXiv:1706.05350_.  arXiv:1706.05350.
  * Leavitt and Morcos (2020) Leavitt, ML and Morcos, A (2020).  Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs.  _arXiv:2003.01262_.  arXiv:arXiv:2003.01262v3.
  * Lecun (2019) Lecun, Y (2019).  The Epistemology of Deep Learning.  In _Talk at the Institute for Advanced Study_.
  * LeCun, Bengio, and Hinton (2015) LeCun, Y, Bengio, Y, and Hinton, G (2015).  Deep learning.  _Nature_ , 521(7553):436–444.  10.1038/nature14539.  arXiv:arXiv:1312.6184v5.
  * LeCun, Bottou, Bengio et al. (1998) LeCun, Y, Bottou, L, Bengio, Y, and Haffner, P (1998).  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2323.  10.1109/5.726791.  arXiv:1102.0183.
  * Li, Isupova, Haghtalab et al. (2021) Li, Y, Isupova, O, Haghtalab, N, White, A, and Granziol, D (2021).  The ICML Debate: Should AI Research and Development Be Controlled by a Regulatory Body or Government Oversight?
  * Liang and Srikant (2017) Liang, S and Srikant, R (2017).  Why Deep Neural Networks for Function Approximation?  In _ICLR_.  arXiv:1610.04161.
  * Lillicrap, Santoro, Marris et al. (2020) Lillicrap, TP, Santoro, A, Marris, L, Akerman, CJ, and Hinton, G (2020).  Backpropagation and the brain.  _Nature Reviews Neuroscience_ , 21(6):335–346.  10.1038/S41583-020-0277-3.
  * Lin, Tegmark, and Rolnick (2017) Lin, HW, Tegmark, M, and Rolnick, D (2017).  Why does deep and cheap learning work so well?  _Journal of Statistical Physics_ , 168:1223–1247.  arXiv:arXiv:1608.08225v2.
  * Linnainmaa (1970) Linnainmaa, S (1970).  _The representation of the cumulative rounding error of an algorithm as a Taylor expansion of the local rounding errors_.  Ph.D. thesis, Univ. Helsinki.
  * Liu, Bernstein, Meister et al. (2021) Liu, Y, Bernstein, J, Meister, M, and Yue, Y (2021).  Learning by Turning: Neural Architecture Aware Optimisation.  In _ICML_ , pages 6748–6758. PMLR.
  * Lowe (1999) Lowe, DG (1999).  Object recognition from local scale-invariant features.  In _ICCV_ , pages 1150–1157.
  * Mangalam and Prabhu (2019) Mangalam, K and Prabhu, VU (2019).  Do deep neural networks learn shallow learnable examples first?  In _Workshop Deep Phenomena, ICML_.
  * Mansur and Kuno (2008) Mansur, A and Kuno, Y (2008).  Improving Recognition through Object Sub-categorization.  In _Advances in Visual Computing_ , pages 851–859. Springer.
  * Martinetz and Schulten (1991) Martinetz, T and Schulten, K (1991).  A ”neural-gas” network learns topologies.  _Arti ficial Neural Networks_.
  * Masters and Luschi (2018) Masters, D and Luschi, C (2018).  Revisiting Small Batch Training for Deep Neural Networks.  _arXiv:1804.07612_.  arXiv:arXiv:1804.07612v1.
  * McCulloch and Pitts (1943) McCulloch, WS and Pitts, W (1943).  A logical calculus of the ideas immanent in nervous activity.  _Bulletin of Mathematical Biology_ , 52(1-2):99–115.  10.1007/BF02459570.
  * Mnih, Heess, Graves et al. (2014) Mnih, V, Heess, N, Graves, A, and Kavukcuoglu, K (2014).  Recurrent Models of Visual Attention.  In _NIPS_.
  * Morcos, Barrett, Rabinowitz et al. (2018) Morcos, AS, Barrett, DGT, Rabinowitz, NC, and Botvinick, M (2018).  On the importance of single directions for generalization.  In _ICLR_.  arXiv:arXiv:1803.06959v4.
  * Müller, Kornblith, and Hinton (2019) Müller, R, Kornblith, S, and Hinton, G (2019).  When Does Label Smoothing Help ?  In _NeurIPS_.  arXiv:arXiv:1906.02629v3.
  * Murphy (2021) Murphy, KP (2021).  _Probabilistic Machine Learning: An introduction_.  MIT Press.
  * Nagarajan, Andreassen, and Neyshabur (2021) Nagarajan, V, Andreassen, A, and Neyshabur, B (2021).  Understanding the failure modes of out-of-distribution generalization.  In _ICLR_.
  * Nair and Hinton (2010) Nair, V and Hinton, GE (2010).  Rectified Linear Units Improve Restricted Boltzmann Machines.  In _ICML_ , pages 807—-814.
  * Neyshabur, Tomioka, and Srebro (2015) Neyshabur, B, Tomioka, R, and Srebro, N (2015).  In search of the real inductive bias: On the role of implicit regularization in deep learning.  In _ICLR_.  arXiv:arXiv:1412.6614v4.
  * Nguyen, Yosinski, and Clune (2015) Nguyen, A, Yosinski, J, and Clune, J (2015).  Deep Neural Networks are Easily Fooled : High Confidence Predictions for Unrecognizable Images.  In _CVPR_ , pages 427–436.  arXiv:arXiv:1412.1897v4.
  * Oquab, Bottou, Laptev et al. (2014) Oquab, M, Bottou, L, Laptev, I, and Sivic, J (2014).  Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks.  In _CVPR_ , pages 1717–1724.
  * Panesar (2019) Panesar, A (2019).  _Machine learning and AI for healthcare_.  Apress Media.  ISBN 9781484265369.  10.1007/978-1-4842-6537-6.
  * Radford, Wu, Amodei et al. (2019) Radford, A, Wu, J, Amodei, D, Amodei, D, Clark, J, et al. (2019).  Better Language Models and Their Implications.
  * Rahimi (2017) Rahimi, A (2017).  Test of Time Award reception.  In _NIPS_.
  * Recht, Roelofs, Schmidt et al. (2019) Recht, B, Roelofs, R, Schmidt, L, and Shankar, V (2019).  Do ImageNet Classifiers Generalize to ImageNet?  In _ICML_ , pages 5389–5400.
  * Rolnick, Donti, Kaack et al. (2019) Rolnick, D, Donti, PL, Kaack, LH, Kochanski, K, Lacoste, A, et al. (2019).  Tackling Climate Change with Machine Learning.  _arXiv:1906.05433_.  arXiv:1906.05433.
  * Rosenblatt (1958) Rosenblatt, F (1958).  The perceptron: a probabilistic model for information storage and organization in the brain.  _Psychological review_ , 65(6):386–408.
  * Rumelhart, Hinton, and Williams (1986) Rumelhart, DE, Hinton, GE, and Williams, RJ (1986).  Learning representations by back-propagating errors.  _Nature_ , 323(6088):533–536.  10.1038/323533a0.
  * Schaffer (1994) Schaffer, C (1994).  A Conservation Law for Generalization Performance.  In _International Conference on Machine Learning_ , pages 259–265. Morgan Kaufmann.  10.1016/B978-1-55860-335-6.50039-8.
  * Schmidhuber (2014) Schmidhuber, J (2014).  Deep Learning in Neural Networks: An Overview.  _arXiv preprint arXiv:1404.7828_ , pages 1–66.  10.1016/j.neunet.2014.09.003.  arXiv:arXiv:1404.7828v1.
  * Shane, Metz, and Wakabayashi (2018) Shane, S, Metz, C, and Wakabayashi, D (2018).  How a Pentagon Contract Became an Identity Crisis for Google.
  * Shankar, Roelofs, Mania et al. (2020) Shankar, V, Roelofs, R, Mania, H, Fang, A, Recht, B, et al. (2020).  Evaluating Machine Accuracy on ImageNet.  In _ICML_ , pages 8634–8644. PMLR.
  * Silver, Huang, Maddison et al. (2016) Silver, D, Huang, A, Maddison, CJ, Guez, A, Sifre, L, et al. (2016).  Mastering the game of Go with deep neural networks and tree search.  _Nature_ , 529(7585):484–489.  10.1038/nature16961.
  * Simonyan, Vedaldi, and Zisserman (2014) Simonyan, K, Vedaldi, A, and Zisserman, A (2014).  Deep Inside Convolutional Networks : Visualising Image Classification Models and Saliency Maps.  _ICLR_.  arXiv:arXiv:1312.6034v2.
  * Simonyan and Zisserman (2015) Simonyan, K and Zisserman, A (2015).  Very Deep Convolutional Networks for Large-Scale Image Recognition.  In _ICLR_.  arXiv:arXiv:1409.1556v6.
  * Smith and Topin (2017) Smith, LN and Topin, N (2017).  Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates.  _arXiv:1708.07120_.  arXiv:arXiv:1708.07120v2.
  * Smith, Dherin, Barrett et al. (2021) Smith, SL, Dherin, B, Barrett, D, and De, S (2021).  On the Origin of Implicit Regularization in Stochastic Gradient Descent.  In _ICLR_.
  * Smith and Le (2017) Smith, SL and Le, QV (2017).  A bayesian perspective on generalization and stochastic gradient descent.  In _Proceedings of Second workshop on Bayesian Deep Learning (NIPS 2017)_.  arXiv:arXiv:1710.06451v3.
  * Srivastava, Hinton, Krizhevsky et al. (2014) Srivastava, N, Hinton, GE, Krizhevsky, A, Sutskever, I, and Salakhutdinov, R (2014).  Dropout : A Simple Way to Prevent Neural Networks from Overfitting.  _Journal of Machine Learning Research_ , 15(1):1929–1958.
  * Su, Vargas, and Kouichi (2019) Su, J, Vargas, DV, and Kouichi, S (2019).  One pixel attack for fooling deep neural networks.  _IEEE Transactions on Evolutionary Computation_ , 23(5):828–841.  10.1109/tevc.2019.2890858.  arXiv:1710.08864.
  * Szegedy, Zaremba, Sutskeveer et al. (2014) Szegedy, C, Zaremba, W, Sutskeveer, I, Bruna, J, Erhan, D, et al. (2014).  Intriguing properties of neural networks.  In _ICLR_.  arXiv:arXiv:1312.6199v4.
  * Telgarsky (2016) Telgarsky, M (2016).  benefits of depth in neural networks.  In _Annual Conference on Learning Theory_ , volume 49, pages 1517–1539. PMLR.
  * Tieleman and Hinton (2012) Tieleman, T and Hinton, G (2012).  Lecture 6.5—RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning.
  * Torralba and Efros (2011) Torralba, A and Efros, AA (2011).  Unbiased look at dataset bias.  In _Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition_ , pages 1521–1528. IEEE Computer Society.  10.1109/CVPR.2011.5995347.
  * Vapnik (1989) Vapnik, V (1989).  _Statistical learning theory_.  Wiley New York.
  * Vapnik and Izmailov (2019) Vapnik, V and Izmailov, R (2019).  Rethinking statistical learning theory: learning using statistical invariants.  _Machine Learning_ , 108(3):381–423.  10.1007/s10994-018-5742-0.
  * Vapnik and Chervonenkis (1968) Vapnik, VN and Chervonenkis, AY (1968).  On the uniform convergence of relative frequencies of events to their probabilities.  _Dokl. Akad. Nauk._ , 181(4):781.
  * Wang and Raj (2017) Wang, H and Raj, B (2017).  On the Origin of Deep Learning.  _arXiv:1702.07800_ , pages 1–72.  arXiv:arXiv:1702.07800v4.
  * Werbos (1982) Werbos, PJ (1982).  Applications of advances in nonlinear sensitivity analysis.  In _System Modeling and Optimization_ , pages 762–770. Springer-Verlag.  10.1007/BFB0006203.
  * Widrow and Hoff (1960) Widrow, B and Hoff, ME (1960).  Adaptive switching circuits.  _IRE WESCON Convention Record_ , 4:96–104.
  * Wigner (1960) Wigner, E (1960).  The unreasonable effectiveness of mathematics in the natural sciences.  _Communications in Pure and Applied Mathematics_ , 13:1–14.
  * Wilson, Roelofs, Stern et al. (2017) Wilson, AC, Roelofs, R, Stern, M, Srebro, N, and Recht, B (2017).  The Marginal Value of Adaptive Gradient Methods in Machine Learning.  In _NIPS_ , pages 4151–4161.
  * Wolpert (1996) Wolpert, DH (1996).  The lack of a prior distinctions between learning algorithms and the existence of a priori distinctions between learning algorithms.  _Neural Computation_ , 8:1341–1421.
  * Wu, Zou, Braverman et al. (2021) Wu, J, Zou, D, Braverman, V, and Quanquan Gu (2021).  Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate.  In _ICLR_.
  * Yosinski, Clune, Nguyen et al. (2015) Yosinski, J, Clune, J, Nguyen, A, Fuchs, T, and Lipson, H (2015).  Understanding Neural Networks Through Deep Visualization.  In _Deep Learning Workshop at ICML_.  arXiv:arXiv:1506.06579v1.
  * Young, Hazarika, Poria et al. (2018) Young, T, Hazarika, D, Poria, S, and Cambria, E (2018).  Recent trends in deep learning based natural language processing.  _ieee Computational intelligence magazine_ , 13(3):55–75.  10.1038/nature14539.
  * Yu, Lin, Salakhutdinov et al. (2017) Yu, AW, Lin, Q, Salakhutdinov, R, and Carbonell, J (2017).  Normalized gradient with adaptive stepsize method for deep neural network training.  _arXiv:1707.04822_.
  * Yuan (2020) Yuan, Y (2020).  YOLO Creator Joseph Redmon Stopped CV Research Due to Ethical Concerns.  _Medium_.
  * Yun, Krishnan, and Mobahi (2021) Yun, C, Krishnan, S, and Mobahi, H (2021).  A unifying view on implicit bias in training linear neural networks.  In _ICLR_.
  * Zagoruyko (2015) Zagoruyko, S (2015).  92.45% on CIFAR-10 in Torch.  _http://torch.ch/blog/2015/07/30/cifar.html_.
  * Zagoruyko and Komodakis (2016) Zagoruyko, S and Komodakis, N (2016).  Wide Residual Networks.  In _BMVC_.  arXiv:arXiv:1605.07146v2.
  * Zeiler and Fergus (2014) Zeiler, MD and Fergus, R (2014).  Visualizing and Understanding Convolutional Networks.  _ECCV_ , pages 818–833.  arXiv:arXiv:1311.2901v3.
  * Zhang, Bengio, Hardt et al. (2017) Zhang, C, Bengio, S, Hardt, M, Recht, B, and Vinyals, O (2017).  Understanding deep learning requires re-thinking generalization.  In _ICLR_.  arXiv:arXiv:1611.03530v1.
  * Zhang, Bengio, Hardt et al. (2021) Zhang, C, Bengio, S, Hardt, M, Recht, B, and Vinyals, O (2021).  Understanding deep learning (still) requires rethinking generalization.  _Communications of the ACM_ , 64(3):107–115.  10.1145/3446776.
  * Zhang, Wang, Xu et al. (2019) Zhang, G, Wang, C, Xu, B, and Grosse, R (2019).  Three mechanisms of weight decay regularization.  In _ICLR_.
  * Zhang, Cisse, Dauphin et al. (2018) Zhang, H, Cisse, M, Dauphin, YN, and Lopez-Paz, D (2018).  mixup: Beyond Empirical Risk Minimization.  In _ICLR_. International Conference on Learning Representations, ICLR.  arXiv:1710.09412.
  * Zhao, Wu, Lau et al. (2021) Zhao, N, Wu, Z, Lau, RWH, and Lin, S (2021).  What makes instance discrimination good for transfer learning?  In _ICLR_.  arXiv:2006.06606.
  * Zhou, Khosla, Lapedriza et al. (2015) Zhou, B, Khosla, A, Lapedriza, A, Oliva, A, and Torralba, A (2015).  Object detectors emerge in Deep Scene CNNs.  In _ICLR_.  arXiv:arXiv:1412.6856v2.
  * Zielinski, Krishnan, and Chatterjee (2020) Zielinski, P, Krishnan, S, and Chatterjee, S (2020).  Weak and Strong Gradient Directions: Explaining Memorization, Generalization, and Hardness of Examples at Scale.  _arXiv:2003.07422_.  arXiv:2003.07422.
  * Zoran and Weiss (2011) Zoran, D and Weiss, Y (2011).  From learning models of natural image patches to whole image restoration.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 479–486.  10.1109/ICCV.2011.6126278.

## Publications

This thesis wraps up a series of works that have been previously published in
several conference venues:

  * •

Simon Carbonnelle, C. De Vleeschouwer. Intraclass clustering: an implicit
learning ability that regularizes DNNs, ICLR 2021.

  * •

Simon Carbonnelle, C. De Vleeschouwer. Experimental study of the neuron-level
mechanisms emerging from backpropagation, ESANN 2019.

  * •

Simon Carbonnelle, C. De Vleeschouwer. Layer rotation: a surprisingly simple
indicator of generalization in deep networks, Workshop Deep Phenomena, ICML
2019.

My journey as a PhD student started with a collaboration with Claire Gosse and
Marie Van Reybroeck, two researchers in speech and language pathology. This
collaboration initiated an ongoing research project that studies the
interactions between handwriting, spelling and dyslexia through the use of
digital tablets and signal processing tools. My work in this context also
contributed to the following two publications:

  * •

C. Gosse, S. Carbonnelle, C. De Vleeschouwer, M. Van Reybroeck. Specifying the
graphic characteristics of words that influence children’s handwriting,
Reading and Writing, 31 (5), 1181-1207, 2018.

  * •

C. Gosse, S. Carbonnelle, C. De Vleeschouwer, M. Van Reybroeck. The influence
of graphic complexities of words on the handwriting of children of 2nd grade,
SIG WRITING, 16th international conference of the EARLI special interest group
on writing, Liverpool, 2016.