Title: Generalization Mechanics in Deep Learning. Abstract: Deep neural nets are well known for their ability to generalize, but understanding the mechanics of generalization remains an important open question. We analyze the probability-flow through a deep net, especially what happens to this flow during the training process.
Quantitatively, this study leads to a new approach for obtaining generalization bounds, using which we derive a non-vacuous bound for an over-parameterized deep network that learns relatively small dataset such as CIFAR-10. On the qualitative side, this reveals that
the process of learning imposes an implicit regularization, \viz orbit-regularization, as an essential ingredient for generalization. 
