Title: How deep convolutional neural networks lose spatial information with training. Abstract: A central question of machine learning is how deep nets  manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building a representation of the data where information  irrelevant to the task is lost. For image data sets, this view is supported by the observation that after (and not before) training,  the neural representation becomes less and less sensitive to diffeomorphisms acting on images as the signal propagates through the net.  
This loss of sensitivity correlates with performance, and surprisingly  also correlates  with a gain of sensitivity to white noise acquired during training. These facts are unexplained, and as we demonstrate still hold when white noise is added to the images of the training set. Here, we (i) show empirically for various architectures that stability to image diffeomorphisms is achieved by spatial pooling in the first half of the net, and by channel pooling in the second half, (ii) introduce a scale-detection task for a simple model of data where pooling is learnt during training, which captures all empirical observations above and (iii) compute in this model how stability to diffeomorphisms and noise scale with depth. The scalings are found to depend on the presence of strides in the net architecture. We find that the increased sensitivity to noise is due to the perturbing noise piling up during pooling, after a ReLU non-linearity is applied to the noise in the internal layers.