<p>You do not provide deep nets with supervision on each layer, this would be too complex in terms of building the dataset. What you see on these slides is interpretation of what is happening <strong>on its own</strong>, not what <strong>we enforce</strong>. There are both layer-by-layer techniques (less popular now) and everything-jointly (popular know), but neither of them use additional supervision, you do not tell network to extract edges, it simply emerges from the optimization problem and network structure in practise. </p>

<p>However, there are also deep architectures that do not have this property, like <a href="https://arxiv.org/abs/1603.09382" rel="nofollow">https://arxiv.org/abs/1603.09382</a> or in general - recurrent nets (which are also "deep" in this sense). Thus do not treat this as a property of the deep learning, this is simply <strong>a common empirical observation when dealing with particular data</strong>, nothing less and nothing more.</p>
Are deep neural networks taught layer by layer or all layers at once?
<p>I'm trying to grasp the concepts of deep neural networks. When they are explained, they basically say that each layer of the network represents one level of abstraction, for example, the first layer is about edges, next layer is about shapes, like wheels and the next layer about what the wheels add up to, like a car.</p>

<p>This image pretty much represents the concept:
<a href="https://i.stack.imgur.com/795lQ.jpg" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/795lQ.jpg" alt="enter image description here"></a></p>

<p>When figuring out the weights for each layer, is this done one layer at the time or all layers together. Do you first run the AI on a set of images labeled with different kinds of edges and then a set of images labeled with things like wheels and then on a set of images labeled with cars or do you let the network figure that out for itself?</p>