540
CHAPTER 13. NEURAL NETS AND DEEP LEARNING
VGGnet.9 This simple network strictly alternates convolutional and pooling
layers. In practice, high-performing networks are ﬁnely tuned to their task,
and may stack convolutional layers directly on top of one another with the
occasional pooling layer in between. Moreover, there is often more than one
fully-connected layer before the ﬁnal output. The input to the ﬁrst layer is the
224-by-224, 3-channel image. The input to each subsequent layer is the output
of the previous layer.
Layer Type
Size
Stride
Pad
Filter Count
Output Size
Convolution
3
1
1
64
224 × 224 × 64
Max Pool
2
2
0
64
112 × 112 × 64
Convolution
3
1
1
128
112 × 112 × 128
Max Pool
2
2
0
128
56 × 56 × 128
Convolution
3
1
1
256
56 × 56 × 256
Max Pool
2
2
0
256
28 × 28 × 256
Convolution
3
1
1
512
14 × 14 × 512
Max Pool
2
2
0
512
14 × 14 × 512
Convolution
3
1
1
1024
14 × 14 × 1024
Max Pool
2
2
0
1024
7 × 7 × 1024
Fully Connected
1 × 1 × 1000
Figure 13.11: Layers of a Convolutional Neural Network
The ﬁrst layer is a convolutional layer, consisting of 64 ﬁlters, each with
three channels, as would be the case for any color-image processor. The ﬁlters
are 3-by-3, and the stride is 1, so every 3-by-3 square of the image is an input
to the ﬁlter. There is one unit of zero-padding, so the number of outputs of
this layer equals the number of inputs. Further, notice that we can view the
output as another 224-by-224 array. Each element of this array consists of 64
ﬁlters, each of which is a 3-channel pixel.
The output of the ﬁrst layer is fed to a max-pool layer, in which we divide
the 224-by-224 array into 2-by-2 squares (because both the size and stride are
2). Thus, the 224-by-224 array has become a 112-by-112 array, and there are
still the same 64 ﬁlters.
At the third layer, which is again convolutional, we take the 112-by-112
array of pixels from the second layer as input.
This layer has more ﬁlters
than the ﬁrst layer – 128 to be exact. The intuitive reason for the increase is
that the ﬁrst layer recognizes very simple structures, like edges, and there are
not too many diﬀerent simple structures. However, the third layer should be
recognizing somewhat more complex features, and these features can involve a
6-by-6 square of the input image, because of the pooling done at the second
layer.
Similarly, each subsequent convolutional layer takes inputs from the
9K. Simonyan and A. Zussman,“Very Deep Convolutional Networks for Large-Scale Image
Recognition,” arXiv:1409–1556v6.
