13.4. CONVOLUTIONAL NEURAL NETWORKS
535
13.4.1
Convolutional Layers
Convolutional layers make use of the fact that image features often are described
by small contiguous areas in the image. For example, at the ﬁrst convolutional
layer, we might recognize small sections of edges in the image. At later layers,
more complex structures, such as features that look like ﬂower petals or eyes
might be recognized. The idea that simpliﬁes the calculation is the fact that
the recognition of features such as edges does not depend on where in the image
the edge is. Thus, we can train a single node to recognize a small section of
edge, say an edge through a 5-by-5 region of pixels. This idea beneﬁts us in
two ways.
1. The node in question needs only inputs from 25 pixels corresponding to
any 5-by-5 square, not inputs from all 224-by-224 pixels. That saves us a
lot in the representation of the trained CNN.
2. The number of weights that we need to learn in the training process is
greatly reduced. For each node in the layer, we require only one weight
for each input to that node – say 75 weights if a pixel is represented by
RGB values, not the 150,528 weights that we suggested above would be
needed for an ordinary, fully connected layer.
We shall think of the nodes in a convolutional layer as ﬁlters for learning
features. A ﬁlter examines a small spatially contiguous area of the image –
traditionally, a small square area such as a 5 × 5 square of pixels. Moreover,
since many features of interest may occur anywhere in the input image (and
possibly in more than one location), we apply the same ﬁlter at many locations
on the input.
To keep things simple, suppose the input consists of monochromatic images,
that is, grey-scale images whose pixels are each a single value. Each image is
thus encoded by a 2-dimensional pixel array of size 224 × 224. A 5 × 5 ﬁlter F
is encoded by a 5 × 5 weight matrix W and single bias parameter b. When the
ﬁlter is applied on a similarly-sized square region of input image X with the
top left corner of the ﬁlter aligned with the image pixel xij, the response of the
ﬁlter at this position, denoted rij, is given by:
rij =
4
X
k=0
4
X
l=0
xi+k,j+lwkl + b
(13.1)
We now slide the ﬁlter along the length and width of the input image, applying
the ﬁlter at each position, so that we capture every possible 5 × 5 square region
of pixels in the image. Notice that we can apply the ﬁlter at input locations
1 ≤i ≤220 and 1 ≤j ≤220, although it does not “ﬁt” at positions with a
higher i or j. The resulting set of responses rij are then passed through an
activation function to form the activation map R of the ﬁlter. In most cases,
the activation function is ReLU or one of its variants. When trained, i.e., the
