536
CHAPTER 13. NEURAL NETS AND DEEP LEARNING
weights wij of the ﬁlter are determined, the ﬁlter will recognize some feature of
the image, and the activation map tells whether (or to what degree) this feaure
is present at each position of the image.
Example 13.8 : In Fig. 13.10(b), we see a 2 × 2 ﬁlter, which is to be applied
to the 4 × 4 image in Fig. 13.10(b). To do so, we lay the ﬁlter over all nine of
the 2 × 2 squares of the image. In the ﬁgure, we suggest the ﬁlter being placed
over the 2 × 2 square in the upper-right corner. After overlaying the ﬁlter, we
multiply each of the ﬁlter elements by the corresponding image element and
then take the sum of the products. In principle, we then need to add in a bias
term, but in this example, we shall assume the bias is 0.
1   0   1   0
0   1   0   1
1   1   0   0
0   0   0   1
1    0
0   −1
0   0   0
(b) 2−by−2 filter
1   1  −1
(a) 4−by−4 image
(c) 3−by−3 response
−1  1   0
Figure 13.10: Applying a ﬁlter to an image
Another way to look at this process is that we turn the ﬁlter into a vector
by concatenating its rows, in order, and we do the same to the square of the
image. Then, we take the dot product of the vectors. For instance, the ﬁlter
can be thought of as the vector [1, 0, 0, −1], and the square in the upper-left
corner of the image can be thought of as the vector [1, 0, 0, 1]. The dot product
of these vectors is 1 × 1 + 0 × 0 + 0 × 0 + (−1) × 1 = 0. Thus, the result, shown
in Fig. 13.10(c), has a 0 for its upper-left entry.
For another example, if we slide the ﬁlter down one row, the dot product of
the ﬁlter as a vector and the vector formed from the ﬁrst two elements of the
second and third rows of the image is 1 × 0 + 0 × 1 + 0 × 1 + (−1) × 1 = −1.
Thus, the ﬁrst element of the second row of the convolution is −1.
✷
When we deal with color images, the input has three channels. That is, each
pixel is represented by three values, one for each color. Suppose we have a color
image of size 224 × 224 × 3. The ﬁlter’s output will also have three channels,
and so the ﬁlter is now encoded by a 5 × 5 × 3 weight matrix W and single
bias parameter b. The activation map R still remains 5 × 5, with each response
given by:
rij =
4
X
k=0
4
X
l=0
3
X
d=1
xi+k,j+l,dwkld + b
(13.2)
In our example, we imagined a ﬁlter of size 5. In general, the size of the ﬁlter is
a hyperparameter of the convolutional layer. Filters of size 3, 4, or 5 are most
