12.2. PERCEPTRONS
473
x1
x1
w.x = 0
w
w’
w’.x = 0
η
Figure 12.5: A misclassiﬁed point x1 moves the vector w
Example 12.4 : Let us consider training a perceptron to recognize spam email.
The training set consists of pairs (x, y) where x is a vector of 0’s and 1’s, with
each component xi corresponding to the presence (xi = 1) or absence (xi = 0)
of a particular word in the email. The value of y is +1 if the email is known
to be spam and −1 if it is known not to be spam. While the number of words
found in the training set of emails is very large, we shall use a simpliﬁed example
where there are only ﬁve words: “and,” “viagra,” “the,” “of,” and “nigeria.”
Figure 12.6 gives the training set of six vectors and their corresponding classes.
and
viagra
the
of
nigeria
y
a
1
1
0
1
1
+1
b
0
0
1
1
0
−1
c
0
1
1
0
0
+1
d
1
0
0
1
0
−1
e
1
0
1
0
1
+1
f
1
0
1
1
0
−1
Figure 12.6: Training data for spam emails
In this example, we shall use learning rate η = 1/2, and we shall visit
each training example once, in the order shown in Fig. 12.6. We begin with
w = [0, 0, 0, 0, 0] and compute w.a = 0. Since 0 is not positive, we move w in
the direction of a by performing w := w + (1/2)(+1)a. The new value of w
is thus
w = [0, 0, 0, 0, 0] + [1
2, 1
2, 0, 1
2, 1
2] = [1
2, 1
2, 0, 1
2, 1
2]
Next, consider b. w.b = [ 1
2, 1
2, 0, 1
2, 1
2].[0, 0, 1, 1, 0] = 1
2. Since the associated
