482
CHAPTER 12. LARGE-SCALE MACHINE LEARNING
examples that must be classiﬁed in the future, the classiﬁer will not perform
well on new data.
?
Figure 12.12: Generally, more that one hyperplane can separate the classes if
they can be separated at all
Another problem is illustrated in Fig. 12.12. Usually, if classes can be sep-
arated by one hyperplane, then there are many diﬀerent hyperplanes that will
separate the points. However, not all hyperplanes are equally good. For in-
stance, if we choose the hyperplane that is furthest clockwise, then the point
indicated by “?” will be classiﬁed as a circle, even though we intuitively see it as
closer to the squares. When we meet support-vector machines in Section 12.3,
we shall see that there is a way to insist that the hyperplane chosen be the one
that in a sense divides the space most fairly.
Yet another problem is illustrated by Fig. 12.13. Most rules for training
a perceptron stop as soon as there are no misclassiﬁed points.
As a result,
the chosen hyperplane will be one that just manages to classify some of the
points correctly. For instance, the upper line in Fig. 12.13 has just managed
to accommodate two of the squares, and the lower line has just managed to
accommodate two of the circles.
If either of these lines represent the ﬁnal
weight vector, then the weights are biased toward one of the classes.
That
is, they correctly classify the points in the training set, but the upper line
would classify new squares that are just below it as circles, while the lower line
would classify circles just above it as squares. Again, a more equitable choice
of separating hyperplane will be shown in Section 12.3.
12.2.8
Parallel Implementation of Perceptrons
The training of a perceptron is an inherently sequential process. If the num-
ber of dimensions of the vectors involved is huge, then we might obtain some
