12.2. PERCEPTRONS
481
straight line that separates day trips from excursions. However, if we replace
the Cartesian coordinates by polar coordinates, then in the transformed space of
polar coordinates, the dashed circle shown in Fig. 12.10 becomes a hyperplane.
Formally, we transform the vector x = [x1, x2] into [
p
x2
1 + x2
2, arctan(x2/x1)].
In fact, we can also do dimensionality reduction of the data. The angle of
the point is irrelevant, and only the radius
p
x2
1 + x2
2 matters. Thus, we can
turn the point vectors into one-component vectors giving the distance of the
point from the origin. Associated with the small distances will be the class
label “day trip,” while the larger distances will all be associated with the label
“excursion.” Training the perceptron is extremely easy.
✷
12.2.7
Problems With Perceptrons
Despite the extensions discussed above, there are some limitations to the ability
of perceptrons to classify some data. The biggest problem is that sometimes
the data is inherently not separable by a hyperplane. An example is shown in
Fig. 12.11. In this example, points of the two classes mix near the boundary so
that any line through the points will have points of both classes on at least one
of the sides.
Figure 12.11: A training set may not allow the existence of any separating
hyperplane
As mentioned in Section 12.2.6, it is, in principle, possible to ﬁnd some
function on the points that transforms them to another space where they are
linearly separable. However, doing so could well lead to overﬁtting, the situation
where the classiﬁer works very well on the training set, because it has been
carefully designed to handle each training example correctly. However, because
the classiﬁer is exploiting details of the training set that do not apply to other
