494
CHAPTER 12. LARGE-SCALE MACHINE LEARNING
for u
for v
for b
if
u + 4v + b ≥+1
then
0
else
−1
−4
−1
if
2u + 2v + b ≥+1
then
0
else
−2
−2
−1
if
3u + 4v + b ≥+1
then
0
else
−3
−4
−1
if
u + v + b ≤−1
then
0
else
+1
+1
+1
if
2u + v + b ≤−1
then
0
else
+2
+1
+1
if
3u + v + b ≤−1
then
0
else
+3
+1
+1
Figure 12.19: Sum each of these terms and multiply by C to get the contribution
of bad points to the derivatives of f with respect to u, v, and b
w = [u, v]
b
Bad
∂/∂u
∂/∂v
∂/∂b
(1)
[0.000, 1.000]
−2.000
oxoooo
−0.200
0.800
−2.100
(2)
[0.040, 0.840]
−1.580
oxoxxx
0.440
0.940
−1.380
(3)
[−0.048, 0.652]
−1.304
oxoxxx
0.352
0.752
−1.104
(4)
[−0.118, 0.502]
−1.083
xxxxxx
−0.118
−0.198
−1.083
(5)
[−0.094, 0.542]
−0.866
oxoxxx
0.306
0.642
−0.666
(6)
[−0.155, 0.414]
−0.733
xxxxxx
Figure 12.20: Beginning of the process of gradient descent
Consider line (1). It shows the initial value of w = [0, 1], as we suggested
in Fig. 12.18 was the initial value of w. Recall that we use u and v for the
components of w, so u = 0 and v = 1. We also see the initial value of b = −2,
which is the appropriate value for the initial hyperplane shown in Fig. 12.18.
We must use these values of u, v, and b to evaluate the conditions in Fig. 12.19.
The ﬁrst of the conditions in Fig. 12.19 is u + 4v + b ≥+1. The left side is
0 + 4 + (−2) = 2, so the condition is satisﬁed. However, the second condition,
2u+2v+b ≥+1 fails. The left side is 0+2+(−2) = 0. The fact that the sum is
0 means the second point (2, 2) is exactly on the separating hyperplane, and not
outside the margin. The third condition is satisﬁed, since 0+4+(−2) = 2 ≥+1.
The last three conditions are also satisﬁed, and in fact are satisﬁed exactly. For
instance, the fourth condition is u+v+b ≤−1. The left side is 0+1+(−2) = −1.
Thus, the pattern oxoooo represents the outcome of these six conditions, as we
see in the ﬁrst line of Fig. 12.20.
We use these conditions to compute the partial derivatives. For ∂f/∂u, we
use u in place of wj in Equation 12.6. This expression thus becomes
u + C
0 + (−2) + 0 + 0 + 0 + 0

= 0 + 1
10(−2) = −0.2
The sum multiplying C can be explained this way. For each of the six conditions
of Fig. 12.19, take 0 if the condition is satisﬁed, and take the value in the
column labeled “for u” if it is not satisﬁed. Similarly, for v in place of wj we
