13.5. RECURRENT NEURAL NETWORKS
547
as an exercise for the reader. Now, noting that dst−1
dW
is just Rt−1, we have the
recurrence:
Rt = A(B + W TRt−1)
Setting Pt = AB and Qt = AW T, we end up with:
Rt = Pt + QtRt−1
(13.7)
We can use this recurrence to set up an iterative evaluation of Rt, and thence
de
dW . We initialize the iteration by setting R0 to the matrix with all zeros. This
iterative method for computing the gradients of an RNN is called Backpropa-
gation Through Time (BPTT), since it reveals the eﬀects of earlier time-steps
on later time-steps.
13.5.2
Vanishing and Exploding Gradients
RNN’s are a simple and appealing model for learning from sequences, and
the BPTT algorithm is straightforward to implement. Unfortunately, RNN’s
have a fatal ﬂaw that limits their use in many practical applications. They
are eﬀective only at learning short-term connections between nearby elements
in the sequence and ineﬀective at learning long-distance connections. Long-
distance connections are crucial in many applications; for example, there can
be an arbitrary number of words or clauses separating a verb or a pronoun from
the subject with which it is associated.
In order to understand the cause of this limitation, let us unroll Equa-
tion 13.7:
Rt
=
Pt + QtRt−1
=
Pt + Qt(Pt−1 + Qt−1Rt−2)
=
Pt + QtPt−1 + QtQt−1Rt−2
. . .
Ultimately yielding:
Rt = Pt +
t−1
X
j=0
Pj
tY
k=j+1
Qk
(13.8)
From Equation 13.8, it is clear that the contribution of step i to Rt is given by:
Ri
t = Pi
tY
k=i+1
Qk
(13.9)
Equation 13.9 includes the product of several matrices that look like diagonal
matrix A. Each entry in A is strictly less than 1. Just as the product of many
numbers, each strictly less than 1, approaches zero as we add more multipli-
cands, the term Qt
k=i+1 Qk approaches zero for i ≪t. In other words, the
gradient at step t is determined entirely by the preceding few time steps, with
