13.1. INTRODUCTION TO NEURAL NETS
511
recognize are those that end with 11 but do not have 11 elsewhere. these are
0011 and 1011.
Fortunately, the second node in the ﬁrst layer, with weights [0, 0, 1, 1] and
threshold 1.5 gives output 1 whenever x3 = x4 = 1, and not otherwise. This
node thus recognizes the inputs 0011 and 1011, as well as some other good
inputs that are also recognized by the ﬁrst node.
Now, let us turn to the second layer, with a single node; that node has
weights [1, 1] and threshold 0.5. It thus behaves as an “OR-gate.” It gives
output y = 1 whenever either or both of the nodes in the ﬁrst layer have output
1, but gives output y = 0 if both of the ﬁrst-layer nodes give output 0. Thus,
the neural net of Fig. 13.1 gives output 1 for all the good inputs but none of
the bad inputs.
✷
x 1
x 2
x 3
x 4
x 1
x 2
x 3
x 4
y
1
1
1
2
0
0
0
1
1
1
1
−0.5
1
−2.5
1
−1.5
Figure 13.2: Making the threshold 0 for all nodes
It is useful in many situations to assume that nodes have a threshold of
0. Recall from Section 12.2.4 that we can always convert a perceptron with a
nonzero threshold t to one with a 0 threshold if we add an additional input.
That input always has value 1 and a weight equal to −t. For example, we can
convert the net of Fig. 13.1 to that in Fig. 13.2.
13.1.1
Neural Nets, in General
Example 13.1 and its net of Fig. 13.1 is much simpler than anything that would
be a useful application of neural nets. The general case is suggested by Fig. 13.3.
The ﬁrst, or input layer, is the input, which is presumed to be a vector of some
length n. Each component of the vector [x1, x2, . . . , xn] is an input to the net.
There are one or more hidden layers and ﬁnally, at the end, an output layer,
which gives the result of the net. Each of the layers can have a diﬀerent number
of nodes, and in fact, choosing the right number of nodes at each layer is an
