9.2. CONTENT-BASED RECOMMENDATIONS
325
9.2.2
Discovering Features of Documents
There are other classes of items where it is not immediately apparent what the
values of features should be. We shall consider two of them: document collec-
tions and images. Documents present special problems, and we shall discuss
the technology for extracting features from documents in this section. Images
will be discussed in Section 9.2.3 as an important example where user-supplied
features have some hope of success.
There are many kinds of documents for which a recommendation system can
be useful. For example, there are many news articles published each day, and
we cannot read all of them. A recommendation system can suggest articles on
topics a user is interested in, but how can we distinguish among topics? Web
pages are also a collection of documents. Can we suggest pages a user might
want to see? Likewise, blogs could be recommended to interested users, if we
could classify blogs by topics.
Unfortunately, these classes of documents do not tend to have readily avail-
able information giving features. A substitute that has been useful in practice is
the identiﬁcation of words that characterize the topic of a document. How we do
the identiﬁcation was outlined in Section 1.3.1. First, eliminate stop words –
the several hundred most common words, which tend to say little about the
topic of a document. For the remaining words, compute the TF.IDF score for
each word in the document. The ones with the highest scores are the words
that characterize the document.
We may then take as the features of a document the n words with the highest
TF.IDF scores. It is possible to pick n to be the same for all documents, or to
let n be a ﬁxed percentage of the words in the document. We could also choose
to make all words whose TF.IDF scores are above a given threshold to be a
part of the feature set.
Now, documents are represented by sets of words. Intuitively, we expect
these words to express the subjects or main ideas of the document. For example,
in a news article, we would expect the words with the highest TF.IDF score to
include the names of people discussed in the article, unusual properties of the
event described, and the location of the event. To measure the similarity of two
documents, there are several natural distance measures we can use:
1. We could use the Jaccard distance between the sets of words (recall Sec-
tion 3.5.3).
2. We could use the cosine distance (recall Section 3.5.4) between the sets,
treated as vectors.
To compute the cosine distance in option (2), think of the sets of high-
TF.IDF words as a vector, with one component for each possible word. The
vector has 1 if the word is in the set and 0 if not. Since between two docu-
ments there are only a ﬁnite number of words among their two sets, the inﬁnite
dimensionality of the vectors is unimportant. Almost all components are 0 in
