Chapter 12
Large-Scale Machine
Learning
Many algorithms are today classiﬁed as “machine learning.” These algorithms
share, with the other algorithms studied in this book, the goal of extracting
information from data.
All algorithms for analysis of data are designed to
produce a useful summary of the data, from which decisions are made. Among
many examples, the frequent-itemset analysis that we did in Chapter 6 produces
information like association rules, which can then be used for planning a sales
strategy or for many other purposes.
However, algorithms called “machine learning” not only summarize our
data; they are perceived as learning a model or classiﬁer from the data, and thus
discover something about data that will be seen in the future. For instance, the
clustering algorithms discussed in Chapter 7 produce clusters that not only tell
us something about the data being analyzed (the training set), but they allow
us to classify future data into one of the clusters that result from the clustering
algorithm. Thus, machine-learning enthusiasts often speak of clustering with
the neologism “unsupervised learning”; the term unsupervised refers to the fact
that the input data does not tell the clustering algorithm what the clusters
should be. In supervised machine learning,, which is the subject of this chapter,
the available data includes information about the correct way to classify at least
some of the data. The data classiﬁed already is called the training set.
This chapter is not intended to be a complete discussion of machine learning.
We concentrate on a small number of ideas, and emphasize how to deal with
very large data sets. Especially important is how we exploit parallelism to build
models of the data. We consider the classical “perceptron” approach to learning
a data classiﬁer, where a hyperplane that separates two classes is sought. Then,
we look at more modern techniques involving support-vector machines. Similar
to perceptrons, these methods look for hyperplanes that best divide the classes,
so that few, if any, members of the training set lie close to the hyperplane. We
next consider nearest-neighbor techniques, where data is classiﬁed according to
463
