typical 4 would be open.

Figure \[fig:mnistproj\] shows for each of the ten classes the distribution of values obtained by projecting the training examples in that class onto the first eigenvector for class pair $(3, 2)$, i.e., the top left image in Figure \[fig:mnistfeats\]. The projection pattern inspires two comments. First, while the magnitude of the projection is itself discriminative for distinguishing between 2s and 3s, there is additional information in knowing the sign of the projection. This motivates our particular choice of nonlinear expansion in Algorithm \[alg:gem\]. Second, the detector is discriminative for class 3 vs. class 2 as per design, but also useful for distinguishing other classes from 2s. However certain classes such as 1s and 7s would be completely confused with 2s were this the only feature. The number of classes in MNIST is modest ($k=10$) so we can easily afford to extract features for all $k(k-1)$ class pairs for excellent discrimination. For problems with a large number of classes, however, we need to carefully pick the subproblems we need to solve so that the resulting set of features is discriminative, diverse, and complete. We revisit this topic in Section \[sec:discussion\].

Table \[tab:mnistres\] contains results for algorithm 