LDA and SIR are discriminative but they work with rank-$k$ matrices so the number of directions that could be extracted is limited by the number of classes. Furthermore both of these methods lose valuable fidelity about the data by using the conditional means.

Oriented PCA is sufficiently general to encompass our technique as a special case. Nonetheless, to the best of our knowledge, the specific signal and noise models in this paper are novel and, as we show in Section \[sec:experiments\], they empirically work very well.

Experiments {#sec:experiments}
===========

MNIST
-----

![Pictures of the top 5 generalized eigenvectors for MNIST for class pairs $(3, 2)$ (top row), $(8, 5)$ (second row), $(3, 5)$ (third row), $(8, 0)$ (fourth row), and $(4, 9)$ (bottom row) with $\gamma = 0.5$. Filters have large response on the first class and small response on the second class. Best viewed in color.[]{data-label="fig:mnistfeats"}](mnist_select.png){width="\linewidth"}

We begin with the MNIST database of handwritten digits [@mnistlecun], for which we can visualize the generalized eigenvectors, providing intuition regarding the discriminative nature of the computed directions. For each of the ten classes, we estimated $C_m = {\mathbb{E}}[xx^\top|y=m]$ using and then extracted generalized eigenvectors for each class pair $(i, j)$ by solving 