gives us a direction along which we expect to be able to discriminate the two classes by simply using the magnitude of the projection. Moreover, if there are many eigenvalues substantially different from 1 all associated eigenvectors can be used as feature detectors.

Useful Properties
-----------------

The feature detectors resulting from maximizing equation have two useful properties which we list below. For simplicity we state the results assuming full rank exact conditional moment matrices, and then discuss the impact of regularization and finite samples.

\[prop:invariance\] (Invariance) Under the above assumptions, the embedding $v^\top x$ is invariant to invertible linear transformations of $x$.

Let $A \in {\mathbb{R}}^{d \times d}$ be invertible and $x'=Ax$ be the transformed input. Let $C_m={\mathbb{E}}[xx^\top|y=m]$ be the second moment matrix given $y=m$ for the original data. For a class pair $(i,j)$, a generalized eigenvector $v$ satisfies $C_i v= \lambda C_j v$. Using the Cholesky factorization $C_j=L_j L_j^\top$ and setting $v=L_j^{-\top}u$ we have $$\label{eq:gevreduction}
L_j^{-1}C_i L_j^{-\top} u=\lambda u,$$ i.e., $u$ is an eigenvector of $L_j^{-1}C_i L_j^{-\top}$. Moreover, the embedding for the original data involves only $$\label{eq:embed}
v^\top x=u^\top L_j^{-1} x.$$ For the transformed data, the conditional second moments are ${\mathbb{E}}[x'x'^\top|y=m]=A{\mathbb{E}}[xx^\top|y=m]A^\top=AC_m A^\top$ and the corresponding generalized eigenvector $v'$ satisfies 