$AC_i A^\top v'= \lambda AC_j A^\top v'$. Letting $v'=A^{-\top} L_j^{-\top} u'$ we see that $u'$ satisfies $L_j^{-1}C_i L_j^{-\top} u'=\lambda u'$ which is the same as . Therefore $u'$ can be chosen such that $u'=u$. Finally, the embedding involves only $v'^\top x' = u'^\top L_j^{-1}A^{-1} A x = u^\top L_j^{-1} x$ which is the same as the embedding for the original data.

It is worth pointing out that the results of some popular methods, such as PCA, are not invariant to linear transformations of the inputs. For such methods, differences in preprocessing and normalization can lead to vastly different results. The practical utility of an “off the shelf” classifier is greatly improved by this invariance, which provides robustness to data specification, e.g., differing units of measurement across the original features.

(Diversity) Two feature detectors $v_1$ and $v_2$ extracted from the same ordered class pair $(i,j)$ have uncorrelated responses ${\mathbb{E}}[(v_1^\top x)(v_2^\top x)|y=j]=0.$

This follows from the orthogonality of the eigenvectors in the induced problem $L_j^{-1}C_i L_j^{-\top} u=\lambda u$ (c.f. proof of Proposition \[prop:invariance\]) and the connection $v= L_j^{-\top}u$. If $u_1$ and $u_2$ are eigenvectors of $L_j^{-1}C_i L_j^{-\top}$ then $ 0 = u_1^\top u_2 = v_1^\top L_j L_j^\top v_2 = v_1^\top {\mathbb{E}}[xx^\top|y=j] v_2 