to come up with a classifier $h:
{\mathbb{R}}^d \to [k]$ with low generalization error ${\mathbb{P}}_{\mathcal{D}}(h(x)\neq y)$. Abusing notation, we will sometimes use $y$ to refer to the one hot encoding of $y$ that identifies each class with one of the vertices of the standard $k-1$-simplex. To keep the focus on the quality of our feature representation we will restrict ourselves to $h$ being linear, such as a multiclass linear SVM or multinomial logistic regression. We suspect representations that improve the performance of linear classifiers will also beneficially compose with nonlinear techniques.

Method
======

One of the simplest possible statistics involving both features and labels is the matrix ${\mathbb{E}}[x y^\top]$, which in multiclass classification is the collection of class-conditional mean feature vectors. This statistic has been thoroughly explored, e.g., Fisher LDA [@fisher1936use] and Sliced Inverse Regression [@li1991sliced]. However, in many practical applications we expect that the data distribution contains much more information than that contained in the first moment statistics. The natural next object of study is the tensor ${\mathbb{E}}[x \otimes x \otimes y]$.

In multiclass classification, the tensor ${\mathbb{E}}[x\otimes x \otimes
y]$ is simply a collection of the conditional second moment matrices $C_i={\mathbb{E}}[xx^\top | y=i]$. There are many standard ways 