architecture as a form of pre-training.

We remark that each step in Algorithm \[alg:gem\] is highly amenable to distributed implementation: empirical class-conditional second moment matrices can be computed using map-reduce techniques, the generalized eigenvalue problems can be solved independently in parallel, and the logistic regression optimization is convex and therefore highly scalable [@agarwal2011].

Related Work {#sec:related}
============

      Method                               Signal                                           Noise
  -------------- ----------------------------------------------------------- -----------------------------------
       PCA                         ${\mathbb{E}}[xx^\top]$                                   $I$
       VCA                                   $I$                                   ${\mathbb{E}}[xx^\top]$
    Fisher LDA    ${\mathbb{E}}_y[{\mathbb{E}}[x|y]{\mathbb{E}}[x|y]^\top]$   $\sum_y \operatorname*{Cov}[x|y]$
       SIR            $\sum_y {\mathbb{E}}[w|y]{\mathbb{E}}[w|y]^\top$                       $I$
   Oriented PCA                    ${\mathbb{E}}[xx^\top]$                         ${\mathbb{E}}[zz^\top]$
    Our method                   ${\mathbb{E}}[xx^\top|y=i]$                     ${\mathbb{E}}[xx^\top|y=j]$

  : Table of related methods (assuming ${\mathbb{E}}[x]=0$) for finding directions that maximize the signal to noise ratio. $\operatorname*{Cov}[x|y]$ refers to the conditional covariance matrix of $x$ given $y$, $w$ is a whitened version of $x$, and $z$ is any type of noise meaningful to the task at hand. []{data-label="tab:signalnoise"}

Our approach resembles many existing methods that work by finding eigenvectors of matrices constructed from data. One can think of all these approaches as procedures for finding directions $v$ that maximize the signal to noise ratio $$R(v)=\frac{v^\top S v}{v^\top N v},
\label{eqn:signoise}$$ where the symmetric matrices $S$ and $N$ are such that the quadratic forms $v^\top S v$ and $v^\top N v$ represent the signal 