that minimizes its average KL-divergence to each element of the ensemble). The result matches the classification error and improves upon the cross-entropy loss of the best published T-DSN. This is remarkable considering the T-DSN is a deep architecture employing between 8 and 13 stacked layers of nonlinear transformations, whereas the GEM procedure produces a shallow architecture with a single nonlinear layer.

  ---------------- -------------------- ------------------
                          Frame               Cross
                     State Error (%)         Entropy
        GEM         $41.87 \pm 0.073 $   $1.637\pm 0.001$
       T-DSN               40.9                2.02
   GEM (ensemble)         40.86               1.581
  ---------------- -------------------- ------------------

  : Results on TIMIT test set. T-DSN is the best result from [@hutchinson2012deep].[]{data-label="tab:timitres"}

Discussion {#sec:discussion}
==========

Given the simplicity and empirical success of our method, we were surprised to find considerable work on methods that only extract the first generalized eigenvector [@mika2003constructing] but very little work on using the top $m$ generalized eigenvectors. Our experience is that additional eigenvectors provide complementary information. Empirically, their inclusion in the final classifier far outweighs the necessary increase in sample complexity, especially given typical modern data set sizes. Thus we believe this technique should be valuable in other domains.

Of course our method will not be able to extract anything useful if all classes have the same 