-------------
     Dropout         120
   DropConnect       112
       GEM           108
    deep GEM         96
     Maxout          94

  : Test errors on MNIST. All techniques are permutation invariant and do not augment the training set.[]{data-label="tab:mnistres"}

Covertype
---------

Covertype is a multiclass data set whose task is to predict one of 7 forest cover types using 54 cartographic variables [@blackard1999comparative]. RBF kernels provide state of the art performance on Covertype, and consequently it has been a benchmark dataset for fast approximate kernel techniques [@rahimi2007random; @manik]. Here, we demonstrate that generalized eigenvector extraction composes well with randomized feature maps in the primal. This approximates generalized eigenfunction extraction in the RKHS, while retaining the speed and compactness of primal approaches.

Covertype does not come with a designated test set, so we randomly permuted the data set and used the last 10% for testing, utilizing the same train-test split for all experiments. We followed the same experimental protocol as the previous section, i.e., held out a portion of the training set for validation to select hyperparameters.

Table \[tab:covertyperes\] summarizes the results.[^2] GEM and deep GEM are exactly the same as in the previous section, i.e., Algorithm \[alg:gem\] without and with self-composition respectively. RFF stands for Random Fourier Features [@rahimi2007random], in 