each class, we used the default behavior or inferring the prior from the training data. For logistic regression, we used the `saga` solver with the elastic net penalty at an L1 ratio of 0.25. For k-NN, we used 18 neighbors and the L3 distance. For Multilayer Perceptron, we employed hidden layers with 50, 25, 10, and 5 nodes respectively, the ReLU activation function and an initial learning rate of 0.01. For the Random Forest classifier, we allowed 200 trees with a maximum of 20 leaf nodes and weighted each observation to balance the classes. For the linear Support Vector Classifier we set a cost of 0.5, and 0.8 for the radial basis function version.

Validation
----------

As in [@compare], we measure the performance of each model via a Leave-One-Subject-Out jackknife estimate of accuracy, F1 score, and the Matthews correlation coefficient. One subject (three observations) is removed from the dataset and held in reserve. Each model is trained on the remaining data and then tested on the subject held in reserve. This is repeated for each subject in turn until we have as many predictions as we have samples.

Ordinary bootstrap and jackknife estimates of performance cannot be used since there 