results are significant at $\alpha=0.1$. In terms of CPU-time, HIPAD consistently outperformed ADMM-ENSVM by several times on dense data. The feature support sizes selected by HIPAD were also very competitive or even better than the ones selected by ADMM-ENSVM. In most cases, HIPAD was able to shrink the feature space to below 10$\%$ of the original size.

Simulation for Knowledge Incorporation {#sec:exp_ksvm}
--------------------------------------

We generated synthetic data to simulate the example presented at the beginning of Section \[sec:enk-svm\] in the high dimensional feature space. Specifically, four groups of multi-variate Gaussians $K_1,\cdots,K_4$ were sampled from $\mathcal{N}(\mu_1^+, \Sigma_1), \cdots, \mathcal{N}(\mu_4^+, \Sigma_4)$ and $\mathcal{N}(\mu_1^-, \Sigma_1), \cdots, \mathcal{N}(\mu_4^-, \Sigma_4)$ for four disjoint blocks of feature values (${\textbf{x}}_{K_1}, \cdots, {\textbf{x}}_{K_4}$). For positive class samples, $\mu_1^+ = 2, \mu_2^+ = 0.5, \mu_3^+ = -0.2, \mu_4^+ = -1$; for negative class samples, $\mu_1^- = -2, \mu_2^- = -0.5, \mu_3^- = 0.2, \mu_4^- = 1$. All the covariance matrices have 1 on the diagonal and 0.8 everywhere else. The training samples contain blocks $K_2$ and $K_3$, while all four blocks are present in the test samples. A random fraction (5%-10%) of the remaining entries in all the samples are generated from the standard Gaussian distribution.

The training samples 