feature dimensions but are highly sparse. The rest of the seven data sets are all dense data. **rcv1** and **real-sim** are subsets of the original data sets, where we randomly sampled 500 training instances and 1,000 test instances. **gisette** is a handwriting digit recognition problem from NIPS 2003 Feature Selection Challenge, and we also sub-sampled 500 instances for training. (For testing, we used the original test set of 1,000 instances.) **duke**, **leukemia**, and **colon-cancer** are data sets of gene expression profiles for breast cancer, leukemia, and colon cancer respectively. **fMRIa**, **fMRIb**, and **fMRIc** are functional MRI (fMRI) data of brain activities when the subjects are presented with pictures and text paragraphs. The data was compiled and made available by Tom Mitchell’s neuroinformatics research group [^1]. Except the three fMRI data sets, all the other data sets and their references are available at the LIBSVM website [^2].

The parameters of HIPAD, ADMM-ENSVM, and LIBSVM were selected through cross validation on the training data. We summarize the experiment results in Table \[tab:real\_data\_results\]. Clearly, HIPAD produced the best overall predication performance. In order to test the significance of the difference, we used the test statistic in [@iman1980approximations] based on Friedman’s $\chi^2_F$, and the 