the above challenges and requirements [*simultaneously*]{}. Our framework combines the advantages of a first-order optimization algorithm (through the use of ADMM) and a second-order method (via IPM) to achieve both superior speed and accuracy. Through a novel algorithmic approach that is able to incorporate expert knowledge, our proposed framework is able to exploit domain knowledge to improve feature selection, and hence, prediction accuracy. Besides efficiency and generalization performance, we demonstrate through experiments on both synthetic and real data that our method is also more robust to inaccuracy in the supplied knowledge than existing approaches.

A Two-phase Hybrid Optimization Algorithm {#sec:two_phase_alg}
=========================================

As previously mentioned, for data sets with many features, the high dimensionality of the feature space still poses a computational challenge for IPM. Fortunately, many data sets of this kind are very sparse, and the resulting classifier $w$ is also expected to be sparse, i.e. only a small subset of the features are expected to carry significant weights in classification. Naturally, it is ideal for IPM to train a classifier on the most important features only.

Inspired by the Hybrid Iterative Shrinkage (HIS) [@shi2010fast] algorithm for training large-scale sparse logistic regression classifiers, we propose a two-phase algorithm to shrink 