has 50 samples with 300 features each, ADMM converged in 558 iterations. The output classifier ${\textbf{w}}$ contained only 13 non-zero features, and the feature support converged in approximately 50 iteration (see Figure \[fig:fea\_supp\_plots\] in the Appendix for illustrative plots showing the early convergence of ADMM). Although the remaining more than 500 iterations are needed by ADMM in order to satisfy the optimality criteria, they do not offer any additional information regarding the feature selection process. Hence, it is important to monitor the change in the signs and indices of the support and terminate the first phase promptly. In our implementation, we adopt the criterion used in [@shi2010fast] and monitor the relative change in the iterates as a surrogate of the change in the support, i.e. $$\label{eq:hipad_transition}
    \frac{\|{\textbf{w}}^{k+1}-{\textbf{w}}^k\|}{\max(\|{\textbf{w}}^k\|,1)} < \epsilon_{tol}.$$ We have observed in our experiments that when the change over the iterates is small, the evolution of the support indices stabilizes too.

Upon starting the second phase, it is desirable for IPM to warm-start from the corresponding sub-vector of the solution returned by ADMM. It should also be noted that we apply IPM during the second phase to solve the classical $L_2$-regularized SVM (\[eq:unconstr\_svm\]), instead of the ENSVM (\[eq:en\_svm\]) in 