we consider the following model for domain knowledge incorporation. $$\begin{aligned}
   \textrm{(KSVM-P)} \qquad
    \min_{{\textbf{w}},b,\xi,{\textbf{u}},{\textbf{v}},\eta_u,\eta_v} && \frac{1}{2}{\textbf{w}}^T{\textbf{w}}+ c{\textbf{e}}^T\xi + \frac{\rho_1}{2}\|B^T{\textbf{u}}+{\textbf{w}}\|_2^2 \\
    && + \rho_2\eta_u + \frac{\rho_3}{2}\|D^T{\textbf{v}}-{\textbf{w}}\|_2^2 + \rho_4\eta_v \\
  \nonumber s.t. && y_i({\textbf{w}}^T{\textbf{x}_i}+ b) \geq 1 - \xi_i, \quad i = 1,\cdots,N, \\
   \nonumber && \textbf{d}^T{\textbf{u}}- b + 1 \leq \eta_u, \\
   \nonumber && \textbf{g}^T{\textbf{v}}+ b + 1 \leq \eta_v,\\
   \nonumber && (\xi,{\textbf{u}},{\textbf{v}},\eta_u,\eta_v) \geq {\textbf{0}}.\end{aligned}$$

We are now ready to propose a novel combination of ENSVM and KSVM, and we will explain in the next section how the combined problem can be solved in our HIPAD framework. The main motivation behind this combination is to exploit domain knowledge to improve the feature selection, and hence, the generalization performance of HIPAD. To the best of our knowledge, this is the first method of this kind.

ADMM Phase
----------

Our strategy for solving the elastic-net SVM with domain knowledge incorporation is still to apply the ADMM method. First, we combine problems (\[eq:en\_svm\]) and (KSVM-P) and write the resulting optimization problem in an equivalent unconstrained form (by penalizing the violation of the inequality constraints through hinge losses in the objective function) $$    \textrm{(ENK-SVM)} \quad \min_{{\textbf{w}},b,{\textbf{u}}\geq {\textbf{0}},{\textbf{v}}\geq {\textbf{0}}} F({\textbf{w}},b,{\textbf{u}},{\textbf{v}}),$$ where $F({\textbf{w}},b,{\textbf{u}},{\textbf{v}}) \equiv \frac{\lambda_2}{2}\|{\textbf{w}}\|_2^2 + \lambda_1\|{\textbf{w}}\|_1 + 