=
        \left(
          \begin{array}{c}
             \textbf{r}_{{\textbf{w}}}\\
             \textbf{r}_b\\
          \end{array}
        \right),$$ where $\kappa_1 = \lambda_2 + \mu_2 + \rho_1 + \rho_3, \kappa_2 = \mu_3 + \mu_4, \textbf{r}_{{\textbf{w}}} = X^TY\gamma_1^k + \mu_1 X^TY({\textbf{e}}-{\textbf{a}}^k) - \gamma_2^k + \mu_2{\textbf{c}}^k + \rho_3D^T{\textbf{v}}^k - \rho_1B^T{\textbf{u}}^k$ and $\textbf{r}_{b} = {\textbf{e}}^TY\gamma_1^k + \mu_1{\textbf{e}}^TY({\textbf{e}}-{\textbf{a}}^k) + \gamma_3^k + \mu_3({\textbf{d}}^T{\textbf{u}}^k + 1 - q^k) - \gamma_4^k - \mu_4({\textbf{g}}^T{\textbf{v}}^k+1-p^k)$. Similar to solving the linear system in Algorithm \[alg:admm-ensvm\] ADMM-ENSVM, we can compute the solution to the above linear system through a few PCG iterations, taking advantage of the fact that the left-hand-side matrix is of low-rank.

To minimize the augmented Lagrangian with respect to ${\textbf{u}}$, we need to solve a convex quadratic problem with non-negative constraints $$\label{eq:ksvm_admm_usub}
    \min_{{\textbf{u}}\geq {\textbf{0}}} \quad \frac{\rho_1}{2}\|B^T{\textbf{u}}+ {\textbf{w}}^{k+1}\|_2^2 + \gamma_3^k{\textbf{d}}^T{\textbf{u}}+ \frac{\mu_3}{2}\|{\textbf{d}}^T{\textbf{u}}- b^{k+1} + 1 - q^{k}\|_2^2.$$ Solving problem efficiently is crucial for the efficiency of the overal algorithm. We describe a novel way to do so. Introducing a slack variable ${\textbf{s}}$ and transferring the non-negative constraint on ${\textbf{u}}$ to ${\textbf{s}}$, we decompose the problem into two parts which are easy to solve. Specifically, we reformulate (\[eq:ksvm\_admm\_usub\]) as $$\begin{aligned}
    \min_{{\textbf{u}},{\textbf{s}}\geq {\textbf{0}}} && \>\>\> \frac{\rho_1}{2}\|B^T{\textbf{u}}+ {\textbf{w}}^{k+1}\|_2^2 + \gamma_3^k{\textbf{d}}^T{\textbf{u}}+ \frac{\mu_3}{2}\|{\textbf{d}}^T{\textbf{u}}- b^{k+1} + 1 - q^{k}\|_2^2 \\
    \nonumber s.t. && \>\>\>  {\textbf{u}}- {\textbf{s}}= {\textbf{0}}.\end{aligned}$$ Penalizing 