elastic net splits as well, and the elastic net coefficients will be identical to those of the component lasso before the NNLS step. In this case, the model chosen by the component lasso will involve splitting the predictors into components if the NNLS reweighting is useful in minimizing the validation MSE.

Details of connected component estimation via the graphical lasso
-----------------------------------------------------------------

Given an observed covariance $S={X^TX/n}$ with $X \sim \mathcal{N}(0,\Sigma)$, the graphical lasso estimates $\Theta=\Sigma^{-1}$ by maximizing the penalized log-likelihood $$\begin{aligned}
\ell(\Theta)=\log{\rm det} \Theta-{\rm tr}(S\Theta)-\tau ||\Theta||_1\end{aligned}$$ over all non-negative definite matrices $\Theta$ . The KKT conditions for this problem are $$\begin{aligned}
\Theta^{-1}=S-\tau \Gamma(\Theta)=0\end{aligned}$$ where $\Gamma(\Theta)$ is a matrix of componentwise subgradients $\text{sign}(\Theta_{ij})$. If $C_1, C_2 \ldots C_K$ are a partition of $1,2,\ldots p$, then and show that the corresponding arrangement of $\hat\Theta(\tau)$ is block diagonal if and only if $S_{ii'} \leq \tau$ for all $i \in C_k, i' \in C_{k'}, k\neq k'$. This means that soft-thresholding of $S$ at level $\tau$ into its connected components yields the connected components of $\hat\Theta(\tau)$.

Furthermore, there is an interesting connection to hierarchical clustering. Specifically the connected components correspond to the subtrees from when we apply single linkage agglomerative clustering to $S$ and then 