observations in the training, validation and test sets respectively.

[Orthogonal components example]{} We generate an example with two connected components, where the predictors in different components are orthogonal. The corresponding sample covariance matrix is block diagonal with 2 blocks. As mentioned earlier, the subgradient equations of the lasso and elastic net split naturally when the components are orthogonal. Therefore, the component lasso only differs from the non naive elastic net in the NNLS reweighting step.

We generate the example as follows: $p=8$, $\sigma=3$ and $\beta=(3,1.5,0,0,2,3,0,0)$. We simulate 100 20/20/200 sets of observations such that the correlations within a component are equal to 0.8, and force the correlations between the components to be exactly 0. We then check the performance of the component lasso in two settings: when the number of components it uses is fixed to 2, and, when the optimal number of components is chosen in the validation step. The corresponding test MSEs are given in Table \[table:orthog\].

  **Method**                       **Median MSE**   **Median FP**   **Median FN**
  -------------------------------- ---------------- --------------- ---------------
  Lasso                            7.16 (0.50)      0.40 (0.02)     0 (0.03)
  Rescaled Lasso                   7.26 (0.49)      0.33 (0.02)     0 (0.02)
  Lasso-OLS Hybrid                 7.64 (0.46)      0.33 (0.02)     0 (0.02)
  Naive Elastic Net                6.04 (0.45)      0.43 (0.01)     