---
abstract: 'A number of recent work studied the effectiveness of feature selection using Lasso. It is known that under the restricted isometry properties (RIP), Lasso does not generally lead to the exact recovery of the set of nonzero coefficients, due to the looseness of convex relaxation. This paper considers the feature selection property of nonconvex regularization, where the solution is given by a multi-stage convex relaxation scheme. Under appropriate conditions, we show that the local solution obtained by this procedure recovers the set of nonzero coefficients without suffering from the bias of Lasso relaxation, which complements parameter estimation results of this procedure in [@Zhang09-multistage].'
author:
- |
    Tong Zhang\
    Statistics Department\
    Rutgers University\
    Piscataway, NJ 08854\
    `tzhang@stat.rutgers.edu`
bibliography:
- 'L1.bib'
- 'learning.bib'
- 'pubj.bib'
title: 'Multi-stage Convex Relaxation for Feature Selection'
---

Introduction
============

We consider the linear regression problem, where we observe a set of input vectors ${{\mathbf x}}_1, \ldots, {{\mathbf x}}_n \in R^p$, with corresponding desired output variables $y_1, \ldots, y_n$. In a statistical linear model, it is common to assume that there exists a target coefficient vector $\bar{{{\mathbf w}}} \in R^p$ such that $$y_i = \bar{{{\mathbf w}}}^\top {{\mathbf x}}_i  + \epsilon_i
\qquad (i=1,\ldots,n) , \label{eq:noise-def}$$ where 