Near-ideal behavior of compressed sensing algorithms

Compressed sensing theory addresses the problem of recovering a nearly sparse signal from a noise-corrupted linear measurement of far smaller dimension. In some recent papers, it is shown that the LASSO algorithm exhibits near-ideal behavior, in the following sense: If x is a sparse signal, and if an estimate x-hat of x is found using LASSO, then the Euclidean norm of the residual error is bounded by a universal constant times the error achieved by an “oracle” that knows the support set of x. The LASSO algorithm has been generalized in several directions such as the group LASSO, the sparse group LASSO, either without or with tree-structured overlapping groups, and most recently, the sorted LASSO. This raises the question as to which if any of these algorithms also exhibits near-ideal behavior. In this paper we present a unified theory by showing that any algorithm exhibits near-ideal behavior in the above sense, provided that three conditions are satisfied: (i) the norm used to define the sparsity index is “decomposable,” (ii) the penalty norm that is minimized in an effort to enforce sparsity is gamma-decomposable, and (iii) a “compressibility condition” in terms of a group restricted isometry property is satisfied. Our results imply that the group LASSO, and the sparse group LASSO (with some permissible overlap in the groups), as well as the sorted ℓ₁-norm minimization all exhibit near-ideal behavior. Explicit bounds on the residual error are derived that contain previously known results as special cases.