$l_{q}$ Sparsity Penalized Linear Regression With Cyclic Descent

Recently, there has been a lot of focus on penalized least squares problems for noisy sparse signal estimation. The penalty induces sparsity and a very common choice has been the convex l₁ norm. However, to improve sparsity and reduce the biases associated with the l₁ norm, one must move to non-convex penalties such as the l_q norm . In this paper we present a novel cyclic descent algorithm for optimizing the resulting l_q penalized least squares problem. Optimality conditions for this problem are derived and competing ones clarified. Coordinate-wise convergence as well as convergence to a local minimizer of the algorithm, which is highly non-trivial, is proved and we illustrate with simulations comparing the signal reconstruction quality with three penalty functions: l₀, l₁ and l_q with 0 <; q <; 1.