Dictionary learning with ℓ1/2 regularizer for sparsity based on proximal operator

In this study, we propose a fast and efficient algorithm for learning overcomplete dictionary for sparse representation of signals using the nonconvex ℓ_1/2 regularizer for sparsity. The special importance of ℓ_1/2 regularizer has been recognized in recent studies on sparse modeling. The ℓ_1/2-norm, however, leads to a nonconvex and nonsmooth optimization problem that is difficult to solve efficiently. In this paper, we propose a method based on a decomposition scheme and alternating optimization that can turn the whole problem into a set of subminimizations of univariate functions, each of which is dependent on only one dictionary atom or the coefficient vector. Although the subproblem with respect to the coefficient vector is still nonsmooth and nonconvex due to the ℓ_1/2 regularizer, remarkably, it becomes much simpler and it has a closed-form solution by introducing a technique that is proximal operator. The main advantages of the proposed algorithm is that, as suggested by the simulation study, it is faster and more efficient than state-of-the-art algorithms with different sparsity constraints.