Dictionary learning for sparse decomposition: A new criterion and algorithm

During the last decade, there has been a growing interest toward the problem of sparse decomposition. A very important task in this field is dictionary learning, which is designing a suitable dictionary that can sparsely represent a group of training signals. In most dictionary learning algorithms, the cost function to determine the the optimum dictionary is the ℓ₀ norm of the matrix of decomposition coefficients of the training signals. However, we believe that this cost function fails to fully express the goal of dictionary learning, because it only sparsifies the whole set of coefficients for all training signals, rather than the coefficients for each training signal individually. Thus, in this paper we present a new criterion for dictionary learning. We then propose a new dictionary learning algorithm that solves our proposed optimization problem for the case of complete dictionaries. The proposed algorithm follows the idea of smoothed ℓ₀ (SL0) algorithm for sparse recovery. Simulation results emphasize the efficiency of the proposed cost function and algorithm.