Learning overcomplete dictionaries with ℓ0-sparse Non-negative Matrix Factorisation

Non-negative Matrix Factorisation (NMF) is a popular tool in which a `parts-based´ representation of a non-negative matrix is sought. NMF tends to produce sparse decompositions. This sparsity is a desirable property in many applications, and Sparse NMF (S-NMF) methods have been proposed to enhance this feature. Typically these enforce sparsity through use of a penalty term, and a ℓ₁ norm penalty term is often used. However an ℓ₁ penalty term may not be appropriate in a non-negative framework. In this paper the use of a ℓ₀ norm penalty for NMF is proposed, approximated using backwards elimination from an initial NNLS decomposition. Dictionary recovery experiments using overcomplete dictionaries show that this method outperforms both NMF and a state of the art S-NMF method, in particular when the dictionary to be learnt is dense.