A unified FOCUSS framework for learning sparse dictionaries and non-squared error

FOCUSS is an Iteratively Reweighted Least Squares approximation used to find the inverse solution of an underdetermined linear system when the source vector is assumed to be sparse. It also provides an iterative descent method used to solve for an unknown dictionary. We describe three extensions to the FOCUSS model: First a choice of generalized p-norm reconstruction error which corresponds to differing assumptions on the cost of errors. Second the use of a constraint which encourages sparsity on the dictionary atoms, and third the combination of both sparsity on dictionary atoms and generalized reconstruction error to form one unified framework for solving a wide set of sparsity requirements on sources, on loadings, and on error. Finally, we describe a practical set of algorithms for learning dictionaries and source vectors under each of these model assumptions, and show experimental results using these algorithms.