Supervised Matrix Factorization with sparseness constraints and fast inference

Non-negative Matrix Factorization is a technique for decomposing large data sets into bases and code words, where all entries of the occurring matrices are non-negative. A recently proposed technique also incorporates sparseness constraints, in such a way that the amount of nonzero entries in both bases and code words becomes controllable. This paper extends the Non-negative Matrix Factorization with Sparseness Constraints. First, a modification of the optimization criteria ensures fast inference of the code words. Thus, the approach is real-time capable for use in time critical applications. Second, in case a teacher signal is associated with the samples, it is considered in order to ensure that inferred code words of different classes can be well distinguished. Thus, the derived bases generate discriminative code words, which is a crucial prerequisite for training powerful classifiers. Experiments on natural image patches show, similar to recent results in the field of sparse coding algorithms, that Gabor-like filters are minimizing the reconstruction error while retaining inference capabilities. However, applying the approach with incorporation of the teacher signal to handwritten digits yields morphologically completely different bases, while achieving superior classification results.