Deep Matrix Factorization for social image tag refinement and assignment

The number of images associated with user-provided tags has increased dramatically in recent years. User-provided tags are incomplete, subjective and noisy. In this work, we focus on the problem of image tag refinement and assignment. Different from previous work, we propose a novel Deep Matrix Factorization (DMF) algorithm, which uncovers the latent image representations and tag representations embedded in the latent subspace by exploiting the weakly-supervised tagging information and visual information. Due to the well-known semantic gap, the hidden representations of images are learned by a hierarchical model, which are progressively transformed from the visual feature space. It can naturally embed new images into the subspace using the learned deep architecture. Besides, to remove the noisy or redundant visual features, a sparse model is imposed on the transformation matrix of the first layer in the deep architecture. Finally, a unified optimization problem with a well-defined objective function is developed to formulate the proposed problem. Extensive experiments on real-world social image databases are conducted on the tasks of image tag refinement and assignment. Encouraging results are achieved with comparison to the state-of-the-art algorithms, which demonstrates the effectiveness of the proposed method.