Bayesian Overlapping Subspace Clustering

Given a data matrix, the problem of finding dense/uniform sub-blocks in the matrix is becoming important in several applications. The problem is inherently combinatorial since the uniform sub-blocks may involve arbitrary subsets of rows and columns and may even be overlapping. While there are a few existing methods based on co-clustering or subspace clustering, they typically rely on local search heuristics and in general do not have a systematic model for such data. We present a Bayesian Overlapping Subspace Clustering (BOSC) model which is a hierarchical generative model for matrices with potentially overlapping uniform sub-block structures. The BOSC model can also handle matrices with missing entries. We propose an EM-style algorithm based on approximate inference using Gibbs sampling and parameter estimation using coordinate descent for the BOSC model. Through experiments on both simulated and real datasets, we demonstrate that the proposed algorithm outperforms the state-of-the-art.