Sparse linear regression with beta process priors

A Bayesian approximation to finding the minimum ℓ₀ norm solution for an underdetermined linear system is proposed that is based on the beta process prior. The beta process linear regression (BP-LR) model finds sparse solutions to the underdetermined model y = Φx + ϵ, by modeling the vector x as an element-wise product of a non-sparse weight vector, w, and a sparse binary vector, z, that is drawn from the beta process prior. The hierarchical model is fully conjugate and therefore is amenable to fast inference methods. We demonstrate the model on a compressive sensing problem and on a correlated-feature problem, where we show the ability of the BP-LR to selectively remove the irrelevant features, while preserving the relevant groups of correlated features.