Covariance sketching

Learning covariance matrices from high-dimensional data is an important problem that has received a lot of attention recently. We are particularly interested in the high-dimensional setting, where the number of samples one has access to is fewer than the number of variates. Fortunately, in many applications of interest, the underlying covariance matrix is sparse and hence has limited degrees of freedom. In most existing work however, it is assumed that one can obtain samples of all the variates simultaneously. This could be very expensive or physically infeasible in some applications. As a means of overcoming this limitation, we propose a new procedure that “pools” the covariates into a small number of groups and then samples each pooled group. We show that in certain cases it is possible to recover the covariance matrix from the pooled samples using an efficient convex optimization program, and so we call the procedure “covariance sketching”.