Randomized sketches of convex programs with sharp guarantees

Random projection (RP) is a classical technique for reducing storage and computational costs. We analyze RP-based approximations of convex programs, in which the original optimization problem is approximated by the solution of a lower-dimensional problem. Such dimensionality reduction is essential in computation and memory limited settings, since the complexity of general convex programming can be quite high (e.g., cubic for quadratic programs, and substantially higher for semidefinite programs). We prove that the approximation ratio of this procedure can be bounded in terms of the geometry of constraint set. When using Gaussian random projections, the data matrix defining the cost function can be projected down to the statistical dimension of the tangent cone of the constraints at the original solution, which is often substantially smaller than the original dimension. We illustrate consequences of our theory for various cases, including unconstrained and ℓ₁-constrained least squares, support vector machines and discuss connections with denoising and compressed sensing.