Complexity of learning subspace juntas and ICA

Inspired by feature selection problems in machine learning and statistics, we study classification problems where the label function depends only on an unknown low dimensional relevant subspace of the data (we call this a k-subspace junta). Assuming that the relevant subspace is truly independent of the irrelevant subspace, and that the distribution over the irrelevant subspace is Gaussian, we give a polynomial-time algorithm for recovering the relevant subspace and for learning; additionally, we require only a polynomial number of samples. Our main tool is the solution of a tensor optimization problem. In general, finding the global optimum of a tensor is NP-hard, but we avoid this difficulty by using only local optima.