Learning mixtures of product distributions over discrete domains

We consider the problem of learning mixtures of product distributions over discrete domains in the distribution learning framework introduced by Kearns et al. (1994). We give a poly(n/ε) time algorithm for learning a mixture of k arbitrary product distributions over the n-dimensional Boolean cube {0, 1}ⁿ to accuracy ε, for any constant k. Previous poly(n)-time algorithms could only achieve this for k = 2 product distributions; our result answers an open question stated independently in M. Cryan (1999) and Y. Freund and Y. Mansour (1999). We further give evidence that no polynomial time algorithm can succeed when k is superconstant, by reduction from a notorious open problem in PAC learning. Finally, we generalize our poly(n/ε) time algorithm to learn any mixture of k = O(1) product distributions over {0, 1,... , b }ⁿ,for any b = O(1).