The importance of convexity in learning with squared loss

We show that if the closure of a function class F under the metric induced by some probability distribution is not convex, then the sample complexity for agnostically learning F with squared loss (using only hypotheses in F) is Ω(ln(1/δ)/ε²) where 1-δ is the probability of success and ε is the required accuracy. In comparison, if the class F is convex and has finite pseudodimension, then the sample complexity is O(1/ε(ln(1/ε)+ln(1/b)). If a nonconvex class F has finite pseudodimension, then the sample complexity for agnostically learning the closure of the convex hull of F, is O(1/ε(1/ε(ln(1/ε)+ln(1/δ)). Hence, for agnostic learning, learning the convex hull provides better approximation capabilities with little sample complexity penalty