On the efficient minimization of convex surrogates in supervised learning

Bartlett et al (2006) recently proved that a ground condition for convex surrogates, classification calibration, ties up the minimization of the surrogates and classification risks, and left as important open problems the algorithmic questions about the minimization of these surrogates. Our paper gives an answer for a wide subset of these surrogates that we call ldquobalanced surrogatesrdquo, a set with popular members (logistic loss, squared loss), that contains all surrogates meeting three important requirements about classification. We propose an algorithm that fits linear separators to the minimization of any such surrogate, with guaranteed convergence bounds under a so-called ldquoweak learning assumptionrdquo, a generalization of the one that grounds celebrated boosting algorithms. Experiments on more than 50 readily available domains of 10 flavors of the algorithm display the performances of new surrogates.