Quadratic Discriminant Revisited

In this study, we revisit quadratic discriminant analysis (QDA). For this purpose, we present a majorize-minimize (MM) optimization algorithm to estimate parameters for generative classifiers, of which conditional distributions are from the exponential family. Furthermore, we propose a block-coordinate descent algorithm to sequentially update parameters of QDA in each iteration of the MM algorithm, for each update, we apply a trust region method, of which each iteration has a simple closed form solution. Numerical experiments show that: when compared with conjugate gradient method, the new proposed method is faster in 9 of 10 benchmark data sets, when compared with other widely used quadratic classifiers in the literature, QDA trained with the proposed method is either the best or not statistically significantly different from the best ones in 8 of 10 benchmark data sets.