Objective functions for maximum likelihood classifier design

This paper reports research into maximum likelihood parameter estimation for classification of data modelled as mixtures of multivariate Gaussian distributions. Two likelihood metrics are compared: the log conditional probability of the feature data (the non-discriminative log likelihood, L_n), and the log conditional probability of the class labels (the discriminative log likelihood, L_d). Results on some simple data sets indicate that L_d yields poorer classification accuracy, as measured by the average log probability l¯_c of obtaining the correct classification of a set of labelled test data. Analysis of the score equations and the information matrices derived from L_d and L _n reveals that L_d produces estimates of class means with larger bias and variance, and hence larger mean-square error (E¯²), than those from L_n. Some experimental results on simple data sets are given as illustration