Goodness of Fit: Measures for a Fuzzy Classifier

The understandability of rule sets is an important issue in knowledge discovery, where classification rules, for example, are extracted from large data sets. An important criterion in this context is the goodness of fit of a given classifier, i.e., a measure that gives an quantitative answer to the question, how good a classifier fits to the data it has to classify. In this article we provide an appropriate measure for a Mamdani-type fuzzy classifier with Gaussians and singletons as membership functions, sum-prod inference, and height method for defuzzification. That is, goodness of fit must be measured for multivariate Gaussian mixture models. Therefore, we adopt conventional test methods for univariate, unimodal probability distributions (e.g., Kolmogorov-Smirnov for chi-square), provide a measure for the goodness of fit of our fuzzy classifier, and discuss its properties. In a second step we go even beyond this point by showing how this measure could be extended to an analysis tool that gives detailed hints which rules or which membership functions are not suitably realized.