The capacity of monotonic functions Original Research Article

We consider the class M of monotonically increasing binary output functions. M has considerable practical significance in machine learning and pattern recognition because prior information often suggests a monotonic relationship between input and output variables. The decision boundaries of monotonic classifiers are compared and contrasted with those of linear classifiers. M is shown to have a VC dimension of ∞, meaning that the VC bounds cannot guarantee generalization independent of input distribution. We demonstrate that when the input distribution is taken into account, however, the VC bounds become useful because the annealed VC entropy of M is modest for many distributions. Techniques for estimating the capacity and bounding the annealed VC entropy of M given the input distribution are presented and implemented.