Multi-class classifiers based on binary classifiers: Performance, efficiency, and minimum coding matrix distances

Using multiple binary classifiers is a popular way to construct multi-class classifiers. There exist several strategies to construct multi-class classifiers from binary classifiers. An important question is which strategy offers the highest probability of successful classification given the number of N binary classifiers used. The first result presented in this work is a method to approximate how many classes can be distinguished using N binary classifiers in practical systems rather than theoretical setups. We come to the conclusion that in this formulation, all methods share the same performance limit, which is determined using the first result. The next question is what the smallest number of binary classifiers is that is needed to attain a given probability of success. To investigate this, we introduce the concept of efficiency, which is the ratio between the number bits needed to count the number of distinguishable classes and the number of bits used. The last contribution concerns the conclusion that methods should exist that are more efficient than those currently employed.