A Hybrid System for Probability Estimation in Multiclass Problems Combining SVMs and Neural Networks

This paper addresses the problem of probability estimation in multiclass classification tasks combining two well known data mining techniques: support vector machines and neural networks. We present an algorithm which uses both techniques in a two-step procedure. The first step employs support vector machines within a one-vs-all reduction from multiclass to binary approach to obtain the distances between each observation and the support vectors representing the classes. The second step uses these distances as inputs for a neural network, built with an entropy cost function and softmax transfer function for the output layer where class membership is used for training. Consequently, this network estimates probabilities of class membership for new observations. A benchmark using different databases demonstrates that the proposed algorithm is highly competitive with the most recent techniques for multiclass probability estimation.