The Influence Machine: Nonnegative Instance-Space Learning with Differentiated Regularization

We introduce a new method for classification called the influence machine. The influence machine assigns influence powers to the instances in the training sample so that they can apply their influence to other instances through the connections between the instances specified by a connection matrix. A new instance is classified to be positive if the overall influence it receives is positive and vice versa. Similar to support vector machine (SVM), the influence machine selects a small subset of the training instances to give influence power. However, this selection is very different from how the support vectors are selected by SVM. Experiment results show that the classification performance of the influence machine is comparable to that of the SVM. In a few cases, the influence machine shows much better classification accuracy. The influence machine has other advantages: any similarity matrix can be applied with the influence machine, not like SVM which requires that the kernel be positive definite. Furthermore, the influence machine uses linear optimization, instead of the quadratic optimization used by SVM. It may be more suitable for large scale learning problems.