Learning from soft partitions of data: reducing the variance

Distributed machine learning can be realized using a divide and conquer methodology. One such divide and conquer method is learning from soft partitions of data. By examining the decomposition of classifier error into bias and variance terms, we see that learning from smaller partitions of data introduces higher variance. In this paper, we investigate the use of a particular variance reduction technique, randomized C4.5, when learning from soft partitions of data. This approach maintains the distributed nature of the learning algorithm while boosting the overall classification accuracy. Experiments on six machine learning datasets demonstrate the improved accuracy gains by reducing classifier variance. In particular, learning from soft partitions of data can produce more accurate classifiers than using an ensemble of randomized decision trees constructed from the entire dataset, which in turn results in a more accurate classifier than building a single decision tree.