A distributed instance-weighted SVM algorithm on large-scale imbalanced datasets

When huge amounts of data are processed to extract knowledge, the situation becomes a challenge because the data mining techniques are not adapted to the space and time requirements. This challenge is more significant when the data is class imbalanced. Like many other machine learning algorithms, the success of the support vector machine (SVM) is limited when it is applied to the problem of learning from imbalanced datasets, especially on big datasets. In this paper, we are trying to apply an instance-weighted variant of the SVM, with a parallel Meta-learning algorithm using MapReduce, to deal with the big data class imbalance problem. We develop a symmetric weight boosting method to optimize the instance-weighted SVM. Experimental results on benchmark datasets and real application big datasets show that the proposed algorithm not only is effective on big data class imbalanced problem, but also reduces the training computational complexity significantly when the number of computing nodes increases.