A Two-Stage Data Preprocessing Approach for Software Fault Prediction

Software fault prediction is valuable in predicting fault proneness of software modules and then limited test resources can be effectively allocated for software quality assurance. Researchers have proved that either feature selection or instance reduction can improve the performance of classification models used for fault prediction. However, to the best of our knowledge, few researchers have combined them to study the effects on classification models. Therefore we propose a novel two-stage data preprocessing approach, which incorporates both feature selection and instance reduction. In particular, in the feature selection stage, we propose a new algorithm using both feature selection and threshold-based clustering which contains both relevance analysis and redundancy control. Then in the instance reduction stage, we apply random sampling to keep the balance between the faulty and non-faulty classes. In empirical studies, we implemented five different data preprocessing schemes based on our proposed approach, and performed a comparative study on the prediction performance of the commonly used classification models. The final results demonstrate the effectiveness of our approach and further provide a guideline for achieving cost-effective data preprocessing when using our approach.