Investigating the use of chronological split for software effort estimation

In previous studies, the authors investigated separately the use of two different types of chronological splits (project-by-project split and date-based split) for assigning projects to training sets and testing sets. The aim of this study is to compare the two types of chronological splits against each other, to see whether either leads to better prediction accuracy. Estimation models were built and evaluated using training and testing sets formed using project-by-project chronological splitting and date-based splitting using two different splitting dates. The authors used 906 projects from the ISBSG Release 10 repository. The authors found no significant differences between the accuracy of models built and evaluated with either of the different splitting methods. However, models built using later splitting dates were more accurate than models built using earlier splitting dates. Different accuracy with different splitting dates suggests that chronological splitting is useful. Therefore the authors recommend that training and testing sets should be formed with regard to chronology, and a date-based split appears sensible for researchers in this field.