Feature selection for detecting language levels in L2 English Learners

This study analyses different features that would enable classifiers to detect language levels in adult second language (L2) English Learners. 46 different speech samples from users speaking 15 different L1 native languages were selected from the Learning Prosody in a Foreign Language (LeaP) corpus [1]. Using different groupings of features from the spoken L2 secondary language (English), a Support Vector Machine (SVM), was trained and the speakers were classified into three different categories: c1, c2, and s1. These categories used correspond to beginner, intermediate, and advanced levels of the target secondary language, English. The categories in the automatic system correspond to the same category names given by the human annotators of the LeaP corpus. The features are grouped into four different sub-categories: sentence, syllable, duration, and pitch. Count features, such as sentence word count, sentence article count, etc. had the greatest influence on the system, while the sentence features had the second most influence. Surprisingly, most of the pitch features had no effect on the accuracy. A small common word list was also used, that proved to be very helpful. The edit distance measure of the sentences with the common words removed had a positive effect; measurable differences could be found with and without the common words included in the sentences. Due to the small size of the training and testing sets, it was found that the different groupings of the L1 languages of the speakers had a significant effect on the accuracy of the classification predictions. Certain combinations of L1 training and test sets had a higher accuracy rating depending on the L1 languages used in training or test. The classification predictions had a variance as much as 40%.