An Automatic Scoring System for PDA Integrated Ecology Observation Worksheet

This study aims to develop an automatic scoring system for evaluating the ecology observation worksheets in mobile learning activities. The feedbacks for constructed response tasks were provided online to motivate students searching for detailed information in e-library, such that their inquiry abilities could be enhanced. Three PDA integrated outdoor observation activities and worksheets were arranged for elementary school´s ecology learning. A scoring rubric and two scoring methods, i.e. keyword pattern-matching (PM) and Latent Semantic Analysis (LSA), have been developed for rating the construct responses of worksheet. The scoring consistencies between human raters and the scoring methods were examined. Twenty-seven 5th to 6th graders from Tainan district participated in this study. The inter-rater reliability between two human raters was higher than 0.90. The correlation coefficients between keyword PM and LSA and human rating were around 0.88 and 0.74, respectively. That is, keyword PM had better performance than LSA in scoring the task specific construct responses.