Coverage Metrics for Learning-Event Datasets Based on Client-Side Monitoring

The collection of learner events within a server-client architecture occurs either at server, client or both complementarily. Such collection may be incomplete due to various factors, particularly for client-based monitoring, where learners can disable, delete or even modify their event logs due to privacy policies. The quality and accuracy of any analysis based on such data collections depends critically on the quality of the subjacent dataset. We propose three initial metrics to evaluate the completeness of a learning dataset: client-to-server ratio, event-to-activity ratio and subjective ratio. These metrics provide a glimpse on the coverage rate of the monitoring and can be applied to distinguish subsets of data with a minimum level of reliability to be used in a learning analytics study.