Formalization of clinical trial eligibility criteria: Evaluation of a pattern-based approach

The semi-automatic evaluation of eligibility criteria can facilitate the recruitment for clinical trials, timely completion of studies and generation of clinical evidence about new approaches to treatment, prevention and diagnosis. Because eligibility criteria are represented as free text, automatically extracting their meaning and evaluating them for a particular patient is challenging. This paper presents our approach to the problem of automatic interpretation of criteria meaning. It is based on detecting in text semantic entities (diseases, treatment, measurements etc.) using ontology annotators and semantic taggers, and detecting predefined patterns providing the contextual information in which these entities occur. Evaluation of the approach is the main subject of the paper. It covers several aspects: precision and recall of the pattern detection algorithm and the assessment of the implications of using the identified patterns to find potential candidates. It was performed manually using a subset of patterns and randomly selected 33 trials from ClinicalTrials.gov. The average precision and recall of pattern detection algorithm calculated for selected patterns is 0.9 and 0.91, meaning that in most cases using the patterns can lead to correct interpretation of criteria and can support patient recruitment.