Co-training and ensemble based duplicate detection in adverse drug event reporting systems

Nowadays, many countries have established spontaneous reporting systems (SRSs) to facilitate postmarketing surveillance of listed drugs and collect enough data for detecting unknown adverse drug reactions. Due to data in SRSs coming from different sources of reporters, there heralds the problem of duplicate reporting; even a small amount of duplicate records would bias the detection results. Although lots of works have been conducted on duplicate record detection, very few of them have been devoted to dataset about adverse drug reactions, and none of them have considered the existence of follow-up reports. In this study, we investigated the problem of identifying duplicate ADR reports in SRSs with the presence of follow-ups. We proposed an ensemble and co-training based detection method that is capable of detecting for a given report not only its duplicates but also its initial or earlier linkage cases.