Reliability of Standard Health Assessment Instruments in a Large, Population-Based Cohort Study

Purpose The Millennium Cohort Study began in 2001 using mail and Internet questionnaires to gather occupational and environmental exposure, behavioral risk factor, and health outcome data from a large, population-based US military cohort. Standardized instruments, including the Patient Health Questionnaire, the Medical Outcomes Study Short Form-36 for Veterans, and the Posttraumatic Stress Disorder (PTSD) Checklist–Civilian Version, have been validated in various populations. The purpose of this study was to investigate internal consistency of standardized instruments and concordance of responses in a test-retest setting. Methods Cronbach alpha coefficients were used to investigate the internal consistency of standardized instruments among 76,742 participants. Kappa statistics were calculated to measure stability of aggregated responses in a subgroup of 470 participants who voluntarily submitted an additional survey within 6 months of their original submission. Results High internal consistency was found for 14 of 16 health components, with lower internal consistency found among two alcohol components. Substantial test-retest stability was observed for stationary variables, while moderate stability was found for more dynamic variables that measured conditions with low prevalence. Conclusions These results substantiate internal consistency and stability of several standard health instruments applied to this large cohort. Such reliability analyses are vital to the integrity of long-term outcome studies.