Reliability of assigning correct current procedural terminology—4 E/M codes

Study objectives: We determine the degree of interagency and intra-agency agreement on Current Procedural Terminology-4 (CPT-4) coding levels for emergency department medical records. We hypothesized that the level of agreement among coding agencies would be poor, but the distribution of codes would be similar and not significantly different. Methods: We performed a prospective observational study consisting of 3 separate randomized, blinded trials: 2 interagency comparisons and 1 intra-agency comparison. The setting was 2 suburban academic EDs. In the 2 interagency audits, 4 coding agencies were used. In the intra-agency comparison, we used 5 individual coders from within our own internal agency. The main outcome measures were the level of agreement not due to chance estimated with the multiple-rater weighted κ statistic and the Kendall τ-b. We measured the distribution of codes by agency or by individual coders with the χ2 test. Results: Our sample for the 2 interagency audits consisted of 194 and 195 records, respectively. We observed poor agreement in the level of coding assigned to individual charts among the 4 coding agencies, with κ values of 0.28 (95% confidence interval [CI] 0.275 to 0.285) and 0.287 (95% CI 0.283 to 0.291). Our intra-agency comparison consisted of 100 records. The agreement in the intra-agency review was significantly better but still fair (κ=0.436; 95% CI 0.428 to 0.444). The distribution of CPT-4 codes was significantly different for all 3 comparisons (P<.001). Conclusion: For our patient population, group of physicians, and methods of documentation, we identified poor-to-fair agreement in coding of emergency charts between coding agencies. Only fair agreement was measured in the intra-agency sample. The distribution of assigned CPT-4 codes was significantly different in each comparison. These findings have important financial and legal implications regarding the reliability of coding methods. [Ann Emerg Med. 2002;40:269-274.]