Methodology for establishing a `gold standard´ [for a medical expert system]

Presents a statistical method for establishing a consensus diagnosis among several experts, which is the common `gold standard´ used for evaluating the diagnostic performance of a medical expert system. This method involves the use of the extended kappa statistic developed by J.L. Fleiss (1971) and R.J. Light (1971) in the social sciences for the study of a similar problem. The method is carried out in two stages. First, the existence of an overall agreement or disagreement among the medical experts in their diagnoses of a sample of patients is established. Second, in the case of overall agreement, one selects the particular disorders that have a significant level of agreement among the experts, and uses the experts´ diagnoses as the `gold standard´ for the sample of patients that were classified in those disorders