Practice guidelines developed by specialty societies: the need for a critical appraisal

Background There is increasing concern about the quality, reliability, and independence of practice guidelines. Because no information is available on the methodological quality of the guidelines developed by specialty societies, we undertook a survey on those published in peer-reviewed journals. Methods Practice guidelines produced by specialty societies and published in English between January, 1988, and July, 1998, where identified through MEDLINE. Their quality was assessed in terms of whether they reported: the type of professionals and stakeholders involved in the development process; the strategy to identify primary evidence; and an explicit grading of recommendations according to the quality of supporting evidence. Findings Overall, 431 guidelines were eligible for the study. Most did not meet the criteria: 67% did not report any description of the type of stakeholders, 88% gave no information on searches for published studies, and 82% did not give any explicit grading of the strength of recommendations. There was improvement over time for searches (from 2% to 18%, p<0•001) and explicit grading of evidence (from 6% to 27%, p<0•001). All three criteria for quality were met in only 22 (5%) guidelines. Interpretation Despite improvement over time, the quality of practice guidelines developed by specialty societies is unsatisfactory. Explicit methodological criteria for the production of guidelines shared among public agencies, scientific societies, and patientsʹ associations need to be set up. Common standards of reporting, following the same principles that led to the CONSORT statement for randomised clinical trials, should be promoted.