Randomized trial for answers to clinical questions: Evaluating a pre-appraised versus a MEDLINE search protocol

Objective: The paper compares the speed, validity, and applicability of two different protocols for searching the primary medical literature. Design: A randomized trial involving medicine residents was performed. Setting: An inpatient general medicine rotation was used. Participants: Thirty-two internal medicine residents were block randomized into four groups of eight. Main Outcome Measures: Success rate of each search protocol was measured by perceived search time, number of questions answered, and proportion of articles that were applicable and valid. Results: Residents randomized to the MEDLINE-first (protocol A) group searched 120 questions, and residents randomized to the MEDLINE-last (protocol B) searched 133 questions. In protocol A, 104 answers (86.7%) and, in protocol B, 117 answers (88%) were found to clinical questions. In protocol A, residents reported that 26 (25.2%) of the answers were obtained quickly or rated as ʹʹfastʹʹ (<5 minutes) as opposed to 55 (51.9%) in protocol B, (P = 0.0004). A subset of questions and articles (n = 79) were reviewed by faculty who found that both protocols identified similar numbers of answer articles that addressed the questions and were felt to be valid using critical appraisal criteria. Conclusion: For resident-generated clinical questions, both protocols produced a similarly high percentage of applicable and valid articles. The MEDLINE-last search protocol was perceived to be faster. However, in the MEDLINE-last protocol, a significant portion of questions (23%) still required searching MEDLINE to find an answer.