Balancing Test Length with Sufficiently Reliable Scores.

One of the most important indicators of a quality examination is the reliability of the scores. In low to moderate stakes scenarios it is desirable for scores to achieve a minimum estimate of .70. Unfortunately, for many medical educators it is difficult to attain this minimum threshold for “acceptable” statistical reproducibility. A common approach is to include additional items to an exam, but this process can become cumbersome and misguided without clear direction. Fortunately, the Spearman-Brown Prophecy formula can help medical educators identify approximately how many additional items will be necessary to achieve a minimum reliability estimate of .70. This article describes a case in which we were presented with a less than desirable reliability estimate from a previous medical school examination, utilized the Spearman-Brown Prophecy formula, and was able to achieve the minimum estimate of .70 on the next iteration of the examination. We encourage others to make use of this technique rarely utilized outside of the psychometric arena as well.