Detecting cumulative watershed effects: the statistical power of pairing

The statistical power for detecting change in water quality should be a primary consideration when designing monitoring studies. However, some of the standard approaches for estimating sample size result in a power of less than 50%, and doubling the pre- and post-treatment sample size are necessary to increase the power to 80%. The ability to detect change can be improved by including an additional explanatory variable such as paired watershed measurements. However, published guidelines have not explicitly quantified the benefits of including an explanatory variable or the specific conditions that favor a paired watershed design. This paper (1) presents a power analysis for the statistical model (analysis of covariance) commonly used in paired watershed studies; (2) discusses the conditions under which it is beneficial to include an explanatory variable; and (3) quantifies the benefits of the paired watershed design. The results show that it is beneficial to include an explanatory variable when its correlation to the water quality variable of concern is as low as about 0.3. The ability to detect change increases non-linearly as the correlation increases. Power curves quantify sample size requirements as a function of the correlation and intrinsic variability. In general, the temporal and spatial variability of many watershed-scale characteristics, such as annual sediment loads, makes it very difficult to detect changes within time spans that are useful for land managers or conducive to adaptive management.