An investigation into expectation-led interviewer effects in health surveys

Many large-scale health surveys use interviewers to obtain standardised information about the health of the general population. To improve response rates and data quality, the researchers/designers usually brief the interviewers to familiarise them with the survey procedures and stimulate their interest in the survey. However, it is possible that interviewers, having been exposed to researchers’ expectations, may inadvertently influence respondents to produce outcomes consistent with those expectations. Such expectations are referred to here as ‘expectation-led interviewer effects’. In this paper, the design and results from an experiment to test for expectation-led interviewer effects are described. The experiment involved conducting two health surveys, called the ‘experimental’ and the ‘control’, which were identical in every way except that researchers made a reference to a supposed link between childhood and adult health at the experimental survey briefing. The testing procedure was designed prior to data collection to preclude accusations of data dredging and to ensure that the type I error probability was less than 5 percent. No consistent evidence of expectation-led interviewer effects was found, bar a statistically significant effect for health questions requiring the recall of detailed quantitative information. This effect was small, however.