Parameter estimation validity and relationship robustness: A comparison of telephone and internet survey techniques

With the expansion of telecommunication and online technologies for the purpose of survey administration, the issue of measurement validity has come to the fore. The proliferation of automated audio services and computer-based survey techniques has been matched by a corresponding denigration of the quality of traditional phone survey data, most notably as an outcome of falling response rates. This trend, combined with the introduction of screening technologies and answering machines, represents a barrier to the proper execution of survey research. Whereas the question was once, “can technology-assisted surveys achieve the same level of validity as traditional phone surveys?”, the question now becomes, “what are the relative advantages and disadvantages of technology-assisted and phone surveys?” Each has its own challenges and opportunities, and this paper begins to explore these. The present study provides further insight into the validity of telephone and Internet survey data, and explores whether or not the robustness of relationships between variables varies by survey mode. Study data were provided by two surveys, the first of which was conducted in a metropolitan area of the Midwestern US, with interviews of 505 adults using a computer-aided telephone-interviewing (CATI) system. The second was a national survey of 2172 respondents conducted over the Internet by a commercial research firm that sends requests to a diverse set of potential respondents, who logged onto the survey site to participate. Results suggest that weighting in an attempt to achieve parametric matching does seem to increase robustness of relationships and, in this age of poor response rates, this seems to demand an increased use of parametric weightings. Implications of study findings for telematic survey practitioners are discussed.