Investigating Differences between Internet and Mail Implementation of a Stated-Preference Study While Controlling for Differences in Sample Frames and Self-Selection Effects

The increasing use of internet surveys for stated preference studies raises questions about the effect of the survey mode on sample representativeness, value estimates and data quality. A number of studies have now been conducted that compare the use of the internet with other survey modes, but all of these comparisons apart from one have confounded testing of mode effects with sample frame effects, and no previous studies have separated these two effects when testing the consequences of using an online panel. In addition, no previous studies have adjusted for survey self-selection across treatments during model estimation which has also confounded testing in previous studies. In this study we seek to identify the effects of using an online panel, and conduct three treatments to decompose model and sample frame effects. Using choice modelling and a case study focusing on riverine health we find that there are some differences in response rates and differences in mean values of the socio-demographics, experience of the river and knowledge. We also use sample weights developed through raking to adjust for potential self-selection effects due to item non-response on key survey questions. We find that self-selection effects are negligible; however in contrast to the previous literature we identify substantive and statistically significant differences in values due to sample frame but especially survey mode, with internet surveys producing values that are on average 30% lower than mail.