IDEAS home Printed from https://ideas.repec.org/a/bla/ajarec/v66y2022i1p44-71.html
   My bibliography  Save this article

Comparing Internet and phone survey mode effects across countries and research contexts

Author

Listed:
  • Jed J. Cohen
  • Johannes Reichl

Abstract

We compare Internet and telephone survey responses across 27 European Union nations and two research contexts: one, a choice experiment of willingness to pay to avoid power outages, and the other, a public acceptance of energy infrastructure question. The various forms of survey mode effects and the challenges of survey mode choice are documented and developed in the context of statistical theory and an application to an economics survey. We find evidence that survey mode effects vary across research contexts, and to a lesser extent, across nations. We suggest that the degree of measurement bias may be varying between research contexts, for example based on the availability of a perceived socially correct response within a given context. Future survey‐based research should evaluate the choice of survey mode in a context‐ and region‐specific manner.

Suggested Citation

  • Jed J. Cohen & Johannes Reichl, 2022. "Comparing Internet and phone survey mode effects across countries and research contexts," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 66(1), pages 44-71, January.
  • Handle: RePEc:bla:ajarec:v:66:y:2022:i:1:p:44-71
    DOI: 10.1111/1467-8489.12451
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/1467-8489.12451
    Download Restriction: no

    File URL: https://libkey.io/10.1111/1467-8489.12451?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    2. Sandorf, Erlend Dancke & Campbell, Danny & Hanley, Nick, 2017. "Disentangling the influence of knowledge on attribute non-attendance," Journal of choice modelling, Elsevier, vol. 24(C), pages 36-50.
    3. Boxall, Peter C. & Adamowicz, Wiktor L. & Moon, Amanda, 2009. "Complexity in choice experiments: choice of the status quo alternative and implications for welfare measurement," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 53(4), pages 1-17.
    4. Silvio R. Rendon, 2013. "Fixed and Random Effects in Classical and Bayesian Regression," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 75(3), pages 460-476, June.
    5. Scarpa, Riccardo & Rose, John M., 2008. "Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 52(3), pages 1-30.
    6. Carlsson, Fredrik & Martinsson, Peter, 2008. "Does it matter when a power outage occurs? -- A choice experiment study on the willingness to pay to avoid power outages," Energy Economics, Elsevier, vol. 30(3), pages 1232-1245, May.
    7. Wustenhagen, Rolf & Wolsink, Maarten & Burer, Mary Jean, 2007. "Social acceptance of renewable energy innovation: An introduction to the concept," Energy Policy, Elsevier, vol. 35(5), pages 2683-2691, May.
    8. Jill Windle & John Rolfe, 2011. "Comparing Responses from Internet and Paper-Based Collection Methods in more Complex Stated Preference Environmental Valuation Surveys," Economic Analysis and Policy, Elsevier, vol. 41(1), pages 83-97, March.
    9. Hyslop, Dean R & Imbens, Guido W, 2001. "Bias from Classical and Other Forms of Measurement Error," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(4), pages 475-481, October.
    10. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    11. Jelke Bethlehem, 2010. "Selection Bias in Web Surveys," International Statistical Review, International Statistical Institute, vol. 78(2), pages 161-188, August.
    12. Paul Dunn & Anamitra Shome, 2009. "Cultural Crossvergence and Social Desirability Bias: Ethical Evaluations by Chinese and Canadian Business Students," Journal of Business Ethics, Springer, vol. 85(4), pages 527-543, April.
    13. Scarpa, R. & Thiene, M. & Train, K., 2008. "Appendix to Utility in WTP space: a tool to address confounding random scale effects in destination choice to the Alps," American Journal of Agricultural Economics APPENDICES, Agricultural and Applied Economics Association, vol. 90(4), pages 1-9, January.
    14. Javid Shabbir & Sat Gupta, 2017. "Estimation of finite population mean in simple and stratified random sampling using two auxiliary variables," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 46(20), pages 10135-10148, October.
    15. Garrett Sonnier & Andrew Ainslie & Thomas Otter, 2007. "Heterogeneity distributions of willingness-to-pay in choice models," Quantitative Marketing and Economics (QME), Springer, vol. 5(3), pages 313-331, September.
    16. Jed Cohen & Klaus Moeltner & Johannes Reichl & Michael Schmidthaler, 2018. "Effect of global warming on willingness to pay for uninterrupted electricity supply in European nations," Nature Energy, Nature, vol. 3(1), pages 37-45, January.
    17. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    18. de Nooij, Michiel & Koopmans, Carl & Bijvoet, Carlijn, 2007. "The value of supply security: The costs of power interruptions: Economic input for damage reduction and investment in networks," Energy Economics, Elsevier, vol. 29(2), pages 277-295, March.
    19. Cohen, Jed J. & Reichl, Johannes & Schmidthaler, Michael, 2014. "Re-focussing research efforts on the public acceptance of energy infrastructure: A critical review," Energy, Elsevier, vol. 76(C), pages 4-9.
    20. Fredrik Carlsson & Peter Martinsson, 2007. "Willingness to Pay among Swedish Households to Avoid Power Outages: A Random Parameter Tobit Model Approach," The Energy Journal, International Association for Energy Economics, vol. 0(Number 1), pages 75-90.
    21. Robert Breunig & Rebecca McKibbin, 2011. "The effect of survey design on household reporting of financial difficulty," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 991-1005, October.
    22. Robert G. Ethier & Gregory L. Poe & William D. Schulze & Jeremy Clark, 2000. "Comparison of Hypothetical Phone and Mail Contingent Valuation Responses for Green-Pricing Electricity Programs," Land Economics, University of Wisconsin Press, vol. 76(1), pages 54-67.
    23. Peter Elliott & David Wadley, 2012. "Coming to Terms with Power Lines," International Planning Studies, Taylor & Francis Journals, vol. 17(2), pages 179-201.
    24. Jed Cohen, Klaus Moeltner, Johannes Reichl and Michael Schmidthaler, 2016. "An Empirical Analysis of Local Opposition to New Transmission Lines Across the EU-27," The Energy Journal, International Association for Energy Economics, vol. 0(Number 3).
    25. Leahy, Eimear & Tol, Richard S.J., 2011. "An estimate of the value of lost load for Ireland," Energy Policy, Elsevier, vol. 39(3), pages 1514-1520, March.
    26. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    27. Nielsen, Jytte Seested, 2011. "Use of the Internet for willingness-to-pay surveys: A comparison of face-to-face and web-based interviews," Resource and Energy Economics, Elsevier, vol. 33(1), pages 119-129, January.
    28. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    29. Spanos,Aris, 1999. "Probability Theory and Statistical Inference," Cambridge Books, Cambridge University Press, number 9780521424080.
    30. Campbell, Danny, 2007. "Combining mixed logit models and random effects models to identify the determinants of willingness to pay for rural landscape improvements," 81st Annual Conference, April 2-4, 2007, Reading University, UK 7975, Agricultural Economics Society.
    31. Danny Campbell, 2007. "Willingness to Pay for Rural Landscape Improvements: Combining Mixed Logit and Random‐Effects Models," Journal of Agricultural Economics, Wiley Blackwell, vol. 58(3), pages 467-483, September.
    32. Siraj Muneer & Javid Shabbir & Alamgir Khalil, 2017. "Estimation of finite population mean in simple random sampling and stratified random sampling using two auxiliary variables," Communications in Statistics - Theory and Methods, Taylor & Francis Journals, vol. 46(5), pages 2181-2192, March.
    33. Søren Olsen, 2009. "Choosing Between Internet and Mail Survey Modes for Choice Experiment Surveys Considering Non-Market Goods," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 44(4), pages 591-610, December.
    34. Cohen, Jed & Moeltner, Klaus & Reichl, Johannes & Schmidthaler, Michael, 2018. "Valuing electricity-dependent infrastructure: An essential-input approach," Energy Economics, Elsevier, vol. 73(C), pages 258-273.
    35. Reichl, Johannes & Schmidthaler, Michael & Schneider, Friedrich, 2013. "The value of supply security: The costs of power outages to Austrian households, firms and the public sector," Energy Economics, Elsevier, vol. 36(C), pages 256-261.
    36. Schwappach, David L.B. & Strasmann, Thomas J., 2006. ""Quick and dirty numbers"?: The reliability of a stated-preference technique for the measurement of preferences for resource allocation," Journal of Health Economics, Elsevier, vol. 25(3), pages 432-448, May.
    37. Baarsma, Barbara E. & Hop, J. Peter, 2009. "Pricing power outages in the Netherlands," Energy, Elsevier, vol. 34(9), pages 1378-1386.
    38. Thomas Klausch & Joop Hox & Barry Schouten, 2015. "Selection error in single- and mixed mode surveys of the Dutch general population," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 945-961, October.
    39. Olga Maslovskaya & Gabriele B. Durrant & Peter W.F. Smith & Tim Hanson & Ana Villar, 2019. "What are the Characteristics of Respondents using Different Devices in Mixed‐device Online Surveys? Evidence from Six UK Surveys," International Statistical Review, International Statistical Institute, vol. 87(2), pages 326-346, August.
    40. Riccardo Scarpa & Mara Thiene & Kenneth Train, 2008. "Utility in Willingness to Pay Space: A Tool to Address Confounding Random Scale Effects in Destination Choice to the Alps," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 90(4), pages 994-1010.
    41. Praktiknjo, Aaron J. & Hähnel, Alexander & Erdmann, Georg, 2011. "Assessing energy supply security: Outage costs in private households," Energy Policy, Elsevier, vol. 39(12), pages 7825-7833.
    42. Matthew Cotton & Patrick Devine-Wright, 2013. "Putting pylons into place: a UK case study of public perspectives on the impacts of high voltage overhead transmission lines," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 56(8), pages 1225-1245, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cohen, Jed J. & Moeltner, Klaus & Reichl, Johannes & Schmidthaler, Michael, 2016. "Linking the value of energy reliability to the acceptance of energy infrastructure: Evidence from the EU," Resource and Energy Economics, Elsevier, vol. 45(C), pages 124-143.
    2. Morrissey, Karyn & Plater, Andrew & Dean, Mary, 2018. "The cost of electric power outages in the residential sector: A willingness to pay approach," Applied Energy, Elsevier, vol. 212(C), pages 141-150.
    3. Liebe, Ulf & Glenk, Klaus & Oehlmann, Malte & Meyerhoff, Jürgen, 2015. "Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?," Journal of choice modelling, Elsevier, vol. 14(C), pages 17-31.
    4. Amoah, Anthony & Ferrini, Silvia & Schaafsma, Marije, 2019. "Electricity outages in Ghana: Are contingent valuation estimates valid?," Energy Policy, Elsevier, vol. 135(C).
    5. Zhifeng Gao & Lisa A. House & Jing Xie, 2016. "Online Survey Data Quality and Its Implication for Willingness-to-Pay: A Cross-Country Comparison," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(2), pages 199-221, June.
    6. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    7. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    8. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    9. Motz, Alessandra, 2021. "Security of supply and the energy transition: The households' perspective investigated through a discrete choice model with latent classes," Energy Economics, Elsevier, vol. 97(C).
    10. Sandorf, Erlend Dancke & Aanesen, Margrethe & Navrud, Ståle, 2016. "Valuing unfamiliar and complex environmental goods: A comparison of valuation workshops and internet panel surveys with videos," Ecological Economics, Elsevier, vol. 129(C), pages 50-61.
    11. Zawojska, Ewa & Czajkowski, Mikotaj, 2017. "Are preferences stated in web vs. personal interviews different? A comparison of willingness to pay results for a large multi-country study of the Baltic Sea eutrophication reduction," Annual Meeting, 2017, June 18-21, Montreal, Canada 258604, Canadian Agricultural Economics Society.
    12. Carlsson, Fredrik & Kataria, Mitesh & Lampi, Elina & Martinsson, Peter, 2021. "Past and present outage costs – A follow-up study of households’ willingness to pay to avoid power outages," Resource and Energy Economics, Elsevier, vol. 64(C).
    13. Doherty, Edel & Campbell, Danny & Hynes, Stephen, 2012. "Exploring cost heterogeneity in recreational demand," Working Papers 148832, National University of Ireland, Galway, Socio-Economic Marine Research Unit.
    14. Landegren, Finn & Johansson, Jonas & Samuelsson, Olof, 2019. "Quality of supply regulations versus societal priorities regarding electricity outage consequences: Case study in a Swedish context," International Journal of Critical Infrastructure Protection, Elsevier, vol. 26(C).
    15. Kim, Kayoung & Nam, Heekoo & Cho, Youngsang, 2015. "Estimation of the inconvenience cost of a rolling blackout in the residential sector: The case of South Korea," Energy Policy, Elsevier, vol. 76(C), pages 76-86.
    16. Jacobsen, Jette Bredahl & Lundhede, Thomas Hedemark & Martinsen, Louise & Hasler, Berit & Thorsen, Bo Jellesmark, 2011. "Embedding effects in choice experiment valuations of environmental preservation projects," Ecological Economics, Elsevier, vol. 70(6), pages 1170-1177, April.
    17. Mikołaj Czajkowski & Ewa Zawojska & Norman Meade & Ronaldo Seroa da Motta & Mike Welsh & Ramon Arigoni Ortiz, 2022. "On the inference about willingness to pay distribution using contingent valuation data," Working Papers 2022-08, Faculty of Economic Sciences, University of Warsaw.
    18. Majid Hashemi & Glenn P. Jenkins & Roop Jyoti & Aygul Ozbafli, 2018. "Evaluating the Cost to Industry of Electricity Outages," Development Discussion Papers 2018-14, JDI Executive Programs.
    19. Tobias Börger, 2016. "Are Fast Responses More Random? Testing the Effect of Response Time on Scale in an Online Choice Experiment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 65(2), pages 389-413, October.
    20. Woo, C.K. & Ho, T. & Shiu, A. & Cheng, Y.S. & Horowitz, I. & Wang, J., 2014. "Residential outage cost estimation: Hong Kong," Energy Policy, Elsevier, vol. 72(C), pages 204-210.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ajarec:v:66:y:2022:i:1:p:44-71. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/aaresea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.