IDEAS home Printed from https://ideas.repec.org/a/now/jirere/101.00000045.html
   My bibliography  Save this article

Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes

Author

Listed:
  • Lindhjem, Henrik
  • Navrud, Ståle

Abstract

Internet is quickly becoming the survey mode of choice for stated preference (SP) surveys in environmental economics. However, this choice is being made with relatively little consideration of its potential influence on survey results. This paper reviews the theory and emerging evidence of mode effects in the survey methodology and SP literatures, summarizes the findings, and points out implications for Internet SP practice and research. The SP studies that compare Internet with other modes do generally not find substantial difference. The majority of welfare estimates are equal; or somewhat lower for the Internet surveys. Further, there is no clear evidence of substantially lower quality or validity of Internet responses. However, the degree of experimental control is often low in comparative studies across survey modes, and they often confound measurement and sample composition effects. Internet offers a huge potential for experimentation and innovation in SP research, but when used to derive reliable welfare estimates for policy assessment, issues like representation and nonresponse bias for different Internet panels should receive more attention.

Suggested Citation

  • Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    • Handle: RePEc:now:jirere:101.00000045
    DOI: 10.1561/101.00000045
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1561/101.00000045
    Download Restriction: no
    File URL: https://libkey.io/10.1561/101.00000045?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Maria Loureiro & Justus Lotade, 2005. "Interviewer Effects on the Valuation of Goods with Ethical and Environmental Attributes," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 30(1), pages 49-72, January.
    2. Malhotra, Neil & Krosnick, Jon A., 2007. "The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples," Political Analysis, Cambridge University Press, vol. 15(3), pages 286-323, July.
    3. Jürgen Meyerhoff & Ulf Liebe, 2009. "Status Quo Effect in Choice Experiments: Empirical Evidence on Attitudes and Choice Task Complexity," Land Economics, University of Wisconsin Press, vol. 85(3), pages 515-528.
    4. Boxall, Peter C. & Adamowicz, Wiktor L. & Moon, Amanda, 2009. "Complexity in choice experiments: choice of the status quo alternative and implications for welfare measurement," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 53(4), pages 1-17.
    5. Nele Lienhoop & Anke Fischer, 2009. "Can you be bothered? The role of participant motivation in the valuation of species conservation measures," Journal of Environmental Planning and Management, Taylor & Francis Journals, vol. 52(4), pages 519-534.
    6. Ladenburg, Jacob & Olsen, Søren Bøye, 2008. "Gender-specific starting point bias in choice experiments: Evidence from an empirical study," Journal of Environmental Economics and Management, Elsevier, vol. 56(3), pages 275-285, November.
    7. Anna Alberini & James R. Kahn (ed.), 2006. "Handbook on Contingent Valuation," Books, Edward Elgar Publishing, number 1893.
    8. Darren Hudson & Lee-Hong Seah & Diane Hite & Tim Haab, 2004. "Telephone presurveys, self-selection, and non-response bias to mail and Internet surveys in economic research," Applied Economics Letters, Taylor & Francis Journals, vol. 11(4), pages 237-240.
    9. Hui Li & R. P. Berrens & A. K. Bohara & H. C. Jenkins-Smith & C. L. Silva & L. Weimer, 2004. "Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences the same?," Applied Economics Letters, Taylor & Francis Journals, vol. 11(3), pages 173-176.
    10. DeShazo, J. R. & Fermo, German, 2002. "Designing Choice Sets for Stated Preference Methods: The Effects of Complexity on Choice Consistency," Journal of Environmental Economics and Management, Elsevier, vol. 44(1), pages 123-143, July.
    11. Christopher G. Leggett & Naomi S. Kleckner & Kevin J. Boyle & John W. Dufield & Robert Cameron Mitchell, 2003. "Social Desirability Bias in Contingent Valuation Surveys Administered Through In-Person Interviews," Land Economics, University of Wisconsin Press, vol. 79(4), pages 561-575.
    12. Fredrik Carlsson & Mitesh Kataria & Elina Lampi, 2010. "Dealing with Ignored Attributes in Choice Experiments on Valuation of Sweden’s Environmental Quality Objectives," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 47(1), pages 65-89, September.
    13. Shogren, Jason F., 2006. "Experimental Methods and Valuation," Handbook of Environmental Economics, in: K. G. Mäler & J. R. Vincent (ed.), Handbook of Environmental Economics, edition 1, volume 2, chapter 19, pages 969-1027, Elsevier.
    14. Jill Windle & John Rolfe, 2011. "Comparing Responses from Internet and Paper-Based Collection Methods in more Complex Stated Preference Environmental Valuation Surveys," Economic Analysis and Policy, Elsevier, vol. 41(1), pages 83-97, March.
    15. Bateman, Ian J. & Burgess, Diane & Hutchinson, W. George & Matthews, David I., 2008. "Learning design contingent valuation (LDCV): NOAA guidelines, preference learning and coherent arbitrariness," Journal of Environmental Economics and Management, Elsevier, vol. 55(2), pages 127-141, March.
    16. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    17. Davis, Jennifer, 2004. "Assessing Community Preferences for Development Projects: Are Willingness-to-Pay Studies Robust to Mode Effects?," World Development, Elsevier, vol. 32(4), pages 655-672, April.
    18. Gabaix, Xavier & Laibson, David Isaac & Moloche, Guillermo & Stephen, Weinberg, 2003. "The allocation of attention: theory and evidence," MPRA Paper 47339, University Library of Munich, Germany.
    19. Blomquist, Glenn C. & Whitehead, John C., 1998. "Resource quality information and validity of willingness to pay in contingent valuation," Resource and Energy Economics, Elsevier, vol. 20(2), pages 179-196, June.
    20. Andreoni, James, 1990. "Impure Altruism and Donations to Public Goods: A Theory of Warm-Glow Giving?," Economic Journal, Royal Economic Society, vol. 100(401), pages 464-477, June.
    21. Bateman, Ian J. & Mawby, James, 2004. "First impressions count: interviewer appearance and information effects in stated preference studies," Ecological Economics, Elsevier, vol. 49(1), pages 47-55, May.
    22. Lindhjem, Henrik, 2007. "20 years of stated preference valuation of non-timber benefits from Fennoscandian forests: A meta-analysis," Journal of Forest Economics, Elsevier, vol. 12(4), pages 251-277, February.
    23. Fredrik Carlsson, 2010. "Design of Stated Preference Surveys: Is There More to Learn from Behavioral Economics?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(2), pages 167-177, June.
    24. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    25. David J. Bjornstad & James R. Kahn (ed.), 1996. "The Contingent Valuation of Environmental Resources," Books, Edward Elgar Publishing, number 731.
    26. Nielsen, Jytte Seested, 2011. "Use of the Internet for willingness-to-pay surveys: A comparison of face-to-face and web-based interviews," Resource and Energy Economics, Elsevier, vol. 33(1), pages 119-129, January.
    27. Smith, V. Kerry, 2000. "JEEM and Non-market Valuation: 1974-1998," Journal of Environmental Economics and Management, Elsevier, vol. 39(3), pages 351-374, May.
    28. Bateman, Ian J. & Day, Brett H. & Jones, Andrew P. & Jude, Simon, 2009. "Reducing gain-loss asymmetry: A virtual reality choice experiment valuing land use change," Journal of Environmental Economics and Management, Elsevier, vol. 58(1), pages 106-118, July.
    29. Li, Hui & Jenkins-Smith, Hank C. & Silva, Carol L. & Berrens, Robert P. & Herron, Kerry G., 2009. "Public support for reducing US reliance on fossil fuels: Investigating household willingness-to-pay for energy research and development," Ecological Economics, Elsevier, vol. 68(3), pages 731-742, January.
    30. MacMillan, Douglas & Hanley, Nick & Lienhoop, Nele, 2006. "Contingent valuation: Environmental polling or preference engine?," Ecological Economics, Elsevier, vol. 60(1), pages 299-307, November.
    31. Berrens, Robert P. & Bohara, Alok K. & Jenkins-Smith, Hank C. & Silva, Carol L. & Weimer, David L., 2004. "Information and effort in contingent valuation surveys: application to global climate change using national internet samples," Journal of Environmental Economics and Management, Elsevier, vol. 47(2), pages 331-363, March.
    32. Søren Olsen, 2009. "Choosing Between Internet and Mail Survey Modes for Choice Experiment Surveys Considering Non-Market Goods," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 44(4), pages 591-610, December.
    33. STÅLE Navrud & GERALD Pruckner, 1997. "Environmental Valuation – To Use or Not to Use? A Comparative Study of the United States and Europe," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 10(1), pages 1-26, July.
    34. Lindberg, Kreg & Johnson, Rebecca L. & Berrens, Robert P., 1997. "Contingent Valuation Of Rural Tourism Development With Tests Of Scope And Mode Stability," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 22(1), pages 1-17, July.
    35. Henrik Lindhjem & Ståle Navrud, 2009. "Asking for Individual or Household Willingness to Pay for Environmental Goods?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 43(1), pages 11-29, May.
    36. Cameron, Trudy Ann & Poe, Gregory L. & Ethier, Robert G. & Schulze, William D., 2002. "Alternative Non-market Value-Elicitation Methods: Are the Underlying Preferences the Same?," Journal of Environmental Economics and Management, Elsevier, vol. 44(3), pages 391-425, November.
    37. Schwappach, David L.B. & Strasmann, Thomas J., 2006. ""Quick and dirty numbers"?: The reliability of a stated-preference technique for the measurement of preferences for resource allocation," Journal of Health Economics, Elsevier, vol. 25(3), pages 432-448, May.
    38. Berrens, Robert P. & Bohara, Alok K. & Jenkins-Smith, Hank & Silva, Carol & Weimer, David L., 2003. "The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples," Political Analysis, Cambridge University Press, vol. 11(1), pages 1-22, January.
    39. Arvin B. Vista & Randall S. Rosenberger & Alan R. Collins, 2009. "If You Provide It, Will They Read It? Response Time Effects in a Choice Experiment," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 57(3), pages 365-377, September.
    40. van der Heide, C. Martijn & van den Bergh, Jeroen C.J.M. & van Ierland, Ekko C. & Nunes, Paulo A.L.D., 2008. "Economic valuation of habitat defragmentation: A study of the Veluwe, the Netherlands," Ecological Economics, Elsevier, vol. 67(2), pages 205-216, September.
    41. Whitehead, John C. & Groothuis, Peter A. & Blomquist, Glenn C., 1993. "Testing for non-response and sample selection bias in contingent valuation : Analysis of a combination phone/mail survey," Economics Letters, Elsevier, vol. 41(2), pages 215-220.
    42. Søren Olsen & Thomas Lundhede & Jette Jacobsen & Bo Thorsen, 2011. "Tough and Easy Choices: Testing the Influence of Utility Difference on Stated Certainty-in-Choice in Choice Experiments," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 49(4), pages 491-510, August.
    43. Vera Toepoel & Corrie Vis & Marcel Das & Arthur van Soest, 2009. "Design of Web Questionnaires," Sociological Methods & Research, , vol. 37(3), pages 371-392, February.
    44. Whittington, Dale & Smith, V. Kerry & Okorafor, Apia & Okore, Augustine & Liu, Jin Long & McPhail, Alexander, 1992. "Giving respondents time to think in contingent valuation studies: A developing country application," Journal of Environmental Economics and Management, Elsevier, vol. 22(3), pages 205-225, May.
    45. Shaikh, Sabina L. & Sun, Lili & Cornelis van Kooten, G., 2007. "Treating respondent uncertainty in contingent valuation: A comparison of empirical treatments," Ecological Economics, Elsevier, vol. 62(1), pages 115-125, April.
    46. Beilei Cai & Trudy Cameron & Geoffrey Gerdes, 2010. "Distributional Preferences and the Incidence of Costs and Benefits in Climate Change Policy," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(4), pages 429-458, August.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lindhjem, Henrik & Navrud, Ståle, 2008. "Internet CV surveys – a cheap, fast way to get large samples of biased values?," MPRA Paper 11471, University Library of Munich, Germany.
    2. Lindhjem, Henrik & Navrud, Ståle, 2010. "Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?," MPRA Paper 24069, University Library of Munich, Germany.
    3. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    4. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    5. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    6. Menegaki, Angeliki, N. & Olsen, Søren Bøye & Tsagarakis, Konstantinos P., 2016. "Towards a common standard – A reporting checklist for web-based stated preference valuation surveys and a critique for mode surveys," Journal of choice modelling, Elsevier, vol. 18(C), pages 18-50.
    7. Guimarães, Maria Helena & Nunes, Luís Catela & Madureira, Lívia & Santos, José Lima & Boski, Tomasz & Dentinho, Tomaz, 2015. "Measuring birdwatchers preferences: A case for using online networks and mixed-mode surveys," Tourism Management, Elsevier, vol. 46(C), pages 102-113.
    8. Jed J. Cohen & Johannes Reichl, 2022. "Comparing Internet and phone survey mode effects across countries and research contexts," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 66(1), pages 44-71, January.
    9. Tobias Börger, 2016. "Are Fast Responses More Random? Testing the Effect of Response Time on Scale in an Online Choice Experiment," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 65(2), pages 389-413, October.
    10. Liebe, Ulf & Glenk, Klaus & Oehlmann, Malte & Meyerhoff, Jürgen, 2015. "Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?," Journal of choice modelling, Elsevier, vol. 14(C), pages 17-31.
    11. Bruno Lanz & Allan Provins, 2012. "Do status quo choices reflect preferences? Evidence from a discrete choice experiment in the context of water utilities' investment planning," CEPE Working paper series 12-87, CEPE Center for Energy Policy and Economics, ETH Zurich.
    12. Fredrik Carlsson, 2010. "Design of Stated Preference Surveys: Is There More to Learn from Behavioral Economics?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 46(2), pages 167-177, June.
    13. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    14. Jürgen Meyerhoff & Morten Mørkbak & Søren Olsen, 2014. "A Meta-study Investigating the Sources of Protest Behaviour in Stated Preference Surveys," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 58(1), pages 35-57, May.
    15. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    16. Hui Li & Robert P. Berrens & Alok K. Bohara & Hank C. Jenkins-Smith & Carol L. Silva & David L. Weimer, 2005. "Exploring the Beta Model Using Proportional Budget Information in a Contingent Valuation Study," Economics Bulletin, AccessEcon, vol. 17(8), pages 1-9.
    17. Mozumder, Pallab & Vásquez, William F. & Marathe, Achla, 2011. "Consumers' preference for renewable energy in the southwest USA," Energy Economics, Elsevier, vol. 33(6), pages 1119-1126.
    18. Bruno Lanz & Allan Provins, 2015. "Using discrete choice experiments to regulate the provision of water services: do status quo choices reflect preferences?," Journal of Regulatory Economics, Springer, vol. 47(3), pages 300-324, June.
    19. Ladenburg, Jacob & Olsen, Søren Bøye, 2014. "Augmenting short Cheap Talk scripts with a repeated Opt-Out Reminder in Choice Experiment surveys," Resource and Energy Economics, Elsevier, vol. 37(C), pages 39-63.
    20. Thijs Dekker & Paul Koster & Roy Brouwer, 2014. "Changing with the Tide: Semiparametric Estimation of Preference Dynamics," Land Economics, University of Wisconsin Press, vol. 90(4), pages 717-745.

    More about this item

    Keywords

    Internet; Survey mode; Contingent valuation; Stated preferences;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D62 - Microeconomics - - Welfare Economics - - - Externalities
    • H41 - Public Economics - - Publicly Provided Goods - - - Public Goods
    • Q51 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Valuation of Environmental Effects

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:now:jirere:101.00000045. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Lucy Wiseman (email available below). General contact details of provider: http://www.nowpublishers.com/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.