IDEAS home Printed from
   My bibliography  Save this article

Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames


  • Kevin J. Boyle

    (Virginia Tech)

  • Mark Morrison

    (School of Management and Marketing)

  • Darla Hatton MacDonald


  • Roderick Duncan

    (Charles Sturt University)

  • John Rose

    (University of South Australia)


The increasing use of internet surveys for stated preference studies raises questions about the effect of the survey mode on welfare estimates. A number of studies have conducted convergent validity investigations that compare the use of the internet with other survey implementation modes such as mail, telephone and in-person. All, but one, of these comparison studies is confounded different sample frames for the different modes of survey implementation. In this study we investigate differences in internet and mail survey modes holding the sample frame constant. This is done in the context of a choice-modelling study of improvements in the River Murray in Australia. We also investigate sample frame holding the survey mode (mail) constant. We find that survey mode (internet vs. mail) influences welfare estimates, and the internet welfare estimates are about 78 % of the mail welfare estimates on average. There is not a significant effect of sample frame (internet panel vs. postal addresses) on welfare estimates for implementation of a mail survey.

Suggested Citation

  • Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
  • Handle: RePEc:kap:enreec:v:64:y:2016:i:3:d:10.1007_s10640-015-9876-2
    DOI: 10.1007/s10640-015-9876-2

    Download full text from publisher

    File URL:
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL:
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    1. Christian A. Vossler & Maurice Doyon & Daniel Rondeau, 2012. "Truth in Consequentiality: Theory and Field Evidence on Discrete Choice Experiments," American Economic Journal: Microeconomics, American Economic Association, vol. 4(4), pages 145-171, November.
    2. Yuan Yuan & Kevin J. Boyle & Wen You, 2015. "Sample Selection, Individual Heterogeneity, and Regional Heterogeneity in Valuing Farmland Conservation Easements," Land Economics, University of Wisconsin Press, vol. 91(4), pages 627-649.
    3. Hui Li & R. P. Berrens & A. K. Bohara & H. C. Jenkins-Smith & C. L. Silva & L. Weimer, 2004. "Telephone versus Internet samples for a national advisory referendum: are the underlying stated preferences the same?," Applied Economics Letters, Taylor & Francis Journals, vol. 11(3), pages 173-176.
    4. Jill Windle & John Rolfe, 2011. "Comparing Responses from Internet and Paper-Based Collection Methods in more Complex Stated Preference Environmental Valuation Surveys," Economic Analysis and Policy, Elsevier, vol. 41(1), pages 83-97, March.
    5. Bliemer, Michiel C.J. & Rose, John M. & Hensher, David A., 2009. "Efficient stated choice experiments for estimating nested logit models," Transportation Research Part B: Methodological, Elsevier, vol. 43(1), pages 19-35, January.
    6. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    7. Train,Kenneth E., 2009. "Discrete Choice Methods with Simulation," Cambridge Books, Cambridge University Press, number 9780521747387, February.
    8. Judith Covey & Angela Robinson & Michael Jones-Lee & Graham Loomes, 2010. "Responsibility, scale and the valuation of rail safety," Journal of Risk and Uncertainty, Springer, vol. 40(1), pages 85-108, February.
    9. H. Spencer Banzhaf & Dallas Burtraw & David Evans & Alan Krupnick, 2006. "Valuation of Natural Resource Improvements in the Adirondacks," Land Economics, University of Wisconsin Press, vol. 82(3), pages 445-464.
    10. Nielsen, Jytte Seested, 2011. "Use of the Internet for willingness-to-pay surveys: A comparison of face-to-face and web-based interviews," Resource and Energy Economics, Elsevier, vol. 33(1), pages 119-129, January.
    11. Li, Hui & Jenkins-Smith, Hank C. & Silva, Carol L. & Berrens, Robert P. & Herron, Kerry G., 2009. "Public support for reducing US reliance on fossil fuels: Investigating household willingness-to-pay for energy research and development," Ecological Economics, Elsevier, vol. 68(3), pages 731-742, January.
    12. Vossler, Christian A. & Evans, Mary F., 2009. "Bridging the gap between the field and the lab: Environmental goods, policy maker input, and consequentiality," Journal of Environmental Economics and Management, Elsevier, vol. 58(3), pages 338-345, November.
    13. Søren Olsen, 2009. "Choosing Between Internet and Mail Survey Modes for Choice Experiment Surveys Considering Non-Market Goods," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 44(4), pages 591-610, December.
    14. Berrens, Robert P. & Bohara, Alok K. & Jenkins-Smith, Hank & Silva, Carol & Weimer, David L., 2003. "The Advent of Internet Surveys for Political Research: A Comparison of Telephone and Internet Samples," Political Analysis, Cambridge University Press, vol. 11(1), pages 1-22, January.
    15. Gregory L. Poe & Kelly L. Giraud & John B. Loomis, 2005. "Computational Methods for Measuring the Difference of Empirical Distributions," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 87(2), pages 353-365.
    16. van der Heide, C. Martijn & van den Bergh, Jeroen C.J.M. & van Ierland, Ekko C. & Nunes, Paulo A.L.D., 2008. "Economic valuation of habitat defragmentation: A study of the Veluwe, the Netherlands," Ecological Economics, Elsevier, vol. 67(2), pages 205-216, September.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Kiriaki M. Keramitsoglou & Katja Lozar Manfreda & Charalampia Anastasiou & Knut Kalgraff Skjak & Konstantinos P. Tsagarakis, 2018. "Mode comparison study on willingness to buy and willingness to pay for organic foods: paper-and-pencil versus computerized questionnaire," Electronic Commerce Research, Springer, vol. 18(3), pages 587-603, September.
    2. Bhattacharyya, Aditi & Kutlu, Levent & Sickles, Robin C., 2018. "Pricing Inputs and Outputs: Market prices versus shadow prices, market power, and welfare analysis," Working Papers 18-009, Rice University, Department of Economics.
    3. Huynh, Elisabeth & Coast, Joanna & Rose, John & Kinghorn, Philip & Flynn, Terry, 2017. "Values for the ICECAP-Supportive Care Measure (ICECAP-SCM) for use in economic evaluation at end of life," Social Science & Medicine, Elsevier, vol. 189(C), pages 114-128.
    4. Skeie, Magnus Aa. & Lindhjem, Henrik & Skjeflo, Sofie & Navrud, Ståle, 2019. "Smartphone and tablet effects in contingent valuation web surveys – No reason to worry?," Ecological Economics, Elsevier, vol. 165(C), pages 1-1.
    5. Yukichika Kawata & Masahide Watanabe, 2018. "Economic feasibility of Campylobacter†reduced chicken: Do consumers have high willingness to pay?," Agribusiness, John Wiley & Sons, Ltd., vol. 34(2), pages 222-239, March.
    6. Jones, Benjamin A. & Berrens, Robert P. & Jenkins-Smith, Hank & Silva, Carol & Ripberger, Joe & Carlson, Deven & Gupta, Kuhika & Wehde, Wesley, 2018. "In search of an inclusive approach: Measuring non-market values for the effects of complex dam, hydroelectric and river system operations," Energy Economics, Elsevier, vol. 69(C), pages 225-236.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lindhjem, Henrik & Navrud, Ståle, 2010. "Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?," MPRA Paper 24069, University Library of Munich, Germany.
    2. Menegaki, Angeliki, N. & Olsen, Søren Bøye & Tsagarakis, Konstantinos P., 2016. "Towards a common standard – A reporting checklist for web-based stated preference valuation surveys and a critique for mode surveys," Journal of choice modelling, Elsevier, vol. 18(C), pages 18-50.
    3. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    4. Zhifeng Gao & Lisa A. House & Jing Xie, 2016. "Online Survey Data Quality and Its Implication for Willingness-to-Pay: A Cross-Country Comparison," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(2), pages 199-221, June.
    5. Murakami, Kayo & Ida, Takanori & Tanaka, Makoto & Friedman, Lee, 2015. "Consumers' willingness to pay for renewable and nuclear energy: A comparative analysis between the US and Japan," Energy Economics, Elsevier, vol. 50(C), pages 178-189.
    6. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    7. Liebe, Ulf & Glenk, Klaus & Oehlmann, Malte & Meyerhoff, Jürgen, 2015. "Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?," Journal of choice modelling, Elsevier, vol. 14(C), pages 17-31.
    8. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    9. Chèze, Benoît & David, Maia & Martinet, Vincent, 2020. "Understanding farmers' reluctance to reduce pesticide use: A choice experiment," Ecological Economics, Elsevier, vol. 167(C).
    10. Gillespie Rob & Kragt Marit E., 2012. "Accounting for Nonmarket Impacts in a Benefit-Cost Analysis of Underground Coal Mining in New South Wales, Australia," Journal of Benefit-Cost Analysis, De Gruyter, vol. 3(2), pages 1-29, May.
    11. Guimarães, Maria Helena & Nunes, Luís Catela & Madureira, Lívia & Santos, José Lima & Boski, Tomasz & Dentinho, Tomaz, 2015. "Measuring birdwatchers preferences: A case for using online networks and mixed-mode surveys," Tourism Management, Elsevier, vol. 46(C), pages 102-113.
    12. Punel, Aymeric & Stathopoulos, Amanda, 2017. "Modeling the acceptability of crowdsourced goods deliveries: Role of context and experience effects," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 105(C), pages 18-38.
    13. Börger, Tobias & Hattam, Caroline & Burdon, Daryl & Atkins, Jonathan P. & Austen, Melanie C., 2014. "Valuing conservation benefits of an offshore marine protected area," Ecological Economics, Elsevier, vol. 108(C), pages 229-241.
    14. Sandorf, Erlend Dancke & Aanesen, Margrethe & Navrud, Ståle, 2016. "Valuing unfamiliar and complex environmental goods: A comparison of valuation workshops and internet panel surveys with videos," Ecological Economics, Elsevier, vol. 129(C), pages 50-61.
    15. Robert J. Johnston & Kevin J. Boyle & Wiktor (Vic) Adamowicz & Jeff Bennett & Roy Brouwer & Trudy Ann Cameron & W. Michael Hanemann & Nick Hanley & Mandy Ryan & Riccardo Scarpa & Roger Tourangeau & Ch, 2017. "Contemporary Guidance for Stated Preference Studies," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 4(2), pages 319-405.
    16. Chavez, Daniel E. & Palma, Marco A. & Nayga, Rodolfo M. & Mjelde, James W., 2020. "Product availability in discrete choice experiments with private goods," Journal of choice modelling, Elsevier, vol. 36(C).
    17. Dugstad, Anders & Grimsrud, Kristine & Kipperberg, Gorm & Lindhjem, Henrik & Navrud, Ståle, 2020. "Acceptance of wind power development and exposure – Not-in-anybody's-backyard," Energy Policy, Elsevier, vol. 147(C).
    18. Nguyen, Thanh Cong & Le, Hoa Thu & Nguyen, Hang Dieu & Ngo, Mai Thanh & Nguyen, Hong Quang, 2021. "Examining ordering effects and strategic behaviour in a discrete choice experiment," Economic Analysis and Policy, Elsevier, vol. 70(C), pages 394-413.
    19. Mozumder, Pallab & Vásquez, William F. & Marathe, Achla, 2011. "Consumers' preference for renewable energy in the southwest USA," Energy Economics, Elsevier, vol. 33(6), pages 1119-1126.
    20. Kiriaki M. Keramitsoglou & Katja Lozar Manfreda & Charalampia Anastasiou & Knut Kalgraff Skjak & Konstantinos P. Tsagarakis, 2018. "Mode comparison study on willingness to buy and willingness to pay for organic foods: paper-and-pencil versus computerized questionnaire," Electronic Commerce Research, Springer, vol. 18(3), pages 587-603, September.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:enreec:v:64:y:2016:i:3:d:10.1007_s10640-015-9876-2. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.