IDEAS home Printed from https://ideas.repec.org/p/ags/feemcl/163585.html
   My bibliography  Save this paper

Facing the Experts: Survey Mode and Expert Elicitation

Author

Listed:
  • Baker, Erin
  • Bosetti, Valentina
  • Jenni, Karen E.
  • Ricci, Elena Claire

Abstract

In this paper we compare the results of two different expert elicitation methods: in-person interviews and a self-administered web-based survey. Traditional expert elicitation has been done face to face, with an elicitor meeting with an expert for a few hours to several days, depending on the complexity of the analysis. Recently, however, some groups have been using other methods to solicit expert judgments, including self-administered surveys (written, emailed, and web-based), and the use of interactive web tools to facilitate interactions during an elicitation. These elicitations require fewer resources from the assessment team than in-person interviews, and often allow participating experts to provide input on their own schedules, perhaps with additional time to think about their responses. Thus they open up the possibility of using expert elicitation to obtain inputs relevant to a broader set of decisions. To our knowledge, these newer survey-based methods have not been rigorously evaluated for efficacy. We find, much like the results in the literature on different survey modes, different results from two different modes we examined, but no clear indication of which method might be preferred. We suggest future work including some controlled, lab-based experiments and real EEs well designed to avoid sample selection biases and specifically targeted to capture survey mode effects. Such studies would help us determine whether and when the different survey modes are most effective.

Suggested Citation

  • Baker, Erin & Bosetti, Valentina & Jenni, Karen E. & Ricci, Elena Claire, 2014. "Facing the Experts: Survey Mode and Expert Elicitation," Climate Change and Sustainable Development 163585, Fondazione Eni Enrico Mattei (FEEM).
  • Handle: RePEc:ags:feemcl:163585
    DOI: 10.22004/ag.econ.163585
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/163585/files/NDL2014-001R.pdf
    Download Restriction: no

    File URL: https://libkey.io/10.22004/ag.econ.163585?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Judith Covey & Angela Robinson & Michael Jones-Lee & Graham Loomes, 2010. "Responsibility, scale and the valuation of rail safety," Journal of Risk and Uncertainty, Springer, vol. 40(1), pages 85-108, February.
    2. Baker, Erin & Chon, Haewon & Keisler, Jeffrey, 2009. "Advanced solar R&D: Combining economic analysis with expert elicitations to inform climate policy," Energy Economics, Elsevier, vol. 31(Supplemen), pages 37-49.
    3. Nielsen, Jytte Seested, 2011. "Use of the Internet for willingness-to-pay surveys: A comparison of face-to-face and web-based interviews," Resource and Energy Economics, Elsevier, vol. 33(1), pages 119-129, January.
    4. Fiorese, Giulia & Catenacci, Michela & Verdolini, Elena & Bosetti, Valentina, 2013. "Advanced biofuels: Future perspectives from an expert elicitation survey," Energy Policy, Elsevier, vol. 56(C), pages 293-311.
    5. Bosetti, Valentina & Catenacci, Michela & Fiorese, Giulia & Verdolini, Elena, 2012. "The future prospect of PV and CSP solar technologies: An expert elicitation survey," Energy Policy, Elsevier, vol. 49(C), pages 308-317.
    6. Lindhjem, Henrik & Navrud, Ståle, 2011. "Are Internet surveys an alternative to face-to-face interviews in contingent valuation?," Ecological Economics, Elsevier, vol. 70(9), pages 1628-1637, July.
    7. Nemet, Gregory F. & Baker, Erin & Jenni, Karen E., 2013. "Modeling the future costs of carbon capture using experts' elicited probabilities under policy scenarios," Energy, Elsevier, vol. 56(C), pages 218-228.
    8. Marta-Pedroso, Cristina & Freitas, Helena & Domingos, Tiago, 2007. "Testing for the survey mode effect on contingent valuation data quality: A case study of web based versus in-person interviews," Ecological Economics, Elsevier, vol. 62(3-4), pages 388-398, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. World Health Organization, Foodborne Epidemiology Reference Group, Source Attribution Task Force, 2016. "Research Synthesis Methods in an Age of Globalized Risks: Lessons from the Global Burden of Foodborne Disease Expert Elicitation," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 191-202, February.
    2. Elena Verdolini & Laura Díaz Anadón & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2018. "Future Prospects for Energy Technologies: Insights from Expert Elicitations," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 133-153.
    3. Martínez-Cruz, Adán L. & Juárez-Torres, Miriam & Guerrero, Santiago, 2017. "Assessing Impacts From Climate Change on Local Social-ecological Systems in Contexts Where Information is Lacking: An Expert Elicitation in the Bolivian Altiplano," Ecological Economics, Elsevier, vol. 137(C), pages 70-82.
    4. Mazzanti, Massimiliano & Rizzo, Ugo, 2017. "Diversely moving towards a green economy: Techno-organisational decarbonisation trajectories and environmental policy in EU sectors," Technological Forecasting and Social Change, Elsevier, vol. 115(C), pages 111-116.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zawojska, Ewa & Czajkowski, Mikotaj, 2017. "Are preferences stated in web vs. personal interviews different? A comparison of willingness to pay results for a large multi-country study of the Baltic Sea eutrophication reduction," Annual Meeting, 2017, June 18-21, Montreal, Canada 258604, Canadian Agricultural Economics Society.
    2. Baker, Erin & Bosetti, Valentina & Salo, Ahti, 2016. "Finding Common Ground when Experts Disagree: Belief Dominance over Portfolios of Alternatives," MITP: Mitigation, Innovation and Transformation Pathways 243147, Fondazione Eni Enrico Mattei (FEEM).
    3. Kevin J. Boyle & Mark Morrison & Darla Hatton MacDonald & Roderick Duncan & John Rose, 2016. "Investigating Internet and Mail Implementation of Stated-Preference Surveys While Controlling for Differences in Sample Frames," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 64(3), pages 401-419, July.
    4. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    5. Laura Diaz Anadon & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2016. "Expert views - and disagreements - about the potential of energy technology R&D," Climatic Change, Springer, vol. 136(3), pages 677-691, June.
    6. Erin Baker & Olaitan Olaleye & Lara Aleluia Reis, 2015. "Decision Frameworks and the Investment in R&D," Working Papers 2015.42, Fondazione Eni Enrico Mattei.
    7. Guimarães, Maria Helena & Nunes, Luís Catela & Madureira, Lívia & Santos, José Lima & Boski, Tomasz & Dentinho, Tomaz, 2015. "Measuring birdwatchers preferences: A case for using online networks and mixed-mode surveys," Tourism Management, Elsevier, vol. 46(C), pages 102-113.
    8. Liebe, Ulf & Glenk, Klaus & Oehlmann, Malte & Meyerhoff, Jürgen, 2015. "Does the use of mobile devices (tablets and smartphones) affect survey quality and choice behaviour in web surveys?," Journal of choice modelling, Elsevier, vol. 14(C), pages 17-31.
    9. Fiorese, Giulia & Catenacci, Michela & Bosetti, Valentina & Verdolini, Elena, 2014. "The power of biomass: Experts disclose the potential for success of bioenergy technologies," Energy Policy, Elsevier, vol. 65(C), pages 94-114.
    10. Laura Diaz Anadon & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2016. "Too Early to Pick Winners: Disagreement across Experts Implies the Need to Diversify R&D Investment," Working Papers 2016.22, Fondazione Eni Enrico Mattei.
    11. Catenacci, Michela & Verdolini, Elena & Bosetti, Valentina & Fiorese, Giulia, 2013. "Going electric: Expert survey on the future of battery technologies for electric vehicles," Energy Policy, Elsevier, vol. 61(C), pages 403-413.
    12. Stergios Athanassoglou & Valentina Bosetti, 2015. "Setting Environmental Policy When Experts Disagree," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 61(4), pages 497-516, August.
    13. Lindhjem, Henrik & Navrud, Ståle, 2010. "Can cheap panel-based internet surveys substitute costly in-person interviews in CV surveys?," MPRA Paper 24069, University Library of Munich, Germany.
    14. Milford, James & Henrion, Max & Hunter, Chad & Newes, Emily & Hughes, Caroline & Baldwin, Samuel F., 2022. "Energy sector portfolio analysis with uncertainty," Applied Energy, Elsevier, vol. 306(PA).
    15. Zhifeng Gao & Lisa A. House & Jing Xie, 2016. "Online Survey Data Quality and Its Implication for Willingness-to-Pay: A Cross-Country Comparison," Canadian Journal of Agricultural Economics/Revue canadienne d'agroeconomie, Canadian Agricultural Economics Society/Societe canadienne d'agroeconomie, vol. 64(2), pages 199-221, June.
    16. Penn, Jerrod & Hu, Wuyang, 2016. "Making the Most of Cheap Talk in an Online Survey," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 236171, Agricultural and Applied Economics Association.
    17. Baker, Erin & Bosetti, Valentina & Anadon, Laura Diaz & Henrion, Max & Aleluia Reis, Lara, 2015. "Future costs of key low-carbon energy technologies: Harmonization and aggregation of energy technology expert elicitation data," Energy Policy, Elsevier, vol. 80(C), pages 219-232.
    18. Jed J. Cohen & Johannes Reichl, 2022. "Comparing Internet and phone survey mode effects across countries and research contexts," Australian Journal of Agricultural and Resource Economics, Australian Agricultural and Resource Economics Society, vol. 66(1), pages 44-71, January.
    19. Ricci, Elena Claire & Bosetti, Valentina & Baker, Erin & Jenni, Karen E., 2014. "From Expert Elicitations to Integrated Assessment: Future Prospects of Carbon Capture Technologies," Climate Change and Sustainable Development 172451, Fondazione Eni Enrico Mattei (FEEM).
    20. Gregory F. Nemet & Laura Diaz Anadon & Elena Verdolini, 2017. "Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in Their Judgments About Future Energy Technologies," Risk Analysis, John Wiley & Sons, vol. 37(2), pages 315-330, February.

    More about this item

    Keywords

    Environmental Economics and Policy;

    JEL classification:

    • D81 - Microeconomics - - Information, Knowledge, and Uncertainty - - - Criteria for Decision-Making under Risk and Uncertainty
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D
    • Q55 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Environmental Economics - - - Environmental Economics: Technological Innovation
    • Q40 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Energy - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:feemcl:163585. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/feemmit.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.