IDEAS home Printed from https://ideas.repec.org/p/fem/femwpa/2016.61.html
   My bibliography  Save this paper

Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in their Judgments about Future Energy Technologies

Author

Listed:
  • Gregory F. Nemet

    (University of Wisconsin-Madison)

  • Laura Diaz Anadon

    (John F. Kennedy School of Government, Harvard University)

  • Elena Verdolini

    (Fondazione Eni Enrico Mattei and Centro Euro-Mediterraneo per i Cambiamenti Climatici)

Abstract

Expert elicitations are frequently used to characterize future technology outcomes. However their usefulness is limited, in part because: estimates across studies are not easily comparable; choices in survey design and expert selection may bias results; and over-confidence is a persistent problem. We provide quantitative evidence of how these choices affect experts’ estimates of the costs of future energy technologies. We harmonize data from 19 elicitations, involving 215 experts, on the 2030 costs of 5 energy technologies: nuclear, biofuels, bioelectricity, solar, and carbon capture. We control for expert characteristics, survey design, and public R&D investment levels on which the elicited values are conditional. We find that, on average, when experts respond to elicitations in person, they ascribe lower confidence (larger uncertainty) to their estimates than when responding via mail or online. In-person interviews also produce more optimistic assessments of best-case (10th percentile) outcomes. The impacts of expert affiliation—government, private sector, or academic—and geography—US or EU—are also significant; academics and US experts have lower confidence than other types of experts. Higher R&D investment levels have no effect on the confidence of experts’ judgments. R&D reduces both the median and breakthrough (10th percentile) cost estimates, although the size of the effect varies across technologies. These results indicate the source, direction, and size of bias in energy technology elicitations. They also point to the technology specificity of some of the effects. These biases should be seriously considered, both in interpreting the results of existing elicitations and in designing new ones.

Suggested Citation

  • Gregory F. Nemet & Laura Diaz Anadon & Elena Verdolini, 2016. "Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in their Judgments about Future Energy Technologies," Working Papers 2016.61, Fondazione Eni Enrico Mattei.
  • Handle: RePEc:fem:femwpa:2016.61
    as

    Download full text from publisher

    File URL: https://www.feem.it/m/publications_pages/NDL2016-061.pdf
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Luca Zamparini & Aura Reggiani, 2007. "Meta-Analysis and the Value of Travel Time Savings: A Transatlantic Perspective in Passenger Transport," Networks and Spatial Economics, Springer, vol. 7(4), pages 377-396, December.
    2. Terry Barker and Katie Jenkins, 2007. "The Costs of Avoiding Dangerous Climate Change: Estimates Derived from a Meta-Analysis of the Literature," Human Development Occasional Papers (1992-2007) HDOCPA-2007-02, Human Development Report Office (HDRO), United Nations Development Programme (UNDP).
    3. Bosetti, Valentina & Catenacci, Michela & Fiorese, Giulia & Verdolini, Elena, 2012. "The future prospect of PV and CSP solar technologies: An expert elicitation survey," Energy Policy, Elsevier, vol. 49(C), pages 308-317.
    4. Havranek, Tomas & Irsova, Zuzana & Janda, Karel, 2012. "Demand for gasoline is more price-inelastic than commonly thought," Energy Economics, Elsevier, vol. 34(1), pages 201-207.
    5. Adam Rose & Noah Dormady, 2011. "A Meta-Analysis of the Economic Impacts of Climate Change Policy in the United States," The Energy Journal, International Association for Energy Economics, vol. 0(Number 2), pages 143-166.
    6. Havranek, Tomas & Irsova, Zuzana & Janda, Karel, 2012. "Demand for gasoline is more price-inelastic than commonly thought," Energy Economics, Elsevier, vol. 34(1), pages 201-207.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giacomo Marangoni & Gauthier De Maere & Valentina Bosetti, 2017. "Optimal Clean Energy R&D Investments Under Uncertainty," MITP: Mitigation, Innovation and Transformation Pathways 256056, Fondazione Eni Enrico Mattei (FEEM).
    2. Elena Verdolini & Laura Díaz Anadón & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2018. "Future Prospects for Energy Technologies: Insights from Expert Elicitations," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 133-153.
    3. Laura Diaz Anadon & Erin Baker & Valentina Bosetti & Lara Aleluia Reis, 2016. "Expert views - and disagreements - about the potential of energy technology R&D," Climatic Change, Springer, vol. 136(3), pages 677-691, June.

    More about this item

    Keywords

    Expert Elicitations; Uncertainty; Energy Technologies; Heuristic Biases; Survey Design;

    JEL classification:

    • O13 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Agriculture; Natural Resources; Environment; Other Primary Products
    • O14 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Industrialization; Manufacturing and Service Industries; Choice of Technology
    • Q4 - Agricultural and Natural Resource Economics; Environmental and Ecological Economics - - Energy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:fem:femwpa:2016.61. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (barbara racah). General contact details of provider: http://edirc.repec.org/data/feemmit.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.