IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0022998.html
   My bibliography  Save this article

Expert Status and Performance

Author

Listed:
  • Mark A Burgman
  • Marissa McBride
  • Raquel Ashton
  • Andrew Speirs-Bridge
  • Louisa Flander
  • Bonnie Wintle
  • Fiona Fidler
  • Libby Rumpff
  • Charles Twardy

Abstract

Expert judgements are essential when time and resources are stretched or we face novel dilemmas requiring fast solutions. Good advice can save lives and large sums of money. Typically, experts are defined by their qualifications, track record and experience [1], [2]. The social expectation hypothesis argues that more highly regarded and more experienced experts will give better advice. We asked experts to predict how they will perform, and how their peers will perform, on sets of questions. The results indicate that the way experts regard each other is consistent, but unfortunately, ranks are a poor guide to actual performance. Expert advice will be more accurate if technical decisions routinely use broadly-defined expert groups, structured question protocols and feedback.

Suggested Citation

  • Mark A Burgman & Marissa McBride & Raquel Ashton & Andrew Speirs-Bridge & Louisa Flander & Bonnie Wintle & Fiona Fidler & Libby Rumpff & Charles Twardy, 2011. "Expert Status and Performance," PLOS ONE, Public Library of Science, vol. 6(7), pages 1-7, July.
    • Handle: RePEc:plo:pone00:0022998
    DOI: 10.1371/journal.pone.0022998
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0022998
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0022998&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0022998?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Willy Aspinall, 2010. "A route to more tractable expert advice," Nature, Nature, vol. 463(7279), pages 294-295, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    2. Miles Parker & Andrew Acland & Harry J Armstrong & Jim R Bellingham & Jessica Bland & Helen C Bodmer & Simon Burall & Sarah Castell & Jason Chilvers & David D Cleevely & David Cope & Lucia Costanzo & , 2014. "Identifying the Science and Technology Dimensions of Emerging Public Policy Issues through Horizon Scanning," PLOS ONE, Public Library of Science, vol. 9(5), pages 1-17, May.
    3. Yang Song & Yawen Wang & Dahai Jin, 2020. "A Bayesian Approach Based on Bayes Minimum Risk Decision for Reliability Assessment of Web Service Composition," Future Internet, MDPI, vol. 12(12), pages 1-20, December.
    4. Xu, Yingchun & Yao, Wen & Zheng, Xiaohu & Chen, Xiaoqian, 2020. "An iterative information integration method for multi-level system reliability analysis based on Bayesian Melding Method," Reliability Engineering and System Safety, Elsevier, vol. 204(C).
    5. A. Charisse Farr & Kerrie Mengersen & Fabrizio Ruggeri & Daniel Simpson & Paul Wu & Prasad Yarlagadda, 2020. "Combining Opinions for Use in Bayesian Networks: A Measurement Error Approach," International Statistical Review, International Statistical Institute, vol. 88(2), pages 335-353, August.
    6. Fergus Bolger & Gene Rowe, 2015. "The Aggregation of Expert Judgment: Do Good Things Come to Those Who Weight?," Risk Analysis, John Wiley & Sons, vol. 35(1), pages 5-11, January.
    7. repec:cup:judgdm:v:15:y:2020:i:5:p:783-797 is not listed on IDEAS
    8. Lyon, Aidan & Wintle, Bonnie C. & Burgman, Mark, 2015. "Collective wisdom: Methods of confidence interval aggregation," Journal of Business Research, Elsevier, vol. 68(8), pages 1759-1767.
    9. Kylie Davies & Max K. Bulsara & Anne‐Sylvie Ramelet & Leanne Monterosso, 2018. "Reliability and criterion‐related validity testing (construct) of the Endotracheal Suction Assessment Tool (ESAT©)," Journal of Clinical Nursing, John Wiley & Sons, vol. 27(9-10), pages 1891-1900, May.
    10. Warwick Smith & Anca M. Hanea & Mark A. Burgman, 2022. "Can Groups Improve Expert Economic and Financial Forecasts?," Forecasting, MDPI, vol. 4(3), pages 1-18, August.
    11. Bolger, Fergus & Rowe, Gene & Belton, Ian & Crawford, Megan M & Hamlin, Iain & Sissons, Aileen & Taylor Browne Lūka, Courtney & Vasilichi, Alexandrina & Wright, George, 2020. "The Simulated Group Response Paradigm: A new approach to the study of opinion change in Delphi and other structured-group techniques," OSF Preprints 4ufzg, Center for Open Science.
    12. Jean Hugé & Nibedita Mukherjee & Camille Fertel & Jean-Philippe Waaub & Thomas Block & Tom Waas & Nico Koedam & Farid Dahdouh-Guebas, 2015. "Conceptualizing the Effectiveness of Sustainability Assessment in Development Cooperation," Sustainability, MDPI, vol. 7(5), pages 1-17, May.
    13. Christopher W. Karvetski & Kenneth C. Olson & David R. Mandel & Charles R. Twardy, 2013. "Probabilistic Coherence Weighting for Optimizing Expert Forecasts," Decision Analysis, INFORMS, vol. 10(4), pages 305-326, December.
    14. Mauksch, Stefanie & von der Gracht, Heiko A. & Gordon, Theodore J., 2020. "Who is an expert for foresight? A review of identification methods," Technological Forecasting and Social Change, Elsevier, vol. 154(C).
    15. Anca M. Hanea & Marissa F. McBride & Mark A. Burgman & Bonnie C. Wintle, 2018. "The Value of Performance Weights and Discussion in Aggregated Expert Judgments," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1781-1794, September.
    16. Woike, Jan K. & Hoffrage, Ulrich & Petty, Jeffrey S., 2015. "Picking profitable investments: The success of equal weighting in simulated venture capitalist decision making," Journal of Business Research, Elsevier, vol. 68(8), pages 1705-1716.
    17. Abigail R Colson & Roger M Cooke, 2018. "Expert Elicitation: Using the Classical Model to Validate Experts’ Judgments," Review of Environmental Economics and Policy, Association of Environmental and Resource Economists, vol. 12(1), pages 113-132.
    18. David R. Mandel & Robert N. Collins & Evan F. Risko & Jonathan A. Fugelsang, 2020. "Effect of confidence interval construction on judgment accuracy," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 15(5), pages 783-797, September.
    19. Galparsoro, I. & Korta, M. & Subirana, I. & Borja, Á. & Menchaca, I. & Solaun, O. & Muxika, I. & Iglesias, G. & Bald, J., 2021. "A new framework and tool for ecological risk assessment of wave energy converters projects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 151(C).
    20. Anna Chrysafi & Vili Virkki & Mika Jalava & Vilma Sandström & Johannes Piipponen & Miina Porkka & Steven J. Lade & Kelsey Mere & Lan Wang-Erlandsson & Laura Scherer & Lauren S. Andersen & Elena Bennet, 2022. "Quantifying Earth system interactions for sustainable food production via expert elicitation," Nature Sustainability, Nature, vol. 5(10), pages 830-842, October.
    21. McKellar, Jennifer M. & Sleep, Sylvia & Bergerson, Joule A. & MacLean, Heather L., 2017. "Expectations and drivers of future greenhouse gas emissions from Canada's oil sands: An expert elicitation," Energy Policy, Elsevier, vol. 100(C), pages 162-169.
    22. Thomas Allen & Paolo Prosperi & Bruce Cogill & Martine Padilla & Iuri Peri, 2019. "A Delphi Approach to Develop Sustainable Food System Metrics," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(3), pages 1307-1339, February.
    23. James L Anderson & Christopher M Anderson & Jingjie Chu & Jennifer Meredith & Frank Asche & Gil Sylvia & Martin D Smith & Dessy Anggraeni & Robert Arthur & Atle Guttormsen & Jessica K McCluney & Tim W, 2015. "The Fishery Performance Indicators: A Management Tool for Triple Bottom Line Outcomes," PLOS ONE, Public Library of Science, vol. 10(5), pages 1-20, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Meissner, Philip & Brands, Christian & Wulf, Torsten, 2017. "Quantifiying blind spots and weak signals in executive judgment: A structured integration of expert judgment into the scenario development process," International Journal of Forecasting, Elsevier, vol. 33(1), pages 244-253.
    2. Stapleton, L.M. & Hanna, P. & Ravenscroft, N. & Church, A., 2014. "A flexible ecosystem services proto-typology based on public opinion," Ecological Economics, Elsevier, vol. 106(C), pages 83-90.
    3. Hanea, A.M. & McBride, M.F. & Burgman, M.A. & Wintle, B.C. & Fidler, F. & Flander, L. & Twardy, C.R. & Manning, B. & Mascaro, S., 2017. "I nvestigate D iscuss E stimate A ggregate for structured expert judgement," International Journal of Forecasting, Elsevier, vol. 33(1), pages 267-279.
    4. Gruetzemacher, Ross & Paradice, David & Lee, Kang Bok, 2020. "Forecasting extreme labor displacement: A survey of AI practitioners," Technological Forecasting and Social Change, Elsevier, vol. 161(C).
    5. Martine J. Barons & Lael E. Walsh & Edward E. Salakpi & Linda Nichols, 2024. "A Decision Support System for Sustainable Agriculture and Food Loss Reduction under Uncertain Agricultural Policy Frameworks," Agriculture, MDPI, vol. 14(3), pages 1-21, March.
    6. Colson, Abigail R. & Cooke, Roger M., 2017. "Cross validation for the classical model of structured expert judgment," Reliability Engineering and System Safety, Elsevier, vol. 163(C), pages 109-120.
    7. Kragt, M.E. & Pannell, D.J. & McVittie, A. & Stott, A.W. & Vosough Ahmadi, B. & Wilson, P., 2016. "Improving interdisciplinary collaboration in bio-economic modelling for agricultural systems," Agricultural Systems, Elsevier, vol. 143(C), pages 217-224.
    8. Anca M. Hanea & Marissa F. McBride & Mark A. Burgman & Bonnie C. Wintle, 2018. "The Value of Performance Weights and Discussion in Aggregated Expert Judgments," Risk Analysis, John Wiley & Sons, vol. 38(9), pages 1781-1794, September.
    9. Jeremy Rohmer & Eric Chojnacki, 2021. "Forecast of environment systems using expert judgements: performance comparison between the possibilistic and the classical model," Environment Systems and Decisions, Springer, vol. 41(1), pages 131-146, March.
    10. Bolger, Fergus & Wright, George, 2017. "Use of expert knowledge to anticipate the future: Issues, analysis and directions," International Journal of Forecasting, Elsevier, vol. 33(1), pages 230-243.
    11. Hanea, Anca & Wilkinson, David Peter & McBride, Marissa & Lyon, Aidan & van Ravenzwaaij, Don & Singleton Thorn, Felix & Gray, Charles T. & Mandel, David R. & Willcox, Aaron & Gould, Elliot, 2021. "Mathematically aggregating experts' predictions of possible futures," MetaArXiv rxmh7, Center for Open Science.
    12. Eric Libby & Leon Glass, 2010. "The Calculus of Committee Composition," PLOS ONE, Public Library of Science, vol. 5(9), pages 1-8, September.
    13. E. Schuur & B. Abbott & W. Bowden & V. Brovkin & P. Camill & J. Canadell & J. Chanton & F. Chapin & T. Christensen & P. Ciais & B. Crosby & C. Czimczik & G. Grosse & J. Harden & D. Hayes & G. Hugelius, 2013. "Expert assessment of vulnerability of permafrost carbon to climate change," Climatic Change, Springer, vol. 119(2), pages 359-374, July.
    14. Alvarado-Valencia, Jorge & Barrero, Lope H. & Önkal, Dilek & Dennerlein, Jack T., 2017. "Expertise, credibility of system forecasts and integration methods in judgmental demand forecasting," International Journal of Forecasting, Elsevier, vol. 33(1), pages 298-313.
    15. Flandoli, F. & Giorgi, E. & Aspinall, W.P. & Neri, A., 2011. "Comparison of a new expert elicitation model with the Classical Model, equal weights and single experts, using a cross-validation technique," Reliability Engineering and System Safety, Elsevier, vol. 96(10), pages 1292-1310.
    16. Rohit Aggarwal & Michael J Lee & Braxton Osting & Harpreet Singh, 2021. "Improving Funding Operations of Equity‐based Crowdfunding Platforms," Production and Operations Management, Production and Operations Management Society, vol. 30(11), pages 4121-4139, November.
    17. Döpke, Jörg & Müller, Karsten & Tegtmeier, Lars, 2018. "The economic value of business cycle forecasts for potential investors – Evidence from Germany," Research in International Business and Finance, Elsevier, vol. 46(C), pages 445-461.
    18. World Health Organization, Foodborne Epidemiology Reference Group, Source Attribution Task Force, 2016. "Research Synthesis Methods in an Age of Globalized Risks: Lessons from the Global Burden of Foodborne Disease Expert Elicitation," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 191-202, February.
    19. Julia R. Falconer & Eibe Frank & Devon L. L. Polaschek & Chaitanya Joshi, 2022. "Methods for Eliciting Informative Prior Distributions: A Critical Review," Decision Analysis, INFORMS, vol. 19(3), pages 189-204, September.
    20. Alexander M. R. Bakker & Domitille Louchard & Klaus Keller, 2017. "Sources and implications of deep uncertainties surrounding sea-level projections," Climatic Change, Springer, vol. 140(3), pages 339-347, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0022998. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.