IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v27y2007i1p225-240.html
   My bibliography  Save this article

Combining Experts' Risk Judgments on Technology Performance of Phytoremediation: Self‐Confidence Ratings, Averaging Procedures, and Formative Consensus Building

Author

Listed:
  • Roland W. Scholz
  • Ralf Hansmann

Abstract

Expert panels and averaging procedures are common means for coping with the uncertainty of effects of technology application in complex environments. We investigate the connection between confidence and the validity of expert judgment. Moreover, a formative consensus building procedure (FCB) is introduced that generates probability statements on the performance of technologies, and we compare different algorithms for the statistical aggregation of individual judgments. The case study refers to an expert panel of 10 environmental scientists assessing the performance of a soil cleanup technology that uses the capability of certain plants to accumulate heavy metals from the soil in the plant body (phytoremediation). The panel members first provided individual statements on the effectiveness of a phytoremediation. Such statements can support policymakers, answering the questions concerning the expected performance of the new technology in contaminated areas. The present study reviews (1) the steps of the FCB, (2) the constraints of technology application (contaminants, soil structure, etc.), (3) the measurement of expert knowledge, (4) the statistical averaging and the discursive agreement procedures, and (5) the boundaries of application for the FCB method. The quantitative statement oriented part of FCB generates terms such as: “The probability that the concentration of soil contamination will be reduced by at least 50% is 0.8.” The data suggest that taking the median of the individual expert estimates provides the most accurate aggregated estimate. The discursive agreement procedure of FCB appears suitable for deriving politically relevant singular statements rather than for obtaining comprehensive information about uncertainties as represented by probability distributions.

Suggested Citation

  • Roland W. Scholz & Ralf Hansmann, 2007. "Combining Experts' Risk Judgments on Technology Performance of Phytoremediation: Self‐Confidence Ratings, Averaging Procedures, and Formative Consensus Building," Risk Analysis, John Wiley & Sons, vol. 27(1), pages 225-240, February.
    • Handle: RePEc:wly:riskan:v:27:y:2007:i:1:p:225-240
    DOI: 10.1111/j.1539-6924.2006.00871.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.2006.00871.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1539-6924.2006.00871.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Laughlin, Patrick R. & Bonner, Bryan L. & Miner, Andrew G. & Carnevale, Peter J., 1999. "Frames of Reference in Quantity Estimations by Groups and Individuals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 80(2), pages 103-117, November.
    2. Hurley, W. J. & Lior, D. U., 2002. "Combining expert judgment: On the performance of trimmed mean vote aggregation procedures in the presence of strategic voting," European Journal of Operational Research, Elsevier, vol. 140(1), pages 142-147, July.
    3. Dennis P. Slevin & Larry W. Boone & Eileen M. Russo & Richard S. Allen, 1998. "CONFIDE: A Collective Decision-Making Procedure Using Confidence Estimates of Individual Judgements," Group Decision and Negotiation, Springer, vol. 7(2), pages 179-194, March.
    4. Arno Willems & Mart Janssen ** & Chris Verstegen ** & Tim Bedford, 2005. "Expert quantification of uncertainties in a risk analysis for an infrastructure project," Journal of Risk Research, Taylor & Francis Journals, vol. 8(1), pages 3-17, January.
    5. Crott, Helmut W. & Szilvas, Klaus & Zuber, Johannes A., 1991. "Group decision, choice shift, and polarization in consulting, political, and local political scenarios: An experimental investigation and theoretical analysis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 49(1), pages 22-41, June.
    6. Harry Otway & Detlof von Winterfeldt, 1992. "Expert Judgment in Risk Analysis and Management: Process, Context, and Pitfalls," Risk Analysis, John Wiley & Sons, vol. 12(1), pages 83-93, March.
    7. Sniezek, Janet A. & Henry, Rebecca A., 1989. "Accuracy and confidence in group judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(1), pages 1-28, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Weiping Tan & Jose Ramirez‐Marquez & Brian Sauser, 2011. "A probabilistic approach to system maturity assessment," Systems Engineering, John Wiley & Sons, vol. 14(3), pages 279-293, September.
    2. Scholz, Roland W. & Czichos, Reiner & Parycek, Peter & Lampoltshammer, Thomas J., 2020. "Organizational vulnerability of digital threats: A first validation of an assessment method," European Journal of Operational Research, Elsevier, vol. 282(2), pages 627-643.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bonner, Bryan L. & Sillito, Sheli D. & Baumann, Michael R., 2007. "Collective estimation: Accuracy, expertise, and extroversion as sources of intra-group influence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 121-133, May.
    2. Bonner, Bryan L. & Bolinger, Alexander R., 2013. "Separating the confident from the correct: Leveraging member knowledge in groups to improve decision making and performance," Organizational Behavior and Human Decision Processes, Elsevier, vol. 122(2), pages 214-221.
    3. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40, January.
    4. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40.
    5. David V. Budescu & Hsiu-Ting Yu, 2006. "To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation," Decision Analysis, INFORMS, vol. 3(3), pages 145-162, September.
    6. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    7. Schultze, Thomas & Mojzisch, Andreas & Schulz-Hardt, Stefan, 2012. "Why groups perform better than individuals at quantitative judgment tasks: Group-to-individual transfer as an alternative to differential weighting," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 24-36.
    8. Alexandra Gheondea-Eladi, 2016. "The Evolution of Certainty in a Small Decision-Making Group by Consensus," Group Decision and Negotiation, Springer, vol. 25(1), pages 127-155, January.
    9. Akfırat, Serap & Bayrak, Fatih & Üzümçeker, Emir & Ergiyen, Tolga & Yurtbakan, Taylan & Uysal, Mete Sefa, 2023. "The roles of social norms and leadership in health communication in the context of COVID-19," Social Science & Medicine, Elsevier, vol. 323(C).
    10. Koriat, Asher, 2008. "Alleviating inflation of conditional predictions," Organizational Behavior and Human Decision Processes, Elsevier, vol. 106(1), pages 61-76, May.
    11. Demont, Matty & Rutsaert, Pieter & Ndour, Maimouna & Verbeke, Wim & Seck, Papa Abdoulaye & Tollens, Eric, 2012. "Experimental auctions, collective induction and choice shift: Willingness-to-pay for rice quality in Senegal," 2012 Conference, August 18-24, 2012, Foz do Iguacu, Brazil 126861, International Association of Agricultural Economists.
    12. Keller, Jonas & von der Gracht, Heiko A., 2014. "The influence of information and communication technology (ICT) on future foresight processes — Results from a Delphi survey," Technological Forecasting and Social Change, Elsevier, vol. 85(C), pages 81-92.
    13. Patrick Afflerbach & Christopher Dun & Henner Gimpel & Dominik Parak & Johannes Seyfried, 2021. "A Simulation-Based Approach to Understanding the Wisdom of Crowds Phenomenon in Aggregating Expert Judgment," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(4), pages 329-348, August.
    14. Julia A. Minson & Jennifer S. Mueller & Richard P. Larrick, 2018. "The Contingent Wisdom of Dyads: When Discussion Enhances vs. Undermines the Accuracy of Collaborative Judgments," Management Science, INFORMS, vol. 64(9), pages 4177-4192, September.
    15. Nguyen, Son & Chen, Peggy Shu-Ling & Du, Yuquan & Shi, Wenming, 2019. "A quantitative risk analysis model with integrated deliberative Delphi platform for container shipping operational risks," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 129(C), pages 203-227.
    16. Ine H. J. Van Der Fels‐Klerx & Louis H. J. Goossens & Helmut W. Saatkamp & Suzan H. S. Horst, 2002. "Elicitation of Quantitative Data from a Heterogeneous Expert Panel: Formal Process and Application in Animal Health," Risk Analysis, John Wiley & Sons, vol. 22(1), pages 67-81, February.
    17. Lukas Meub & Till Proeger, 2018. "Are groups ‘less behavioral’? The case of anchoring," Theory and Decision, Springer, vol. 85(2), pages 117-150, August.
    18. Aimée A. Kane, 2010. "Unlocking Knowledge Transfer Potential: Knowledge Demonstrability and Superordinate Social Identity," Organization Science, INFORMS, vol. 21(3), pages 643-660, June.
    19. Laurence Aitchison & Dan Bang & Bahador Bahrami & Peter E Latham, 2015. "Doubly Bayesian Analysis of Confidence in Perceptual Decision-Making," PLOS Computational Biology, Public Library of Science, vol. 11(10), pages 1-23, October.
    20. Benbasat, Izak & Lim, John, 2000. "Information Technology Support For Debiasing Group Judgments: An Empirical Evaluation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(1), pages 167-183, September.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:27:y:2007:i:1:p:225-240. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.