IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v27y2007i1p225-240.html

Combining Experts' Risk Judgments on Technology Performance of Phytoremediation: Self‐Confidence Ratings, Averaging Procedures, and Formative Consensus Building

Author

Listed:
  • Roland W. Scholz
  • Ralf Hansmann

Abstract

Expert panels and averaging procedures are common means for coping with the uncertainty of effects of technology application in complex environments. We investigate the connection between confidence and the validity of expert judgment. Moreover, a formative consensus building procedure (FCB) is introduced that generates probability statements on the performance of technologies, and we compare different algorithms for the statistical aggregation of individual judgments. The case study refers to an expert panel of 10 environmental scientists assessing the performance of a soil cleanup technology that uses the capability of certain plants to accumulate heavy metals from the soil in the plant body (phytoremediation). The panel members first provided individual statements on the effectiveness of a phytoremediation. Such statements can support policymakers, answering the questions concerning the expected performance of the new technology in contaminated areas. The present study reviews (1) the steps of the FCB, (2) the constraints of technology application (contaminants, soil structure, etc.), (3) the measurement of expert knowledge, (4) the statistical averaging and the discursive agreement procedures, and (5) the boundaries of application for the FCB method. The quantitative statement oriented part of FCB generates terms such as: “The probability that the concentration of soil contamination will be reduced by at least 50% is 0.8.” The data suggest that taking the median of the individual expert estimates provides the most accurate aggregated estimate. The discursive agreement procedure of FCB appears suitable for deriving politically relevant singular statements rather than for obtaining comprehensive information about uncertainties as represented by probability distributions.

Suggested Citation

  • Roland W. Scholz & Ralf Hansmann, 2007. "Combining Experts' Risk Judgments on Technology Performance of Phytoremediation: Self‐Confidence Ratings, Averaging Procedures, and Formative Consensus Building," Risk Analysis, John Wiley & Sons, vol. 27(1), pages 225-240, February.
    • Handle: RePEc:wly:riskan:v:27:y:2007:i:1:p:225-240
    DOI: 10.1111/j.1539-6924.2006.00871.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.2006.00871.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1539-6924.2006.00871.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Laughlin, Patrick R. & Bonner, Bryan L. & Miner, Andrew G. & Carnevale, Peter J., 1999. "Frames of Reference in Quantity Estimations by Groups and Individuals," Organizational Behavior and Human Decision Processes, Elsevier, vol. 80(2), pages 103-117, November.
    2. Dennis P. Slevin & Larry W. Boone & Eileen M. Russo & Richard S. Allen, 1998. "CONFIDE: A Collective Decision-Making Procedure Using Confidence Estimates of Individual Judgements," Group Decision and Negotiation, Springer, vol. 7(2), pages 179-194, March.
    3. Kara M. Morgan & Michael L. DeKay & Paul S. Fischbeck & M. Granger Morgan & Baruch Fischhoff & H. Keith Florig, 2001. "A Deliberative Method for Ranking Risks (II): Evaluation of Validity and Agreement among Risk Managers," Risk Analysis, John Wiley & Sons, vol. 21(5), pages 923-923, October.
    4. Crott, Helmut W. & Szilvas, Klaus & Zuber, Johannes A., 1991. "Group decision, choice shift, and polarization in consulting, political, and local political scenarios: An experimental investigation and theoretical analysis," Organizational Behavior and Human Decision Processes, Elsevier, vol. 49(1), pages 22-41, June.
    5. Harry Otway & Detlof von Winterfeldt, 1992. "Expert Judgment in Risk Analysis and Management: Process, Context, and Pitfalls," Risk Analysis, John Wiley & Sons, vol. 12(1), pages 83-93, March.
    6. Hurley, W. J. & Lior, D. U., 2002. "Combining expert judgment: On the performance of trimmed mean vote aggregation procedures in the presence of strategic voting," European Journal of Operational Research, Elsevier, vol. 140(1), pages 142-147, July.
    7. Arno Willems & Mart Janssen ** & Chris Verstegen ** & Tim Bedford, 2005. "Expert quantification of uncertainties in a risk analysis for an infrastructure project," Journal of Risk Research, Taylor & Francis Journals, vol. 8(1), pages 3-17, January.
    8. Sniezek, Janet A. & Henry, Rebecca A., 1989. "Accuracy and confidence in group judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 43(1), pages 1-28, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Scholz, Roland W. & Czichos, Reiner & Parycek, Peter & Lampoltshammer, Thomas J., 2020. "Organizational vulnerability of digital threats: A first validation of an assessment method," European Journal of Operational Research, Elsevier, vol. 282(2), pages 627-643.
    2. Weiping Tan & Jose Ramirez‐Marquez & Brian Sauser, 2011. "A probabilistic approach to system maturity assessment," Systems Engineering, John Wiley & Sons, vol. 14(3), pages 279-293, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bonner, Bryan L. & Sillito, Sheli D. & Baumann, Michael R., 2007. "Collective estimation: Accuracy, expertise, and extroversion as sources of intra-group influence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 121-133, May.
    2. Bonner, Bryan L. & Bolinger, Alexander R., 2013. "Separating the confident from the correct: Leveraging member knowledge in groups to improve decision making and performance," Organizational Behavior and Human Decision Processes, Elsevier, vol. 122(2), pages 214-221.
    3. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40, January.
    4. Kerr, Norbert L. & Tindale, R. Scott, 2011. "Group-based forecasting?: A social psychological analysis," International Journal of Forecasting, Elsevier, vol. 27(1), pages 14-40.
    5. David V. Budescu & Hsiu-Ting Yu, 2006. "To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation," Decision Analysis, INFORMS, vol. 3(3), pages 145-162, September.
    6. Robert T. Clemen & Robert L. Winkler, 1999. "Combining Probability Distributions From Experts in Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 19(2), pages 187-203, April.
    7. Schultze, Thomas & Mojzisch, Andreas & Schulz-Hardt, Stefan, 2012. "Why groups perform better than individuals at quantitative judgment tasks: Group-to-individual transfer as an alternative to differential weighting," Organizational Behavior and Human Decision Processes, Elsevier, vol. 118(1), pages 24-36.
    8. Nicolás C. Bronfman & Luis Abdón Cifuentes & Michael L. deKay & Henry H. Willis, 2007. "Accounting for Variation in the Explanatory Power of the Psychometric Paradigm: The Effects of Aggregation and Focus," Journal of Risk Research, Taylor & Francis Journals, vol. 10(4), pages 527-554, June.
    9. Alexandra Gheondea-Eladi, 2016. "The Evolution of Certainty in a Small Decision-Making Group by Consensus," Group Decision and Negotiation, Springer, vol. 25(1), pages 127-155, January.
    10. Akfırat, Serap & Bayrak, Fatih & Üzümçeker, Emir & Ergiyen, Tolga & Yurtbakan, Taylan & Uysal, Mete Sefa, 2023. "The roles of social norms and leadership in health communication in the context of COVID-19," Social Science & Medicine, Elsevier, vol. 323(C).
    11. Demont, Matty & Rutsaert, Pieter & Ndour, Maimouna & Verbeke, Wim & Seck, Papa Abdoulaye & Tollens, Eric, 2012. "Experimental auctions, collective induction and choice shift: Willingness-to-pay for rice quality in Senegal," 2012 Conference, August 18-24, 2012, Foz do Iguacu, Brazil 126861, International Association of Agricultural Economists.
    12. Henry H. Willis & Michael L. DeKay & Baruch Fischhoff & M. Granger Morgan, 2005. "Aggregate, Disaggregate, and Hybrid Analyses of Ecological Risk Perceptions," Risk Analysis, John Wiley & Sons, vol. 25(2), pages 405-428, April.
    13. Julia A. Minson & Jennifer S. Mueller & Richard P. Larrick, 2018. "The Contingent Wisdom of Dyads: When Discussion Enhances vs. Undermines the Accuracy of Collaborative Judgments," Management Science, INFORMS, vol. 64(9), pages 4177-4192, September.
    14. Hua Li & George E. Apostolakis & Joseph Gifun & William VanSchalkwyk & Susan Leite & David Barber, 2009. "Ranking the Risks from Multiple Hazards in a Small Community," Risk Analysis, John Wiley & Sons, vol. 29(3), pages 438-456, March.
    15. Ine H. J. Van Der Fels‐Klerx & Louis H. J. Goossens & Helmut W. Saatkamp & Suzan H. S. Horst, 2002. "Elicitation of Quantitative Data from a Heterogeneous Expert Panel: Formal Process and Application in Animal Health," Risk Analysis, John Wiley & Sons, vol. 22(1), pages 67-81, February.
    16. Benbasat, Izak & Lim, John, 2000. "Information Technology Support For Debiasing Group Judgments: An Empirical Evaluation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(1), pages 167-183, September.
    17. Prather, Larry J. & Middleton, Karen L., 2006. "Timing and selectivity of mutual fund managers: An empirical test of the behavioral decision-making theory," Journal of Empirical Finance, Elsevier, vol. 13(3), pages 249-273, June.
    18. Sjöberg, Lennart, 2003. "Risk communication between experts and the public: perceptions and intentions," SSE/EFI Working Paper Series in Business Administration 2003:13, Stockholm School of Economics.
    19. Philippos Louis & Matías Núñez & Dimitrios Xefteris, 2019. "Trimming Extreme Opinions in Preference Aggregation," University of Cyprus Working Papers in Economics 12-2019, University of Cyprus Department of Economics.
    20. Henry H. Willis & Michael L. DeKay, 2007. "The Roles of Group Membership, Beliefs, and Norms in Ecological Risk Perception," Risk Analysis, John Wiley & Sons, vol. 27(5), pages 1365-1380, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:27:y:2007:i:1:p:225-240. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.