IDEAS home Printed from
   My bibliography  Save this article

Assessing Dependence: Some Experimental Results


  • Robert T. Clemen

    () (Fuqua School of Business, Duke University, Durham, North Carolina 27708-0120)

  • Gregory W. Fischer

    () (Fuqua School of Business, Duke University, Durham, North Carolina 27708-0120)

  • Robert L. Winkler

    () (Fuqua School of Business, Duke University, Durham, North Carolina 27708-0120)


Constructing decision- and risk-analysis probability models often requires measures of dependence among variables. Although data are sometimes available to estimate such measures, in many applications they must be obtained by means of subjective judgment by experts. We discuss two experimental studies that compare the accuracy of six different methods for assessing dependence. Our results lead to several conclusions: First, simply asking experts to report a correlation is a reasonable approach. Direct estimation is more accurate than the other methods studied, is not prone to mathematically inconsistent responses (as are some other measures), and is judged to be less difficult than alternate methods. In addition, directly assessed correlations showed less variability than the correlations derived from other assessment methods. Our results also show that experience with the variables can improve performance somewhat, as can training in a given assessment method. Finally, if a judge uses several different assessment methods, an average of the resulting estimates can also lead to better performance.

Suggested Citation

  • Robert T. Clemen & Gregory W. Fischer & Robert L. Winkler, 2000. "Assessing Dependence: Some Experimental Results," Management Science, INFORMS, vol. 46(8), pages 1100-1115, August.
  • Handle: RePEc:inm:ormnsc:v:46:y:2000:i:8:p:1100-1115
    DOI: 10.1287/mnsc.46.8.1100.12023

    Download full text from publisher

    File URL:
    Download Restriction: no

    References listed on IDEAS

    1. H. V. Ravinder & Don N. Kleinmuntz & James S. Dyer, 1988. "The Reliability of Subjective Probabilities Obtained Through Decomposition," Management Science, INFORMS, vol. 34(2), pages 186-199, February.
    2. Spyros Makridakis & Robert L. Winkler, 1983. "Averages of Forecasts: Some Empirical Results," Management Science, INFORMS, vol. 29(9), pages 987-996, September.
    3. Robert T. Clemen & Terence Reilly, 1999. "Correlations and Copulas for Decision and Risk Analysis," Management Science, INFORMS, vol. 45(2), pages 208-224, February.
    4. Pechmann, Cornelia & Ratneshwar, S, 1992. "Consumer Covariation Judgments: Theory or Data Driven?," Journal of Consumer Research, Oxford University Press, vol. 19(3), pages 373-386, December.
    Full references (including those not matched with items on IDEAS)


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Anil Gaba & Dana G. Popescu & Zhi Chen, 2019. "Assessing Uncertainty from Point Forecasts," Management Science, INFORMS, vol. 65(1), pages 90-106, January.
    2. James K. Hammitt & Yifan Zhang, 2013. "Combining Experts’ Judgments: Comparison of Algorithmic Methods Using Synthetic Data," Risk Analysis, John Wiley & Sons, vol. 33(1), pages 109-120, January.
    3. Werner, Christoph & Bedford, Tim & Cooke, Roger M. & Hanea, Anca M. & Morales-Nápoles, Oswaldo, 2017. "Expert judgement for dependence in probabilistic modelling: A systematic literature review and future research directions," European Journal of Operational Research, Elsevier, vol. 258(3), pages 801-819.
    4. Tianyang Wang & James S. Dyer & John C. Butler, 2016. "Modeling Correlated Discrete Uncertainties in Event Trees with Copulas," Risk Analysis, John Wiley & Sons, vol. 36(2), pages 396-410, February.
    5. Jing Ai & Patrick L. Brockett & Tianyang Wang, 2017. "Optimal Enterprise Risk Management and Decision Making With Shared and Dependent Risks," Journal of Risk & Insurance, The American Risk and Insurance Association, vol. 84(4), pages 1127-1169, December.
    6. Herbert Hove & Frank Beichelt & Parmod K. Kapur, 2017. "Estimation of the Frank copula model for dependent competing risks in accelerated life testing," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 8(4), pages 673-682, December.
    7. Ebrahimi, Nader & Jalali, Nima Y. & Soofi, Ehsan S., 2014. "Comparison, utility, and partition of dependence under absolutely continuous and singular distributions," Journal of Multivariate Analysis, Elsevier, vol. 131(C), pages 32-50.
    8. Tianyang Wang & James S. Dyer, 2012. "A Copulas-Based Approach to Modeling Dependence in Decision Trees," Operations Research, INFORMS, vol. 60(1), pages 225-242, February.
    9. Jesus Palomo & David Rios Insua & Fabrizio Ruggeri, 2007. "Modeling External Risks in Project Management," Risk Analysis, John Wiley & Sons, vol. 27(4), pages 961-978, August.
    10. Donald L. Keefer & Craig W. Kirkwood & James L. Corner, 2004. "Perspective on Decision Analysis Applications, 1990–2001," Decision Analysis, INFORMS, vol. 1(1), pages 4-22, March.
    11. Robert L. Winkler & Robert T. Clemen, 2004. "Multiple Experts vs. Multiple Methods: Combining Correlation Assessments," Decision Analysis, INFORMS, vol. 1(3), pages 167-176, September.
    12. Hanea, Anca & Morales Napoles, Oswaldo & Ababei, Dan, 2015. "Non-parametric Bayesian networks: Improving theory and reviewing applications," Reliability Engineering and System Safety, Elsevier, vol. 144(C), pages 265-284.
    13. Morales, O. & Kurowicka, D. & Roelen, A., 2008. "Eliciting conditional and unconditional rank correlations from conditional probabilities," Reliability Engineering and System Safety, Elsevier, vol. 93(5), pages 699-710.
    14. Marie-Sophie Denner & Louis Christian Püschel & Maximilian Röglinger, 2018. "How to Exploit the Digitalization Potential of Business Processes," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 60(4), pages 331-349, August.
    15. Gillian Anderson & Lesley Walls & Matthew Revie & Euan Fenelon & Calum Storie, 2015. "Quantifying intra-organisational risks: An analysis of practice-theory tensions in probability elicitation to improve technical risk management in an energy utility," Journal of Risk and Reliability, , vol. 229(3), pages 171-180, June.
    16. J. Eric Bickel & James E. Smith, 2006. "Optimal Sequential Exploration: A Binary Learning Model," Decision Analysis, INFORMS, vol. 3(1), pages 16-32, March.
    17. M Revie & T Bedford & L Walls, 2010. "Evaluation of elicitation methods to quantify Bayes linear models," Journal of Risk and Reliability, , vol. 224(4), pages 322-332, December.
    18. Tianyang Wang & James S. Dyer & Warren J. Hahn, 2017. "Sensitivity analysis of decision making under dependent uncertainties using copulas," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 5(1), pages 117-139, November.
    19. Ilia Tsetlin & Robert L. Winkler, 2005. "Risky Choices and Correlated Background Risk," Management Science, INFORMS, vol. 51(9), pages 1336-1345, September.
    20. Huifen Chen, 2001. "Initialization for NORTA: Generation of Random Vectors with Specified Marginals and Correlations," INFORMS Journal on Computing, INFORMS, vol. 13(4), pages 312-331, November.
    21. Achille N. Njike & Mustafa Kumral, 2019. "Mining corporate portfolio optimization model with company’s operational performance level and international risk," Mineral Economics, Springer;Raw Materials Group (RMG);Luleå University of Technology, vol. 32(3), pages 307-315, November.
    22. James E. Smith & Detlof von Winterfeldt, 2004. "Anniversary Article: Decision Analysis in Management Science," Management Science, INFORMS, vol. 50(5), pages 561-574, May.
    23. Robin L. Dillon & Richard John & Detlof von Winterfeldt, 2002. "Assessment of Cost Uncertainties for Large Technology Projects: A Methodology and an Application," Interfaces, INFORMS, vol. 32(4), pages 52-66, August.
    24. Ali E. Abbas & David V. Budescu & Yuhong (Rola) Gu, 2010. "Assessing Joint Distributions with Isoprobability Contours," Management Science, INFORMS, vol. 56(6), pages 997-1011, June.
    25. Wang, Fan & Li, Heng & Dong, Chao & Ding, Lieyun, 2019. "Knowledge representation using non-parametric Bayesian networks for tunneling risk analysis," Reliability Engineering and System Safety, Elsevier, vol. 191(C).


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:46:y:2000:i:8:p:1100-1115. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Matthew Walls). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.