IDEAS home Printed from
   My bibliography  Save this article

Evaluating and Combining Physicians' Probabilities of Survival in an Intensive Care Unit


  • Robert L. Winkler

    (The Fuqua School of Business, Duke University, Durham, North Carolina 27708)

  • Roy M. Poses

    (Division of General Medicine, Medical College of Virginia, P.O. Box 102, Richmond, Virginia 23298)


In this paper, probabilities of survival assessed by physicians for patients admitted to an intensive care unit are studied. The probabilities from each of four types of physicians are evaluated on an overall basis and in terms of specific attributes, and the groups are compared. The physicians with the most experience and expertise perform better overall. All four groups appear to be reasonably well calibrated, and the key factor in relative overall performance is the level of discrimination provided by the probabilities. Averages of two, three, and four probabilities for each individual patient are also analyzed. As the number of the probabilities in the average increases, performance improves on average on all dimensions, although the best overall performance is exhibited by a combination of probabilities from the two physician types performing best individually. Some comparisons are made with previous work, and implications for probability assessment and combination in medicine and more generally in other areas of application are discussed. Important characteristics of the study are the fact that it was conducted on-line in a real setting, the involvement of individuals with different levels of expertise, the use of a true predictive situation with a clearly-defined event, the consideration of multiple dimensions of the quality of judgments, and the collection of multiple probabilities for each case to permit the investigation of a variety of possible combinations of probabilities.

Suggested Citation

  • Robert L. Winkler & Roy M. Poses, 1993. "Evaluating and Combining Physicians' Probabilities of Survival in an Intensive Care Unit," Management Science, INFORMS, vol. 39(12), pages 1526-1543, December.
  • Handle: RePEc:inm:ormnsc:v:39:y:1993:i:12:p:1526-1543

    Download full text from publisher

    File URL:
    Download Restriction: no


    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.

    Cited by:

    1. Hammitt, James K. & Zhang, Yifan, 2012. "Combining Experts’ Judgments: Comparison of Algorithmic Methods using Synthetic Data," TSE Working Papers 12-293, Toulouse School of Economics (TSE).
    2. Yaniv, Ilan & Kleinberger, Eli, 2000. "Advice Taking in Decision Making: Egocentric Discounting and Reputation Formation," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(2), pages 260-281, November.
    3. R. Winkler & Javier Muñoz & José Cervera & José Bernardo & Gail Blattenberger & Joseph Kadane & Dennis Lindley & Allan Murphy & Robert Oliver & David Ríos-Insua, 1996. "Scoring rules and the evaluation of probabilities," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 5(1), pages 1-60, June.
    4. Jain, Kriti & Bearden, J. Neil & Filipowicz, Allan, 2013. "Depression and forecast accuracy: Evidence from the 2010 FIFA World Cup," International Journal of Forecasting, Elsevier, vol. 29(1), pages 69-79.
    5. Ozer, Muammer, 2011. "Understanding the impacts of product knowledge and product type on the accuracy of intentions-based new product predictions," European Journal of Operational Research, Elsevier, vol. 211(2), pages 359-369, June.
    6. Fifić, Mario & Gigerenzer, Gerd, 2014. "Are two interviewers better than one?," Journal of Business Research, Elsevier, vol. 67(8), pages 1771-1779.
    7. Roopesh Ranjan & Tilmann Gneiting, 2010. "Combining probability forecasts," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 71-91.
    8. Lessmann, Stefan & Sung, Ming-Chien & Johnson, Johnnie E.V. & Ma, Tiejun, 2012. "A new methodology for generating and combining statistical forecasting models to enhance competitive event prediction," European Journal of Operational Research, Elsevier, vol. 218(1), pages 163-174.
    9. Sarkar, Sumit & Mendelson, Haim & Storey, Veda C., 1996. "Approximate representation of probabilistic data in expert systems," European Journal of Operational Research, Elsevier, vol. 94(3), pages 488-504, November.
    10. Yaniv, Ilan, 2004. "Receiving other people's advice: Influence and benefit," Organizational Behavior and Human Decision Processes, Elsevier, vol. 93(1), pages 1-13, January.
    11. Simon, Mark & Shrader, Rodney C., 2012. "Entrepreneurial actions and optimistic overconfidence: The role of motivated reasoning in new product introductions," Journal of Business Venturing, Elsevier, vol. 27(3), pages 291-309.
    12. Joseph Lipscomb & Giovanni Parmigiani & Vic Hasselblad, 1998. "Combining Expert Judgment by Hierarchical Modeling: An Application to Physician Staffing," Management Science, INFORMS, vol. 44(2), pages 149-161, February.
    13. David V. Budescu & Hsiu-Ting Yu, 2006. "To Bayes or Not to Bayes? A Comparison of Two Classes of Models of Information Aggregation," Decision Analysis, INFORMS, vol. 3(3), pages 145-162, September.
    14. James E. Smith & Detlof von Winterfeldt, 2004. "Anniversary Article: Decision Analysis in Management Science," Management Science, INFORMS, vol. 50(5), pages 561-574, May.
    15. Ozer, Muammer, 2007. "Reducing the demand uncertainties at the fuzzy-front-end of developing new online services," Research Policy, Elsevier, vol. 36(9), pages 1372-1387, November.
    16. Ozer, Muammer, 2009. "The roles of product lead-users and product experts in new product evaluation," Research Policy, Elsevier, vol. 38(8), pages 1340-1349, October.
    17. Yaniv, Ilan & Milyavsky, Maxim, 2007. "Using advice from multiple sources to revise and improve judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 104-120, May.


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ormnsc:v:39:y:1993:i:12:p:1526-1543. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Mirko Janc). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.