IDEAS home Printed from https://ideas.repec.org/a/kap/theord/v73y2012i4p671-685.html
   My bibliography  Save this article

The Group Calibration Index: a group-based approach for assessing forecasters’ expertise when external outcome data are missing

Author

Listed:
  • Ilan Fischer
  • Ravid Bogaire

Abstract

The Group Calibration Index (GCI) provides a means of assessing the quality of forecasters’ predictions in situations that lack external feedback or outcome data. The GCI replaces the missing outcome data with aggregated ratings of a well-defined reference group. A simulation study and two experiments show how the GCI classifies forecaster performance and distinguishes between forecasters with restricted information and those with complete information. The results also show that under certain circumstances, where members of the reference group have high-quality information, the new GCI will outperform expert classification that is based on traditional calibration indices. Copyright Springer Science+Business Media, LLC. 2012

Suggested Citation

  • Ilan Fischer & Ravid Bogaire, 2012. "The Group Calibration Index: a group-based approach for assessing forecasters’ expertise when external outcome data are missing," Theory and Decision, Springer, vol. 73(4), pages 671-685, October.
    • Handle: RePEc:kap:theord:v:73:y:2012:i:4:p:671-685
    DOI: 10.1007/s11238-011-9265-4
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s11238-011-9265-4
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s11238-011-9265-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Fischer, Ilan & Budescu, David V., 2005. "When do those who know more also know more about how much they know? The development of confidence and performance in categorical decision tasks," Organizational Behavior and Human Decision Processes, Elsevier, vol. 98(1), pages 39-53, September.
    2. Sterman, John., 1994. "Learning in and about complex systems," Working papers 3660-94., Massachusetts Institute of Technology (MIT), Sloan School of Management.
    3. Keren, Gideon, 1987. "Facing uncertainty in the game of bridge: A calibration study," Organizational Behavior and Human Decision Processes, Elsevier, vol. 39(1), pages 98-114, February.
    4. Wagenaar, Willem A. & Keren, Gideon B., 1985. "Calibration of probability assessments by professional blackjack dealers, statistical experts, and lay people," Organizational Behavior and Human Decision Processes, Elsevier, vol. 36(3), pages 406-416, December.
    5. Richard P. Larrick & Jack B. Soll, 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science, INFORMS, vol. 52(1), pages 111-127, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Don A. Moore & Samuel A. Swift & Angela Minster & Barbara Mellers & Lyle Ungar & Philip Tetlock & Heather H. J. Yang & Elizabeth R. Tenney, 2017. "Confidence Calibration in a Multiyear Geopolitical Forecasting Competition," Management Science, INFORMS, vol. 63(11), pages 3552-3565, November.
    2. Fischer, Ilan & Budescu, David V., 2005. "When do those who know more also know more about how much they know? The development of confidence and performance in categorical decision tasks," Organizational Behavior and Human Decision Processes, Elsevier, vol. 98(1), pages 39-53, September.
    3. Lawrence, Michael & Goodwin, Paul & O'Connor, Marcus & Onkal, Dilek, 2006. "Judgmental forecasting: A review of progress over the last 25 years," International Journal of Forecasting, Elsevier, vol. 22(3), pages 493-518.
    4. Lu, Jinfeng & Dimov, Dimo, 2023. "A system dynamics modelling of entrepreneurship and growth within firms," Journal of Business Venturing, Elsevier, vol. 38(3).
    5. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    6. Day Yang Liu & Wen Chun Tsai & Pei Leen Liu & Chung Yi Fang, 2021. "Determinants of sales revenue in innovation diffusion effects of Taiwan sports lottery during the FIFA World Cup 2018," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 10(4), pages 43-58, June.
    7. Hazhir Rahmandad & Nelson Repenning, 2016. "Capability erosion dynamics," Strategic Management Journal, Wiley Blackwell, vol. 37(4), pages 649-672, April.
    8. Katarzyna Tworek & Katarzyna Walecka-Jankowska & Anna Zgrzywa-Ziemak, 2019. "The role of information systems in shaping integrative logic for business sustainability," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 29(4), pages 125-146.
    9. Brenner, Lyle & Griffin, Dale & Koehler, Derek J., 2005. "Modeling patterns of probability calibration with random support theory: Diagnosing case-based judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 97(1), pages 64-81, May.
    10. McKenzie, Craig R.M. & Liersch, Michael J. & Yaniv, Ilan, 2008. "Overconfidence in interval estimates: What does expertise buy you?," Organizational Behavior and Human Decision Processes, Elsevier, vol. 107(2), pages 179-191, November.
    11. Meri Duryan & Dragan Nikolik & Godefridus Merode & Leopold M. G. Curfs, 2015. "Reflecting on the efficacy of cognitive mapping for decision-making in intellectual disability care: a case study," International Journal of Health Planning and Management, Wiley Blackwell, vol. 30(2), pages 127-144, April.
    12. repec:cup:judgdm:v:8:y:2013:i:2:p:91-105 is not listed on IDEAS
    13. Erik Pruyt & Jan H. Kwakkel, 2014. "Radicalization under deep uncertainty: a multi-model exploration of activism, extremism, and terrorism," System Dynamics Review, System Dynamics Society, vol. 30(1-2), pages 1-28, January.
    14. Oberlechner, Thomas & Osler, Carol, 2012. "Survival of Overconfidence in Currency Markets," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 47(1), pages 91-113, February.
    15. Sarah Gerritsen & Sophia Harré & David Rees & Ana Renker-Darby & Ann E. Bartos & Wilma E. Waterlander & Boyd Swinburn, 2020. "Community Group Model Building as a Method for Engaging Participants and Mobilising Action in Public Health," IJERPH, MDPI, vol. 17(10), pages 1-12, May.
    16. David C. Lane, 2012. "What Is a ‘Policy Insight’?," Systems Research and Behavioral Science, Wiley Blackwell, vol. 29(6), pages 590-595, November.
    17. Julia A. Minson & Jennifer S. Mueller & Richard P. Larrick, 2018. "The Contingent Wisdom of Dyads: When Discussion Enhances vs. Undermines the Accuracy of Collaborative Judgments," Management Science, INFORMS, vol. 64(9), pages 4177-4192, September.
    18. Iván Barreda-Tarrazona & Nikolaos Georgantzís & Constantine Manasakis & Evangelos Mitrokostas & Emmanuel Petrakis, 2012. "Managerial compensation contracts in quantity-setting duopoly," Working Papers 2012/17, Economics Department, Universitat Jaume I, Castellón (Spain).
    19. Charles F. Manski, 2010. "When consensus choice dominates individualism: Jensen's inequality and collective decisions under uncertainty," Quantitative Economics, Econometric Society, vol. 1(1), pages 187-202, July.
    20. Katarzyna Ostasiewicz & Michal H. Tyc & Piotr Goliczewski & Piotr Magnuszewski & Andrzej Radosz & Jan Sendzimir, 2006. "Integrating economic and psychological insights in binary choice models with social interactions," Papers physics/0609170, arXiv.org.
    21. Auping, Willem L. & Pruyt, Erik & de Jong, Sijbren & Kwakkel, Jan H., 2016. "The geopolitical impact of the shale revolution: Exploring consequences on energy prices and rentier states," Energy Policy, Elsevier, vol. 98(C), pages 390-399.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:theord:v:73:y:2012:i:4:p:671-685. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.