IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v11y2014i2p133-145.html
   My bibliography  Save this article

Two Reasons to Make Aggregated Probability Forecasts More Extreme

Author

Listed:
  • Jonathan Baron

    (Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Barbara A. Mellers

    (Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Philip E. Tetlock

    (Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Eric Stone

    (Department of Psychology, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

  • Lyle H. Ungar

    (Department of Computer and Information Science, University of Pennsylvania, Philadelphia, Pennsylvania 19104)

Abstract

When aggregating the probability estimates of many individuals to form a consensus probability estimate of an uncertain future event, it is common to combine them using a simple weighted average. Such aggregated probabilities correspond more closely to the real world if they are transformed by pushing them closer to 0 or 1. We explain the need for such transformations in terms of two distorting factors: The first factor is the compression of the probability scale at the two ends, so that random error tends to push the average probability toward 0.5. This effect does not occur for the median forecast, or, arguably, for the mean of the log odds of individual forecasts. The second factor---which affects mean, median, and mean of log odds---is the result of forecasters taking into account their individual ignorance of the total body of information available. Individual confidence in the direction of a probability judgment (high/low) thus fails to take into account the wisdom of crowds that results from combining different evidence available to different judges. We show that the same transformation function can approximately eliminate both distorting effects with different parameters for the mean and the median. And we show how, in principle, use of the median can help distinguish the two effects.

Suggested Citation

  • Jonathan Baron & Barbara A. Mellers & Philip E. Tetlock & Eric Stone & Lyle H. Ungar, 2014. "Two Reasons to Make Aggregated Probability Forecasts More Extreme," Decision Analysis, INFORMS, vol. 11(2), pages 133-145, June.
    • Handle: RePEc:inm:ordeca:v:11:y:2014:i:2:p:133-145
    DOI: 10.1287/deca.2014.0293
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.2014.0293
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.2014.0293?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. A. Drăgulescu & V.M. Yakovenko, 2001. "Evidence for the exponential distribution of income in the USA," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 20(4), pages 585-589, April.
    2. Satopää, Ville A. & Baron, Jonathan & Foster, Dean P. & Mellers, Barbara A. & Tetlock, Philip E. & Ungar, Lyle H., 2014. "Combining multiple probability predictions using a simple logit model," International Journal of Forecasting, Elsevier, vol. 30(2), pages 344-356.
    3. Wallsten, Thomas S. & Diederich, Adele, 2001. "Understanding pooled subjective probability estimates," Mathematical Social Sciences, Elsevier, vol. 41(1), pages 1-18, January.
    4. David J. Johnstone & Victor Richmond R. Jose & Robert L. Winkler, 2011. "Tailored Scoring Rules for Probabilities," Decision Analysis, INFORMS, vol. 8(4), pages 256-268, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ellis Scharfenaker, Markus P.A. Schneider, 2019. "Labor Market Segmentation and the Distribution of Income: New Evidence from Internal Census Bureau Data," Working Paper Series, Department of Economics, University of Utah 2019_08, University of Utah, Department of Economics.
    2. Anwar Shaikh & Amr Ragab, 2009. "Ingresos de la Gran Mayoría (IGM): Una Nueva Medida de la Desigualdad Mundial," Policy Research Brief 7, International Policy Centre for Inclusive Growth.
    3. Karimi, Majid & Zaerpour, Nima, 2022. "Put your money where your forecast is: Supply chain collaborative forecasting with cost-function-based prediction markets," European Journal of Operational Research, Elsevier, vol. 300(3), pages 1035-1049.
    4. Costas Efthimiou & Adam Wearne, 2016. "Household Income Distribution in the USA," Papers 1602.06234, arXiv.org.
    5. Hernández-Ramírez, E. & del Castillo-Mussot, M. & Hernández-Casildo, J., 2021. "World per capita gross domestic product measured nominally and across countries with purchasing power parity: Stretched exponential or Boltzmann–Gibbs distribution?," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 568(C).
    6. repec:cup:judgdm:v:14:y:2019:i:4:p:395-411 is not listed on IDEAS
    7. Makowsky, Michael, 2009. "Religious Extremism, Clubs, and Civil Liberties: A Model of Religious Populations," MPRA Paper 14358, University Library of Munich, Germany.
    8. Callealta Barroso, Francisco Javier & García-Pérez, Carmelo & Prieto-Alaiz, Mercedes, 2020. "Modelling income distribution using the log Student’s t distribution: New evidence for European Union countries," Economic Modelling, Elsevier, vol. 89(C), pages 512-522.
    9. Schulz, Jan & Mayerhoffer, Daniel M., 2021. "A network approach to consumption," BERG Working Paper Series 173, Bamberg University, Bamberg Economic Research Group.
    10. Ricardo Lopez-Ruiz, 2012. "Complex Systems with Trivial Dynamics," Papers 1210.6481, arXiv.org.
    11. Ville A. Satopää & Robin Pemantle & Lyle H. Ungar, 2016. "Modeling Probability Forecasts via Information Diversity," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(516), pages 1623-1633, October.
    12. Fabio Clementi & Mauro Gallegati, 2005. "Pareto's Law of Income Distribution: Evidence for Grermany, the United Kingdom, and the United States," Microeconomics 0505006, University Library of Munich, Germany.
    13. Tao, Yong, 2015. "Universal laws of human society’s income distribution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 435(C), pages 89-94.
    14. Xavier Gabaix, 2009. "Power Laws in Economics and Finance," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 255-294, May.
    15. Kaldasch, Joachim, 2012. "Evolutionary model of the personal income distribution," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 391(22), pages 5628-5642.
    16. Shaikh, Anwar & Jacobo, Juan Esteban, 2020. "Economic Arbitrage and the Econophysics of Income Inequality," Review of Behavioral Economics, now publishers, vol. 7(4), pages 299–315-2, December.
    17. Inoue, Jun-ichi & Ghosh, Asim & Chatterjee, Arnab & Chakrabarti, Bikas K., 2015. "Measuring social inequality with quantitative methodology: Analytical estimates and empirical data analysis by Gini and k indices," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 429(C), pages 184-204.
    18. Kayal, Suchandan & Kumar, Somesh, 2013. "Estimation of the Shannon’s entropy of several shifted exponential populations," Statistics & Probability Letters, Elsevier, vol. 83(4), pages 1127-1135.
    19. Oksana Melikhova & Jakub Čížek, 2014. "Kuznets Inverted U-Curve Hypothesis Examined on Up-To Date Observations for 145 Countries," Prague Economic Papers, Prague University of Economics and Business, vol. 2014(3), pages 388-410.
    20. Valérie Berenger & Cuauhtémoc Calderón Villarreal & Franck Celestini, 2009. "Modelling the Distribution of Multidimensional Poverty Scores: Evidence from Mexico," Estudios Económicos, El Colegio de México, Centro de Estudios Económicos, vol. 24(1), pages 3-34.
    21. Fabio Clementi & Mauro Gallegati & Giorgio Kaniadakis, 2012. "A new model of income distribution: the κ-generalized distribution," Journal of Economics, Springer, vol. 105(1), pages 63-91, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:11:y:2014:i:2:p:133-145. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.