IDEAS home Printed from https://ideas.repec.org/a/sae/somere/v48y2019i2p400-430.html
   My bibliography  Save this article

The Difference Between Instability and Uncertainty: Comment on Young and Holsteen (2017)

Author

Listed:
  • Adam Slez

Abstract

Young and Holsteen (YH) introduce a number of tools for evaluating model uncertainty. In so doing, they are careful to differentiate their method from existing forms of model averaging. The fundamental difference lies in the way in which the underlying estimates are weighted. Whereas standard approaches to model averaging assign higher weight to better fitting models, the YH method weights all models equally. As I show, this is a nontrivial distinction, in that the two sets of procedures tend to produce radically different results. Drawing on both simulation and real-world examples, I demonstrate that in failing to distinguish between numerical variation and statistical uncertainty, the procedure proposed by YH will tend to overstate the amount of uncertainty resulting from variation across models. In standard circumstances, the quality of estimates produced using this method will tend to be objectively worse than that of conventional alternatives.

Suggested Citation

  • Adam Slez, 2019. "The Difference Between Instability and Uncertainty: Comment on Young and Holsteen (2017)," Sociological Methods & Research, , vol. 48(2), pages 400-430, May.
  • Handle: RePEc:sae:somere:v:48:y:2019:i:2:p:400-430
    DOI: 10.1177/0049124117729704
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0049124117729704
    Download Restriction: no

    File URL: https://libkey.io/10.1177/0049124117729704?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Koop, Gary & Leon-Gonzalez, Roberto & Strachan, Rodney, 2012. "Bayesian model averaging in the instrumental variable regression model," Journal of Econometrics, Elsevier, vol. 171(2), pages 237-250.
    2. David Kaplan & Jianshen Chen, 2012. "A Two-Step Bayesian Approach for Propensity Score Analysis: Simulations and Case Study," Psychometrika, Springer;The Psychometric Society, vol. 77(3), pages 581-609, July.
    3. StevenN. Durlauf & Andros Kourtellos & ChihMing Tan, 2008. "Are Any Growth Theories Robust?," Economic Journal, Royal Economic Society, vol. 118(527), pages 329-346, March.
    4. Crespo Cuaresma, Jesus & Grün, Bettina & Hofmarcher, Paul & Humer, Stefan & Moser, Mathias, 2015. "A Comprehensive Approach to Posterior Jointness Analysis in Bayesian Model Averaging Applications," Department of Economics Working Paper Series 193, WU Vienna University of Economics and Business.
    5. Xavier Sala-I-Martin & Gernot Doppelhofer & Ronald I. Miller, 2004. "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, American Economic Association, vol. 94(4), pages 813-835, September.
    6. Zeugner, Stefan & Feldkircher, Martin, 2015. "Bayesian Model Averaging Employing Fixed and Flexible Priors: The BMS Package for R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 68(i04).
    7. Steven N. Durlauf & Andros Kourtellos & Chih Ming Tan, 2008. "Are Any Growth Theories Robust?," Economic Journal, Royal Economic Society, vol. 118(527), pages 329-346, March.
    8. Claudia Czado & Adrian Raftery, 2006. "Choosing the link function and accounting for link uncertainty in generalized linear models using Bayes factors," Statistical Papers, Springer, vol. 47(3), pages 419-442, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Semken, Christoph & Rossell, David, 2020. "Bayesian Specification Curve Analysis," OSF Preprints cahyq, Center for Open Science.
    2. Cantone, Giulio Giacomo, 2023. "The multiversal methodology as a remedy of the replication crisis," MetaArXiv kuhmz, Center for Open Science.
    3. Christoph Semken & David Rossell, 2022. "Specification analysis for technology use and teenager well‐being: Statistical validity and a Bayesian proposal," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(5), pages 1330-1355, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mark F. J. Steel, 2020. "Model Averaging and Its Use in Economics," Journal of Economic Literature, American Economic Association, vol. 58(3), pages 644-719, September.
    2. Jetter, Michael & Parmeter, Christopher F., 2018. "Sorting through global corruption determinants: Institutions and education matter – Not culture," World Development, Elsevier, vol. 109(C), pages 279-294.
    3. León-González, Roberto & Montolio, Daniel, 2015. "Endogeneity and panel data in growth regressions: A Bayesian model averaging approach," Journal of Macroeconomics, Elsevier, vol. 46(C), pages 23-39.
    4. Bruns, Stephan B. & Ioannidis, John P.A., 2020. "Determinants of economic growth: Different time different answer?," Journal of Macroeconomics, Elsevier, vol. 63(C).
    5. Ampaabeng, Samuel K. & Tan, Chih Ming, 2013. "The long-term cognitive consequences of early childhood malnutrition: The case of famine in Ghana," Journal of Health Economics, Elsevier, vol. 32(6), pages 1013-1027.
    6. Theo S. Eicher & David J. Kuenzel, 2016. "The elusive effects of trade on growth: Export diversity and economic take-off," Canadian Journal of Economics, Canadian Economics Association, vol. 49(1), pages 264-295, February.
    7. Michael Jetter & Christopher F. Parmeter, 2016. "Uncovering the determinants of corruption," Working Papers 2016-02, University of Miami, Department of Economics.
    8. Arin, K. Peren & Braunfels, Elias, 2018. "The resource curse revisited: A Bayesian model averaging approach," Energy Economics, Elsevier, vol. 70(C), pages 170-178.
    9. Horváth, Roman, 2013. "Does trust promote growth?," Journal of Comparative Economics, Elsevier, vol. 41(3), pages 777-788.
    10. Andros Kourtellos & Charalambos G. Tsangarides, 2022. "Robust Correlates of Growth Spells: Do Inequality and Redistribution Matter?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 84(6), pages 1302-1328, December.
    11. Samuel K. Ampaabeng & Chih Ming Tang, 2012. "The Long-Term Cognitive Consequences of Early Childhood Malnutrition: The Case of Famine in Ghana," Working Paper series 64_12, Rimini Centre for Economic Analysis.
    12. Jaroslava Hlouskova & Martin Wagner, 2013. "The Determinants of Long-Run Economic Growth: A Conceptually and Computationally Simple Approach," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 149(IV), pages 445-492, December.
    13. Bettina Grün & Paul Hofmarcher, 2021. "Identifying groups of determinants in Bayesian model averaging using Dirichlet process clustering," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 48(3), pages 1018-1045, September.
    14. Jan Kluge & Sarah Lappöhn & Kerstin Plank, 2023. "Predictors of TFP growth in European countries," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 50(1), pages 109-140, February.
    15. Antonio Ciccone & Marek Jarociński, 2010. "Determinants of Economic Growth: Will Data Tell?," American Economic Journal: Macroeconomics, American Economic Association, vol. 2(4), pages 222-246, October.
    16. Abe, Ryosuke & Kato, Hironori, 2017. "What led to the establishment of a rail-oriented city? Determinants of urban rail supply in Tokyo, Japan, 1950–2010," Transport Policy, Elsevier, vol. 58(C), pages 72-79.
    17. Benecká, Soňa & Komarek, Lubos, 2018. "International reserves: Facing model uncertainty," Economic Systems, Elsevier, vol. 42(3), pages 523-531.
    18. Michael S. Delgado & Daniel J. Henderson & Christopher F. Parmeter, 2014. "Does Education Matter for Economic Growth?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 76(3), pages 334-359, June.
    19. Hasan, Iftekhar & Horvath, Roman & Mares, Jan, 2020. "Finance and wealth inequality," Journal of International Money and Finance, Elsevier, vol. 108(C).
    20. Aart Kraay & Norikazu Tawara, 2013. "Can specific policy indicators identify reform priorities?," Journal of Economic Growth, Springer, vol. 18(3), pages 253-283, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:somere:v:48:y:2019:i:2:p:400-430. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.