IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v19y2022i4p354-383.html
   My bibliography  Save this article

Model Complexity and Accuracy: A COVID-19 Case Study

Author

Listed:
  • Colin Small

    (Operations Research and Industrial Engineering, Cockrell School of Engineering, University of Texas at Austin, Austin, Texas 78712)

  • J. Eric Bickel

    (Operations Research and Industrial Engineering, Cockrell School of Engineering, University of Texas at Austin, Austin, Texas 78712; Department of Information, Risk, and Operations Management, McCombs School of Business, University of Texas at Austin, Austin, Texas 78712)

Abstract

When creating mathematical models for forecasting and decision making, there is a tendency to include more complexity than necessary, in the belief that higher-fidelity models are more accurate than simpler ones. In this paper, we analyze the performance of models that submitted COVID-19 forecasts to the U.S. Centers for Disease Control and Prevention and evaluate them against a simple two-equation model that is specified using simple linear regression. We find that our simple model was comparable in accuracy to highly publicized models and had among the best-calibrated forecasts. This result may be surprising given the complexity of many COVID-19 models and their support by large forecasting teams. However, our result is consistent with the body of research that suggests that simple models perform very well in a variety of settings.

Suggested Citation

  • Colin Small & J. Eric Bickel, 2022. "Model Complexity and Accuracy: A COVID-19 Case Study," Decision Analysis, INFORMS, vol. 19(4), pages 354-383, December.
    • Handle: RePEc:inm:ordeca:v:19:y:2022:i:4:p:354-383
    DOI: 10.1287/deca.2022.0457
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.2022.0457
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.2022.0457?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Brighton, Henry & Gigerenzer, Gerd, 2015. "The bias bias," Journal of Business Research, Elsevier, vol. 68(8), pages 1772-1784.
    2. Armstrong, J Scott, 1978. "Forecasting with Econometric Methods: Folklore versus Fact," The Journal of Business, University of Chicago Press, vol. 51(4), pages 549-564, October.
    3. Smith, Stanley K., 1997. "Further thoughts on simplicity and complexity in population projection models," International Journal of Forecasting, Elsevier, vol. 13(4), pages 557-565, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Green, Kesten C. & Armstrong, J. Scott, 2015. "Simple versus complex forecasting: The evidence," Journal of Business Research, Elsevier, vol. 68(8), pages 1678-1685.
    2. Booth, Heather, 2006. "Demographic forecasting: 1980 to 2005 in review," International Journal of Forecasting, Elsevier, vol. 22(3), pages 547-581.
    3. Armstrong, J. Scott & Green, Kesten C. & Graefe, Andreas, 2015. "Golden rule of forecasting: Be conservative," Journal of Business Research, Elsevier, vol. 68(8), pages 1717-1731.
    4. Guercini, Simone & Milanesi, Matilde, 2020. "Heuristics in international business: A systematic literature review and directions for future research," Journal of International Management, Elsevier, vol. 26(4).
    5. Michael P. Cameron & William Cochrane, 2015. "Using Land-Use Modelling to Statistically Downscale Population Projections to Small Areas," Working Papers in Economics 15/12, University of Waikato.
    6. Rodolfo Garcia Sierra & Álvaro Zerda Sarmiento, 2017. "Caracterización de la función de valor empleada en las decisiones ambientales por las grandes organizaciones: Estudio de los grandes proyectos hidroeléctricos en Colombia," Revista Facultad de Ciencias Económicas, Universidad Militar Nueva Granada, vol. 26(1), pages 69-91, December.
    7. J. Scott Armstrong, 1984. "Forecasting by Extrapolation: Conclusions from 25 Years of Research," Interfaces, INFORMS, vol. 14(6), pages 52-66, December.
    8. Fullerton, Thomas M., Jr. & Walke, Adam G. & Villavicencio, Diana, 2015. "An Econometric Approach for Modeling Population Change in Doña Ana County, New Mexico," MPRA Paper 71141, University Library of Munich, Germany, revised 28 Jan 2015.
    9. Mikko Myrskylä & Joshua R. Goldstein & Yen-hsin Alice Cheng, 2012. "New cohort fertility forecasts for the developed world," MPIDR Working Papers WP-2012-014, Max Planck Institute for Demographic Research, Rostock, Germany.
    10. Osman Gulseven, 2016. "Forecasting Population and Demographic Composition of Kuwait Until 2030," International Journal of Economics and Financial Issues, Econjournals, vol. 6(4), pages 1429-1435.
    11. Brighton, Henry, 2020. "Statistical foundations of ecological rationality," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 14, pages 1-32.
    12. Woike, Jan K. & Hoffrage, Ulrich & Petty, Jeffrey S., 2015. "Picking profitable investments: The success of equal weighting in simulated venture capitalist decision making," Journal of Business Research, Elsevier, vol. 68(8), pages 1705-1716.
    13. Huddleston, Samuel H. & Porter, John H. & Brown, Donald E., 2015. "Improving forecasts for noisy geographic time series," Journal of Business Research, Elsevier, vol. 68(8), pages 1810-1818.
    14. Riedl, Anna & Vervaeke, John, 2022. "Rationality and Relevance Realization," OSF Preprints vymwu, Center for Open Science.
    15. Ashish Sood & Gareth M. James & Gerard J. Tellis, 2009. "Functional Regression: A New Model for Predicting Market Penetration of New Products," Marketing Science, INFORMS, vol. 28(1), pages 36-51, 01-02.
    16. Ville A. Satopää & Marat Salikhov & Philip E. Tetlock & Barbara Mellers, 2021. "Bias, Information, Noise: The BIN Model of Forecasting," Management Science, INFORMS, vol. 67(12), pages 7599-7618, December.
    17. Flyvbjerg, Bent & Ansar, Atif & Budzier, Alexander & Buhl, Søren & Cantarelli, Chantal & Garbuio, Massimo & Glenting, Carsten & Holm, Mette Skamris & Lovallo, Dan & Lunn, Daniel & Molin, Eric & Rønnes, 2018. "Five things you should know about cost overrun," Transportation Research Part A: Policy and Practice, Elsevier, vol. 118(C), pages 174-190.
    18. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    19. Armstrong, J. Scott, 1983. "Strategic Planning and Forecasting Fundamentals," MPRA Paper 81682, University Library of Munich, Germany.
    20. Forbes, William & Hudson, Robert & Skerratt, Len & Soufian, Mona, 2015. "Which heuristics can aid financial-decision-making?," International Review of Financial Analysis, Elsevier, vol. 42(C), pages 199-210.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:19:y:2022:i:4:p:354-383. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.