IDEAS home Printed from https://ideas.repec.org/a/spr/stpapr/v61y2020i5d10.1007_s00362-018-1048-3.html
   My bibliography  Save this article

When and when not to use optimal model averaging

Author

Listed:
  • Michael Schomaker

    (University of Cape Town)

  • Christian Heumann

    (Ludwig-Maximilians Universität München)

Abstract

Traditionally model averaging has been viewed as an alternative to model selection with the ultimate goal to incorporate the uncertainty associated with the model selection process in standard errors and confidence intervals by using a weighted combination of candidate models. In recent years, a new class of model averaging estimators has emerged in the literature, suggesting to combine models such that the squared risk, or other risk functions, are minimized. We argue that, contrary to popular belief, these estimators do not necessarily address the challenges induced by model selection uncertainty, but should be regarded as attractive complements for the machine learning and forecasting literature, as well as tools to identify causal parameters. We illustrate our point by means of several targeted simulation studies.

Suggested Citation

  • Michael Schomaker & Christian Heumann, 2020. "When and when not to use optimal model averaging," Statistical Papers, Springer, vol. 61(5), pages 2221-2240, October.
  • Handle: RePEc:spr:stpapr:v:61:y:2020:i:5:d:10.1007_s00362-018-1048-3
    DOI: 10.1007/s00362-018-1048-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s00362-018-1048-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s00362-018-1048-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gao, Yan & Zhang, Xinyu & Wang, Shouyang & Zou, Guohua, 2016. "Model averaging based on leave-subject-out cross-validation," Journal of Econometrics, Elsevier, vol. 192(1), pages 139-151.
    2. Qingfeng Liu & Ryo Okui & Arihiro Yoshimura, 2016. "Generalized Least Squares Model Averaging," Econometric Reviews, Taylor & Francis Journals, vol. 35(8-10), pages 1692-1752, December.
    3. Schomaker, Michael & Heumann, Christian, 2014. "Model selection and model averaging after multiple imputation," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 758-770.
    4. Paul Kabaila & A. H. Welsh & Waruni Abeysekera, 2016. "Model-Averaged Confidence Intervals," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(1), pages 35-48, March.
    5. Zhang, Xinyu & Liu, Chu-An, 2019. "Inference After Model Averaging In Linear Regression Models," Econometric Theory, Cambridge University Press, vol. 35(4), pages 816-841, August.
    6. Heejung Bang & James M. Robins, 2005. "Doubly Robust Estimation in Missing Data and Causal Inference Models," Biometrics, The International Biometric Society, vol. 61(4), pages 962-973, December.
    7. Turek, Daniel & Fletcher, David, 2012. "Model-averaged Wald confidence intervals," Computational Statistics & Data Analysis, Elsevier, vol. 56(9), pages 2809-2815.
    8. Fletcher, David & Dillingham, Peter W., 2011. "Model-averaged confidence intervals for factorial experiments," Computational Statistics & Data Analysis, Elsevier, vol. 55(11), pages 3041-3048, November.
    9. Yan, Jun, 2007. "Enjoy the Joy of Copulas: With a Package copula," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 21(i04).
    10. Friedman, Jerome H. & Hastie, Trevor & Tibshirani, Rob, 2010. "Regularization Paths for Generalized Linear Models via Coordinate Descent," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 33(i01).
    11. Leeb, Hannes & Pötscher, Benedikt M., 2005. "Model Selection And Inference: Facts And Fiction," Econometric Theory, Cambridge University Press, vol. 21(1), pages 21-59, February.
    12. van der Laan Mark J. & Petersen Maya L, 2007. "Statistical Learning of Origin-Specific Statically Optimal Individualized Treatment Rules," The International Journal of Biostatistics, De Gruyter, vol. 3(1), pages 1-38, March.
    13. Chris Chatfield, 1995. "Model Uncertainty, Data Mining and Statistical Inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 158(3), pages 419-444, May.
    14. Hansen, Bruce E., 2008. "Least-squares forecast averaging," Journal of Econometrics, Elsevier, vol. 146(2), pages 342-350, October.
    15. Gruber, Susan & Laan, Mark van der, 2012. "tmle: An R Package for Targeted Maximum Likelihood Estimation," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 51(i13).
    16. Xinyu Zhang & Guohua Zou & Hua Liang, 2014. "Model averaging and weight choice in linear mixed-effects models," Biometrika, Biometrika Trust, vol. 101(1), pages 205-218.
    17. Wan, Alan T.K. & Zhang, Xinyu & Zou, Guohua, 2010. "Least squares model averaging by Mallows criterion," Journal of Econometrics, Elsevier, vol. 156(2), pages 277-283, June.
    18. Xavier Sala-I-Martin & Gernot Doppelhofer & Ronald I. Miller, 2004. "Determinants of Long-Term Growth: A Bayesian Averaging of Classical Estimates (BACE) Approach," American Economic Review, American Economic Association, vol. 94(4), pages 813-835, September.
    19. Hansen, Bruce E. & Racine, Jeffrey S., 2012. "Jackknife model averaging," Journal of Econometrics, Elsevier, vol. 167(1), pages 38-46.
    20. Chu‐An Liu & Biing‐Shen Kuo, 2016. "Model averaging in predictive regressions," Econometrics Journal, Royal Economic Society, vol. 19(2), pages 203-231, June.
    21. Liang, Hua & Zou, Guohua & Wan, Alan T. K. & Zhang, Xinyu, 2011. "Optimal Weight Choice for Frequentist Model Average Estimators," Journal of the American Statistical Association, American Statistical Association, vol. 106(495), pages 1053-1066.
    22. Michael Schomaker, 2012. "Shrinkage averaging estimation," Statistical Papers, Springer, vol. 53(4), pages 1015-1034, November.
    23. Pötscher, Benedikt M., 2006. "The Distribution of Model Averaging Estimators and an Impossibility Result Regarding Its Estimation," MPRA Paper 73, University Library of Munich, Germany, revised Jul 2006.
    24. Cheng, Tzu-Chang F. & Ing, Ching-Kang & Yu, Shu-Hui, 2015. "Toward optimal model averaging in regression models with time series errors," Journal of Econometrics, Elsevier, vol. 189(2), pages 321-334.
    25. Bruce E. Hansen, 2007. "Least Squares Model Averaging," Econometrica, Econometric Society, vol. 75(4), pages 1175-1189, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Schomaker, Michael & Heumann, Christian, 2014. "Model selection and model averaging after multiple imputation," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 758-770.
    2. Rongjie Jiang & Liming Wang & Yang Bai, 2021. "Optimal model averaging estimator for semi-functional partially linear models," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 84(2), pages 167-194, February.
    3. Shaobo Jin & Sebastian Ankargren, 2019. "Frequentist Model Averaging in Structural Equation Modelling," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 84-104, March.
    4. Liu, Chu-An, 2015. "Distribution theory of the least squares averaging estimator," Journal of Econometrics, Elsevier, vol. 186(1), pages 142-159.
    5. Shou-Yung Yin & Chu-An Liu & Chang-Ching Lin, 2021. "Focused Information Criterion and Model Averaging for Large Panels With a Multifactor Error Structure," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 39(1), pages 54-68, January.
    6. Fang, Fang & Li, Jialiang & Xia, Xiaochao, 2022. "Semiparametric model averaging prediction for dichotomous response," Journal of Econometrics, Elsevier, vol. 229(2), pages 219-245.
    7. Sun, Yuying & Hong, Yongmiao & Wang, Shouyang & Zhang, Xinyu, 2023. "Penalized time-varying model averaging," Journal of Econometrics, Elsevier, vol. 235(2), pages 1355-1377.
    8. Liao, Jun & Zou, Guohua & Gao, Yan & Zhang, Xinyu, 2021. "Model averaging prediction for time series models with a diverging number of parameters," Journal of Econometrics, Elsevier, vol. 223(1), pages 190-221.
    9. Liao, Jun & Zong, Xianpeng & Zhang, Xinyu & Zou, Guohua, 2019. "Model averaging based on leave-subject-out cross-validation for vector autoregressions," Journal of Econometrics, Elsevier, vol. 209(1), pages 35-60.
    10. Liao, Jun & Zou, Guohua, 2020. "Corrected Mallows criterion for model averaging," Computational Statistics & Data Analysis, Elsevier, vol. 144(C).
    11. Yan Gao & Xinyu Zhang & Shouyang Wang & Terence Tai-leung Chong & Guohua Zou, 2019. "Frequentist model averaging for threshold models," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 71(2), pages 275-306, April.
    12. Chu-An Liu & Biing-Shen Kuo & Wen-Jen Tsay, 2017. "Autoregressive Spectral Averaging Estimator," IEAS Working Paper : academic research 17-A013, Institute of Economics, Academia Sinica, Taipei, Taiwan.
    13. Shangwei Zhao & Jun Liao & Dalei Yu, 2020. "Model averaging estimator in ridge regression and its large sample properties," Statistical Papers, Springer, vol. 61(4), pages 1719-1739, August.
    14. Peng, Jingfu & Yang, Yuhong, 2022. "On improvability of model selection by model averaging," Journal of Econometrics, Elsevier, vol. 229(2), pages 246-262.
    15. Hui Xiao & Yiguo Sun, 2019. "On Tuning Parameter Selection in Model Selection and Model Averaging: A Monte Carlo Study," JRFM, MDPI, vol. 12(3), pages 1-16, June.
    16. Ryan Greenaway-McGrevy & Kade Sorensen, 2021. "A spatial model averaging approach to measuring house prices," Journal of Spatial Econometrics, Springer, vol. 2(1), pages 1-32, December.
    17. Lu, Xun & Su, Liangjun, 2015. "Jackknife model averaging for quantile regressions," Journal of Econometrics, Elsevier, vol. 188(1), pages 40-58.
    18. Sun, Yuying & Hong, Yongmiao & Lee, Tae-Hwy & Wang, Shouyang & Zhang, Xinyu, 2021. "Time-varying model averaging," Journal of Econometrics, Elsevier, vol. 222(2), pages 974-992.
    19. Liao, Jun & Wan, Alan T.K. & He, Shuyuan & Zou, Guohua, 2022. "Optimal model averaging for multivariate regression models," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    20. Jingwen Tu & Hu Yang & Chaohui Guo & Jing Lv, 2021. "Model averaging marginal regression for high dimensional conditional quantile prediction," Statistical Papers, Springer, vol. 62(6), pages 2661-2689, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:stpapr:v:61:y:2020:i:5:d:10.1007_s00362-018-1048-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.