IDEAS home Printed from https://ideas.repec.org/a/tpr/restat/v101y2019i5p743-762.html

Choosing Among Regularized Estimators in Empirical Economics: The Risk of Machine Learning

Author

Listed:
  • Alberto Abadie

    (MIT)

  • Maximilian Kasy

    (Harvard University)

Abstract

Many settings in empirical economics involve estimation of a large number of parameters. In such settings, methods that combine regularized estimation and data-driven choices of regularization parameters are useful. We provide guidance to applied researchers on the choice between regularized estimators and data-driven selection of regularization parameters. We characterize the risk and relative performance of regularized estimators as a function of the data-generating process and show that data-driven choices of regularization parameters yield estimators with risk uniformly close to the risk attained under the optimal (unfeasible) choice of regularization parameters. We illustrate using examples from empirical economics.

Suggested Citation

  • Alberto Abadie & Maximilian Kasy, 2019. "Choosing Among Regularized Estimators in Empirical Economics: The Risk of Machine Learning," The Review of Economics and Statistics, MIT Press, vol. 101(5), pages 743-762, December.
    • Handle: RePEc:tpr:restat:v:101:y:2019:i:5:p:743-762
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/rest_a_00812
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alexandre Belloni & Victor Chernozhukov, 2011. "High Dimensional Sparse Econometric Models: An Introduction," Papers 1106.5242, arXiv.org, revised Sep 2011.
    2. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    3. Roger Koenker & Ivan Mizera, 2014. "Convex Optimization, Shape Constraints, Compound Decisions, and Empirical Bayes Rules," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 674-685, June.
    4. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    5. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    6. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    7. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    8. Joshua Angrist & Victor Chernozhukov & Iván Fernández-Val, 2006. "Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure," Econometrica, Econometric Society, vol. 74(2), pages 539-563, March.
    9. Leeb, Hannes & Pötscher, Benedikt M., 2008. "CORRIGENDUM: Correction to “Performance Limits for Estimators of the Risk or Distribution of Shrinkage-Type Estimators, and Some General Lower Risk-Bound Results”," Econometric Theory, Cambridge University Press, vol. 24(2), pages 581-583, April.
    10. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    11. David S. Abrams & Marianne Bertrand & Sendhil Mullainathan, 2012. "Do Judges Vary in Their Treatment of Race?," The Journal of Legal Studies, University of Chicago Press, vol. 41(2), pages 347-383.
    12. Koenker, Roger & Mizera, Ivan, 2014. "Convex Optimization in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i05).
    13. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    2. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    3. Hirano, Keisuke & Porter, Jack R., 2020. "Asymptotic analysis of statistical decision rules in econometrics," Handbook of Econometrics, in: Steven N. Durlauf & Lars Peter Hansen & James J. Heckman & Rosa L. Matzkin (ed.), Handbook of Econometrics, edition 1, volume 7, chapter 0, pages 283-354, Elsevier.
    4. Giuseppe De Luca & Jan R. Magnus & Franco Peracchi, 2022. "Asymptotic properties of the weighted-average least squares (WALS) estimator," EIEF Working Papers Series 2203, Einaudi Institute for Economics and Finance (EIEF), revised Mar 2022.
    5. Zhaoxing Gao & Ruey S. Tsay, 2024. "Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution," Papers 2405.00424, arXiv.org, revised Jul 2024.
    6. Akbas, Ozan E. & Betz, Frank & Gattini, Luca, 2026. "Quantifying credit gaps using survey data on discouraged borrowers," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 107(C).
    7. Isaiah Andrews & Jesse M. Shapiro, 2021. "A Model of Scientific Communication," Econometrica, Econometric Society, vol. 89(5), pages 2117-2142, September.
    8. Julien Chevallier & Dominique Guégan & Stéphane Goutte, 2021. "Is It Possible to Forecast the Price of Bitcoin?," Forecasting, MDPI, vol. 3(2), pages 1-44, May.
    9. Coraggio, Luca & Pagano, Marco & Scognamiglio, Annalisa & Tåg, Joacim, 2025. "JAQ of all trades: Job mismatch, firm productivity and managerial quality," Journal of Financial Economics, Elsevier, vol. 164(C).
    10. Georges, Christophre & Pereira, Javier, 2021. "Market stability with machine learning agents," Journal of Economic Dynamics and Control, Elsevier, vol. 122(C).
    11. Walters, Christopher, 2024. "Empirical Bayes methods in labor economics," Handbook of Labor Economics,, Elsevier.
    12. Battiston, Pietro & Gamba, Simona & Santoro, Alessandro, 2024. "Machine learning and the optimization of prediction-based policies," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    13. Jan Niederreiter, 2023. "Broadening Economics in the Era of Artificial Intelligence and Experimental Evidence," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 9(1), pages 265-294, March.
    14. Juan C. Yamin, 2025. "Poverty Targeting with Imperfect Information," Papers 2506.18188, arXiv.org, revised May 2026.
    15. Timothy B. Armstrong & Michal Koles'ar & Mikkel Plagborg-M{o}ller, 2020. "Robust Empirical Bayes Confidence Intervals," Papers 2004.03448, arXiv.org, revised May 2022.
    16. Mike Gilraine & Jiaying Gu & Robert McMillan, 2022. "A Nonparametric Approach for Studying Teacher Impacts," Working Papers tecipa-716, University of Toronto, Department of Economics.
    17. Christopher R. Walters, 2024. "Empirical Bayes Methods in Labor Economics," NBER Working Papers 33091, National Bureau of Economic Research, Inc.
    18. Michael Gilraine & Jiaying Gu & Robert McMillan, 2020. "A New Method for Estimating Teacher Value-Added," NBER Working Papers 27094, National Bureau of Economic Research, Inc.
    19. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    20. Michael Gilraine & Jiaying Gu & Robert McMillan, 2021. "A Nonparametric Method for Estimating Teacher Value-Added," Working Papers tecipa-689, University of Toronto, Department of Economics.
    21. Soonwoo Kwon, 2023. "Optimal Shrinkage Estimation of Fixed Effects in Linear Panel Data Models," Papers 2308.12485, arXiv.org, revised Sep 2025.
    22. Altındağ, Onur & O'Connell, Stephen D. & Şaşmaz, Aytuğ & Balcıoğlu, Zeynep & Cadoni, Paola & Jerneck, Matilda & Foong, Aimee Kunze, 2021. "Targeting humanitarian aid using administrative data: Model design and validation," Journal of Development Economics, Elsevier, vol. 148(C).
    23. Tatiana de Macedo Nogueira Lima, 2022. "Documento de Trabalho 03/2022 - Aprendizado de máquina e antitruste," Documentos de Trabalho 2022030, Conselho Administrativo de Defesa Econômica (Cade), Departamento de Estudos Econômicos.
    24. Tibor Szendrei & Arnab Bhattacharjee & Mark E. Schaffer, 2024. "Fused LASSO as Non-Crossing Quantile Regression," Papers 2403.14036, arXiv.org, revised Apr 2025.
    25. Brian Asquith & Judith K. Hellerstein & Mark J. Kutzbach & David Neumark, 2021. "Social capital determinants and labor market networks," Journal of Regional Science, Wiley Blackwell, vol. 61(1), pages 212-260, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Walters, Christopher, 2024. "Empirical Bayes methods in labor economics," Handbook of Labor Economics,, Elsevier.
    2. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    3. Jiaying Gu & Roger Koenker, 2023. "Invidious Comparisons: Ranking and Selection as Compound Decisions," Econometrica, Econometric Society, vol. 91(1), pages 1-41, January.
    4. Zhiheng You, 2026. "Using Prior Studies to Design Experiments: An Empirical Bayes Approach," Papers 2602.20581, arXiv.org.
    5. Michael Geruso & Timothy J. Layton & Jacob Wallace, 2023. "What Difference Does a Health Plan Make? Evidence from Random Plan Assignment in Medicaid," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 341-379, July.
    6. Valentin Verdier, 2020. "Estimation and Inference for Linear Models with Two-Way Fixed Effects and Sparsely Matched Data," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 1-16, March.
    7. Eric A. Hanushek & Babs Jacobs & Guido Schwerdt & Rolf van der Velden & Stan Vermeulen & Simon Wiederhold, 2021. "Where Do STEM Graduates Stem From? The Intergenerational Transmission of Comparative Skill Advantages," CESifo Working Paper Series 9388, CESifo.
    8. Mookerjee, Sulagna & Slichter, David, 2023. "Test scores, schools, and the geography of economic opportunity," Journal of Urban Economics, Elsevier, vol. 137(C).
    9. Johannes S. Kunz & Kevin E. Staub & Rainer Winkelmann, 2021. "Predicting individual effects in fixed effects panel probit models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1109-1145, July.
    10. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2011. "Inference for high-dimensional sparse econometric models," CeMMAP working papers CWP41/11, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. Fox, Jeremy T. & Kim, Kyoo il & Yang, Chenyu, 2016. "A simple nonparametric approach to estimating the distribution of random coefficients in structural models," Journal of Econometrics, Elsevier, vol. 195(2), pages 236-254.
    12. Kunz, J.S.; & Staub, K.E.; & Winkelmann, R.;, 2018. "Predicting fixed effects in panel probit models," Health, Econometrics and Data Group (HEDG) Working Papers 18/23, HEDG, c/o Department of Economics, University of York.
    13. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    14. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-Dimensional Econometrics and Regularized GMM," Papers 1806.01888, arXiv.org, revised Jun 2018.
    15. Rosenqvist, Olof & Sauermann, Jan, 2025. "The effects of increasing compensatory resource allocation on student achievement," Working Paper Series 2025:15, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    16. Christos Genakos & Eleni Kyrkopoulou & Elias Papaioannou, 2026. "An Olympic opportunity," CEP Discussion Papers dp2175, Centre for Economic Performance, LSE.
    17. Raffaella Giacomini & Sokbae Lee & Silvia Sarpietro, 2023. "Individual Shrinkage for Random Effects," Papers 2308.01596, arXiv.org, revised May 2026.
    18. Simon Fan & Yu Pang & Pierre Pestieau, 2023. "Nature versus Nurture in Social Mobility Under Private and Public Education Systems," Public Finance Review, , vol. 51(1), pages 132-167, January.
    19. Patrick Kline & Christopher Walters, 2021. "Reasonable Doubt: Experimental Detection of Job‐Level Employment Discrimination," Econometrica, Econometric Society, vol. 89(2), pages 765-792, March.
    20. Lleras-Muney, Adriana & Price, Joseph & Yue, Dahai, 2022. "The association between educational attainment and longevity using individual-level data from the 1940 census," Journal of Health Economics, Elsevier, vol. 84(C).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:restat:v:101:y:2019:i:5:p:743-762. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: The MIT Press (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.