IDEAS home Printed from https://ideas.repec.org/a/tpr/restat/v101y2019i5p743-762.html
   My bibliography  Save this article

Choosing Among Regularized Estimators in Empirical Economics: The Risk of Machine Learning

Author

Listed:
  • Alberto Abadie

    (MIT)

  • Maximilian Kasy

    (Harvard University)

Abstract

Many settings in empirical economics involve estimation of a large number of parameters. In such settings, methods that combine regularized estimation and data-driven choices of regularization parameters are useful. We provide guidance to applied researchers on the choice between regularized estimators and data-driven selection of regularization parameters. We characterize the risk and relative performance of regularized estimators as a function of the data-generating process and show that data-driven choices of regularization parameters yield estimators with risk uniformly close to the risk attained under the optimal (unfeasible) choice of regularization parameters. We illustrate using examples from empirical economics.

Suggested Citation

  • Alberto Abadie & Maximilian Kasy, 2019. "Choosing Among Regularized Estimators in Empirical Economics: The Risk of Machine Learning," The Review of Economics and Statistics, MIT Press, vol. 101(5), pages 743-762, December.
    • Handle: RePEc:tpr:restat:v:101:y:2019:i:5:p:743-762
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/rest_a_00812
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Alan B. Krueger, 1997. "Experimental Estimates of Education Production Functions," Working Papers 758, Princeton University, Department of Economics, Industrial Relations Section..
    2. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1163-1228.
    3. Alexandre Belloni & Victor Chernozhukov, 2011. "High Dimensional Sparse Econometric Models: An Introduction," Papers 1106.5242, arXiv.org, revised Sep 2011.
    4. Raj Chetty & Nathaniel Hendren, 2018. "The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(3), pages 1107-1162.
    5. David S. Abrams & Marianne Bertrand & Sendhil Mullainathan, 2012. "Do Judges Vary in Their Treatment of Race?," The Journal of Legal Studies, University of Chicago Press, vol. 41(2), pages 347-383.
    6. Raj Chetty & John N. Friedman & Jonah E. Rockoff, 2014. "Measuring the Impacts of Teachers II: Teacher Value-Added and Student Outcomes in Adulthood," American Economic Review, American Economic Association, vol. 104(9), pages 2633-2679, September.
    7. Roger Koenker & Ivan Mizera, 2014. "Convex Optimization, Shape Constraints, Compound Decisions, and Empirical Bayes Rules," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(506), pages 674-685, June.
    8. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    9. Leeb, Hannes & Pötscher, Benedikt M., 2006. "Performance Limits For Estimators Of The Risk Or Distribution Of Shrinkage-Type Estimators, And Some General Lower Risk-Bound Results," Econometric Theory, Cambridge University Press, vol. 22(1), pages 69-97, February.
    10. James H. Stock & Mark W. Watson, 2012. "Generalized Shrinkage Methods for Forecasting Using Many Predictors," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 30(4), pages 481-493, June.
    11. Fan J. & Li R., 2001. "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1348-1360, December.
    12. Koenker, Roger & Mizera, Ivan, 2014. "Convex Optimization in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 60(i05).
    13. Alan B. Krueger, 1999. "Experimental Estimates of Education Production Functions," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 497-532.
    14. Joshua Angrist & Victor Chernozhukov & Iván Fernández-Val, 2006. "Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure," Econometrica, Econometric Society, vol. 74(2), pages 539-563, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Philippe Goulet Coulombe & Maxime Leroux & Dalibor Stevanovic & Stéphane Surprenant, 2022. "How is machine learning useful for macroeconomic forecasting?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(5), pages 920-964, August.
    2. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg-Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Working Papers 2022-27, Princeton University. Economics Department..
    3. Altındağ, Onur & O'Connell, Stephen D. & Şaşmaz, Aytuğ & Balcıoğlu, Zeynep & Cadoni, Paola & Jerneck, Matilda & Foong, Aimee Kunze, 2021. "Targeting humanitarian aid using administrative data: Model design and validation," Journal of Development Economics, Elsevier, vol. 148(C).
    4. Julien Chevallier & Dominique Guégan & Stéphane Goutte, 2021. "Is It Possible to Forecast the Price of Bitcoin?," Forecasting, MDPI, vol. 3(2), pages 1-44, May.
    5. Tatiana de Macedo Nogueira Lima, 2022. "Documento de Trabalho 03/2022 - Aprendizado de máquina e antitruste," Documentos de Trabalho 2022030, Conselho Administrativo de Defesa Econômica (Cade), Departamento de Estudos Econômicos.
    6. Coraggio, Luca & Pagano, Marco & Scognamiglio, Annalisa & Tåg, Joacim, 2025. "JAQ of all trades: Job mismatch, firm productivity and managerial quality," Journal of Financial Economics, Elsevier, vol. 164(C).
    7. Domenico Giannone & Michele Lenza & Giorgio E. Primiceri, 2021. "Economic Predictions With Big Data: The Illusion of Sparsity," Econometrica, Econometric Society, vol. 89(5), pages 2409-2437, September.
    8. Szendrei, Tibor & Bhattacharjee, Arnab & Schaffer, Mark E, 2024. "Fused LASSO as Non-crossing Quantile Regression," IZA Discussion Papers 17149, Institute of Labor Economics (IZA).
    9. Georges, Christophre & Pereira, Javier, 2021. "Market stability with machine learning agents," Journal of Economic Dynamics and Control, Elsevier, vol. 122(C).
    10. Battiston, Pietro & Gamba, Simona & Santoro, Alessandro, 2024. "Machine learning and the optimization of prediction-based policies," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    11. Jan Niederreiter, 2023. "Broadening Economics in the Era of Artificial Intelligence and Experimental Evidence," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 9(1), pages 265-294, March.
    12. Isaiah Andrews & Jesse M. Shapiro, 2021. "A Model of Scientific Communication," Econometrica, Econometric Society, vol. 89(5), pages 2117-2142, September.
    13. Mike Gilraine & Jiaying Gu & Robert McMillan, 2022. "A Nonparametric Approach for Studying Teacher Impacts," Working Papers tecipa-716, University of Toronto, Department of Economics.
    14. Giuseppe De Luca & Jan Magnus & Franco Peracchi, 2022. "Asymptotic properties of the weighted average least squares (WALS) estimator," Tinbergen Institute Discussion Papers 22-022/III, Tinbergen Institute.
    15. Christopher Walters, 2024. "Empirical Bayes Methods in Labor Economics," RF Berlin - CReAM Discussion Paper Series 2422, Rockwool Foundation Berlin (RF Berlin) - Centre for Research and Analysis of Migration (CReAM).
    16. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    17. Michael Gilraine & Jiaying Gu & Robert McMillan, 2020. "A New Method for Estimating Teacher Value-Added," NBER Working Papers 27094, National Bureau of Economic Research, Inc.
    18. Timothy B. Armstrong & Michal Koles'ar & Mikkel Plagborg-M{o}ller, 2020. "Robust Empirical Bayes Confidence Intervals," Papers 2004.03448, arXiv.org, revised May 2022.
    19. Zhaoxing Gao & Ruey S. Tsay, 2024. "Optimal Bias-Correction and Valid Inference in High-Dimensional Ridge Regression: A Closed-Form Solution," Papers 2405.00424, arXiv.org, revised Jul 2024.
    20. Michael Gilraine & Jiaying Gu & Robert McMillan, 2021. "A Nonparametric Method for Estimating Teacher Value-Added," Working Papers tecipa-689, University of Toronto, Department of Economics.
    21. Akbas, Ozan E. & Betz, Frank & Gattini, Luca, 2023. "Quantifying credit gaps using survey data on discouraged borrowers," EIB Working Papers 2023/06, European Investment Bank (EIB).
    22. Brian Asquith & Judith K. Hellerstein & Mark J. Kutzbach & David Neumark, 2021. "Social capital determinants and labor market networks," Journal of Regional Science, Wiley Blackwell, vol. 61(1), pages 212-260, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Timothy B. Armstrong & Michal Kolesár & Mikkel Plagborg‐Møller, 2022. "Robust Empirical Bayes Confidence Intervals," Econometrica, Econometric Society, vol. 90(6), pages 2567-2602, November.
    3. Valentin Verdier, 2020. "Estimation and Inference for Linear Models with Two-Way Fixed Effects and Sparsely Matched Data," The Review of Economics and Statistics, MIT Press, vol. 102(1), pages 1-16, March.
    4. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    5. Jiaying Gu & Roger Koenker, 2023. "Invidious Comparisons: Ranking and Selection as Compound Decisions," Econometrica, Econometric Society, vol. 91(1), pages 1-41, January.
    6. Michael Geruso & Timothy J. Layton & Jacob Wallace, 2023. "What Difference Does a Health Plan Make? Evidence from Random Plan Assignment in Medicaid," American Economic Journal: Applied Economics, American Economic Association, vol. 15(3), pages 341-379, July.
    7. Stephen Machin & Sandra McNally & Martina Viarengo, 2018. "Changing How Literacy Is Taught: Evidence on Synthetic Phonics," American Economic Journal: Economic Policy, American Economic Association, vol. 10(2), pages 217-241, May.
    8. Thompson, Paul N., 2021. "Is four less than five? Effects of four-day school weeks on student achievement in Oregon," Journal of Public Economics, Elsevier, vol. 193(C).
    9. Karthik Muralidharan & Venkatesh Sundararaman, 2013. "Contract Teachers: Experimental Evidence from India," NBER Working Papers 19440, National Bureau of Economic Research, Inc.
    10. Johannes S. Kunz & Kevin E. Staub & Rainer Winkelmann, 2021. "Predicting individual effects in fixed effects panel probit models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1109-1145, July.
    11. Pedro Carneiro & Oswald Koussihouèdé & Nathalie Lahire & Costas Meghir & Corina Mommaerts, 2020. "School Grants and Education Quality: Experimental Evidence from Senegal," Economica, London School of Economics and Political Science, vol. 87(345), pages 28-51, January.
    12. Masi, Barbara, 2018. "A ticket to ride: The unintended consequences of school transport subsidies," Economics of Education Review, Elsevier, vol. 63(C), pages 100-115.
    13. Stephen Machin & Sandra McNally & Martina Viarengo, 2016. ""Teaching to teach" literacy," CEP Discussion Papers dp1425, Centre for Economic Performance, LSE.
    14. Kunz, J.S.; & Staub, K.E.; & Winkelmann, R.;, 2018. "Predicting fixed effects in panel probit models," Health, Econometrics and Data Group (HEDG) Working Papers 18/23, HEDG, c/o Department of Economics, University of York.
    15. Paredes, Valentina & Pino, Francisco J. & Díaz, David, 2024. "Does facial structure explain differences in student evaluations of teaching? The role of fWHR as a proxy for perceived dominance," Economics & Human Biology, Elsevier, vol. 54(C).
    16. Bladimir Carrillo, 2020. "Present Bias and Underinvestment in Education? Long-Run Effects of Childhood Exposure to Booms in Colombia," Journal of Labor Economics, University of Chicago Press, vol. 38(4), pages 1127-1265.
    17. Raffaella Giacomini & Sokbae Lee & Silvia Sarpietro, 2023. "A Robust Method for Microforecasting and Estimation of Random Effects," Working Paper Series WP 2023-26, Federal Reserve Bank of Chicago.
    18. Carrell, Scott E. & Hoekstra, Mark, 2014. "Are school counselors an effective education input?," Economics Letters, Elsevier, vol. 125(1), pages 66-69.
    19. Jo Blanden & Matthias Doepke & Jan Stuhler, 2022. "Education inequality," CEP Discussion Papers dp1849, Centre for Economic Performance, LSE.
    20. Jennifer A. Heissel & Claudia Persico & David Simon, 2022. "Does Pollution Drive Achievement? The Effect of Traffic Pollution on Academic Performance," Journal of Human Resources, University of Wisconsin Press, vol. 57(3), pages 747-776.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:restat:v:101:y:2019:i:5:p:743-762. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: The MIT Press (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.