IDEAS home Printed from https://ideas.repec.org/a/aea/jecper/v28y2014i2p29-50.html
   My bibliography  Save this article

High-Dimensional Methods and Inference on Structural and Treatment Effects

Author

Listed:
  • Alexandre Belloni
  • Victor Chernozhukov
  • Christian Hansen

Abstract

Data with a large number of variables relative to the sample size?"high-dimensional data"?are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that many different characteristics per observation are available. For example, the US Census collects information on hundreds of individual characteristics and scanner datasets record transaction-level data for households across a wide range of products. Second, even when the number of available variables is relatively small, researchers rarely know the exact functional form with which the small number of variables enter the model of interest. Researchers are thus faced with a large set of potential variables formed by different ways of interacting and transforming the underlying variables. This paper provides an overview of how innovations in "data mining"? can be adapted and modified to provide high-quality inference about model parameters. Note that we use the term "data mining" in a modern sense which denotes a principled search for "true" predictive power that guards against false discovery and overfitting, does not erroneously equate in-sample fit to out-of-sample predictive ability, and accurately accounts for using the same data to examine many different hypotheses or models.

Suggested Citation

  • Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
  • Handle: RePEc:aea:jecper:v:28:y:2014:i:2:p:29-50
    Note: DOI: 10.1257/jep.28.2.29
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/jep.28.2.29
    Download Restriction: no

    File URL: http://www.aeaweb.org/jep/data/2802/2802-0029_data.zip
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016057, October.
    2. Chen, Xiaohong & Pouzo, Demian, 2009. "Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals," Journal of Econometrics, Elsevier, vol. 152(1), pages 46-60, September.
    3. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107627314, October.
    4. John J. Donohue III & Steven D. Levitt, 2008. "Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz," The Quarterly Journal of Economics, Oxford University Press, vol. 123(1), pages 425-440.
    5. Christopher L. Foote & Christopher F. Goetz, 2008. "The Impact of Legalized Abortion on Crime: Comment," The Quarterly Journal of Economics, Oxford University Press, vol. 123(1), pages 407-423.
    6. Eric Gautier & Alexandre Tsybakov, 2011. "High-Dimensional Instrumental Variables Regression and Confidence Sets," Working Papers 2011-13, Center for Research in Economics and Statistics.
    7. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016040, October.
    8. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    9. Robinson, Peter M, 1988. "Root- N-Consistent Semiparametric Regression," Econometrica, Econometric Society, vol. 56(4), pages 931-954, July.
    10. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    11. Jerry A. Hausman & Whitney K. Newey & Tiemen Woutersen & John C. Chao & Norman R. Swanson, 2012. "Instrumental variable estimation with heteroskedasticity and many instruments," Quantitative Economics, Econometric Society, vol. 3(2), pages 211-255, July.
    12. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107638105, October.
    13. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107674165, October.
    14. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016064, October.
    15. Ng Serena & Bai Jushan, 2009. "Selecting Instrumental Variables in a Data Rich Environment," Journal of Time Series Econometrics, De Gruyter, vol. 1(1), pages 1-34, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Green, Gareth & Richards, Timothy, 2016. "Interpreting Results of Demand Estimation from Machine Learning Models," 2016 Annual Meeting, July 31-August 2, 2016, Boston, Massachusetts 236147, Agricultural and Applied Economics Association.
    2. Carneiro, Pedro & Lee, Sokbae & Wilhelm, Daniel, 2016. "Optimal Data Collection for Randomized Control Trials," IZA Discussion Papers 9908, Institute for the Study of Labor (IZA).
    3. Michael Knaus & Michael Lechner & Anthony Strittmatter, 2017. "Heterogeneous Employment Effects of Job Search Programmes: A Machine Learning Approach," Papers 1709.10279, arXiv.org, revised May 2018.
    4. Anil Kumar, 2018. "Do Restrictions on Home Equity Extraction Contribute to Lower Mortgage Defaults? Evidence from a Policy Discontinuity at the Texas Border," American Economic Journal: Economic Policy, American Economic Association, vol. 10(1), pages 268-297, February.
    5. Ingrid Gould Ellen & Keren Mertens Horn & Davin Reed, 2017. "Has Falling Crime Invited Gentrification?," Working Papers 17-27, Center for Economic Studies, U.S. Census Bureau.
    6. repec:nbr:nberch:14010 is not listed on IDEAS
    7. Panagopoulos, Orestis P. & Pappu, Vijay & Xanthopoulos, Petros & Pardalos, Panos M., 2016. "Constrained subspace classifier for high dimensional datasets," Omega, Elsevier, vol. 59(PA), pages 40-46.
    8. Emanuele Ciani & Guido de Blasio, 2015. "European structural funds during the crisis: evidence from Southern Italy," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-31, December.
    9. Joyce P Jacobsen & Laurence M Levin & Zachary Tausanovitch, 2016. "Comparing Standard Regression Modeling to Ensemble Modeling: How Data Mining Software Can Improve Economists’ Predictions," Eastern Economic Journal, Palgrave Macmillan;Eastern Economic Association, vol. 42(3), pages 387-398, June.
    10. Achim Ahrens, 2015. "Civil conflicts in Africa: Climate, economic shocks, nighttime lights and spill-over effects," SEEC Discussion Papers 1501, Spatial Economics and Econometrics Centre, Heriot Watt University.
    11. Patrick Bajari & Denis Nekipelov & Stephen P. Ryan & Miaoyu Yang, 2015. "Machine Learning Methods for Demand Estimation," American Economic Review, American Economic Association, vol. 105(5), pages 481-485, May.
    12. repec:nbr:nberch:14009 is not listed on IDEAS
    13. Jon Kleinberg & Himabindu Lakkaraju & Jure Leskovec & Jens Ludwig & Sendhil Mullainathan, 2017. "Human Decisions and Machine Predictions," NBER Working Papers 23180, National Bureau of Economic Research, Inc.
    14. Pieter Bakx & Bram Wouterse & Eddy (E.K.A.) van Doorslaer & Albert Wong, 2018. "Better off at home? Effects of a nursing home admission on costs, hospitalizations and survival," Tinbergen Institute Discussion Papers 18-060/V, Tinbergen Institute.
    15. Serena Ng, 2017. "Opportunities and Challenges: Lessons from Analyzing Terabytes of Scanner Data," NBER Working Papers 23673, National Bureau of Economic Research, Inc.
    16. Michael C. Knaus, 2018. "A Double Machine Learning Approach to Estimate the Effects of Musical Practice on Student's Skills," Papers 1805.10300, arXiv.org.
    17. Damian Kozbur, 2017. "Testing-Based Forward Model Selection," American Economic Review, American Economic Association, vol. 107(5), pages 266-269, May.
    18. Krumer, Alex & Lechner, Michael, 2016. "Midweek Effect on Performance: Evidence from the German Soccer Bundesliga," Economics Working Paper Series 1609, University of St. Gallen, School of Economics and Political Science.
    19. repec:eee:econom:v:200:y:2017:i:1:p:59-78 is not listed on IDEAS
    20. repec:eee:jfpoli:v:76:y:2018:i:c:p:70-80 is not listed on IDEAS
    21. Marcel Fafchamps & Julien Labonne, 2016. "Using Split Samples to Improve Inference about Causal Effects," NBER Working Papers 21842, National Bureau of Economic Research, Inc.
    22. Ajay Agrawal & Joshua S. Gans & Avi Goldfarb, 2018. "Prediction, Judgment, and Complexity: A Theory of Decision Making and Artificial Intelligence," NBER Chapters,in: The Economics of Artificial Intelligence: An Agenda National Bureau of Economic Research, Inc.
    23. Jan Bruha & Michal Hlavacek & Lubos Komarek, 2017. "House Prices and Household Consumption: The Case of the Czech Republic," Working Papers 2017/11, Czech National Bank, Research Department.
    24. Marta Auricchio & Emanuele Ciani & Alberto Dalmazzo & Guido de Blasio, 2017. "The consequences of public employment: evidence from Italian municipalities," Temi di discussione (Economic working papers) 1125, Bank of Italy, Economic Research and International Relations Area.

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jecper:v:28:y:2014:i:2:p:29-50. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Michael P. Albert). General contact details of provider: http://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.