IDEAS home Printed from https://ideas.repec.org/a/aea/jecper/v28y2014i2p29-50.html
   My bibliography  Save this article

High-Dimensional Methods and Inference on Structural and Treatment Effects

Author

Listed:
  • Alexandre Belloni
  • Victor Chernozhukov
  • Christian Hansen

Abstract

Data with a large number of variables relative to the sample size?"high-dimensional data"?are readily available and increasingly common in empirical economics. High-dimensional data arise through a combination of two phenomena. First, the data may be inherently high dimensional in that many different characteristics per observation are available. For example, the US Census collects information on hundreds of individual characteristics and scanner datasets record transaction-level data for households across a wide range of products. Second, even when the number of available variables is relatively small, researchers rarely know the exact functional form with which the small number of variables enter the model of interest. Researchers are thus faced with a large set of potential variables formed by different ways of interacting and transforming the underlying variables. This paper provides an overview of how innovations in "data mining"? can be adapted and modified to provide high-quality inference about model parameters. Note that we use the term "data mining" in a modern sense which denotes a principled search for "true" predictive power that guards against false discovery and overfitting, does not erroneously equate in-sample fit to out-of-sample predictive ability, and accurately accounts for using the same data to examine many different hypotheses or models.

Suggested Citation

  • Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2014. "High-Dimensional Methods and Inference on Structural and Treatment Effects," Journal of Economic Perspectives, American Economic Association, vol. 28(2), pages 29-50, Spring.
  • Handle: RePEc:aea:jecper:v:28:y:2014:i:2:p:29-50
    Note: DOI: 10.1257/jep.28.2.29
    as

    Download full text from publisher

    File URL: http://www.aeaweb.org/articles.php?doi=10.1257/jep.28.2.29
    Download Restriction: no

    File URL: http://www.aeaweb.org/jep/data/2802/2802-0029_data.zip
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016057, May.
    2. Chen, Xiaohong & Pouzo, Demian, 2009. "Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals," Journal of Econometrics, Elsevier, vol. 152(1), pages 46-60, September.
    3. John J. Donohue III & Steven D. Levitt, 2008. "Measurement Error, Legalized Abortion, and the Decline in Crime: A Response to Foote and Goetz," The Quarterly Journal of Economics, Oxford University Press, vol. 123(1), pages 425-440.
    4. Eric Gautier & Alexandre Tsybakov, 2011. "High-Dimensional Instrumental Variables Regression and Confidence Sets," Working Papers 2011-13, Center for Research in Economics and Statistics.
    5. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    6. Robinson, Peter M, 1988. "Root- N-Consistent Semiparametric Regression," Econometrica, Econometric Society, vol. 56(4), pages 931-954, July.
    7. Jerry A. Hausman & Whitney K. Newey & Tiemen Woutersen & John C. Chao & Norman R. Swanson, 2012. "Instrumental variable estimation with heteroskedasticity and many instruments," Quantitative Economics, Econometric Society, vol. 3(2), pages 211-255, July.
    8. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107674165, May.
    9. Leeb, Hannes & Pötscher, Benedikt M., 2008. "Can One Estimate The Unconditional Distribution Of Post-Model-Selection Estimators?," Econometric Theory, Cambridge University Press, vol. 24(2), pages 338-376, April.
    10. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016064, May.
    11. Ng Serena & Bai Jushan, 2009. "Selecting Instrumental Variables in a Data Rich Environment," Journal of Time Series Econometrics, De Gruyter, vol. 1(1), pages 1-34, April.
    12. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107627314, May.
    13. Christopher L. Foote & Christopher F. Goetz, 2008. "The Impact of Legalized Abortion on Crime: Comment," The Quarterly Journal of Economics, Oxford University Press, vol. 123(1), pages 407-423.
    14. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107016040, May.
    15. Newey, Whitney K., 1997. "Convergence rates and asymptotic normality for series estimators," Journal of Econometrics, Elsevier, vol. 79(1), pages 147-168, July.
    16. Acemoglu,Daron & Arellano,Manuel & Dekel,Eddie (ed.), 2013. "Advances in Economics and Econometrics," Cambridge Books, Cambridge University Press, number 9781107638105, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daron Acemoglu & Gino Gancia & Fabrizio Zilibotti, 2015. "Offshoring and Directed Technical Change," American Economic Journal: Macroeconomics, American Economic Association, vol. 7(3), pages 84-122, July.
    2. Carlos A. Manzanares & Ying Jiang & Patrick Bajari, 2015. "Improving Policy Functions in High-Dimensional Dynamic Games," NBER Working Papers 21124, National Bureau of Economic Research, Inc.
    3. Antonio Romero-Medina & Matteo Triossi, 2017. "Take-it-or-leave-it contracts in many-to-many matching markets," Documentos de Trabajo 328, Centro de Economía Aplicada, Universidad de Chile.
    4. David Strömberg, 2015. "Media and Politics," Annual Review of Economics, Annual Reviews, vol. 7(1), pages 173-205, August.
    5. Philippe Aghion & Peter Howitt & Susanne Prantl, 2015. "Patent rights, product market reforms, and innovation," Journal of Economic Growth, Springer, vol. 20(3), pages 223-262, September.
    6. Pierpaolo Battigalli & Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci, 2017. "Mixed extensions of decision problems under uncertainty," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 63(4), pages 827-866, April.
    7. Pierpaolo Battigalli & Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci, 2015. "Self-Confirming Equilibrium and Model Uncertainty," American Economic Review, American Economic Association, vol. 105(2), pages 646-677, February.
    8. Simone Cerreia-Vioglio & Fabio Maccheroni & Massimo Marinacci & Aldo Rustichini, 2012. "Niveloids and Their Extensions:Risk Measures on Small Domains," Working Papers 458, IGIER (Innocenzo Gasparini Institute for Economic Research), Bocconi University.
    9. Cao, Shutao, 2017. "Accounting for productivity growth in a small open economy: Sector-specific technological change and relative prices of trade," Working Paper Series 6203, Victoria University of Wellington, School of Economics and Finance.
    10. Massimo Marinacci, 2015. "Model Uncertainty," Journal of the European Economic Association, European Economic Association, vol. 13(6), pages 1022-1100, December.
    11. Song, Suyong, 2015. "Semiparametric estimation of models with conditional moment restrictions in the presence of nonclassical measurement errors," Journal of Econometrics, Elsevier, vol. 185(1), pages 95-109.
    12. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    13. Victor Chernozhukov & Whitney Newey & Rahul Singh & Vasilis Syrgkanis, 2020. "Adversarial Estimation of Riesz Representers," Papers 2101.00009, arXiv.org.
    14. Nishanth Dikkala & Greg Lewis & Lester Mackey & Vasilis Syrgkanis, 2020. "Minimax Estimation of Conditional Moment Models," Papers 2006.07201, arXiv.org.
    15. Escanciano, Juan Carlos & Li, Wei, 2021. "Optimal Linear Instrumental Variables Approximations," Journal of Econometrics, Elsevier, vol. 221(1), pages 223-246.
    16. Yu Zhu, 2020. "Inference in nonparametric/semiparametric moment equality models with shape restrictions," Quantitative Economics, Econometric Society, vol. 11(2), pages 609-636, May.
    17. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2011. "Inference on Treatment Effects After Selection Amongst High-Dimensional Controls," Papers 1201.0224, arXiv.org, revised May 2012.
    18. Xiaohong Chen & Demian Pouzo, 2015. "Sieve Wald and QLR Inferences on Semi/Nonparametric Conditional Moment Models," Econometrica, Econometric Society, vol. 83(3), pages 1013-1079, May.
    19. Chaohua Dong & Jiti Gao & Oliver Linton, 2017. "High dimensional semiparametric moment restriction models," Monash Econometrics and Business Statistics Working Papers 17/17, Monash University, Department of Econometrics and Business Statistics.
    20. Sun, Yanqing & Zhang, Yuanqing & Huang, Jianhua Z., 2019. "Estimation of a semiparametric varying-coefficient mixed regressive spatial autoregressive model," Econometrics and Statistics, Elsevier, vol. 9(C), pages 140-155.

    More about this item

    JEL classification:

    • C55 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Large Data Sets: Modeling and Analysis

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jecper:v:28:y:2014:i:2:p:29-50. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.