IDEAS home Printed from https://ideas.repec.org/p/feb/natura/00763.html
   My bibliography  Save this paper

Using Machine Learning for Efficient Flexible Regression Adjustment in Economic Experiments

Author

Listed:
  • John List
  • Ian Muir
  • Gregory Sun

Abstract

This study investigates how to use regression adjustment to reduce variance in experimental data. We show that the estimators recommended in the literature satisfy an orthogonality property with respect to the parameters of the adjustment. This observation greatly simplifies the derivation of the asymptotic variance of these estimators and allows us to solve for the efficient regression adjustment in a large class of adjustments. Our efficiency results generalize a number of previous results known in the literature. We then discuss how this efficient regression adjustment can be feasibly implemented. We show the practical relevance of our theory in two ways. First, we use our efficiency results to improve common practices currently employed in field experiments. Second, we show how our theory allows researchers to robustly incorporate machine learning techniques into their experimental estimators to minimize variance.

Suggested Citation

  • John List & Ian Muir & Gregory Sun, 2022. "Using Machine Learning for Efficient Flexible Regression Adjustment in Economic Experiments," Natural Field Experiments 00763, The Field Experiments Website.
  • Handle: RePEc:feb:natura:00763
    as

    Download full text from publisher

    File URL: http://s3.amazonaws.com/fieldexperiments-papers2/papers/00763.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Stefano DellaVigna & John A. List & Ulrike Malmendier, 2012. "Testing for Altruism and Social Pressure in Charitable Giving," The Quarterly Journal of Economics, Oxford University Press, vol. 127(1), pages 1-56.
    2. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    3. Pedro Carneiro & Sokbae Lee & Daniel Wilhelm, 2020. "Optimal data collection for randomized control trials [Microcredit impacts: Evidence from a randomized microcredit program placement experiment by Compartamos Banco]," The Econometrics Journal, Royal Economic Society, vol. 23(1), pages 1-31.
    4. Burlig, Fiona & Preonas, Louis & Woerman, Matt, 2020. "Panel data and experimental design," Journal of Development Economics, Elsevier, vol. 144(C).
    5. Jinyong Hahn, 1998. "On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects," Econometrica, Econometric Society, vol. 66(2), pages 315-332, March.
    6. Akanksha Negi & Jeffrey M. Wooldridge, 2021. "Revisiting regression adjustment in experiments with heterogeneous treatment effects," Econometric Reviews, Taylor & Francis Journals, vol. 40(5), pages 504-534, April.
    7. Newey, Whitney K, 1994. "The Asymptotic Variance of Semiparametric Estimators," Econometrica, Econometric Society, vol. 62(6), pages 1349-1382, November.
    8. Greer K. Gosnell & John A. List & Robert D. Metcalfe, 2020. "The Impact of Management Practices on Employee Productivity: A Field Experiment with Airline Captains," Journal of Political Economy, University of Chicago Press, vol. 128(4), pages 1195-1233.
    9. Akanksha Negi & Jeffrey M. Wooldridge, 2020. "Robust and Efficient Estimation of Potential Outcome Means under Random Assignment," Papers 2010.01800, arXiv.org.
    10. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2011. "Inference on Treatment Effects After Selection Amongst High-Dimensional Controls," Papers 1201.0224, arXiv.org, revised May 2012.
    11. Angrist, J D & Imbens, G W & Krueger, A B, 1999. "Jackknife Instrumental Variables Estimation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(1), pages 57-67, Jan.-Feb..
    12. repec:feb:framed:0087 is not listed on IDEAS
    13. Andrews, Donald W K, 1994. "Asymptotics for Semiparametric Econometric Models via Stochastic Equicontinuity," Econometrica, Econometric Society, vol. 62(1), pages 43-72, January.
    14. Roland G. Fryer Jr & Steven D. Levitt & John A. List & Anya Samek, 2020. "Introducing CogX: A New Preschool Education Program Combining Parent and Child Interventions," NBER Working Papers 27913, National Bureau of Economic Research, Inc.
    15. Goodman-Bacon, Andrew, 2021. "Difference-in-differences with variation in treatment timing," Journal of Econometrics, Elsevier, vol. 225(2), pages 254-277.
    16. Kenneth I. Wolpin & Petra E. Todd, 2006. "Assessing the Impact of a School Subsidy Program in Mexico: Using a Social Experiment to Validate a Dynamic Behavioral Model of Child Schooling and Fertility," American Economic Review, American Economic Association, vol. 96(5), pages 1384-1417, December.
    17. Paul J. Ferraro & Michael K. Price, 2013. "Using Nonpecuniary Strategies to Influence Behavior: Evidence from a Large-Scale Field Experiment," The Review of Economics and Statistics, MIT Press, vol. 95(1), pages 64-73, March.
    18. Steven N. Kaplan & Tobias J. Moskowitz & Berk A. Sensoy, 2013. "The Effects of Stock Lending on Security Prices: An Experiment," Journal of Finance, American Finance Association, vol. 68(5), pages 1891-1936, October.
    19. Alexandre Belloni & Victor Chernozhukov & Christian Hansen, 2013. "Supplementary Appendix for "Inference on Treatment Effects After Selection Amongst High-Dimensional Controls"," Papers 1305.6099, arXiv.org, revised Jun 2013.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    2. Taisuke Otsu & Mengshan Xu, 2022. "Isotonic propensity score matching," STICERD - Econometrics Paper Series 623, Suntory and Toyota International Centres for Economics and Related Disciplines, LSE.
    3. Victor Chernozhukov & Juan Carlos Escanciano & Hidehiko Ichimura & Whitney K. Newey & James M. Robins, 2022. "Locally Robust Semiparametric Estimation," Econometrica, Econometric Society, vol. 90(4), pages 1501-1535, July.
    4. Mengshan Xu & Taisuke Otsu, 2022. "Isotonic propensity score matching," Papers 2207.08868, arXiv.org.
    5. Alexandre Belloni & Victor Chernozhukov & Denis Chetverikov & Christian Hansen & Kengo Kato, 2018. "High-dimensional econometrics and regularized GMM," CeMMAP working papers CWP35/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    7. Sant’Anna, Pedro H.C. & Zhao, Jun, 2020. "Doubly robust difference-in-differences estimators," Journal of Econometrics, Elsevier, vol. 219(1), pages 101-122.
    8. Kitagawa, Toru & Muris, Chris, 2016. "Model averaging in semiparametric estimation of treatment effects," Journal of Econometrics, Elsevier, vol. 193(1), pages 271-289.
    9. Stéphane Bonhomme & Martin Weidner, 2020. "Minimizing Sensitivity to Model Misspecification," CeMMAP working papers CWP37/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    10. Mammen, Enno & Rothe, Christoph & Schienle, Melanie, 2016. "Semiparametric Estimation With Generated Covariates," Econometric Theory, Cambridge University Press, vol. 32(5), pages 1140-1177, October.
    11. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney K. Newey, 2016. "Double machine learning for treatment and causal parameters," CeMMAP working papers CWP49/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    12. Alexandre Belloni & Victor Chernozhukov, 2015. "Comment," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1449-1451, December.
    13. Theresa Kuchler & Arlene Wong & Johannes Stroebel, 2018. "Peer effects in product adoption," 2018 Meeting Papers 1001, Society for Economic Dynamics.
    14. Kyle Colangelo & Ying-Ying Lee, 2020. "Double Debiased Machine Learning Nonparametric Inference with Continuous Treatments," Papers 2004.03036, arXiv.org, revised Jul 2022.
    15. Jiafeng Chen & David M. Ritzwoller, 2021. "Semiparametric Estimation of Long-Term Treatment Effects," Papers 2107.14405, arXiv.org, revised Aug 2022.
    16. Susan Athey & Stefan Wager, 2021. "Policy Learning With Observational Data," Econometrica, Econometric Society, vol. 89(1), pages 133-161, January.
    17. Joachim Inkmann, 2010. "Estimating Firm Size Elasticities of Product and Process R&D," Economica, London School of Economics and Political Science, vol. 77(306), pages 384-402, April.
    18. Zhaonan Qu & Ruoxuan Xiong & Jizhou Liu & Guido Imbens, 2021. "Efficient Treatment Effect Estimation in Observational Studies under Heterogeneous Partial Interference," Papers 2107.12420, arXiv.org, revised Jun 2022.
    19. Chaohua Dong & Jiti Gao & Oliver Linton, 2017. "High dimensional semiparametric moment restriction models," Monash Econometrics and Business Statistics Working Papers 17/17, Monash University, Department of Econometrics and Business Statistics.
    20. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.

    More about this item

    JEL classification:

    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:feb:natura:00763. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.fieldexperiments.com .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joe Seidel The email address of this maintainer does not seem to be valid anymore. Please ask Joe Seidel to update the entry or send us the correct address (email available below). General contact details of provider: http://www.fieldexperiments.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.