IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2401.07075.html
   My bibliography  Save this paper

Heterogeneous treatment effect estimation with high-dimensional data in public policy evaluation -- an application to the conditioning of cash transfers in Morocco using causal machine learning

Author

Listed:
  • Patrick Rehill
  • Nicholas Biddle

Abstract

Causal machine learning methods can be used to search for treatment effect heterogeneity in high-dimensional datasets even where we lack a strong enough theoretical framework to select variables or make parametric assumptions about data. This paper uses causal machine learning methods to estimate heterogeneous treatment effects in the case of an experimental study carried out in Morocco which evaluated the effect of conditionalizing a cash transfer program on several outcomes including maths test scores which is the focus of this work. We explore treatment effects across a dataset of 1936 pre-treatment variables. For the most part, heterogeneity is modelled by two different factors, participation in education (at the baseline) and more general measures of poverty. Those who are more disadvantaged at the baseline benefit less from any treatment. While conditioning generally has a negative effect this more disadvantaged group is also hurt more by conditioning. The second purpose of this paper is to demonstrate and reflect upon a causal machine learning approach to policy evaluation. We propose a novel causal tree method for interpretable modelling of causal effects and reflect on the difficulty of explaining atheoretical results.

Suggested Citation

  • Patrick Rehill & Nicholas Biddle, 2024. "Heterogeneous treatment effect estimation with high-dimensional data in public policy evaluation -- an application to the conditioning of cash transfers in Morocco using causal machine learning," Papers 2401.07075, arXiv.org, revised Mar 2024.
  • Handle: RePEc:arx:papers:2401.07075
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2401.07075
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Sarah Baird & Craig McIntosh & Berk Özler, 2011. "Cash or Condition? Evidence from a Cash Transfer Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 126(4), pages 1709-1753.
    2. Elek, Péter & Bíró, Anikó, 2021. "Regional differences in diabetes across Europe – regression and causal forest analyses," Economics & Human Biology, Elsevier, vol. 40(C).
    3. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    4. Stefan Wager & Susan Athey, 2018. "Estimation and Inference of Heterogeneous Treatment Effects using Random Forests," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(523), pages 1228-1242, July.
    5. Victor Chernozhukov & Denis Chetverikov & Mert Demirer & Esther Duflo & Christian Hansen & Whitney Newey & James Robins, 2018. "Double/debiased machine learning for treatment and structural parameters," Econometrics Journal, Royal Economic Society, vol. 21(1), pages 1-68, February.
    6. Lechner, Michael, 2018. "Modified Causal Forests for Estimating Heterogeneous Causal Effects," IZA Discussion Papers 12040, Institute of Labor Economics (IZA).
    7. Karen Macours & Norbert Schady & Renos Vakis, 2012. "Cash Transfers, Behavioral Changes, and Cognitive Development in Early Childhood: Evidence from a Randomized Experiment," American Economic Journal: Applied Economics, American Economic Association, vol. 4(2), pages 247-273, April.
    8. Bruno S. Frey & Reto Jegen, 2000. "Motivation Crowding Theory: A Survey of Empirical Evidence," CESifo Working Paper Series 245, CESifo.
    9. Beatty, Timothy K.M. & Blow, Laura & Crossley, Thomas F. & O'Dea, Cormac, 2014. "Cash by any other name? Evidence on labeling from the UK Winter Fuel Payment," Journal of Public Economics, Elsevier, vol. 118(C), pages 86-96.
    10. Richard H. Thaler, 2008. "Mental Accounting and Consumer Choice," Marketing Science, INFORMS, vol. 27(1), pages 15-25, 01-02.
    11. Moha Ennaji, 2016. "Women, Gender, and Politics in Morocco," Social Sciences, MDPI, vol. 5(4), pages 1-8, November.
    12. Bailey, Stephen J & Connolly, Stephen, 1998. "The Flypaper Effect: Identifying Areas for Further Research," Public Choice, Springer, vol. 95(3-4), pages 335-361, June.
    13. Peter Kooreman, 2000. "The Labeling Effect of a Child Benefit System," American Economic Review, American Economic Association, vol. 90(3), pages 571-583, June.
    14. Ayesha Afzal & Nawazish Mirza & Fatima Arshad, 2019. "Conditional vs unconditional cash transfers: a study of poverty demographics in Pakistan," Economic Research-Ekonomska Istraživanja, Taylor & Francis Journals, vol. 32(1), pages 3366-3383, January.
    15. James R. Hines & Richard H. Thaler, 1995. "The Flypaper Effect," Journal of Economic Perspectives, American Economic Association, vol. 9(4), pages 217-226, Fall.
    16. Vira Semenova & Victor Chernozhukov, 2021. "Debiased machine learning of conditional average treatment effects and other causal functions," The Econometrics Journal, Royal Economic Society, vol. 24(2), pages 264-289.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Becker, Johannes & Hopp, Daniel & Kriebel, Michael, 2020. "Mental accounting of public funds – The flypaper effect in the lab," Journal of Economic Behavior & Organization, Elsevier, vol. 176(C), pages 321-336.
    2. Daniel Goller, 2023. "Analysing a built-in advantage in asymmetric darts contests using causal machine learning," Annals of Operations Research, Springer, vol. 325(1), pages 649-679, June.
    3. Harold Alderman & Ruslan Yemtsov, 2014. "How Can Safety Nets Contribute to Economic Growth?," The World Bank Economic Review, World Bank, vol. 28(1), pages 1-20.
    4. Teresa Molina Millán & Karen Macours, 2017. "Attrition in randomized control trials: Using tracking information to correct bias," FEUNL Working Paper Series novaf:wp1702, Universidade Nova de Lisboa, Faculdade de Economia.
    5. Marta Favara & Catherine Porter & Tassew Woldehanna, 2019. "Smarter through social protection? Evaluating the impact of Ethiopia’s safety-net on child cognitive abilities," Oxford Development Studies, Taylor & Francis Journals, vol. 47(1), pages 79-96, January.
    6. Tia Palermo & Sudhanshu Handa & Amber Peterman & Leah Prencipe & David Seidenfeld, 2016. "Unconditional government social cash transfer in Africa does not increase fertility," Journal of Population Economics, Springer;European Society for Population Economics, vol. 29(4), pages 1083-1111, October.
    7. Beatty, Timothy K.M. & Blow, Laura & Crossley, Thomas F. & O'Dea, Cormac, 2014. "Cash by any other name? Evidence on labeling from the UK Winter Fuel Payment," Journal of Public Economics, Elsevier, vol. 118(C), pages 86-96.
    8. Martin Persson, U. & Alpízar, Francisco, 2013. "Conditional Cash Transfers and Payments for Environmental Services—A Conceptual Framework for Explaining and Judging Differences in Outcomes," World Development, Elsevier, vol. 43(C), pages 124-137.
    9. Michael Lechner, 2023. "Causal Machine Learning and its use for public policy," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 159(1), pages 1-15, December.
    10. Dominic Richardson & UNICEF Office of Research - Innocenti, 2018. "Key Findings on Families, Family Policy and the Sustainable Development Goals: Synthesis Report," Papers inorer948, Innocenti Research Report.
    11. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    12. Waidler, Jennifer, 2016. "On the fungibility of public and private transfers: A mental accounting approach," MERIT Working Papers 2016-060, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    13. Chuan, Amanda & List, John & Samek, Anya, 2021. "Do financial incentives aimed at decreasing interhousehold inequality increase intrahousehold inequality?," Journal of Public Economics, Elsevier, vol. 196(C).
    14. Michael C Knaus, 2022. "Double machine learning-based programme evaluation under unconfoundedness [Econometric methods for program evaluation]," The Econometrics Journal, Royal Economic Society, vol. 25(3), pages 602-627.
    15. Bryan, Gharad & Chowdhury, Shyamal & Mobarak, Ahmed Mushfiq & Morten, Melanie & Smits, Joeri, 2021. "Encouragement and Distortionary Effects of Conditional Cash Transfers," IZA Discussion Papers 14326, Institute of Labor Economics (IZA).
    16. Peters, Jörg & Langbein, Jörg & Roberts, Gareth, 2016. "Policy evaluation, randomized controlled trials, and external validity—A systematic review," Economics Letters, Elsevier, vol. 147(C), pages 51-54.
    17. Cockx, Bart & Lechner, Michael & Bollens, Joost, 2023. "Priority to unemployed immigrants? A causal machine learning evaluation of training in Belgium," Labour Economics, Elsevier, vol. 80(C).
    18. Richard Akresh & Damien de Walque & Harounan Kazianga, 2014. "Alternative Cash Transfer Delivery Mechanisms: Impacts on Routine Preventative Health Clinic Visits in Burkina Faso," NBER Chapters, in: African Successes, Volume II: Human Capital, pages 113-135, National Bureau of Economic Research, Inc.
    19. Heejun Shin & Joseph Antonelli, 2023. "Improved inference for doubly robust estimators of heterogeneous treatment effects," Biometrics, The International Biometric Society, vol. 79(4), pages 3140-3152, December.
    20. Phillip Heiler, 2022. "Heterogeneous Treatment Effect Bounds under Sample Selection with an Application to the Effects of Social Media on Political Polarization," Papers 2209.04329, arXiv.org, revised Jan 2024.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2401.07075. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.