IDEAS home Printed from https://ideas.repec.org/a/tpr/restat/v99y2017i4p683-697.html

Can Variation in Subgroups' Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment

Author

Listed:
  • Marianne P. Bitler

    (University of California, Davis and NBER)

  • Jonah B. Gelbach

    (University of Pennsylvania Law School)

  • Hilary W. Hoynes

    (University of California, Berkeley and NBER)

Abstract

We assess whether welfare reform affects earnings only through mean impacts that are constant within but vary across subgroups. This is important because researchers interested in treatment effect heterogeneity typically focus on estimating mean impacts that only vary across subgroups. Using a novel approach to simulating treatment group earnings under the constant mean impacts within subgroup model, we find this model does a poor job of capturing treatment effect heterogeneity for Connecticut's Jobs First welfare reform experiment. Notably, ignoring within-group heterogeneitywould lead one to miss evidence that treatment effects are consistent with basic labor supply theory.

Suggested Citation

  • Marianne P. Bitler & Jonah B. Gelbach & Hilary W. Hoynes, 2017. "Can Variation in Subgroups' Average Treatment Effects Explain Treatment Effect Heterogeneity? Evidence from a Social Experiment," The Review of Economics and Statistics, MIT Press, vol. 99(4), pages 683-697, July.
  • Handle: RePEc:tpr:restat:v:99:y:2017:i:4:p:683-697
    as

    Download full text from publisher

    File URL: http://www.mitpressjournals.org/doi/pdf/10.1162/REST_a_00662
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or

    for a different version of it.

    Other versions of this item:

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Sam Watson’s journal round up for 16th October 2017
      by Sam Watson in The Academic Health Economists' Blog on 2017-10-16 16:00:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Callaway, Brantly, 2021. "Bounds on distributional treatment effect parameters using panel data with an application on job displacement," Journal of Econometrics, Elsevier, vol. 222(2), pages 861-881.
    2. Lazuka, Volha, 2017. "The lasting health and income effects of public health formation in Sweden," Lund Papers in Economic History 153, Lund University, Department of Economic History.
    3. Afrouz Azadikhah Jahromi & Brantly Callaway, 2022. "Heterogeneous Effects of Job Displacement on Earnings," Empirical Economics, Springer, vol. 62(1), pages 213-245, January.
    4. Chung, EunYi & Olivares, Mauricio, 2021. "Permutation test for heterogeneous treatment effects with a nuisance parameter," Journal of Econometrics, Elsevier, vol. 225(2), pages 148-174.
    5. EunYi Chung & Mauricio Olivares, 2025. "Quantile‐Based Test for Heterogeneous Treatment Effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 40(1), pages 3-17, January.
    6. Gibbons Charles E. & Suárez Serrato Juan Carlos & Urbancic Michael B., 2019. "Broken or Fixed Effects?," Journal of Econometric Methods, De Gruyter, vol. 8(1), pages 1-12, January.
    7. Alejandro Sanchez-Becerra, 2023. "Robust inference for the treatment effect variance in experiments using machine learning," Papers 2306.03363, arXiv.org.
    8. Jeffrey Smith, 2022. "Treatment Effect Heterogeneity," Evaluation Review, , vol. 46(5), pages 652-677, October.
    9. Brigham R. Frandsen & Lars J. Lefgren, 2021. "Partial identification of the distribution of treatment effects with an application to the Knowledge is Power Program (KIPP)," Quantitative Economics, Econometric Society, vol. 12(1), pages 143-171, January.
    10. Horacio Alvarez Marinelli & Samuel Berlinski & Matias Busso, 2024. "Remedial Education: Evidence from a Sequence of Experiments in Colombia," Journal of Human Resources, University of Wisconsin Press, vol. 59(1), pages 141-174.
    11. Xavier D’Haultfoeuille & Pauline Givord, 2014. "La régression quantile en pratique," Économie et Statistique, Programme National Persée, vol. 471(1), pages 85-111.
    12. Maike Hohberg & Peter Pütz & Thomas Kneib, 2020. "Treatment effects beyond the mean using distributional regression: Methods and guidance," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-29, February.
    13. Marianne Bertrand & Bruno Crépon & Alicia Marguerie & Patrick Premand, 2017. "Contemporaneous and Post-Program Impacts of a Public Works Program," World Bank Publications - Reports 28460, The World Bank Group.
    14. Strittmatter, Anthony, 2019. "Heterogeneous earnings effects of the job corps by gender: A translated quantile approach," Labour Economics, Elsevier, vol. 61(C).
    15. Matthew L. Comey & Amanda R. Eng & Pauline Leung & Zhuan Pei, 2022. "Supercompliers," Papers 2212.14105, arXiv.org, revised Dec 2024.
    16. Jaime Ramirez-Cuellar, 2023. "Testing for idiosyncratic Treatment Effect Heterogeneity," Papers 2304.01141, arXiv.org.
    17. Buhl-Wiggers, Julie & Kerwin, Jason T. & Muñoz-Morales, Juan & Smith, Jeffrey & Thornton, Rebecca, 2024. "Some children left behind: Variation in the effects of an educational intervention," Journal of Econometrics, Elsevier, vol. 243(1).
    18. Robert Garlick, 2018. "Academic Peer Effects with Different Group Assignment Policies: Residential Tracking versus Random Assignment," American Economic Journal: Applied Economics, American Economic Association, vol. 10(3), pages 345-369, July.
    19. Strittmatter, Anthony, 2023. "What is the value added by using causal machine learning methods in a welfare experiment evaluation?," Labour Economics, Elsevier, vol. 84(C).
    20. Likai Chen & Georg Keilbar & Liangjun Su & Weining Wang, 2023. "Inference on many jumps in nonparametric panel regression models," Papers 2312.01162, arXiv.org, revised Jan 2025.
    21. Deparade, Darius & Jarmolinski, Lennart & Mohr, Peter, 2025. "Behavioral interventions, tax compliance and consequences on inequality," Discussion Papers 2025/4, Free University Berlin, School of Business & Economics.
    22. Deirdre Bloome & Daniel Schrage, 2021. "Covariance Regression Models for Studying Treatment Effect Heterogeneity Across One or More Outcomes: Understanding How Treatments Shape Inequality," Sociological Methods & Research, , vol. 50(3), pages 1034-1072, August.
    23. Xiao, Zhijie & Xu, Lan, 2019. "What do mean impacts miss? Distributional effects of corporate diversification," Journal of Econometrics, Elsevier, vol. 213(1), pages 92-120.

    More about this item

    JEL classification:

    • H75 - Public Economics - - State and Local Government; Intergovernmental Relations - - - State and Local Government: Health, Education, and Welfare
    • I38 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Government Programs; Provision and Effects of Welfare Programs
    • J18 - Labor and Demographic Economics - - Demographic Economics - - - Public Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:tpr:restat:v:99:y:2017:i:4:p:683-697. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: The MIT Press (email available below). General contact details of provider: https://direct.mit.edu/journals .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.