IDEAS home Printed from https://ideas.repec.org/p/pra/mprapa/80686.html
   My bibliography  Save this paper

Should we drop covariate cells with attrition problems?

Author

Listed:
  • Ferman, Bruno
  • Ponczek, Vladimir

Abstract

It is well known that sample attrition can lead to inconsistent treatment effect estimators even in randomized control trials. Standard solutions to attrition problems either rely on strong assumptions on the attrition mechanisms or consider the estimation of bounds, which may be uninformative if attrition problems are severe. In this paper, we analyze strategies of focusing the analysis on subsets of the data with less observed attrition problems. We show that these strategies are asymptotically valid when the number of observations in each covariate cell goes to infinity. However, they can lead to important distortions when the number of observations per covariate cell is finite.

Suggested Citation

  • Ferman, Bruno & Ponczek, Vladimir, 2017. "Should we drop covariate cells with attrition problems?," MPRA Paper 80686, University Library of Munich, Germany.
    • Handle: RePEc:pra:mprapa:80686
    as

    Download full text from publisher

    File URL: https://mpra.ub.uni-muenchen.de/80686/1/MPRA_paper_80686.pdf
    File Function: original version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    2. Benjamin A. Olken, 2015. "Promises and Perils of Pre-analysis Plans," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 61-80, Summer.
    3. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    4. Miriam Bruhn & David McKenzie, 2009. "In Pursuit of Balance: Randomization in Practice in Development Field Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 1(4), pages 200-232, October.
    5. Donald W. K. Andrews & Marcia M. A. Schafgans, 1998. "Semiparametric Estimation of the Intercept of a Sample Selection Model," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(3), pages 497-517.
    6. Guido W. Imbens & Charles F. Manski, 2004. "Confidence Intervals for Partially Identified Parameters," Econometrica, Econometric Society, vol. 72(6), pages 1845-1857, November.
    7. Lucas C. Coffman & Muriel Niederle, 2015. "Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible," Journal of Economic Perspectives, American Economic Association, vol. 29(3), pages 81-98, Summer.
    8. Peter Z. Schochet & John Burghardt & Sheena McConnell, 2008. "Does Job Corps Work? Impact Findings from the National Job Corps Study," American Economic Review, American Economic Association, vol. 98(5), pages 1864-1886, December.
    9. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 31(3), pages 129-137.
    10. Paolo Frumento & Fabrizia Mealli & Barbara Pacini & Donald B. Rubin, 2012. "Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 450-466, June.
    11. Carlos A. Flores & Alfonso Flores-Lagunes & Arturo Gonzalez & Todd C. Neumann, 2012. "Estimating the Effects of Length of Exposure to Instruction in a Training Program: The Case of Job Corps," The Review of Economics and Statistics, MIT Press, vol. 94(1), pages 153-171, February.
    12. repec:mpr:mprres:6097 is not listed on IDEAS
    13. Mitali Das & Whitney K. Newey & Francis Vella, 2003. "Nonparametric Estimation of Sample Selection Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 70(1), pages 33-58.
    14. Peter Z. Schochet & John Burghardt & Sheena McConnell, "undated". "Does Job Corps Work? Impact Findings from the National Job Corps Study," Mathematica Policy Research Reports 5067e4907eba4a4881f510080, Mathematica Policy Research.
    15. Ahn, Hyungtaik & Powell, James L., 1993. "Semiparametric estimation of censored selection models with a nonparametric selection mechanism," Journal of Econometrics, Elsevier, vol. 58(1-2), pages 3-29, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ferman, Bruno, 2021. "Matching estimators with few treated and many control observations," Journal of Econometrics, Elsevier, vol. 225(2), pages 295-307.
    2. Ferman, Bruno & Lima, Lycia & Riva, Flávio, 2021. "Artificial Intelligence, Teacher Tasks and Individualized Pedagogy," SocArXiv qw249, Center for Open Science.
    3. Ferman, Bruno & Lima, Lycia & Riva, Flavio, 2020. "Experimental Evidence on Artificial Intelligence in the Classroom," MPRA Paper 103934, University Library of Munich, Germany.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. María laura Alzúa & Guillermo Cruces & Carolina Lopez, 2016. "Long-Run Effects Of Youth Training Programs: Experimental Evidence From Argentina," Economic Inquiry, Western Economic Association International, vol. 54(4), pages 1839-1859, October.
    2. McGovern, Mark E. & Canning, David & Bärnighausen, Till, 2018. "Accounting for non-response bias using participation incentives and survey design: An application using gift vouchers," Economics Letters, Elsevier, vol. 171(C), pages 239-244.
    3. Bo E. Honoré & Luojia Hu, 2020. "Selection Without Exclusion," Econometrica, Econometric Society, vol. 88(3), pages 1007-1029, May.
    4. Martin Huber, 2014. "Treatment Evaluation in the Presence of Sample Selection," Econometric Reviews, Taylor & Francis Journals, vol. 33(8), pages 869-905, November.
    5. Chen, Xuan & Flores, Carlos A. & Flores-Lagunes, Alfonso, 2015. "Going Beyond LATE: Bounding Average Treatment Effects of Job Corps Training," IZA Discussion Papers 9511, Institute of Labor Economics (IZA).
    6. German Blanco & Xuan Chen & Carlos A. Flores & Alfonso Flores-Lagunes, 2020. "Bounds on Average and Quantile Treatment Effects on Duration Outcomes Under Censoring, Selection, and Noncompliance," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 901-920, October.
    7. Mark McGovern & David Canning & Till Bärnighausen, 2018. "Accounting for Non-Response Bias using Participation Incentives and Survey Design," CHaRMS Working Papers 18-02, Centre for HeAlth Research at the Management School (CHaRMS).
    8. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    9. Martin Huber & Giovanni Mellace, 2014. "Testing exclusion restrictions and additive separability in sample selection models," Empirical Economics, Springer, vol. 47(1), pages 75-92, August.
    10. Lewbel, Arthur, 2007. "Endogenous selection or treatment model estimation," Journal of Econometrics, Elsevier, vol. 141(2), pages 777-806, December.
    11. Richard Blundell & Monica Costa Dias, 2009. "Alternative Approaches to Evaluation in Empirical Microeconomics," Journal of Human Resources, University of Wisconsin Press, vol. 44(3).
    12. Richard Blundell & Amanda Gosling & Hidehiko Ichimura & Costas Meghir, 2007. "Changes in the Distribution of Male and Female Wages Accounting for Employment Composition Using Bounds," Econometrica, Econometric Society, vol. 75(2), pages 323-363, March.
    13. Manuel Arellano & Stéphane Bonhomme, 2017. "Sample Selection in Quantile Regression: A Survey," Working Papers wp2018_1702, CEMFI.
    14. Manuel Arellano & Stéphane Bonhomme, 2017. "Sample Selection in Quantile Regression: A Survey," Working Papers wp2017_1702, CEMFI.
    15. Amin, Vikesh & Flores, Carlos A. & Flores-Lagunes, Alfonso & Parisian, Daniel J., 2016. "The effect of degree attainment on arrests: Evidence from a randomized social experiment," Economics of Education Review, Elsevier, vol. 54(C), pages 259-273.
    16. John Engberg & Dennis Epple & Jason Imbrogno & Holger Sieg & Ron Zimmer, 2014. "Evaluating Education Programs That Have Lotteried Admission and Selective Attrition," Journal of Labor Economics, University of Chicago Press, vol. 32(1), pages 27-63.
    17. D’Haultfœuille, Xavier & Maurel, Arnaud & Zhang, Yichong, 2018. "Extremal quantile regressions for selection models and the black–white wage gap," Journal of Econometrics, Elsevier, vol. 203(1), pages 129-142.
    18. Arulampalam, Wiji & Corradi, Valentina & Gutknecht, Daniel, 2021. "Intercept Estimation in Nonlinear Selection Models," IZA Discussion Papers 14364, Institute of Labor Economics (IZA).
    19. Liu, Ruixuan & Yu, Zhengfei, 2022. "Sample selection models with monotone control functions," Journal of Econometrics, Elsevier, vol. 226(2), pages 321-342.
    20. Strittmatter, Anthony, 2019. "Heterogeneous earnings effects of the job corps by gender: A translated quantile approach," Labour Economics, Elsevier, vol. 61(C).

    More about this item

    Keywords

    impact evaluation; attrition; partial identification;
    All these keywords.

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:pra:mprapa:80686. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Joachim Winter (email available below). General contact details of provider: https://edirc.repec.org/data/vfmunde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.