IDEAS home Printed from https://ideas.repec.org/p/ags/rffdps/10596.html

The Analysis of Randomized Experiments with Missing Data

Author

Listed:
  • Imbens, Guido W.
  • Pizer, William A.

Abstract

The otherwise straightforward analysis of randomized experiments is often complicated by the presence of missing data. In such situations it is necessary to make assumptions about the dependence of the selection mechanism on treatment, response, and covariates. The widely used approach of assuming that the data is missing at random conditional on treatment and other fully observed covariates is shown to be inadequate to describe data from a randomized experiment when partially observed covariates are also present. This paper presents an alternative to the missing at random model (MAR) which is both consistent with the data and preserves the appeal of MAR. In particular, the proposed family of models minimize the discrepancy with MAR while explaining observed deviations. We apply this approach to data from the Restart job training program in the United Kingdom as well as an artificial data set. Evaluation of the Restart program is not affected by the assumption of MAR; both approaches suggest that the program increased the chances of exiting unemployment by around 9% within six months. However, analysis of the artificial data demonstrates that assuming MAR can easily lead to erroneous conclusions.

Suggested Citation

  • Imbens, Guido W. & Pizer, William A., 2000. "The Analysis of Randomized Experiments with Missing Data," Discussion Papers 10596, Resources for the Future.
    • Handle: RePEc:ags:rffdps:10596
    DOI: 10.22004/ag.econ.10596
    as

    Download full text from publisher

    File URL: https://ageconsearch.umn.edu/record/10596/files/dp000019.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.22004/ag.econ.10596?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Imbens, G.W. & Rubin, D. & Sacerdote, B., 1999. "Estimating the effect of unearned income on labor supply, earnings, savings and consumption : Evidence from a survey of lottery players," Other publications TiSEM 43407245-d868-4cec-b289-c, Tilburg University, School of Economics and Management.
    2. Manski, Charles F., 1992. "Identification Problems In The Social Sciences," SSRI Workshop Series 292716, University of Wisconsin-Madison, Social Systems Research Institute.
    3. Peter Dolton & Donal O'Neill, 1996. "The Restart Effect and the Return to Full‐Time Stable Employment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(2), pages 275-288, March.
    4. Christian Gourieroux & Alain Monfort, 1981. "On the Problem of Missing Data in Linear Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 48(4), pages 579-586.
    5. Dolton, Peter & O'Neill, Donal, 1996. "Unemployment Duration and the Restart Effect: Some Experimental Evidence," Economic Journal, Royal Economic Society, vol. 106(435), pages 387-400, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. is not listed on IDEAS
    2. Molinari, Francesca, 2010. "Missing Treatments," Journal of Business & Economic Statistics, American Statistical Association, vol. 28(1), pages 82-95.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Denis Conniffe & Donal O'Neill, 2011. "Efficient Probit Estimation with Partially Missing Covariates," Advances in Econometrics, in: Missing Data Methods: Cross-sectional Methods and Applications, pages 209-245, Emerald Group Publishing Limited.
    2. van den Berg, Gerard J & Lindeboom, Maarten & Dolton, Peter J, 2004. "Survey non-response and unemployment duration," Working Paper Series 2004:12, IFAU - Institute for Evaluation of Labour Market and Education Policy.
    3. Denis Conniffe & Donal O’Neill, 2008. "An Efficient Estimator for Dealing with Missing Data on Explanatory Variables in a Probit Choice Model," Economics Department Working Paper Series n1960908.pdf, Department of Economics, National University of Ireland - Maynooth.
    4. Gerard J. Van Den Berg & Maarten Lindeboom & Peter J. Dolton, 2006. "Survey non‐response and the duration of unemployment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(3), pages 585-604, July.
    5. Jeffrey Smith, 2000. "A Critical Survey of Empirical Methods for Evaluating Active Labor Market Policies," Swiss Journal of Economics and Statistics (SJES), Swiss Society of Economics and Statistics (SSES), vol. 136(III), pages 247-268, September.
    6. Graversen, B.K. & van Ours, J.C., 2009. "How a Mandatory Activation Program Reduces Unemployment Durations : The Effects of Distance," Other publications TiSEM 0071ddbd-1c91-4a52-b1ac-5, Tilburg University, School of Economics and Management.
    7. Phillips, David C., 2014. "Getting to work: Experimental evidence on job search and transportation costs," Labour Economics, Elsevier, vol. 29(C), pages 72-82.
    8. James Banks & Richard Disney & Alan Duncan & John Van Reenen, 2005. "The Internationalisation of Public Welfare Policy," Economic Journal, Royal Economic Society, vol. 115(502), pages 62-81, March.
    9. Gunnar Andersson & Karsten Hank & Andres Vikat, 2006. "Understanding parental gender preferences in advanced societies: lessons from Sweden and Finland," MPIDR Working Papers WP-2006-019, Max Planck Institute for Demographic Research, Rostock, Germany.
    10. Hainmueller, Jens & Marbach, Moritz & Hangartner, Dominik & Harder, Niklas & Vallizadeh, Ehsan, 2025. "Refugee Labor Market Integration at Scale: Evidence from Germany’s Fast-Track Employment Program," SocArXiv px9ew_v2, Center for Open Science.
    11. Helen Jensen & Justo Manrique, 1998. "Demand for food commodities by income groups in Indonesia," Applied Economics, Taylor & Francis Journals, vol. 30(4), pages 491-501.
    12. John Van Reenen, 2004. "Active Labor Market Policies and the British New Deal for the Young Unemployed in Context," NBER Chapters, in: Seeking a Premier Economy: The Economic Effects of British Economic Reforms, 1980–2000, pages 461-496, National Bureau of Economic Research, Inc.
    13. Micklewright, John & Nagy, Gyula, 2008. "Job Search Monitoring and Unemployment Duration: Evidence from a Randomised Control Trial," CEPR Discussion Papers 6711, C.E.P.R. Discussion Papers.
    14. Giulio Zanella, 2004. "Discrete Choice with Social Interactions and Endogenous Memberships," Department of Economics University of Siena 442, Department of Economics, University of Siena.
    15. Alberto Abadie & Guido W. Imbens, 2002. "Simple and Bias-Corrected Matching Estimators for Average Treatment Effects," NBER Technical Working Papers 0283, National Bureau of Economic Research, Inc.
    16. Bruno Crépon & Muriel Dejemeppe & Marc Gurgand, 2005. "Counseling the unemployed: does it lower unemployment duration and recurrence?," Working Papers halshs-00590769, HAL.
    17. van Soest, Arthur & Hurd, Michael, 2008. "A Test for Anchoring and Yea-Saying in Experimental Consumption Data," Journal of the American Statistical Association, American Statistical Association, vol. 103, pages 126-136, March.
    18. Philip A. Haile & Elie Tamer, 2003. "Inference with an Incomplete Model of English Auctions," Journal of Political Economy, University of Chicago Press, vol. 111(1), pages 1-51, February.
    19. Raj Chetty, 2012. "Bounds on Elasticities With Optimization Frictions: A Synthesis of Micro and Macro Evidence on Labor Supply," Econometrica, Econometric Society, vol. 80(3), pages 969-1018, May.
    20. repec:iza:izadps:102 is not listed on IDEAS
    21. Gerard J. van den Berg & Bas van der Klaauw, 2005. "Job Search Monitoring and Sanctions," ifo DICE Report, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 3(2), pages 26-29, 07.

    More about this item

    Keywords

    ;

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ags:rffdps:10596. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: AgEcon Search (email available below). General contact details of provider: https://edirc.repec.org/data/rffffus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.