IDEAS home Printed from https://ideas.repec.org/p/col/000122/015810.html
   My bibliography  Save this paper

Outliers in semi-parametric Estimation of Treatment Effects

Author

Listed:
  • Darwin Ugarte Ontiveros

    ()

  • Gustavo Canavire-Bacarreza

    ()

  • Luis Castro Peñarrieta

    ()

Abstract

Average treatment effects estimands can present significant bias under the presence of outliers. Moreover, outliers can be particularly hard to detect, creating bias and inconsistency in the semi-parametric ATE estimads. In this paper, we use Monte Carlo simulations to demonstrate that semi-parametric methods, such as matching, are biased in the presence of outliers. Bad and good leverage points outliers are considered. The bias arises because bad leverage points completely change the distribution of the metrics used to define counterfactuals. Whereas good leverage points increase the chance of breaking the common support condition and distort the balance of the covariates and which may push practitioners to misspecify the propensity score. We provide some clues to diagnose the presence of outliers and propose a reweighting estimator that is robust against outliers based on the Stahel-Donoho multivariate estimator of scale and location. An application of this estimator to LaLonde (1986) data allows us to explain the Dehejia and Wahba (2002) and Smith and Todd (2005) debate on the inability of matching estimators to deal with the evaluation problem.

Suggested Citation

  • Darwin Ugarte Ontiveros & Gustavo Canavire-Bacarreza & Luis Castro Peñarrieta, 2017. "Outliers in semi-parametric Estimation of Treatment Effects," DOCUMENTOS DE TRABAJO CIEF 015810, UNIVERSIDAD EAFIT.
  • Handle: RePEc:col:000122:015810
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10784/11750
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. Ashenfelter, Orley & Card, David, 1985. "Using the Longitudinal Structure of Earnings to Estimate the Effect of Training Programs," The Review of Economics and Statistics, MIT Press, vol. 67(4), pages 648-660, November.
    3. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    4. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    5. Rajeev H. Dehejia & Sadek Wahba, 2002. "Propensity Score-Matching Methods For Nonexperimental Causal Studies," The Review of Economics and Statistics, MIT Press, vol. 84(1), pages 151-161, February.
    6. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    7. Jerry A. Hausman & David A. Wise, 1985. "Introduction to "Social Experimentation"," NBER Chapters,in: Social Experimentation, pages 1-10 National Bureau of Economic Research, Inc.
    8. Heckman, James J. & Robb, Richard Jr., 1985. "Alternative methods for evaluating the impact of interventions : An overview," Journal of Econometrics, Elsevier, vol. 30(1-2), pages 239-267.
    9. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    10. Hausman, Jerry A. & Wise, David A. (ed.), 1985. "Social Experimentation," National Bureau of Economic Research Books, University of Chicago Press, number 9780226319407, December.
    11. Richard Blundell & Monica Costa Dias, 2000. "Evaluation methods for non-experimental data," Fiscal Studies, Institute for Fiscal Studies, vol. 21(4), pages 427-468, January.
    12. Bassi, Laurie J, 1984. "Estimating the Effect of Training Programs with Non-Random Selection," The Review of Economics and Statistics, MIT Press, vol. 66(1), pages 36-43, February.
    13. Vincenzo Verardi & Marjorie Gassner & Darwin Ugarte Ontiveros, 2012. "Robustness for Dummies," Working Papers ECARES ECARES 2012-015, ULB -- Universite Libre de Bruxelles.
    14. James J. Heckman & Edward Vytlacil, 2005. "Structural Equations, Treatment Effects, and Econometric Policy Evaluation," Econometrica, Econometric Society, vol. 73(3), pages 669-738, May.
    15. Nada Eissa & Jeffrey B. Liebman, 1996. "Labor Supply Response to the Earned Income Tax Credit," The Quarterly Journal of Economics, Oxford University Press, vol. 111(2), pages 605-637.
    16. Abadie, Alberto & Imbens, Guido W., 2011. "Bias-Corrected Matching Estimators for Average Treatment Effects," Journal of Business & Economic Statistics, American Statistical Association, vol. 29(1), pages 1-11.
    17. Laurie J. Bassi, 1983. "The Effect of CETA on the Postprogram Earnings of Participants," Journal of Human Resources, University of Wisconsin Press, vol. 18(4), pages 539-556.
    18. Vincenzo Verardi & Christophe Croux, 2009. "Robust regression in Stata," Stata Journal, StataCorp LP, vol. 9(3), pages 439-453, September.
    19. Croux, Christophe & Flandre, Cécile & Haesbroeck, Gentiane, 2002. "The breakdown behavior of the maximum likelihood estimator in the logistic regression model," Statistics & Probability Letters, Elsevier, vol. 60(4), pages 377-386, December.
    20. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    21. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    22. Dehejia, Rajeev, 2005. "Practical propensity score matching: a reply to Smith and Todd," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 355-364.
    23. Jerry A. Hausman & David A. Wise, 1985. "Social Experimentation," NBER Books, National Bureau of Economic Research, Inc, number haus85-1.
    24. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    25. Matias Busso & John DiNardo & Justin McCrary, 2014. "New Evidence on the Finite Sample Properties of Propensity Score Reweighting and Matching Estimators," The Review of Economics and Statistics, MIT Press, vol. 96(5), pages 885-897, December.
    26. repec:ags:stataj:143000 is not listed on IDEAS
    27. Angus Deaton & Nancy Cartwright, 2016. "Understanding and Misunderstanding Randomized Controlled Trials," Working Papers august_25.pdf, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    28. Shakeeb Khan & Elie Tamer, 2010. "Irregular Identification, Support Conditions, and Inverse Weight Estimation," Econometrica, Econometric Society, vol. 78(6), pages 2021-2042, November.
    29. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    30. Croux, Christophe & Haesbroeck, Gentiane, 2003. "Implementing the Bianco and Yohai estimator for logistic regression," Computational Statistics & Data Analysis, Elsevier, vol. 44(1-2), pages 273-295, October.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Treatment effects; Outliers; Propensity score; Mahalanobis distance;

    JEL classification:

    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C13 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Estimation: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:col:000122:015810. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Centro de Investigaciones Económicas y Financieras (CIEF)). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.