IDEAS home Printed from https://ideas.repec.org/p/boc/asug06/6.html
   My bibliography  Save this paper

A Simulation-Based Sensitivity Analysis for Matching Estimators

Author

Listed:
  • Tommaso Nannicini

    (Universidad Carlos III de Madrid)

Abstract

This paper presents a Stata program ("sensatt") that implements the sensitivity analysis for propensity-score matching estimators proposed by Ichino, Mealli and Nannicini (2005). The proposed analysis builds on Rosenbaum and Rubin (1983) and Rosenbaum (1987), and simulates a potential confounder in order to assess the robustness of the estimated treatment effect with respect to specific deviations from the Conditional Independence Assumption (CIA). The program "sensatt" makes use of the user-written Stata commands for propensity-score matching estimation (att*) developed by Becker and Ichino (2002). An example of the implementation of the proposed sensitivity analysis is given by using the National Supported Work (NSW) demonstration, widely known in the program evaluation literature.

Suggested Citation

  • Tommaso Nannicini, 2006. "A Simulation-Based Sensitivity Analysis for Matching Estimators," North American Stata Users' Group Meetings 2006 6, Stata Users Group.
  • Handle: RePEc:boc:asug06:6
    as

    Download full text from publisher

    File URL: http://repec.org/nasug2006/pres_Stata_1.pdf
    Download Restriction: no

    File URL: http://repec.org/nasug2006/sensatt_wp.pdf
    Download Restriction: no

    File URL: http://fmwww.bc.edu/repec/bocode/s/sensatt.ado
    File Function: program code
    Download Restriction: no

    File URL: http://fmwww.bc.edu/repec/bocode/s/sensatt.hlp
    File Function: help file
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    2. A. Smith, Jeffrey & E. Todd, Petra, 2005. "Does matching overcome LaLonde's critique of nonexperimental estimators?," Journal of Econometrics, Elsevier, vol. 125(1-2), pages 305-353.
    3. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    4. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 605-654.
    5. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    6. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    7. Black, Dan A. & Smith, J.A.Jeffrey A., 2004. "How robust is the evidence on the effects of college quality? Evidence from matching," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 99-124.
    8. Barbara Sianesi, 2001. "Propensity score matching," United Kingdom Stata Users' Group Meetings 2001 12, Stata Users Group, revised 23 Aug 2001.
    9. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    10. Alberto Abadie & David Drukker & Jane Leber Herr & Guido W. Imbens, 2004. "Implementing matching estimators for average treatment effects in Stata," Stata Journal, StataCorp LP, vol. 4(3), pages 290-311, September.
    11. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    12. Charles Michalopoulos & Howard S. Bloom & Carolyn J. Hill, 2004. "Can Propensity-Score Methods Match the Findings from a Random Assignment Evaluation of Mandatory Welfare-to-Work Programs?," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 156-179, February.
    13. Guido W. Imbens, 2004. "Nonparametric Estimation of Average Treatment Effects Under Exogeneity: A Review," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 4-29, February.
    14. James J. Heckman & Hidehiko Ichimura & Petra Todd, 1998. "Matching As An Econometric Evaluation Estimator," Review of Economic Studies, Oxford University Press, vol. 65(2), pages 261-294.
    15. Alberto Abadie & Guido W. Imbens, 2006. "Large Sample Properties of Matching Estimators for Average Treatment Effects," Econometrica, Econometric Society, vol. 74(1), pages 235-267, January.
    16. Sascha O. Becker & Andrea Ichino, 2002. "Estimation of average treatment effects based on propensity scores," Stata Journal, StataCorp LP, vol. 2(4), pages 358-377, November.
    17. Edwin Leuven & Barbara Sianesi, 2003. "PSMATCH2: Stata module to perform full Mahalanobis and propensity score matching, common support graphing, and covariate imbalance testing," Statistical Software Components S432001, Boston College Department of Economics, revised 01 Feb 2018.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    3. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    4. Olivier Dagnelie & Philippe Lemay‐Boucher, 2012. "Rosca Participation in Benin: A Commitment Issue," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 74(2), pages 235-252, April.
    5. Carlos A. Flores & Oscar A. Mitnik, 2009. "Evaluating Nonexperimental Estimators for Multiple Treatments: Evidence from Experimental Data," Working Papers 2010-10, University of Miami, Department of Economics.
    6. Arpino, Bruno & Mealli, Fabrizia, 2011. "The specification of the propensity score in multilevel observational studies," Computational Statistics & Data Analysis, Elsevier, vol. 55(4), pages 1770-1780, April.
    7. Maoyong Fan & Yanhong Jin, 2015. "The Supplemental Nutrition Assistance Program and Childhood Obesity in the United States: Evidence from the National Longitudinal Survey of Youth 1997," American Journal of Health Economics, MIT Press, vol. 1(4), pages 432-460, Fall.
    8. Christian K.M. Kingombe, 2012. "The Linkage between Outcome Differences in Cotton Production and Rural Roads Improvements - A Matching Approach," IHEID Working Papers 12-2012, Economics Section, The Graduate Institute of International Studies.
    9. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2006. "Moving the Goalposts: Addressing Limited Overlap in the Estimation of Average Treatment Effects by Changing the Estimand," NBER Technical Working Papers 0330, National Bureau of Economic Research, Inc.
    10. Peter R. Mueser & Kenneth R. Troske & Alexey Gorislavsky, 2007. "Using State Administrative Data to Measure Program Performance," The Review of Economics and Statistics, MIT Press, vol. 89(4), pages 761-783, November.
    11. Eliasson, Kent, 2006. "How Robust is the Evidence on the Returns to College Choice? Results Using Swedish Administrative Data," Umeå Economic Studies 692, Umeå University, Department of Economics.
    12. Wendimu, Mengistu Assefa & Henningsen, Arne & Gibbon, Peter, 2016. "Sugarcane Outgrowers in Ethiopia: “Forced” to Remain Poor?," World Development, Elsevier, vol. 83(C), pages 84-97.
    13. Eliasson, Kent, 2006. "College Choice And Earnings Among University Graduates In Sweden," Umeå Economic Studies 693, Umeå University, Department of Economics.
    14. Claire Bonnard, 2011. "Les incitations à l'innovation dans le secteur privé," Post-Print halshs-00599700, HAL.
    15. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    16. Helena Holmlund & Olmo Silva, 2014. "Targeting Noncognitive Skills to Improve Cognitive Outcomes: Evidence from a Remedial Education Intervention," Journal of Human Capital, University of Chicago Press, vol. 8(2), pages 126-160.
    17. Essama-Nssah, B., 2006. "Propensity score matching and policy impact analysis - a demonstration in EViews," Policy Research Working Paper Series 3877, The World Bank.
    18. Schilling, Brian J. & Attavanich, Witsanu & Jin, Yanhong, 2014. "Does Agritourism Enhance Farm Profitability?," Journal of Agricultural and Resource Economics, Western Agricultural Economics Association, vol. 39(1), pages 1-28, April.
    19. Andrea Pufahl & Christoph R. Weiss, 2009. "Evaluating the effects of farm programmes: results from propensity score matching," European Review of Agricultural Economics, Foundation for the European Review of Agricultural Economics, vol. 36(1), pages 79-101, March.
    20. Paolo Naticchioni & Silvia Loriga, 2011. "Short and Long Term Evaluations of Public Employment Services in Italy," Applied Economics Quarterly (formerly: Konjunkturpolitik), Duncker & Humblot, Berlin, vol. 57(3), pages 201-229.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boc:asug06:6. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/stataea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F Baum (email available below). General contact details of provider: https://edirc.repec.org/data/stataea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.