IDEAS home Printed from
   My bibliography  Save this paper

Data-driven sensitivity analysis for matching estimators


  • Giovanni Cerulli

    (Research Institute on Sustainable Economic Growth, Rome)


Matching is a popular estimator of the Average Treatment Effects (ATEs) within counterfactual observational studies. In recent years, however, many scholars have questioned the validity of this approach for causal inference, as its reliability draws heavily upon the so-called selection-on-observables assumption. When unobservable confounders are possibly at work, they say, it becomes hard to trust matching results, and the analyst should consider alternative methods suitable for tackling unobservable selection. Unfortunately, these alternatives require extra information that may be costly to obtain, or even not accessible. For this reason, some scholars have proposed matching sensitivity tests for the possible presence of unobservable selection. The literature sets out two methods: the Rosenbaum (1987) and the Ichino, Mealli, and Nannicini (2008) tests. Both are implemented in Stata. In this work, I propose a third and different sensitivity test for unobservable selection in Matching estimation based on a ‘leave-covariates-out’ (LCO) approach. Rooted in the machine learning literature, this sensitivity test recalls a bootstrap over different subsets of covariates and simulates various estimation scenarios to be compared with the baseline matching estimated by the analyst. Finally, I will present sensimatch, the Stata routine I developed to run this method, and provide some instructional applications on real datasets.

Suggested Citation

  • Giovanni Cerulli, 2018. "Data-driven sensitivity analysis for matching estimators," London Stata Conference 2018 02, Stata Users Group.
  • Handle: RePEc:boc:usug18:02

    Download full text from publisher

    File URL:
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769, October.
    3. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    4. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, June.
    5. Giovanni Cerulli, 2015. "Econometric Evaluation of Socio-Economic Programs," Advanced Studies in Theoretical and Applied Econometrics, Springer, edition 127, number 978-3-662-46405-2, enero-jun.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joshua D. Angrist & Jörn-Steffen Pischke, 2017. "Undergraduate Econometrics Instruction: Through Our Classes, Darkly," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 125-144, Spring.
    2. W. Bentley MacLeod, 2017. "Viewpoint: The human capital approach to inference," Canadian Journal of Economics, Canadian Economics Association, vol. 50(1), pages 5-39, February.
    3. Art B. Owen & Hal Varian, 2018. "Optimizing the tie-breaker regression discontinuity design," Papers 1808.07563,, revised Jul 2020.
    4. Sloczynski, Tymon, 2020. "Interpreting OLS Estimands When Treatment Effects Are Heterogeneous: Smaller Groups Get Larger Weights," IZA Discussion Papers 13283, Institute of Labor Economics (IZA).
    5. Stefano Carattini & Suphi Sen, 2019. "Carbon Taxes and Stranded Assets: Evidence from Washington State," International Center for Public Policy Working Paper Series, at AYSPS, GSU paper1910, International Center for Public Policy, Andrew Young School of Policy Studies, Georgia State University.
    6. Susan Athey & Raj Chetty & Guido Imbens, 2020. "Combining Experimental and Observational Data to Estimate Treatment Effects on Long Term Outcomes," Papers 2006.09676,
    7. Heinemann, Friedrich & Moessinger, Marc-Daniel & Yeter, Mustafa, 2018. "Do fiscal rules constrain fiscal policy? A meta-regression-analysis," European Journal of Political Economy, Elsevier, vol. 51(C), pages 69-92.
    8. Paul Hunermund & Elias Bareinboim, 2019. "Causal Inference and Data-Fusion in Econometrics," Papers 1912.09104,, revised Dec 2019.
    9. Sloczynski, Tymon, 2018. "A General Weighted Average Representation of the Ordinary and Two-Stage Least Squares Estimands," IZA Discussion Papers 11866, Institute of Labor Economics (IZA).
    10. Candon, David, 2018. "The effect of cancer on the labor supply of employed men over the age of 65," Economics & Human Biology, Elsevier, vol. 31(C), pages 184-199.
    11. Claudio Agostini & Eduardo Engel & Andrea Repetto & Damian Vergara, 2017. "Individual Tax Planning and Small Business Creation: Evidence on the Impact of Special Tax Regimes in Chile," Working Papers wp_054, Adolfo Ibáñez University, School of Government.
    12. Boockmann Bernhard & Buch Claudia M. & Schnitzer Monika, 2014. "Evidenzbasierte Wirtschaftspolitik in Deutschland: Defizite und Potentiale," Perspektiven der Wirtschaftspolitik, De Gruyter, vol. 15(4), pages 307-323, December.
    13. Takano, Keisuke, 2019. "Does visible shock update firms' unrelated trade diversity in anticipation of future shock? Evidence from the Great East Japan Earthquake and expected Nankai Trough Earthquake," TDB-CAREE Discussion Paper Series E-2019-01, Teikoku Databank Center for Advanced Empirical Research on Enterprise and Economy, Graduate School of Economics, Hitotsubashi University.
    14. Ashesh Rambachan & Neil Shephard, 2019. "Econometric analysis of potential outcomes time series: instruments, shocks, linearity and the causal response function," Papers 1903.01637,, revised Feb 2020.
    15. Guido Imbens, 2019. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," NBER Working Papers 26104, National Bureau of Economic Research, Inc.
    16. Claudio A. Agostini & Eduardo Engel & Andrea Repetto & Damián Vergara, 2018. "Using small businesses for individual tax planning: evidence from special tax regimes in Chile," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 25(6), pages 1449-1489, December.
    17. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    18. Bryan S. Graham & Cristine Campos de Xavier Pinto, 2018. "Semiparametrically efficient estimation of the average linear regression function," Papers 1810.12511,
    19. Paik, Myungho & Black, Bernard & Hyman, David A., 2017. "Damage caps and defensive medicine, revisited," Journal of Health Economics, Elsevier, vol. 51(C), pages 84-97.
    20. Chemla, Gilles & Hennessy, Christopher A., 2020. "Rational expectations and the Paradox of policy-relevant natural experiments," Journal of Monetary Economics, Elsevier, vol. 114(C), pages 368-381.

    More about this item

    JEL classification:

    • C01 - Mathematical and Quantitative Methods - - General - - - Econometrics
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection
    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General

    NEP fields

    This paper has been announced in the following NEP Reports:


    Access and download statistics


    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:boc:usug18:02. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher F Baum). General contact details of provider: .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.