IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2012.15716.html
   My bibliography  Save this paper

Assessing Sensitivity to Unconfoundedness: Estimation and Inference

Author

Listed:
  • Matthew A. Masten
  • Alexandre Poirier
  • Linqi Zhang

Abstract

This paper provides a set of methods for quantifying the robustness of treatment effects estimated using the unconfoundedness assumption (also known as selection on observables or conditional independence). Specifically, we estimate and do inference on bounds on various treatment effect parameters, like the average treatment effect (ATE) and the average effect of treatment on the treated (ATT), under nonparametric relaxations of the unconfoundedness assumption indexed by a scalar sensitivity parameter c. These relaxations allow for limited selection on unobservables, depending on the value of c. For large enough c, these bounds equal the no assumptions bounds. Using a non-standard bootstrap method, we show how to construct confidence bands for these bound functions which are uniform over all values of c. We illustrate these methods with an empirical application to effects of the National Supported Work Demonstration program. We implement these methods in a companion Stata module for easy use in practice.

Suggested Citation

  • Matthew A. Masten & Alexandre Poirier & Linqi Zhang, 2020. "Assessing Sensitivity to Unconfoundedness: Estimation and Inference," Papers 2012.15716, arXiv.org.
  • Handle: RePEc:arx:papers:2012.15716
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2012.15716
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    2. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    3. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    4. Emily Oster, 2019. "Unobservable Selection and Coefficient Stability: Theory and Evidence," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 37(2), pages 187-204, April.
    5. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    6. Victor Chernozhukov & Iv·n Fern·ndez-Val & Alfred Galichon, 2010. "Quantile and Probability Curves Without Crossing," Econometrica, Econometric Society, vol. 78(3), pages 1093-1125, May.
    7. Krauth Brian, 2016. "Bounding a Linear Causal Effect Using Relative Correlation Restrictions," Journal of Econometric Methods, De Gruyter, vol. 5(1), pages 117-141, January.
    8. Matthew A. Masten & Alexandre Poirier, 2018. "Identification of Treatment Effects Under Conditional Partial Independence," Econometrica, Econometric Society, vol. 86(1), pages 317-351, January.
    9. James J. Heckman & Jeffrey A. Smith, 1998. "Evaluating the Welfare State," NBER Working Papers 6542, National Bureau of Economic Research, Inc.
    10. repec:hal:wpspec:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    11. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    12. Carlos Cinelli & Chad Hazlett, 2020. "Making sense of sensitivity: extending omitted variable bias," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(1), pages 39-67, February.
    13. Joshua Angrist & Victor Chernozhukov & Iván Fernández-Val, 2006. "Quantile Regression under Misspecification, with an Application to the U.S. Wage Structure," Econometrica, Econometric Society, vol. 74(2), pages 539-563, March.
    14. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    15. repec:hal:spmain:info:hdl:2441/5rkqqmvrn4tl22s9mc4b6ga2g is not listed on IDEAS
    16. Hong, Han & Li, Jessie, 2018. "The numerical delta method," Journal of Econometrics, Elsevier, vol. 206(2), pages 379-394.
    17. Manski, Charles F, 1990. "Nonparametric Bounds on Treatment Effects," American Economic Review, American Economic Association, vol. 80(2), pages 319-323, May.
    18. LaLonde, Robert J, 1986. "Evaluating the Econometric Evaluations of Training Programs with Experimental Data," American Economic Review, American Economic Association, vol. 76(4), pages 604-620, September.
    19. Victor Chernozhukov & Iv'an Fern'andez-Val & Tetsuya Kaji, 2016. "Extremal Quantile Regression: An Overview," Papers 1612.06850, arXiv.org, revised Feb 2017.
    20. Heckman, James J. & Lalonde, Robert J. & Smith, Jeffrey A., 1999. "The economics and econometrics of active labor market programs," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 3, chapter 31, pages 1865-2097, Elsevier.
    21. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paul Diegert & Matthew A. Masten & Alexandre Poirier, 2022. "Assessing Omitted Variable Bias when the Controls are Endogenous," Papers 2206.02303, arXiv.org, revised Jul 2023.
    2. Jacob Dorn & Kevin Guo, 2021. "Sharp Sensitivity Analysis for Inverse Propensity Weighting via Quantile Balancing," Papers 2102.04543, arXiv.org, revised Aug 2023.
    3. Miklin Nikolai & Gachechiladze Mariami & Moreno George & Chaves Rafael, 2022. "Causal inference with imperfect instrumental variables," Journal of Causal Inference, De Gruyter, vol. 10(1), pages 45-63, January.
    4. Santiago Acerenza & Vitor Possebom & Pedro H. C. Sant'Anna, 2023. "Was Javert right to be suspicious? Unpacking treatment effect heterogeneity of alternative sentences on time-to-recidivism in Brazil," Papers 2311.13969, arXiv.org, revised Jan 2024.
    5. Tabe-Ojong, Martin Paul Jr. & Nshakira-Rukundo, Emmanuel, 2021. "Religiosity and parental educational aspirations for children in Kenya," World Development Perspectives, Elsevier, vol. 23(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Paul Diegert & Matthew A. Masten & Alexandre Poirier, 2022. "Assessing Omitted Variable Bias when the Controls are Endogenous," Papers 2206.02303, arXiv.org, revised Jul 2023.
    2. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar A., 2017. "Unobservable, but unimportant? The relevance of usually unobserved variables for the evaluation of labor market policies," Labour Economics, Elsevier, vol. 46(C), pages 14-25.
    3. Andrea Ichino & Fabrizia Mealli & Tommaso Nannicini, 2008. "From temporary help jobs to permanent employment: what can we learn from matching estimators and their sensitivity?," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(3), pages 305-327.
    4. Matthew A. Masten & Alexandre Poirier, 2022. "The Effect of Omitted Variables on the Sign of Regression Coefficients," Papers 2208.00552, arXiv.org, revised Feb 2023.
    5. Brett R. Gordon & Florian Zettelmeyer & Neha Bhargava & Dan Chapsky, 2019. "A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook," Marketing Science, INFORMS, vol. 38(2), pages 193-225, March.
    6. Caliendo, Marco & Mahlstedt, Robert & Mitnik, Oscar, 2014. "Unobservable, but Unimportant? The Influence of Personality Traits (and Other Usually Unobserved Variables) for the Estimation of Treatment Effects," VfS Annual Conference 2014 (Hamburg): Evidence-based Economic Policy 100502, Verein für Socialpolitik / German Economic Association.
    7. Tommaso Nannicini, 2007. "Simulation-based sensitivity analysis for matching estimators," Stata Journal, StataCorp LP, vol. 7(3), pages 334-350, September.
    8. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.
    9. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    10. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    11. Dettmann, E. & Becker, C. & Schmeißer, C., 2011. "Distance functions for matching in small samples," Computational Statistics & Data Analysis, Elsevier, vol. 55(5), pages 1942-1960, May.
    12. Takahashi, Ryo, 2021. "How to stimulate environmentally friendly consumption: Evidence from a nationwide social experiment in Japan to promote eco-friendly coffee," Ecological Economics, Elsevier, vol. 186(C).
    13. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    14. Sascha O. Becker & Marco Caliendo, 2007. "Sensitivity analysis for average treatment effects," Stata Journal, StataCorp LP, vol. 7(1), pages 71-83, February.
    15. Colnet Bénédicte & Josse Julie & Varoquaux Gaël & Scornet Erwan, 2022. "Causal effect on a target population: A sensitivity analysis to handle missing covariates," Journal of Causal Inference, De Gruyter, vol. 10(1), pages 372-414, January.
    16. Myoung Lee & Sang Lee, 2009. "Sensitivity analysis of job-training effects on reemployment for Korean women," Empirical Economics, Springer, vol. 36(1), pages 81-107, February.
    17. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    18. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    19. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    20. Matthew A. Masten & Alexandre Poirier, 2020. "Inference on breakdown frontiers," Quantitative Economics, Econometric Society, vol. 11(1), pages 41-111, January.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2012.15716. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.