IDEAS home Printed from https://ideas.repec.org/p/ifs/cemmap/20-17.html
   My bibliography  Save this paper

Inference on breakdown frontiers

Author

Listed:
  • Matthew Masten

    (Institute for Fiscal Studies and Duke University)

  • Alexandre Poirier

    (Institute for Fiscal Studies and Georgetown University)

Abstract

A breakdown frontier is the boundary between the set of assumptions which lead to a specifi c conclusion and those which do not. In a potential outcomes model with a binary treatment, we consider two conclusions: First, that ATE is at least a specifi c value (e.g., nonnegative) and second that the proportion of units who bene fit from treatment is at least a speci c value (e.g., at least 50%). For these conclusions, we derive the breakdown frontier for two kinds of assumptions: one which indexes deviations from random assignment of treatment, and one which indexes deviations from rank invariance. These classes of assumptions nest both the point identifying assumptions of random assignment and rank invariance and the opposite end of no constraints on treatment selection or the dependence structure between potential outcomes. This frontier provides a quantitative measure of robustness of conclusions to deviations in the point identifying assumptions. We derive vN-consistent sample analog estimators for these frontiers. We then provide an asymptotically valid bootstrap procedure for constructing lower uniform confi dence bands for the breakdown frontier. As a measure of robustness, this confi dence band can be presented alongside traditional point estimates and con fidence intervals obtained under point identifying assumptions. We illustrate this approach in an empirical application to the e ffect of child soldiering on wages. We fi nd that conclusions are fairly robust to failure of rank invariance, when random assignment holds, but conclusions are much more sensitive to both assumptions for small deviations from random assignment.

Suggested Citation

  • Matthew Masten & Alexandre Poirier, 2017. "Inference on breakdown frontiers," CeMMAP working papers CWP20/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
  • Handle: RePEc:ifs:cemmap:20/17
    as

    Download full text from publisher

    File URL: https://www.ifs.org.uk/uploads/cemmap/wps/CWP201717.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Ivan A. Canay & Azeem M. Shaikh, 2016. "Practical and theoretical advances in inference for partially identified models," CeMMAP working papers CWP05/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 151-184, February.
    3. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    4. Guildo W. Imbens, 2003. "Sensitivity to Exogeneity Assumptions in Program Evaluation," American Economic Review, American Economic Association, vol. 93(2), pages 126-132, May.
    5. Patrick Kline & Andres Santos, 2013. "Sensitivity to missing data assumptions: Theory and an evaluation of the U.S. wage structure," Quantitative Economics, Econometric Society, vol. 4(2), pages 231-267, July.
    6. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," Review of Economic Studies, Oxford University Press, vol. 64(4), pages 487-535.
    7. D. Todem & J. Fine & L. Peng, 2010. "A Global Sensitivity Test for Evaluating Statistical Hypotheses with Nonidentifiable Models," Biometrics, The International Biometric Society, vol. 66(2), pages 558-566, June.
    8. Horowitz, Joel L & Manski, Charles F, 1995. "Identification and Robustness with Contaminated and Corrupted Data," Econometrica, Econometric Society, vol. 63(2), pages 281-302, March.
    9. Horowitz, Joel L. & Lee, Sokbae, 2012. "Uniform confidence bands for functions estimated nonparametrically with instrumental variables," Journal of Econometrics, Elsevier, vol. 168(2), pages 175-188.
    10. Matthew Masten & Alexandre Poirier, 2016. "Partial independence in nonseparable models," CeMMAP working papers CWP26/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    11. Juan Carlos Escanciano & Lin Zhu, 2013. "Set inferences and sensitivity analysis in semiparametric conditionally identified models," CeMMAP working papers CWP55/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    12. Victor Chernozhukov & Iv·n Fern·ndez-Val & Alfred Galichon, 2010. "Quantile and Probability Curves Without Crossing," Econometrica, Econometric Society, vol. 78(3), pages 1093-1125, May.
    13. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    14. Rosa L. Matzkin, 2003. "Nonparametric Estimation of Nonadditive Random Functions," Econometrica, Econometric Society, vol. 71(5), pages 1339-1375, September.
    15. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2017. "Measuring the Sensitivity of Parameter Estimates to Estimation Moments," The Quarterly Journal of Economics, Oxford University Press, vol. 132(4), pages 1553-1592.
    16. Yanqin Fan & Sang Soo Park, 2009. "Partial identification of the distribution of treatment effects and its confidence sets," Advances in Econometrics, in: Nonparametric Econometric Methods, pages 3-70, Emerald Group Publishing Limited.
    17. Christopher Blattman & Jeannie Annan, 2010. "The Consequences of Child Soldiering," The Review of Economics and Statistics, MIT Press, vol. 92(4), pages 882-898, November.
    18. Vaart,A. W. van der, 2000. "Asymptotic Statistics," Cambridge Books, Cambridge University Press, number 9780521784504, January.
    19. Francis DiTraglia & Camilo García-Jimeno, 2016. "A Framework for Eliciting, Incorporating, and Disciplining Identification Beliefs in Linear Models," NBER Working Papers 22621, National Bureau of Economic Research, Inc.
    20. Abbring, Jaap H. & Heckman, James J., 2007. "Econometric Evaluation of Social Programs, Part III: Distributional Treatment Effects, Dynamic Treatment Effects, Dynamic Discrete Choice, and General Equilibrium Policy Evaluation," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 72, Elsevier.
    21. Brendan Kline & Elie Tamer, 2016. "Bayesian inference in a class of partially identified models," Quantitative Economics, Econometric Society, vol. 7(2), pages 329-366, July.
    22. Koop, Gary & Poirier, Dale J., 1997. "Learning about the across-regime correlation in switching regression models," Journal of Econometrics, Elsevier, vol. 78(2), pages 217-227, June.
    23. Li, Qi & Racine, Jeffrey S, 2008. "Nonparametric Estimation of Conditional CDF and Quantile Functions With Mixed Categorical and Continuous Data," Journal of Business & Economic Statistics, American Statistical Association, vol. 26, pages 423-434.
    24. Hyungsik Roger Moon & Frank Schorfheide, 2012. "Bayesian and Frequentist Inference in Partially Identified Models," Econometrica, Econometric Society, vol. 80(2), pages 755-782, March.
    25. Rotnitzky Andrea & Daniel Scharfstein & Ting‐Li Su & James Robins, 2001. "Methods for Conducting Sensitivity Analysis of Trials with Potentially Nonignorable Competing Causes of Censoring," Biometrics, The International Biometric Society, vol. 57(1), pages 103-113, March.
    26. John Copas & Shinto Eguchi, 2001. "Local sensitivity approximations for selectivity bias," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 63(4), pages 871-895.
    27. Alan M. Polansky & William. R. Schucany, 1997. "Kernel Smoothing to Improve Bootstrap Confidence Intervals," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 59(4), pages 821-838.
    28. Kaido, Hiroaki, 2016. "A dual approach to inference for partially identified econometric models," Journal of Econometrics, Elsevier, vol. 192(1), pages 269-290.
    29. Christian Léger & Joseph Romano, 1990. "Bootstrap choice of tuning parameters," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 42(4), pages 709-735, December.
    30. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, January.
    31. Yanqin Fan & Andrew J. Patton, 2014. "Copulas in Econometrics," Annual Review of Economics, Annual Reviews, vol. 6(1), pages 179-200, August.
    32. Cao, Ricardo & Cuevas, Antonio & Gonzalez Manteiga, Wensceslao, 1994. "A comparative study of several smoothing methods in density estimation," Computational Statistics & Data Analysis, Elsevier, vol. 17(2), pages 153-176, February.
    33. Fan, Yanqin & Guerre, Emmanuel & Zhu, Dongming, 2017. "Partial identification of functionals of the joint distribution of “potential outcomes”," Journal of Econometrics, Elsevier, vol. 197(1), pages 42-59.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Callaway, Brantly, 2021. "Bounds on distributional treatment effect parameters using panel data with an application on job displacement," Journal of Econometrics, Elsevier, vol. 222(2), pages 861-881.
    2. Gunsilius, Florian F., 2023. "A condition for the identification of multivariate models with binary instruments," Journal of Econometrics, Elsevier, vol. 235(1), pages 220-238.
    3. Kaspar Wüthrich, 2020. "A Comparison of Two Quantile Models With Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(2), pages 443-456, April.
    4. Pedro H. C. Sant'Anna & Xiaojun Song & Qi Xu, 2022. "Covariate distribution balance via propensity scores," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 37(6), pages 1093-1120, September.
    5. Hiroaki Kaido & Jiaxuan Li & Marc Rysman, 2018. "Moment inequalities in the context of simulated and predicted variables," CeMMAP working papers CWP26/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Marco Morucci & Vittorio Orlandi & Harsh Parikh & Sudeepa Roy & Cynthia Rudin & Alexander Volfovsky, 2023. "A Double Machine Learning Approach to Combining Experimental and Observational Data," Papers 2307.01449, arXiv.org.
    7. Firpo, Sergio & Galvao, Antonio F. & Parker, Thomas, 2023. "Uniform inference for value functions," Journal of Econometrics, Elsevier, vol. 235(2), pages 1680-1699.
    8. Roy Allen & John Rehbeck, 2020. "Counterfactual and Welfare Analysis with an Approximate Model," Papers 2009.03379, arXiv.org.
    9. Vitor Possebom, 2021. "Crime and Mismeasured Punishment: Marginal Treatment Effect with Misclassification," Papers 2106.00536, arXiv.org, revised Jul 2023.
    10. Sergio Firpo & Antonio F. Galvao & Martyna Kobus & Thomas Parker & Pedro Rosa-Dias, 2020. "Loss aversion and the welfare ranking of policy interventions," Papers 2004.08468, arXiv.org, revised Sep 2023.
    11. Stéphane Bonhomme & Martin Weidner, 2018. "Minimizing sensitivity to model misspecification," CeMMAP working papers CWP59/18, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    12. Zheng Fang & Juwon Seo, 2019. "A Projection Framework for Testing Shape Restrictions That Form Convex Cones," Papers 1910.07689, arXiv.org, revised Sep 2021.
    13. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "Transparency in Structural Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 711-722, October.
    14. Matthew A. Masten & Alexandre Poirier, 2021. "Salvaging Falsified Instrumental Variable Models," Econometrica, Econometric Society, vol. 89(3), pages 1449-1469, May.
    15. Stéphane Bonhomme & Martin Weidner, 2020. "Minimizing Sensitivity to Model Misspecification," CeMMAP working papers CWP37/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    16. Raffaella Giacomini & Toru Kitagawa & Alessio Volpicella, 2017. "Uncertain identification," CeMMAP working papers CWP18/17, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    17. Tamara Broderick & Ryan Giordano & Rachael Meager, 2020. "An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?," Papers 2011.14999, arXiv.org, revised Jul 2023.
    18. Pietro Emilio Spini, 2021. "Robustness, Heterogeneous Treatment Effects and Covariate Shifts," Papers 2112.09259, arXiv.org.
    19. Claudia Noack, 2021. "Sensitivity of LATE Estimates to Violations of the Monotonicity Assumption," Papers 2106.06421, arXiv.org.
    20. Candelaria, Luis E. & Ura, Takuya, 2023. "Identification and inference of network formation games with misclassified links," Journal of Econometrics, Elsevier, vol. 235(2), pages 862-891.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francesca Molinari, 2020. "Microeconometrics with Partial Identi?cation," CeMMAP working papers CWP15/20, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    2. Matthew Masten & Alexandre Poirier, 2016. "Partial independence in nonseparable models," CeMMAP working papers CWP26/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    3. Firpo, Sergio & Galvao, Antonio F. & Parker, Thomas, 2023. "Uniform inference for value functions," Journal of Econometrics, Elsevier, vol. 235(2), pages 1680-1699.
    4. Ivan A. Canay & Azeem M. Shaikh, 2016. "Practical and theoretical advances in inference for partially identified models," CeMMAP working papers CWP05/16, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    5. Juan Carlos Escanciano & Lin Zhu, 2013. "Set inferences and sensitivity analysis in semiparametric conditionally identified models," CeMMAP working papers CWP55/13, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Callaway, Brantly, 2021. "Bounds on distributional treatment effect parameters using panel data with an application on job displacement," Journal of Econometrics, Elsevier, vol. 222(2), pages 861-881.
    7. Francesca Molinari, 2019. "Econometrics with Partial Identification," CeMMAP working papers CWP25/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    8. Pietro Emilio Spini, 2021. "Robustness, Heterogeneous Treatment Effects and Covariate Shifts," Papers 2112.09259, arXiv.org.
    9. Claudia Noack, 2021. "Sensitivity of LATE Estimates to Violations of the Monotonicity Assumption," Papers 2106.06421, arXiv.org.
    10. Blaise Melly und Kaspar W thrich, 2016. "Local quantile treatment effects," Diskussionsschriften dp1605, Universitaet Bern, Departement Volkswirtschaft.
    11. Juan Carlos Escanciano & Lin Zhu, 2013. "Set inferences and sensitivity analysis in semiparametric conditionally identified models," CeMMAP working papers 55/13, Institute for Fiscal Studies.
    12. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    13. Jiannan Lu & Peng Ding & Tirthankar Dasgupta, 2018. "Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 540-567, October.
    14. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    15. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    16. Arun G. Chandrasekhar & Victor Chernozhukov & Francesca Molinari & Paul Schrimpf, 2019. "Best Linear Approximations to Set Identified Functions: With an Application to the Gender Wage Gap," NBER Working Papers 25593, National Bureau of Economic Research, Inc.
    17. Sungwon Lee, 2024. "Partial identification and inference for conditional distributions of treatment effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(1), pages 107-127, January.
    18. St'ephane Bonhomme & Martin Weidner, 2018. "Minimizing Sensitivity to Model Misspecification," Papers 1807.02161, arXiv.org, revised Oct 2021.
    19. Wenlong Ji & Lihua Lei & Asher Spector, 2023. "Model-Agnostic Covariate-Assisted Inference on Partially Identified Causal Effects," Papers 2310.08115, arXiv.org.
    20. Matthew A. Masten & Alexandre Poirier, 2018. "Identification of Treatment Effects Under Conditional Partial Independence," Econometrica, Econometric Society, vol. 86(1), pages 317-351, January.

    More about this item

    Keywords

    Nonparametric Identification; Partial Identification; Sensitivity Analysis; Selection on Unobservables; Rank Invariance; Treatment Effects;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • C18 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Methodolical Issues: General
    • C21 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Cross-Sectional Models; Spatial Models; Treatment Effect Models
    • C25 - Mathematical and Quantitative Methods - - Single Equation Models; Single Variables - - - Discrete Regression and Qualitative Choice Models; Discrete Regressors; Proportions; Probabilities
    • C51 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Construction and Estimation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ifs:cemmap:20/17. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Emma Hyman (email available below). General contact details of provider: https://edirc.repec.org/data/cmifsuk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.