IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v43y2018i5p540-567.html
   My bibliography  Save this article

Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds

Author

Listed:
  • Jiannan Lu

    (Microsoft Corporation)

  • Peng Ding

    (University of California-Berkeley)

  • Tirthankar Dasgupta

    (Rutgers University)

Abstract

Assessing the causal effects of interventions on ordinal outcomes is an important objective of many educational and behavioral studies. Under the potential outcomes framework, we can define causal effects as comparisons between the potential outcomes under treatment and control. However, unfortunately, the average causal effect, often the parameter of interest, is difficult to interpret for ordinal outcomes. To address this challenge, we propose to use two causal parameters, which are defined as the probabilities that the treatment is beneficial and strictly beneficial for the experimental units. However, although well-defined for any outcomes and of particular interest for ordinal outcomes, the two aforementioned parameters depend on the association between the potential outcomes and are therefore not identifiable from the observed data without additional assumptions. Echoing recent advances in the econometrics and biostatistics literature, we present the sharp bounds of the aforementioned causal parameters for ordinal outcomes, under fixed marginal distributions of the potential outcomes. Because the causal estimands and their corresponding sharp bounds are based on the potential outcomes themselves, the proposed framework can be flexibly incorporated into any chosen models of the potential outcomes and is directly applicable to randomized experiments, unconfounded observational studies, and randomized experiments with noncompliance. We illustrate our methodology via numerical examples and three real-life applications related to educational and behavioral research.

Suggested Citation

  • Jiannan Lu & Peng Ding & Tirthankar Dasgupta, 2018. "Treatment Effects on Ordinal Outcomes: Causal Estimands and Sharp Bounds," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 540-567, October.
  • Handle: RePEc:sae:jedbes:v:43:y:2018:i:5:p:540-567
    DOI: 10.3102/1076998618776435
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998618776435
    Download Restriction: no

    File URL: https://libkey.io/10.3102/1076998618776435?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peng Ding & Tirthankar Dasgupta, 2016. "A Potential Tale of Two-by-Two Tables From Completely Randomized Experiments," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(513), pages 157-168, March.
    2. Charles F. Manski, 1997. "Monotone Treatment Response," Econometrica, Econometric Society, vol. 65(6), pages 1311-1334, November.
    3. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    4. Laber, Eric B. & Murphy, Susan A., 2011. "Adaptive Confidence Intervals for the Test Error in Classification," Journal of the American Statistical Association, American Statistical Association, vol. 106(495), pages 904-913.
    5. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 487-535.
    6. Kreider, Brent & Pepper, John V., 2007. "Disability and Employment: Reevaluating the Evidence in Light of Reporting Errors," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 432-441, June.
    7. Keisuke Hirano & Guido W. Imbens & Geert Ridder, 2003. "Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score," Econometrica, Econometric Society, vol. 71(4), pages 1161-1189, July.
    8. Yanqin Fan & Sang Soo Park, 2009. "Partial identification of the distribution of treatment effects and its confidence sets," Advances in Econometrics, in: Nonparametric Econometric Methods, pages 3-70, Emerald Group Publishing Limited.
    9. David S. Lee, 2009. "Training, Wages, and Sample Selection: Estimating Sharp Bounds on Treatment Effects," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 76(3), pages 1071-1102.
    10. Keisuke Hirano & Jack R. Porter, 2012. "Impossibility Results for Nondifferentiable Functionals," Econometrica, Econometric Society, vol. 80(4), pages 1769-1790, July.
    11. Jing Cheng, 2009. "Estimation and Inference for the Causal Effect of Receiving Treatment on a Multinomial Outcome," Biometrics, The International Biometric Society, vol. 65(1), pages 96-103, March.
    12. Joseph P. Romano & Azeem M. Shaikh, 2010. "Inference for the Identified Set in Partially Identified Econometric Models," Econometrica, Econometric Society, vol. 78(1), pages 169-211, January.
    13. Gary L. Gadbury & Hari K. Iyer, 2000. "Unit–Treatment Interaction and Its Practical Consequences," Biometrics, The International Biometric Society, vol. 56(3), pages 882-885, September.
    14. Leonardo Grilli & Fabrizia Mealli, 2008. "Nonparametric Bounds on the Causal Effect of University Studies on Job Opportunities Using Principal Stratification," Journal of Educational and Behavioral Statistics, , vol. 33(1), pages 111-130, March.
    15. Zhang, Junni L. & Rubin, Donald B. & Mealli, Fabrizia, 2009. "Likelihood-Based Analysis of Causal Effects of Job-Training Programs Using Principal Stratification," Journal of the American Statistical Association, American Statistical Association, vol. 104(485), pages 166-176.
    16. Chuan Ju & Zhi Geng, 2010. "Criteria for surrogate end points based on causal distributions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 72(1), pages 129-142, January.
    17. Stuart G. Baker, 2011. "Estimation and Inference for the Causal Effect of Receiving Treatment on a Multinomial Outcome: An Alternative Approach," Biometrics, The International Biometric Society, vol. 67(1), pages 319-323, March.
    18. R. Bradley & S. Katti & Irma Coons, 1962. "Optimal scaling for ordered categories," Psychometrika, Springer;The Psychometric Society, vol. 27(4), pages 355-374, December.
    19. Alan Agresti & Maria Kateri, 2017. "Ordinal probability effect measures for group comparisons in multinomial cumulative link models," Biometrics, The International Biometric Society, vol. 73(1), pages 214-219, March.
    20. Jing Cheng & Dylan S. Small, 2006. "Bounds on causal effects in three‐arm trials with non‐compliance," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 68(5), pages 815-836, November.
    21. Paolo Frumento & Fabrizia Mealli & Barbara Pacini & Donald B. Rubin, 2012. "Evaluating the Effect of Training on Wages in the Presence of Noncompliance, Nonemployment, and Missing Outcome Data," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 107(498), pages 450-466, June.
    22. Djebbari, Habiba & Smith, Jeffrey, 2008. "Heterogeneous impacts in PROGRESA," Journal of Econometrics, Elsevier, vol. 145(1-2), pages 64-80, July.
    23. Fabrizia Mealli & Barbara Pacini, 2013. "Using Secondary Outcomes to Sharpen Inference in Randomized Experiments With Noncompliance," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(503), pages 1120-1131, September.
    24. Dustin M. Long & Michael G. Hudgens, 2013. "Sharpening Bounds on Principal Effects with Covariates," Biometrics, The International Biometric Society, vol. 69(4), pages 812-819, December.
    25. Fan, Yanqin & Park, Sang Soo, 2010. "Sharp Bounds On The Distribution Of Treatment Effects And Their Statistical Inference," Econometric Theory, Cambridge University Press, vol. 26(3), pages 931-951, June.
    26. Charles F. Manski & John V. Pepper, 2000. "Monotone Instrumental Variables, with an Application to the Returns to Schooling," Econometrica, Econometric Society, vol. 68(4), pages 997-1012, July.
    27. Charles F. Manski & John V. Pepper, 2009. "More on monotone instrumental variables," Econometrics Journal, Royal Economic Society, vol. 12(s1), pages 200-216, January.
    28. Yanqin Fan & Robert Sherman & Matthew Shum, 2014. "Identifying Treatment Effects Under Data Combination," Econometrica, Econometric Society, vol. 82(2), pages 811-822, March.
    29. Iván Díaz & Elizabeth Colantuoni & Michael Rosenblum, 2016. "Enhanced precision in the analysis of randomized trials with ordinal outcomes," Biometrics, The International Biometric Society, vol. 72(2), pages 422-431, June.
    30. Daniel O. Scharfstein & Charles F. Manski & James C. Anthony, 2004. "On the Construction of Bounds in Prospective Studies with Missing Ordinal Outcomes: Application to the Good Behavior Game Trial," Biometrics, The International Biometric Society, vol. 60(1), pages 154-164, March.
    31. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiannan Lu & Yunshu Zhang & Peng Ding, 2020. "Sharp bounds on the relative treatment effect for ordinal outcomes," Biometrics, The International Biometric Society, vol. 76(2), pages 664-669, June.
    2. Fan Yang & Dylan S. Small, 2016. "Using post-outcome measurement information in censoring-by-death problems," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 299-318, January.
    3. Francesca Molinari, 2020. "Microeconometrics with Partial Identification," Papers 2004.11751, arXiv.org.
    4. Callaway, Brantly, 2021. "Bounds on distributional treatment effect parameters using panel data with an application on job displacement," Journal of Econometrics, Elsevier, vol. 222(2), pages 861-881.
    5. Francesca Molinari, 2019. "Econometrics with Partial Identification," CeMMAP working papers CWP25/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Ho, Kate & Rosen, Adam M., 2015. "Partial Identification in Applied Research: Benefits and Challenges," CEPR Discussion Papers 10883, C.E.P.R. Discussion Papers.
    7. Tsunao Okumura & Emiko Usui, 2014. "Concave‐monotone treatment response and monotone treatment selection: With an application to the returns to schooling," Quantitative Economics, Econometric Society, vol. 5, pages 175-194, March.
    8. Victor Chernozhukov & Sokbae Lee & Adam M. Rosen, 2013. "Intersection Bounds: Estimation and Inference," Econometrica, Econometric Society, vol. 81(2), pages 667-737, March.
    9. Vira Semenova, 2023. "Aggregated Intersection Bounds and Aggregated Minimax Values," Papers 2303.00982, arXiv.org, revised Jun 2024.
    10. Chen, Xuan & Flores, Carlos A. & Flores-Lagunes, Alfonso, 2015. "Going Beyond LATE: Bounding Average Treatment Effects of Job Corps Training," IZA Discussion Papers 9511, Institute of Labor Economics (IZA).
    11. Fan, Yanqin & Park, Sang Soo, 2012. "Confidence intervals for the quantile of treatment effects in randomized experiments," Journal of Econometrics, Elsevier, vol. 167(2), pages 330-344.
    12. Lu, Jiannan, 2018. "On the partial identification of a new causal measure for ordinal outcomes," Statistics & Probability Letters, Elsevier, vol. 137(C), pages 1-7.
    13. Pablo Lavado & Gonzalo Rivera, 2016. "Identifying Treatment Effects with Data Combination and Unobserved Heterogeneity," Working Papers 79, Peruvian Economic Association.
    14. Sungwon Lee, 2024. "Partial identification and inference for conditional distributions of treatment effects," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 39(1), pages 107-127, January.
    15. Firpo, Sergio & Galvao, Antonio F. & Parker, Thomas, 2023. "Uniform inference for value functions," Journal of Econometrics, Elsevier, vol. 235(2), pages 1680-1699.
    16. Qian, Hang, 2011. "Bayesian inference with monotone instrumental variables," MPRA Paper 32672, University Library of Munich, Germany.
    17. Fan, Yanqin & Guerre, Emmanuel & Zhu, Dongming, 2017. "Partial identification of functionals of the joint distribution of “potential outcomes”," Journal of Econometrics, Elsevier, vol. 197(1), pages 42-59.
    18. Sungwon Lee, 2021. "Partial Identification and Inference for Conditional Distributions of Treatment Effects," Papers 2108.00723, arXiv.org, revised Nov 2023.
    19. Magnac, Thierry, 2013. "Identification partielle : méthodes et conséquences pour les applications empiriques," L'Actualité Economique, Société Canadienne de Science Economique, vol. 89(4), pages 233-258, Décembre.
    20. Kédagni, Désiré, 2023. "Identifying treatment effects in the presence of confounded types," Journal of Econometrics, Elsevier, vol. 234(2), pages 479-511.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:43:y:2018:i:5:p:540-567. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.