IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v183y2020i4p1363-1385.html

Performance assessment as an application of causal inference

Author

Listed:
  • Nicholas T. Longford

Abstract

Institutions in healthcare, education and other public services are under constant pressure to perform to high standards and with efficiency. Assessment of their performance is often problematic because it either ignores important background variables of their patients, students or clients (the casemix), or adjusts for them in a way that is not equitable or transparent. We apply a method of indirect standardization motivated by the potential outcomes framework, in which we consider the hypothetical scenario of an institution's case‐load being dispersed for treatment throughout the domain of assessment (the country's institutions). The target of estimation is the difference of the means of the outcomes in the realized and hypothetical settings. The method is applied to estimating the prevalence of bronchopulmonary dysplasia in extremely preterm‐born infants in the neonatal units and their networks in Great Britain in 2017. The prevalence of bronchopulmonary dysplasia is an audit measure in the UK National Neonatal Audit Programme.

Suggested Citation

  • Nicholas T. Longford, 2020. "Performance assessment as an application of causal inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(4), pages 1363-1385, October.
    • Handle: RePEc:bla:jorssa:v:183:y:2020:i:4:p:1363-1385
    DOI: 10.1111/rssa.12529
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssa.12529
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssa.12529?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Paul R. Rosenbaum, 2004. "Design sensitivity in observational studies," Biometrika, Biometrika Trust, vol. 91(1), pages 153-164, March.
    2. Rosenbaum, Paul R., 2010. "Design Sensitivity and Efficiency in Observational Studies," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 692-702.
    3. David Spiegelhalter & Christopher Sherlaw‐Johnson & Martin Bardsley & Ian Blunt & Christopher Wood & Olivia Grigg, 2012. "Statistical methods for healthcare regulation: rating, screening and surveillance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(1), pages 1-47, January.
    4. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    5. Jan Hannig & Thomas C. M. Lee, 2009. "Generalized fiducial inference for wavelet regression," Biometrika, Biometrika Trust, vol. 96(4), pages 847-860.
    6. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    7. Imbens,Guido W. & Rubin,Donald B., 2015. "Causal Inference for Statistics, Social, and Biomedical Sciences," Cambridge Books, Cambridge University Press, number 9780521885881, August.
    8. E. I. George & V. Ročková & P. R. Rosenbaum & V. A. Satopää & J. H. Silber, 2017. "Mortality Rate Estimation and Standardization for Public Reporting: Medicare’s Hospital Compare," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 933-947, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marco Doretti & Giorgio E. Montanari, 2024. "Classification and estimation of case-mix adjusted performance indices for binary outcomes," Annals of Operations Research, Springer, vol. 342(3), pages 2201-2225, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nicholas Tibor Longford, 2016. "Decision Theory Applied to Selecting the Winners, Ranking, and Classification," Journal of Educational and Behavioral Statistics, , vol. 41(4), pages 420-442, August.
    2. Hirschauer, Norbert & Grüner, Sven & Mußhoff, Oliver & Becker, Claudia & Jantsch, Antje, 2020. "Can p-values be meaningfully interpreted without random sampling?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 14, pages 71-91.
    3. Jinglong Zhao, 2024. "Experimental Design For Causal Inference Through An Optimization Lens," Papers 2408.09607, arXiv.org, revised Aug 2024.
    4. Sulis, Isabella & Giambona, Francesca & Porcu, Mariano, 2020. "Adjusted indicators of quality and equity for monitoring the education systems over time. Insights on EU15 countries from PISA surveys," Socio-Economic Planning Sciences, Elsevier, vol. 69(C).
    5. Kuha, Jouni & Sturgis, Patrick, 2016. "Comment on ‘what to do instead of significance testing? Calculating the “number of counterfactual cases needed to disturb a finding”’ by Stephen Gorard and Jonathan Gorard," LSE Research Online Documents on Economics 66035, London School of Economics and Political Science, LSE Library.
    6. Isabella Sulis & Mariano Porcu, 2012. "Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 21(2), pages 193-209, June.
    7. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    8. Zhexiao Lin & Peng Ding & Fang Han, 2023. "Estimation Based on Nearest Neighbor Matching: From Density Ratio to Average Treatment Effect," Econometrica, Econometric Society, vol. 91(6), pages 2187-2217, November.
    9. Francesca Giambona & Mariano Porcu & Isabella Sulis, 2017. "Students Mobility: Assessing the Determinants of Attractiveness Across Competing Territorial Areas," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 133(3), pages 1105-1132, September.
    10. Bruno Arpino & Roberta Varriale, 2009. "Assessing the quality of institutions' rankings obtained through multilevel linear regression models," Working Papers 019, "Carlo F. Dondena" Centre for Research on Social Dynamics (DONDENA), Università Commerciale Luigi Bocconi.
    11. JoonHo Lee & Jonathan Che & Sophia Rabe-Hesketh & Avi Feller & Luke Miratrix, 2025. "Improving the Estimation of Site-Specific Effects and Their Distribution in Multisite Trials," Journal of Educational and Behavioral Statistics, , vol. 50(5), pages 731-764, October.
    12. Christoph Baldauf & Fredrik Eng-Larsson & Olov Isaksson, 2024. "Where to Cut the Long Tail? The Value of Carrying Inventory in Online Retail," Management Science, INFORMS, vol. 70(3), pages 1855-1874, March.
    13. Bruno ARPINO & Roberta VARRIALE, 2010. "Assessing The Quality Of Institutions’ Rankings Obtained Through Multilevel Linear Regression Models," Journal of Applied Economic Sciences, Spiru Haret University, Faculty of Financial Management and Accounting Craiova, vol. 5(1(11)_Spr), pages 7-22.
    14. Paul R. Rosenbaum, 2015. "Bahadur Efficiency of Sensitivity Analyses in Observational Studies," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(509), pages 205-217, March.
    15. Tenglong Li & Kenneth A. Frank & Mingming Chen, 2024. "A Conceptual Framework for Quantifying the Robustness of a Regression-Based Causal Inference in Observational Study," Mathematics, MDPI, vol. 12(3), pages 1-14, January.
    16. Tenglong Li & Jordan Lawson, 2021. "A generalized bootstrap procedure of the standard error and confidence interval estimation for inverse probability of treatment weighting," Papers 2109.00171, arXiv.org.
    17. George Leckie & Richard Parker & Harvey Goldstein & Kate Tilling, 2024. "Mixed-Effects Location Scale Models for Joint Modeling School Value-Added Effects on the Mean and Variance of Student Achievement," Journal of Educational and Behavioral Statistics, , vol. 49(6), pages 879-911, December.
    18. Isabella Sulis & Mariano Porcu, 2015. "Assessing Divergences in Mathematics and Reading Achievement in Italian Primary Schools: A Proposal of Adjusted Indicators of School Effectiveness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 122(2), pages 607-634, June.
    19. Isabella Sulis & Mariano Porcu & Vincenza Capursi, 2019. "On the Use of Student Evaluation of Teaching: A Longitudinal Analysis Combining Measurement Issues and Implications of the Exercise," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 142(3), pages 1305-1331, April.
    20. Matias D. Cattaneo & Luke Keele & Rocio Titiunik, 2023. "A Guide to Regression Discontinuity Designs in Medical Applications," Papers 2302.07413, arXiv.org, revised May 2023.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:183:y:2020:i:4:p:1363-1385. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.