IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v27y2018i2pe13-e27.html
   My bibliography  Save this article

Multidimensional performance assessment of public sector organisations using dominance criteria

Author

Listed:
  • Nils Gutacker
  • Andrew Street

Abstract

Public sector organisations pursue multiple objectives and serve a number of stakeholders. But stakeholders are rarely explicit about the valuations they attach to different objectives, nor are these valuations likely to be identical. This complicates the assessment of their performance because no single set of weights can be chosen legitimately to aggregate outputs into unidimensional composite scores. We propose the use of dominance criteria in a multidimensional performance assessment framework to identify best practice and poor performance under relatively weak assumptions about stakeholders' preferences. We use as an example providers of hip replacement surgery in the English National Health Service and estimate multivariate multilevel models to study their performance in terms of length of stay, readmission rates, post‐operative patient‐reported health status and waiting time. We find substantial correlation between objectives and demonstrate that ignoring the correlation can lead to incorrect assessments of performance.

Suggested Citation

  • Nils Gutacker & Andrew Street, 2018. "Multidimensional performance assessment of public sector organisations using dominance criteria," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 13-27, February.
  • Handle: RePEc:wly:hlthec:v:27:y:2018:i:2:p:e13-e27
    DOI: 10.1002/hec.3554
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.3554
    Download Restriction: no

    File URL: https://libkey.io/10.1002/hec.3554?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peter C. Smith & Andrew Street, 2005. "Measuring the efficiency of public services: the limits of analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 401-417, March.
    2. Chew Lian Chua & Alfons Palangkaraya & Jongsay Yong, 2010. "A two‐stage estimation of hospital quality using mortality outcome measures: an application using hospital administrative data," Health Economics, John Wiley & Sons, Ltd., vol. 19(12), pages 1404-1424, December.
    3. France R. M. Portrait & Onno van der Galiën & Bernard Van den Berg, 2016. "Measuring Healthcare Providers' Performances Within Managed Competition Using Multidimensional Quality and Cost Indicators," Health Economics, John Wiley & Sons, Ltd., vol. 25(4), pages 408-423, April.
    4. P.C. Smith, 2002. "Measuring health system performance," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 3(3), pages 145-148, September.
    5. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    6. Martin Gaynor & Kate Ho & Robert J. Town, 2015. "The Industrial Organization of Health-Care Markets," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 235-284, June.
    7. Terza, Joseph V. & Basu, Anirban & Rathouz, Paul J., 2008. "Two-stage residual inclusion estimation: Addressing endogeneity in health econometric modeling," Journal of Health Economics, Elsevier, vol. 27(3), pages 531-543, May.
    8. Katja Grašič & Anne Mason & Andrew Street, 2015. "Paying for the quantity and quality of hospital care: the foundations and evolution of payment policy in England," Health Economics Review, Springer, vol. 5(1), pages 1-10, December.
    9. Timothy Besley & Maitreesh Ghatak, 2003. "Incentives, Choice, and Accountability in the Provision of Public Services," Oxford Review of Economic Policy, Oxford University Press and Oxford Review of Economic Policy Limited, vol. 19(2), pages 235-249, Summer.
    10. Homer J. Hall, 2004. "Quality of information," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(1), pages 92-92, January.
    11. Conrad Kobel & Josselin Thuilliez & Martine Bellanger & Karl-Peter Pfeiffer, 2011. "DRG systems and similar patient classification systems in Europe," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00643049, HAL.
    12. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    13. Gutacker, Nils & Siciliani, Luigi & Cookson, Richard, 2016. "Waiting time prioritisation: Evidence from England," Social Science & Medicine, Elsevier, vol. 159(C), pages 140-151.
    14. F Pedraja-Chaparro & J Salinas-Jiménez & P Smith, 1999. "On the quality of the data envelopment analysis model," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(6), pages 636-644, June.
    15. Nancy Devlin;Jon Sussex, 2011. "Incorporating Multiple Criteria in HTA: Methods and Processes," Monograph 000189, Office of Health Economics.
    16. Marie Kruse & Jan Christensen, 2013. "Is quality costly? Patient and hospital cost drivers in vascular surgery," Health Economics Review, Springer, vol. 3(1), pages 1-13, December.
    17. Peter C. Austin & David A. Alter & Jack V. Tu, 2003. "The Use of Fixed-and Random-Effects Models for Classifying Hospitals as Mortality Outliers: A Monte Carlo Assessment," Medical Decision Making, , vol. 23(6), pages 526-539, November.
    18. Andrew Briggs & Paul Fenn, 1998. "Confidence intervals or surfaces? Uncertainty on the cost‐effectiveness plane," Health Economics, John Wiley & Sons, Ltd., vol. 7(8), pages 723-740, December.
    19. Jacobs,Rowena & Smith,Peter C. & Street,Andrew, 2006. "Measuring Efficiency in Health Care," Cambridge Books, Cambridge University Press, number 9780521851442.
    20. Andrei Shleifer, 1985. "A Theory of Yardstick Competition," RAND Journal of Economics, The RAND Corporation, vol. 16(3), pages 319-327, Autumn.
    21. Nancy J. Devlin & David Parkin & John Browne, 2010. "Patient‐reported outcome measures in the NHS: new methods for analysing and reporting EQ‐5D data," Health Economics, John Wiley & Sons, Ltd., vol. 19(8), pages 886-905, August.
    22. Avinash Dixit, 2002. "# Incentives and Organizations in the Public Sector: An Interpretative Review," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 696-727.
    23. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    24. Hauck, Katharina & Street, Andrew, 2006. "Performance assessment in the context of multiple objectives: A multivariate multilevel analysis," Journal of Health Economics, Elsevier, vol. 25(6), pages 1029-1048, November.
    25. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    26. Häkkinen, Unto & Rosenqvist, Gunnar & Peltola, Mikko & Kapiainen, Satu & Rättö, Hanna & Cots, Francesc & Geissler, Alexander & Or, Zeynep & Serdén, Lisbeth & Sund, Reijo, 2014. "Quality, cost, and their trade-off in treating AMI and stroke patients in European hospitals," Health Policy, Elsevier, vol. 117(1), pages 15-27.
    27. Silvio Daidone & Francesco D’Amico, 2009. "Technical efficiency, specialization and ownership form: evidences from a pooling of Italian hospitals," Journal of Productivity Analysis, Springer, vol. 32(3), pages 203-216, December.
    28. Gowrisankaran, Gautam & Town, Robert J., 1999. "Estimating the quality of care in hospitals using instrumental variables," Journal of Health Economics, Elsevier, vol. 18(6), pages 747-767, December.
    29. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    30. Manuel Gomes & Nils Gutacker & Chris Bojke & Andrew Street, 2016. "Addressing Missing Data in Patient‐Reported Outcome Measures (PROMS): Implications for the Use of PROMS for Comparing Provider Performance," Health Economics, John Wiley & Sons, Ltd., vol. 25(5), pages 515-528, May.
    31. John Geweke & Gautam Gowrisankaran & Robert J. Town, 2003. "Bayesian Inference for Hospital Quality in a Selection Model," Econometrica, Econometric Society, vol. 71(4), pages 1215-1238, July.
    32. Burgess Jr., James F. & Christiansen, Cindy L. & Michalak, Sarah E. & Morris, Carl N., 2000. "Medical profiling: improving standards and risk adjustments using hierarchical models," Journal of Health Economics, Elsevier, vol. 19(3), pages 291-309, May.
    33. Luigi Siciliani & Peter Sivey & Andrew Street, 2013. "Differences In Length Of Stay For Hip Replacement Between Public Hospitals, Specialised Treatment Centres And Private Providers: Selection Or Efficiency?," Health Economics, John Wiley & Sons, Ltd., vol. 22(2), pages 234-242, February.
    34. Hall, Bruce Lee & Hamilton, Barton H., 2004. "New information technology systems and a Bayesian hierarchical bivariate probit model for profiling surgeon quality at a large hospital," The Quarterly Review of Economics and Finance, Elsevier, vol. 44(3), pages 410-429, July.
    35. Bryan Dowd & Tami Swenson & Robert Kane & Shriram Parashuram & Robert Coulam, 2014. "Can Data Envelopment Analysis Provide A Scalar Index Of ‘Value’?," Health Economics, John Wiley & Sons, Ltd., vol. 23(12), pages 1465-1480, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Angela Meggiolaro & Carl Rudolf Blankart & Tom Stargardt & Jonas Schreyögg, 2023. "An econometric approach to aggregating multiple cardiovascular outcomes in German hospitals," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(5), pages 785-802, July.
    2. Yuxi Wang & Simone Ghislandi & Aleksandra Torbica, 2020. "Investigating the geographic disparity in quality of care: the case of hospital readmission after acute myocardial infarction in Italy," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(8), pages 1149-1168, November.
    3. Paul Allanson & Richard Cookson, 2022. "Comparing healthcare quality: A common framework for both ordinal and cardinal data with an application to primary care variation in England," Health Economics, John Wiley & Sons, Ltd., vol. 31(12), pages 2593-2608, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nils Gutacker & Andrew Street, 2015. "Multidimensional performance assessment using dominance criteria," Working Papers 115cherp, Centre for Health Economics, University of York.
    2. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2018. "Effects of Market Structure and Patient Choice on Hospital Quality for Planned Patients," School of Economics Discussion Papers 1118, School of Economics, University of Surrey.
    3. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    4. Nils Gutacker & Chris Bojke & Silvio Daidone & Nancy J. Devlin & David Parkin & Andrew Street, 2013. "Truly Inefficient Or Providing Better Quality Of Care? Analysing The Relationship Between Risk‐Adjusted Hospital Costs And Patients' Health Outcomes," Health Economics, John Wiley & Sons, Ltd., vol. 22(8), pages 931-947, August.
    5. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2021. "Hospital competition and quality for non‐emergency patients in the English NHS," RAND Journal of Economics, RAND Corporation, vol. 52(2), pages 382-414, June.
    6. Laudicella, Mauro & Olsen, Kim Rose & Street, Andrew, 2010. "Examining cost variation across hospital departments-a two-stage multi-level approach using patient-level data," Social Science & Medicine, Elsevier, vol. 71(10), pages 1872-1881, November.
    7. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2016. "Market structure, patient choice and hospital quality for elective patients," Working Papers 139cherp, Centre for Health Economics, University of York.
    8. Martin Gaynor & Carol Propper & Stephan Seiler, 2012. "Free to Choose? Reform and Demand Response in the English National Health Service," The Centre for Market and Public Organisation 12/297, The Centre for Market and Public Organisation, University of Bristol, UK.
    9. Roberto Colombi & Gianmaria Martini & Giorgio Vittadini, 2017. "Determinants of transient and persistent hospital efficiency: The case of Italy," Health Economics, John Wiley & Sons, Ltd., vol. 26(S2), pages 5-22, September.
    10. Yijuan Chen & Juergen Meinecke & Peter Sivey, 2013. "Can hospital waiting times be reduced by being published?," ANU Working Papers in Economics and Econometrics 2013-614, Australian National University, College of Business and Economics, School of Economics.
    11. Angela Meggiolaro & Carl Rudolf Blankart & Tom Stargardt & Jonas Schreyögg, 2023. "An econometric approach to aggregating multiple cardiovascular outcomes in German hospitals," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 24(5), pages 785-802, July.
    12. Nathan E. Wilson, 2021. "The Impact of Competition on Investment: Evidence From California Hospitals," Journal of Industrial Economics, Wiley Blackwell, vol. 69(1), pages 1-32, March.
    13. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    14. Gutacker, Nils & Bloor, Karen & Bojke, Chris & Walshe, Kieran, 2018. "Should interventions to reduce variation in care quality target doctors or hospitals?," Health Policy, Elsevier, vol. 122(6), pages 660-666.
    15. Yijuan Chen & Juergen Meinecke & Peter Sivey, 2016. "A Theory of Waiting Time Reporting and Quality Signaling," Health Economics, John Wiley & Sons, Ltd., vol. 25(11), pages 1355-1371, November.
    16. Gaughan, James & Siciliani, Luigi & Gravelle, Hugh & Moscelli, Giuseppe, 2020. "Do small hospitals have lower quality? Evidence from the English NHS," Social Science & Medicine, Elsevier, vol. 265(C).
    17. Gutacker, Nils & Siciliani, Luigi & Moscelli, Giuseppe & Gravelle, Hugh, 2016. "Choice of hospital: Which type of quality matters?," Journal of Health Economics, Elsevier, vol. 50(C), pages 230-246.
    18. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    19. Simon Burgess & Carol Propper & Marisa Ratto & Emma Tominey, 2017. "Incentives in the Public Sector: Evidence from a Government Agency," Economic Journal, Royal Economic Society, vol. 127(605), pages 117-141, October.
    20. Ferreira, D.C. & Marques, R.C., 2019. "Do quality and access to hospital services impact on their technical efficiency?," Omega, Elsevier, vol. 86(C), pages 218-236.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:27:y:2018:i:2:p:e13-e27. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.