IDEAS home Printed from https://ideas.repec.org/a/wly/hlthec/v27y2018i2pe13-e27.html
   My bibliography  Save this article

Multidimensional performance assessment of public sector organisations using dominance criteria

Author

Listed:
  • Nils Gutacker
  • Andrew Street

Abstract

Public sector organisations pursue multiple objectives and serve a number of stakeholders. But stakeholders are rarely explicit about the valuations they attach to different objectives, nor are these valuations likely to be identical. This complicates the assessment of their performance because no single set of weights can be chosen legitimately to aggregate outputs into unidimensional composite scores. We propose the use of dominance criteria in a multidimensional performance assessment framework to identify best practice and poor performance under relatively weak assumptions about stakeholders' preferences. We use as an example providers of hip replacement surgery in the English National Health Service and estimate multivariate multilevel models to study their performance in terms of length of stay, readmission rates, post‐operative patient‐reported health status and waiting time. We find substantial correlation between objectives and demonstrate that ignoring the correlation can lead to incorrect assessments of performance.

Suggested Citation

  • Nils Gutacker & Andrew Street, 2018. "Multidimensional performance assessment of public sector organisations using dominance criteria," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 13-27, February.
  • Handle: RePEc:wly:hlthec:v:27:y:2018:i:2:p:e13-e27
    DOI: 10.1002/hec.3554
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/hec.3554
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter C. Smith & Andrew Street, 2005. "Measuring the efficiency of public services: the limits of analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 401-417, March.
    2. Chew Lian Chua & Alfons Palangkaraya & Jongsay Yong, 2010. "A two‐stage estimation of hospital quality using mortality outcome measures: an application using hospital administrative data," Health Economics, John Wiley & Sons, Ltd., vol. 19(12), pages 1404-1424, December.
    3. France R. M. Portrait & Onno van der Galiën & Bernard Van den Berg, 2016. "Measuring Healthcare Providers' Performances Within Managed Competition Using Multidimensional Quality and Cost Indicators," Health Economics, John Wiley & Sons, Ltd., vol. 25(4), pages 408-423, April.
    4. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    5. Katja Grašič & Anne Mason & Andrew Street, 2015. "Paying for the quantity and quality of hospital care: the foundations and evolution of payment policy in England," Health Economics Review, Springer, vol. 5(1), pages 1-10, December.
    6. Timothy Besley & Maitreesh Ghatak, 2003. "Incentives, Choice, and Accountability in the Provision of Public Services," Oxford Review of Economic Policy, Oxford University Press, vol. 19(2), pages 235-249, Summer.
    7. Conrad Kobel & Josselin Thuilliez & Martine Bellanger & Karl-Peter Pfeiffer, 2011. "DRG systems and similar patient classification systems in Europe," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-00643049, HAL.
    8. Anders Skrondal & Sophia Rabe‐Hesketh, 2009. "Prediction in multilevel generalized linear models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 659-687, June.
    9. Gutacker, Nils & Siciliani, Luigi & Cookson, Richard, 2016. "Waiting time prioritisation: Evidence from England," Social Science & Medicine, Elsevier, vol. 159(C), pages 140-151.
    10. F Pedraja-Chaparro & J Salinas-Jiménez & P Smith, 1999. "On the quality of the data envelopment analysis model," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 50(6), pages 636-644, June.
    11. Marie Kruse & Jan Christensen, 2013. "Is quality costly? Patient and hospital cost drivers in vascular surgery," Health Economics Review, Springer, vol. 3(1), pages 1-13, December.
    12. Andrew Briggs & Paul Fenn, 1998. "Confidence intervals or surfaces? Uncertainty on the cost-effectiveness plane," Health Economics, John Wiley & Sons, Ltd., vol. 7(8), pages 723-740.
    13. Andrei Shleifer, 1985. "A Theory of Yardstick Competition," RAND Journal of Economics, The RAND Corporation, vol. 16(3), pages 319-327, Autumn.
    14. Nancy J. Devlin & David Parkin & John Browne, 2010. "Patient‐reported outcome measures in the NHS: new methods for analysing and reporting EQ‐5D data," Health Economics, John Wiley & Sons, Ltd., vol. 19(8), pages 886-905, August.
    15. Avinash Dixit, 2002. "# Incentives and Organizations in the Public Sector: An Interpretative Review," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 696-727.
    16. Mundlak, Yair, 1978. "On the Pooling of Time Series and Cross Section Data," Econometrica, Econometric Society, vol. 46(1), pages 69-85, January.
    17. Hauck, Katharina & Street, Andrew, 2006. "Performance assessment in the context of multiple objectives: A multivariate multilevel analysis," Journal of Health Economics, Elsevier, vol. 25(6), pages 1029-1048, November.
    18. Martin Gaynor & Kate Ho & Robert J. Town, 2015. "The Industrial Organization of Health-Care Markets," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 235-284, June.
    19. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    20. Häkkinen, Unto & Rosenqvist, Gunnar & Peltola, Mikko & Kapiainen, Satu & Rättö, Hanna & Cots, Francesc & Geissler, Alexander & Or, Zeynep & Serdén, Lisbeth & Sund, Reijo, 2014. "Quality, cost, and their trade-off in treating AMI and stroke patients in European hospitals," Health Policy, Elsevier, vol. 117(1), pages 15-27.
    21. Silvio Daidone & Francesco D’Amico, 2009. "Technical efficiency, specialization and ownership form: evidences from a pooling of Italian hospitals," Journal of Productivity Analysis, Springer, vol. 32(3), pages 203-216, December.
    22. Gowrisankaran, Gautam & Town, Robert J., 1999. "Estimating the quality of care in hospitals using instrumental variables," Journal of Health Economics, Elsevier, vol. 18(6), pages 747-767, December.
    23. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    24. Manuel Gomes & Nils Gutacker & Chris Bojke & Andrew Street, 2016. "Addressing Missing Data in Patient‐Reported Outcome Measures (PROMS): Implications for the Use of PROMS for Comparing Provider Performance," Health Economics, John Wiley & Sons, Ltd., vol. 25(5), pages 515-528, May.
    25. John Geweke & Gautam Gowrisankaran & Robert J. Town, 2003. "Bayesian Inference for Hospital Quality in a Selection Model," Econometrica, Econometric Society, vol. 71(4), pages 1215-1238, July.
    26. Burgess Jr., James F. & Christiansen, Cindy L. & Michalak, Sarah E. & Morris, Carl N., 2000. "Medical profiling: improving standards and risk adjustments using hierarchical models," Journal of Health Economics, Elsevier, vol. 19(3), pages 291-309, May.
    27. Luigi Siciliani & Peter Sivey & Andrew Street, 2013. "Differences In Length Of Stay For Hip Replacement Between Public Hospitals, Specialised Treatment Centres And Private Providers: Selection Or Efficiency?," Health Economics, John Wiley & Sons, Ltd., vol. 22(2), pages 234-242, February.
    28. Hall, Bruce Lee & Hamilton, Barton H., 2004. "New information technology systems and a Bayesian hierarchical bivariate probit model for profiling surgeon quality at a large hospital," The Quarterly Review of Economics and Finance, Elsevier, vol. 44(3), pages 410-429, July.
    29. Bryan Dowd & Tami Swenson & Robert Kane & Shriram Parashuram & Robert Coulam, 2014. "Can Data Envelopment Analysis Provide A Scalar Index Of ‘Value’?," Health Economics, John Wiley & Sons, Ltd., vol. 23(12), pages 1465-1480, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yuxi Wang & Simone Ghislandi & Aleksandra Torbica, 2020. "Investigating the geographic disparity in quality of care: the case of hospital readmission after acute myocardial infarction in Italy," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 21(8), pages 1149-1168, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nils Gutacker & Andrew Street, 2015. "Multidimensional performance assessment using dominance criteria," Working Papers 115cherp, Centre for Health Economics, University of York.
    2. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2018. "Effects of Market Structure and Patient Choice on Hospital Quality for Planned Patients," School of Economics Discussion Papers 1118, School of Economics, University of Surrey.
    3. Martin Gaynor & Carol Propper & Stephan Seiler, 2012. "Free to Choose? Reform and Demand Response in the English National Health Service," CEP Discussion Papers dp1179, Centre for Economic Performance, LSE.
    4. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    5. Nils Gutacker & Chris Bojke & Silvio Daidone & Nancy J. Devlin & David Parkin & Andrew Street, 2013. "Truly Inefficient Or Providing Better Quality Of Care? Analysing The Relationship Between Risk‐Adjusted Hospital Costs And Patients' Health Outcomes," Health Economics, John Wiley & Sons, Ltd., vol. 22(8), pages 931-947, August.
    6. Yijuan Chen & Juergen Meinecke & Peter Sivey, 2016. "A Theory of Waiting Time Reporting and Quality Signaling," Health Economics, John Wiley & Sons, Ltd., vol. 25(11), pages 1355-1371, November.
    7. Yijuan Chen & Juergen Meinecke & Peter Sivey, 2013. "Can hospital waiting times be reduced by being published?," ANU Working Papers in Economics and Econometrics 2013-614, Australian National University, College of Business and Economics, School of Economics.
    8. Gaughan, James & Siciliani, Luigi & Gravelle, Hugh & Moscelli, Giuseppe, 2020. "Do small hospitals have lower quality? Evidence from the English NHS," Social Science & Medicine, Elsevier, vol. 265(C).
    9. Gutacker, Nils & Siciliani, Luigi & Moscelli, Giuseppe & Gravelle, Hugh, 2016. "Choice of hospital: Which type of quality matters?," Journal of Health Economics, Elsevier, vol. 50(C), pages 230-246.
    10. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    11. Matilde P. Machado & Ricardo Mora & Antonio Romero-Medina, 2012. "Can We Infer Hospital Quality From Medical Graduates’ Residency Choices?," Journal of the European Economic Association, European Economic Association, vol. 10(6), pages 1400-1424, December.
    12. Simon Burgess & Carol Propper & Marisa Ratto & Emma Tominey, 2017. "Incentives in the Public Sector: Evidence from a Government Agency," Economic Journal, Royal Economic Society, vol. 127(605), pages 117-141, October.
    13. Laudicella, Mauro & Olsen, Kim Rose & Street, Andrew, 2010. "Examining cost variation across hospital departments-a two-stage multi-level approach using patient-level data," Social Science & Medicine, Elsevier, vol. 71(10), pages 1872-1881, November.
    14. Sheheryar Banuri & Philip Keefer, 2016. "Mellowing with Tenure? Socialization Increases Prosocial Behavior in Public Organizations," Research in Experimental Economics, in: Sebastian J. Goerg & John R. Hamman (ed.), Experiments in Organizational Economics, volume 19, pages 127-140, Emerald Publishing Ltd.
    15. Mirco Tonin & Michael Vlassopoulos, 2015. "Are public sector workers different? Cross-European evidence from elderly workers and retirees," IZA Journal of Labor Economics, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 4(1), pages 1-21, December.
    16. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2016. "Market structure, patient choice and hospital quality for elective patients," Working Papers 139cherp, Centre for Health Economics, University of York.
    17. Michael Freeman & Nicos Savva & Stefan Scholtes, 2021. "Economies of Scale and Scope in Hospitals: An Empirical Study of Volume Spillovers," Management Science, INFORMS, vol. 67(2), pages 673-697, February.
    18. Francesco Longo & Luigi Siciliani & Andrew Street, 2019. "Are cost differences between specialist and general hospitals compensated by the prospective payment system?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(1), pages 7-26, February.
    19. Laurent Gobillon & Carine Milcent, 2013. "Spatial disparities in hospital performance," Journal of Economic Geography, Oxford University Press, vol. 13(6), pages 1013-1040, November.
    20. Jean-Michel Plassard & Nhu Tran Thi Thanh, 2009. "Liberté de choix des élèves et concurrence des établissements : un survey de l'analyse du pilotage des systèmes éducatifs par les quasi-marchés," Revue d'économie industrielle, De Boeck Université, vol. 0(2), pages 99-130.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:hlthec:v:27:y:2018:i:2:p:e13-e27. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www3.interscience.wiley.com/cgi-bin/jhome/5749 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.