IDEAS home Printed from https://ideas.repec.org/p/chy/respap/115cherp.html
   My bibliography  Save this paper

Multidimensional performance assessment using dominance criteria

Author

Listed:
  • Nils Gutacker

    (Centre for Health Economics, University of York, UK)

  • Andrew Street

    (Centre for Health Economics, University of York, UK)

Abstract

Public sector organisations pursue multiple objectives and serve a number of stakeholders. But stakeholders are rarely explicit about the valuations they attach to different objectives, nor are these valuations likely to be identical. This complicates the assessment of their performance because no single set of weights can be legitimately chosen by regulators to aggregate outputs into unidimensional composite scores. We propose the use of dominance criteria in a multidimensional performance assessment framework to identify best practice and poor performance under relatively weak assumptions about stakeholders’ preferences. We estimate multivariate multilevel models to study providers of hip replacement surgery in the English NHS with respect to their performance in terms of length of stay, readmission rates, post-operative patient-reported health status and waiting time. We find substantial correlation between objectives and demonstrate that ignoring the correlation can lead to incorrect assessments of performance.

Suggested Citation

  • Nils Gutacker & Andrew Street, 2015. "Multidimensional performance assessment using dominance criteria," Working Papers 115cherp, Centre for Health Economics, University of York.
  • Handle: RePEc:chy:respap:115cherp
    as

    Download full text from publisher

    File URL: http://www.york.ac.uk/media/che/documents/papers/researchpapers/CHERP115_Multidimensional_performance_assessment_dominance_criteria.pdf
    File Function: First version, 2015
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Peter C. Smith & Andrew Street, 2005. "Measuring the efficiency of public services: the limits of analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(2), pages 401-417, March.
    2. Chew Lian Chua & Alfons Palangkaraya & Jongsay Yong, 2010. "A two‐stage estimation of hospital quality using mortality outcome measures: an application using hospital administrative data," Health Economics, John Wiley & Sons, Ltd., vol. 19(12), pages 1404-1424, December.
    3. P.C. Smith, 2002. "Measuring health system performance," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 3(3), pages 145-148, September.
    4. Terza, Joseph V. & Basu, Anirban & Rathouz, Paul J., 2008. "Two-stage residual inclusion estimation: Addressing endogeneity in health econometric modeling," Journal of Health Economics, Elsevier, vol. 27(3), pages 531-543, May.
    5. Homer J. Hall, 2004. "Quality of information," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(1), pages 92-92, January.
    6. Devlin, N. & Sussex, J., 2011. "Incorporating Multiple Criteria in HTA: Methods and Processes," Monographs, Office of Health Economics, number 000189.
    7. Andrei Shleifer, 1985. "A Theory of Yardstick Competition," RAND Journal of Economics, The RAND Corporation, vol. 16(3), pages 319-327, Autumn.
    8. Keeler, Emmett B., 1990. "What proportion of hospital cost differences is justifiable?," Journal of Health Economics, Elsevier, vol. 9(3), pages 359-365, November.
    9. Nancy J. Devlin & David Parkin & John Browne, 2010. "Patient‐reported outcome measures in the NHS: new methods for analysing and reporting EQ‐5D data," Health Economics, John Wiley & Sons, Ltd., vol. 19(8), pages 886-905, August.
    10. Avinash Dixit, 2002. "# Incentives and Organizations in the Public Sector: An Interpretative Review," Journal of Human Resources, University of Wisconsin Press, vol. 37(4), pages 696-727.
    11. Trevor C. Bailey & Paul J. Hewson, 2004. "Simultaneous modelling of multiple traffic safety performance indicators by using a multivariate generalized linear mixed model," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 167(3), pages 501-517, August.
    12. Maria Goddard & Russell Mannion & Peter Smith, 2000. "Enhancing performance in health care: a theoretical perspective on agency and the role of information," Health Economics, John Wiley & Sons, Ltd., vol. 9(2), pages 95-107, March.
    13. Hauck, Katharina & Street, Andrew, 2006. "Performance assessment in the context of multiple objectives: A multivariate multilevel analysis," Journal of Health Economics, Elsevier, vol. 25(6), pages 1029-1048, November.
    14. Nils Gutacker & Chris Bojke & Silvio Daidone & Nancy Devlin & Andrew Street, 2013. "Hospital Variation in Patient-Reported Outcomes at the Level of EQ-5D Dimensions," Medical Decision Making, , vol. 33(6), pages 804-818, August.
    15. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    16. Häkkinen, Unto & Rosenqvist, Gunnar & Peltola, Mikko & Kapiainen, Satu & Rättö, Hanna & Cots, Francesc & Geissler, Alexander & Or, Zeynep & Serdén, Lisbeth & Sund, Reijo, 2014. "Quality, cost, and their trade-off in treating AMI and stroke patients in European hospitals," Health Policy, Elsevier, vol. 117(1), pages 15-27.
    17. Gowrisankaran, Gautam & Town, Robert J., 1999. "Estimating the quality of care in hospitals using instrumental variables," Journal of Health Economics, Elsevier, vol. 18(6), pages 747-767, December.
    18. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    19. Luigi Siciliani & Peter Sivey & Andrew Street, 2013. "Differences In Length Of Stay For Hip Replacement Between Public Hospitals, Specialised Treatment Centres And Private Providers: Selection Or Efficiency?," Health Economics, John Wiley & Sons, Ltd., vol. 22(2), pages 234-242, February.
    20. Hall, Bruce Lee & Hamilton, Barton H., 2004. "New information technology systems and a Bayesian hierarchical bivariate probit model for profiling surgeon quality at a large hospital," The Quarterly Review of Economics and Finance, Elsevier, vol. 44(3), pages 410-429, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Sam Watson’s journal round-up for 21st August 2017
      by Sam Watson in The Academic Health Economists' Blog on 2017-08-21 16:00:35

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Valerie Moran & Rowena Jacobs, 2018. "Investigating the relationship between costs and outcomes for English mental health providers: a bi-variate multi-level regression analysis," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 19(5), pages 709-718, June.
    2. Yelin Fu & Yubing Sui & Hao Luo & Biao Han, 2020. "Application of Social Choice Theory to Modify the Value Measure of Health Systems," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 148(3), pages 1005-1019, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Nils Gutacker & Andrew Street, 2018. "Multidimensional performance assessment of public sector organisations using dominance criteria," Health Economics, John Wiley & Sons, Ltd., vol. 27(2), pages 13-27, February.
    2. Peter C. Smith & Andrew Street, 2012. "Concepts and Challenges in Measuring the Performance of Health Care Organizations," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 32, Edward Elgar Publishing.
    3. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    4. Gianluca Fiorentini & Elisa Iezzi & Matteo Lippi Bruni & Cristina Ugolini, 2011. "Incentives in primary care and their impact on potentially avoidable hospital admissions," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 12(4), pages 297-309, August.
    5. Helen Simpson, 2009. "Productivity In Public Services," Journal of Economic Surveys, Wiley Blackwell, vol. 23(2), pages 250-276, April.
    6. Kristensen, Søren Rud & Siciliani, Luigi & Sutton, Matt, 2016. "Optimal price-setting in pay for performance schemes in health care," Journal of Economic Behavior & Organization, Elsevier, vol. 123(C), pages 57-77.
    7. Nicholas Longford & D. B. Rubin, 2006. "Performance assessment and league tables. Comparing like with like," Economics Working Papers 994, Department of Economics and Business, Universitat Pompeu Fabra.
    8. Laudicella, Mauro & Olsen, Kim Rose & Street, Andrew, 2010. "Examining cost variation across hospital departments-a two-stage multi-level approach using patient-level data," Social Science & Medicine, Elsevier, vol. 71(10), pages 1872-1881, November.
    9. Michael Kuhn & Luigi Siciliani, 2009. "Performance Indicators for Quality with Costly Falsification," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 18(4), pages 1137-1154, December.
    10. repec:dau:papers:123456789/12066 is not listed on IDEAS
    11. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2016. "Market structure, patient choice and hospital quality for elective patients," Working Papers 139cherp, Centre for Health Economics, University of York.
    12. Michel Mougeot & Florence Naegelen, 1997. "La réglementation hospitalière : tarification par pathologie ou achat de soins ?," Économie et Prévision, Programme National Persée, vol. 129(3), pages 207-220.
    13. Jin, Ying & Su, Meng, 2009. "Recommendation and repurchase intention thresholds: A joint heterogeneity response estimation," International Journal of Research in Marketing, Elsevier, vol. 26(3), pages 245-255.
    14. Giuseppe Moscelli & Hugh Gravelle & Luigi Siciliani, 2018. "Effects of Market Structure and Patient Choice on Hospital Quality for Planned Patients," School of Economics Discussion Papers 1118, School of Economics, University of Surrey.
    15. Jonathan Karnon & Orla Caffrey & Clarabelle Pham & Richard Grieve & David Ben‐Tovim & Paul Hakendorf & Maria Crotty, 2013. "Applying Risk Adjusted Cost‐Effectiveness (Rac‐E) Analysis To Hospitals: Estimating The Costs And Consequences Of Variation In Clinical Practice," Health Economics, John Wiley & Sons, Ltd., vol. 22(6), pages 631-642, June.
    16. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    17. Gutacker, Nils & Bloor, Karen & Bojke, Chris & Walshe, Kieran, 2018. "Should interventions to reduce variation in care quality target doctors or hospitals?," Health Policy, Elsevier, vol. 122(6), pages 660-666.
    18. Brigitte Dormont & Carine Milcent, 2005. "How to Regulate Heterogeneous Hospitals?," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 14(3), pages 591-621, September.
    19. Kuhn, Michael & Siciliani, Luigi, 2007. "Performance Indicators for Quality with Adverse Selection, Gaming and Inequality Aversion," CEPR Discussion Papers 6261, C.E.P.R. Discussion Papers.
    20. Tom Brijs & Dimitris Karlis & Filip Van den Bossche & Geert Wets, 2007. "A Bayesian model for ranking hazardous road sites," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 1001-1017, October.
    21. Nils Gutacker & Chris Bojke & Silvio Daidone & Nancy J. Devlin & David Parkin & Andrew Street, 2013. "Truly Inefficient Or Providing Better Quality Of Care? Analysing The Relationship Between Risk‐Adjusted Hospital Costs And Patients' Health Outcomes," Health Economics, John Wiley & Sons, Ltd., vol. 22(8), pages 931-947, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:chy:respap:115cherp. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/chyoruk.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Gill Forder (email available below). General contact details of provider: https://edirc.repec.org/data/chyoruk.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.