IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v172y2009i1p161-190.html
   My bibliography  Save this article

Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK

Author

Listed:
  • Gwyn Bevan
  • Richard Hamblin

Abstract

Summary. Following devolution, differences developed between UK countries in systems of measuring performance against a common target that ambulance services ought to respond to 75% of calls for what may be immediately life threatening emergencies (category A calls) within 8 minutes. Only in England was this target integral to a ranking system of ‘star rating’, which inflicted reputational damage on services that failed to hit targets, and only in England has this target been met. In other countries, the target has been missed by such large margins that services would have been publicly reported as failing, if they had been covered by the English system of star ratings. The paper argues that this case‐study adds to evidence from comparisons of different systems of hospital performance measurement that, to have an effect, these systems need to be designed to inflict reputational damage on those that have performed poorly; and it explores implications of this hypothesis. The paper also asks questions about the adequacy of systems of performance measurement of ambulance services in UK countries.

Suggested Citation

  • Gwyn Bevan & Richard Hamblin, 2009. "Hitting and missing targets by ambulance services for emergency calls: effects of different systems of performance measurement within the UK," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(1), pages 161-190, January.
  • Handle: RePEc:bla:jorssa:v:172:y:2009:i:1:p:161-190
    DOI: 10.1111/j.1467-985X.2008.00557.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1467-985X.2008.00557.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1467-985X.2008.00557.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Simon Burgess & Denise Gossage & Carol Propper, 2003. "Explaining differences in hospital performance: Does the answer lie in the labour market?," The Centre for Market and Public Organisation 03/091, The Centre for Market and Public Organisation, University of Bristol, UK.
    2. David Dranove & Daniel Kessler & Mark McClellan & Mark Satterthwaite, 2003. "Is More Information Better? The Effects of "Report Cards" on Health Care Providers," Journal of Political Economy, University of Chicago Press, vol. 111(3), pages 555-588, June.
    3. Kelman, Steven & Friedman, John N., 2007. "Performance Improvement and Performance Dysfunction: An Empirical Examination of Impacts of the Emergency Room Wait-Time Target in the English National Health Service," Working Paper Series rwp07-034, Harvard University, John F. Kennedy School of Government.
    4. Peter Smith, 2005. "Performance Measurement in Health Care: History, Challenges and Prospects," Public Money & Management, Taylor & Francis Journals, vol. 25(4), pages 213-220.
    5. Katharina Hauck & Andrew Street, 2007. "Do targets matter? A comparison of English and Welsh National Health priorities," Health Economics, John Wiley & Sons, Ltd., vol. 16(3), pages 275-290, March.
    6. Harvey Goldstein & David J. Spiegelhalter, 1996. "League Tables and Their Limitations: Statistical Issues in Comparisons of Institutional Performance," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 159(3), pages 385-409, May.
    7. Bevan, Gwyn & Cornwell, Jocelyn, 2006. "Structure and logic of regulation and governance of quality of health care: was OFSTED a model for the Commission for Health Improvement?," Health Economics, Policy and Law, Cambridge University Press, vol. 1(4), pages 343-370, October.
    8. Patel, Brijesh & Chaussalet, Thierry & Millard, Peter, 2008. "Balancing the NHS balanced scorecard!," European Journal of Operational Research, Elsevier, vol. 185(3), pages 905-914, March.
    9. Propper Carol & Sutton Matt & Whitnall Carolyn & Windmeijer Frank, 2008. "Did 'Targets and Terror' Reduce Waiting Times in England for Hospital Care?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 8(2), pages 1-27, January.
    10. Geoffrey Heath & James Radcliffe, 2007. "Performance Measurement and the English Ambulance Service," Public Money & Management, Taylor & Francis Journals, vol. 27(3), pages 223-228, June.
    11. Rowena Jacobs & Maria Goddard, 2007. "How Do Performance Indicators Add Up? An Examination of Composite Indicators in Public Services," Public Money & Management, Taylor & Francis Journals, vol. 27(2), pages 103-110, April.
    12. D. J. Spiegelhalter, 1999. "Surgical audit: statistical lessons from Nightingale and Codman," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 162(1), pages 45-58.
    13. Gwyn Bevan, 2006. "Setting Targets for Health Care Performance: Lessons from a Case Study of the English NHS," National Institute Economic Review, National Institute of Economic and Social Research, vol. 197(1), pages 67-79, January.
    14. Alvarez-Rosete, Arturo & Bevan, Gwyn & Mays, Nicholas & Dixon, Jennifer, 2005. "Effect of diverging policy across the NHS," LSE Research Online Documents on Economics 15513, London School of Economics and Political Science, LSE Library.
    15. Philip Stevens & Lucy Stokes & Mary O’Mahony, 2006. "Metrics, Targets and Performance," National Institute Economic Review, National Institute of Economic and Social Research, vol. 197(1), pages 80-92, January.
    16. Friedman, John N. & Kelman, Steven, 2007. "Effort as Investment: Analyzing the Response to Incentives," Working Paper Series rwp07-024, Harvard University, John F. Kennedy School of Government.
    17. Christopher Hood, 2007. "Public Service Management by Numbers: Why Does it Vary? Where Has it Come From? What Are the Gaps and the Puzzles?," Public Money & Management, Taylor & Francis Journals, vol. 27(2), pages 95-102, April.
    18. Le Grand, Julian, 2003. "Motivation, Agency, and Public Policy: Of Knights and Knaves, Pawns and Queens," OUP Catalogue, Oxford University Press, number 9780199266999.
    19. Rowena Jacobs & Peter Smith, 2004. "A descriptive analysis of general acute Trust star ratings," Working Papers 189chedp, Centre for Health Economics, University of York.
    20. Sheila M. Bird & Cox Sir David & Vern T. Farewell & Goldstein Harvey & Holt Tim & Smith Peter C., 2005. "Performance indicators: good, bad, and ugly," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 168(1), pages 1-27, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. Incorporating reputational concerns in public sector reform: it may be effective but needs creative monitoring
      by Jed Friedman in Development Impact on 2012-06-06 19:02:57

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carol Propper & Deborah Wilson, 2012. "The Use of Performance Measures in Health Care Systems," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 33, Edward Elgar Publishing.
    2. Schang, Laura & Hynninen, Yrjänä & Morton, Alec & Salo, Ahti, 2016. "Developing robust composite measures of healthcare quality – Ranking intervals and dominance relations for Scottish Health Boards," Social Science & Medicine, Elsevier, vol. 162(C), pages 59-67.
    3. Burgess, Simon & Wilson, Deborah & Worth, Jack, 2013. "A natural experiment in school accountability: The impact of school performance information on pupil progress," Journal of Public Economics, Elsevier, vol. 106(C), pages 57-67.
    4. De Rosis, Sabina & Guidotti, Elisa & Zuccarino, Sara & Venturi, Giulia & Ferré, Francesca, 2020. "Waiting time information in the Italian NHS: A citizen perspective," Health Policy, Elsevier, vol. 124(8), pages 796-804.
    5. Bevan, Gwyn & Evans, Alice & Nuti, Sabina, 2018. "Reputations count: why benchmarking performance is improving health care across the world," LSE Research Online Documents on Economics 86469, London School of Economics and Political Science, LSE Library.
    6. Knight, V.A. & Harper, P.R. & Smith, L., 2012. "Ambulance allocation for maximal survival with heterogeneous outcome measures," Omega, Elsevier, vol. 40(6), pages 918-926.
    7. van de Ven, Wynand P.M.M. & Beck, Konstantin & Buchner, Florian & Schokkaert, Erik & Schut, F.T. (Erik) & Shmueli, Amir & Wasem, Juergen, 2013. "Preconditions for efficiency and affordability in competitive healthcare markets: Are they fulfilled in Belgium, Germany, Israel, the Netherlands and Switzerland?," Health Policy, Elsevier, vol. 109(3), pages 226-245.
    8. Kristian Niemietz, 2015. "Internal Markets, Management by Targets, and Quasi-Markets: An Analysis of Health Care Reforms in the English NHS," Economic Affairs, Wiley Blackwell, vol. 35(1), pages 93-108, February.
    9. Nick Manning & Joanna Watkins, 2013. "Targeting Results, Diagnosing the Means," World Bank Publications - Reports 25488, The World Bank Group.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Besley, Timothy & Burchardi, Konrad B. & Bevan, Gwen, 2009. "Naming and shaming: the impacts of different regimes on hospital waiting times in England and Wales," LSE Research Online Documents on Economics 33775, London School of Economics and Political Science, LSE Library.
    2. Propper, Carol & Sutton, Matt & Whitnall, Carolyn & Windmeijer, Frank, 2010. "Incentives and targets in hospital care: Evidence from a natural experiment," Journal of Public Economics, Elsevier, vol. 94(3-4), pages 318-335, April.
    3. Carol Propper & Deborah Wilson, 2012. "The Use of Performance Measures in Health Care Systems," Chapters, in: Andrew M. Jones (ed.), The Elgar Companion to Health Economics, Second Edition, chapter 33, Edward Elgar Publishing.
    4. Matilde P. Machado & Ricardo Mora & Antonio Romero-Medina, 2012. "Can We Infer Hospital Quality From Medical Graduates’ Residency Choices?," Journal of the European Economic Association, European Economic Association, vol. 10(6), pages 1400-1424, December.
    5. Breton, Mylaine & Smithman, Mélanie Ann & Sasseville, Martin & Kreindler, Sara A. & Sutherland, Jason M. & Beauséjour, Marie & Green, Michael & Marshall, Emily Gard & Jbilou, Jalila & Shaw, Jay & Brou, 2020. "How the design and implementation of centralized waiting lists influence their use and effect on access to healthcare - A realist review," Health Policy, Elsevier, vol. 124(8), pages 787-795.
    6. Shimaa Elkomy & Graham Cookson, 2020. "Performance Management Strategy: Waiting Time in the English National Health Services," Public Organization Review, Springer, vol. 20(1), pages 95-112, March.
    7. Martin Gaynor & Carol Propper & Stephan Seiler, 2016. "Free to Choose? Reform, Choice, and Consideration Sets in the English National Health Service," American Economic Review, American Economic Association, vol. 106(11), pages 3521-3557, November.
    8. George Leckie & Harvey Goldstein, 2009. "The limitations of using school league tables to inform school choice," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 835-851, October.
    9. David I. Ohlssen & Linda D. Sharples & David J. Spiegelhalter, 2007. "A hierarchical modelling framework for identifying unusual performance in health care providers," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 170(4), pages 865-890, October.
    10. repec:zbw:rwirep:0331 is not listed on IDEAS
    11. Martin Gaynor & Carol Propper & Stephan Seiler, 2012. "Free to Choose? Reform and Demand Response in the English National Health Service," CEP Discussion Papers dp1179, Centre for Economic Performance, LSE.
    12. Besley, Timothy J. & Bevan, Gwyn & Burchardi, Konrad B., 2009. "Naming & Shaming: The impacts of different regimes on hospital waiting times in England and Wales," CEPR Discussion Papers 7306, C.E.P.R. Discussion Papers.
    13. Juliette Malley & José‐Luis Fernández, 2010. "Measuring Quality In Social Care Services: Theory And Practice," Annals of Public and Cooperative Economics, Wiley Blackwell, vol. 81(4), pages 559-582, December.
    14. Morton, Alec & Bevan, Gwyn, 2008. "What's in a wait: Contrasting management science and economic perspectives on waiting for emergency care," Health Policy, Elsevier, vol. 85(2), pages 207-217, February.
    15. Nazarov, Vladimir (Назаров, Владимир) & Davis, Christopher Mark (Дэвис, Кристофер Марк) & Gerry, Christopher J. (Гэрри, Кристофер Дж.) & Polyakova, Aleksandra (Полякова, Александра) & Sisigina, Natali, 2015. "Evaluating the Effectiveness and Efficiency of the Health System [Оценка Эффективности И Результативности Системы Здравоохранения]," Published Papers mn41, Russian Presidential Academy of National Economy and Public Administration.
    16. Kuhn, Michael & Siciliani, Luigi, 2007. "Performance Indicators for Quality with Adverse Selection, Gaming and Inequality Aversion," CEPR Discussion Papers 6261, C.E.P.R. Discussion Papers.
    17. Nadja Kairies, 2012. "P4P, Reputation and the Reduction of Costly Overprovision," Ruhr Economic Papers 0331, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen.
    18. Vainieri, Milena & Vola, Federico & Gomez Soriano, Gregorio & Nuti, Sabina, 2016. "How to set challenging goals and conduct fair evaluation in regional public health systems. Insights from Valencia and Tuscany Regions," Health Policy, Elsevier, vol. 120(11), pages 1270-1278.
    19. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    20. Nuscheler, Robert & Roeder, Kerstin, 2015. "Financing and funding health care: Optimal policy and political implementability," Journal of Health Economics, Elsevier, vol. 42(C), pages 197-208.
    21. Tahir Andrabi & Jishnu Das & Asim Ijaz Khwaja, 2017. "Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets," American Economic Review, American Economic Association, vol. 107(6), pages 1535-1563, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:172:y:2009:i:1:p:161-190. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.