IDEAS home Printed from https://ideas.repec.org/a/kap/hcarem/v28y2025i1d10.1007_s10729-024-09693-4.html
   My bibliography  Save this article

Assessing the performance of Portuguese public hospitals before and during COVID-19 outbreak, with optimistic and pessimistic benchmarking approaches

Author

Listed:
  • Guilherme Mendes Vara

    (CEGIST, Instituto Superior Técnico, Universidade de Lisboa)

  • Marta Castilho Gomes

    (CERIS, Instituto Superior Técnico, Universidade de Lisboa)

  • Diogo Cunha Ferreira

    (CEGIST, Instituto Superior Técnico, Universidade de Lisboa
    Centre for Public Administration and Public Policies, Institute of Social and Political Sciences, Universidade de Lisboa)

Abstract

The COVID-19 pandemic had a profound impact on the tertiary sector, particularly in healthcare, which faced unprecedented demand despite the existence of limited resources, such as hospital beds, staffing resources, and funding. The magnitude and global scale of this crisis provide a compelling incentive to thoroughly analyse its effects. This study aims to identify best practices within the Portuguese national healthcare service, with the goal of improving preparedness for future crises and informing policy decisions. Using a Benefit-of-the-Doubt (BoD) approach, this research constructs composite indicators to assess the pandemic's impact on the Portuguese public hospitals. The study analyzes monthly data from 2017 to May 2022, highlighting critical trends and performance fluctuations during this period. The findings reveal that each COVID-19 wave led to a decline in hospital performance, with the first wave being the most severe due to a lack of preparedness. Furthermore, the pandemic worsened the disparities among examined hospitals. Pre-pandemic top performers in each group improved their performance and were more consistently recognized as benchmarks, with their average benchmark frequency increasing from 66.5% to 83.5%. These top entities demonstrated greater resilience and adaptability, further distancing themselves from underperforming hospitals, which saw declines in both performance scores and benchmark frequency, widening the performance gap. The superior performance of top entities can be attributed to pre-existing strategic tools and contextual factors that enabled them to withstand the pandemic's challenges more effectively. Highlights • The pandemic aggravated the differences between the hospitals examined. • The top-performing entities further distanced themselves from the remaining entities after the pandemic • Entities considered benchmarks before the pandemic remained the same, and became even more consistent during the pandemic. • The top-performing entities achieved higher scores than their pre-pandemic performance levels. • Benchmarking models for composite indicators with diverse decision-making preferences, and treatment of imperfect knowledge of data.

Suggested Citation

  • Guilherme Mendes Vara & Marta Castilho Gomes & Diogo Cunha Ferreira, 2025. "Assessing the performance of Portuguese public hospitals before and during COVID-19 outbreak, with optimistic and pessimistic benchmarking approaches," Health Care Management Science, Springer, vol. 28(1), pages 1-27, March.
    • Handle: RePEc:kap:hcarem:v:28:y:2025:i:1:d:10.1007_s10729-024-09693-4
    DOI: 10.1007/s10729-024-09693-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10729-024-09693-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10729-024-09693-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ferreira, D.C. & Marques, R.C., 2021. "Public-private partnerships in health care services: Do they outperform public hospitals regarding quality and access? Evidence from Portugal," Socio-Economic Planning Sciences, Elsevier, vol. 73(C).
    2. Zanella, Andreia & Camanho, Ana S. & Dias, Teresa G., 2015. "Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 245(2), pages 517-530.
    3. Robert Stefko & Beata Gavurova & Kristina Kocisova, 2018. "Healthcare efficiency assessment using DEA analysis in the Slovak Republic," Health Economics Review, Springer, vol. 8(1), pages 1-12, December.
    4. Pedro Simões & Rui Marques, 2011. "Performance and congestion analysis of the portuguese hospital services," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 19(1), pages 39-63, March.
    5. Ferreira, D.C. & Marques, R.C. & Nunes, A.M., 2018. "Economies of scope in the health sector: The case of Portuguese hospitals," European Journal of Operational Research, Elsevier, vol. 266(2), pages 716-735.
    6. Färe, Rolf & Karagiannis, Giannis & Hasannasab, Maryam & Margaritis, Dimitris, 2019. "A benefit-of-the-doubt model with reverse indicators," European Journal of Operational Research, Elsevier, vol. 278(2), pages 394-400.
    7. Rita Matos & Diogo Ferreira & Maria Isabel Pedro, 2021. "Economic Analysis of Portuguese Public Hospitals Through the Construction of Quality, Efficiency, Access, and Financial Related Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 157(1), pages 361-392, August.
    8. Alvaro Almeida & Joana Vales, 2020. "The impact of primary health care reform on hospital emergency department overcrowding: Evidence from the Portuguese reform," International Journal of Health Planning and Management, Wiley Blackwell, vol. 35(1), pages 368-377, January.
    9. Alexandre Morais Nunes & Diogo Filipe da Cunha Ferreira, 2022. "Evaluating Portuguese Public Hospitals Performance: Any Difference before and during COVID-19?," Sustainability, MDPI, vol. 15(1), pages 1-23, December.
    10. Laurens Cherchye & Willem Moesen & Nicky Rogge & Tom Puyenbroeck, 2007. "An Introduction to ‘Benefit of the Doubt’ Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 82(1), pages 111-145, May.
    11. Dan Lupu & Ramona Tiganasu, 2022. "COVID-19 and the efficiency of health systems in Europe," Health Economics Review, Springer, vol. 12(1), pages 1-15, December.
    12. Diogo Ferreira & Rui Marques, 2015. "Did the corporatization of Portuguese hospitals significantly change their productivity?," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 16(3), pages 289-303, April.
    13. Pourmahmoud, Jafar & Bagheri, Narges, 2023. "Uncertain Malmquist productivity index: An application to evaluate healthcare systems during COVID-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    14. Fusco, Elisa & Vidoli, Francesco & Sahoo, Biresh K., 2018. "Spatial heterogeneity in composite indicator: A methodological proposal," Omega, Elsevier, vol. 77(C), pages 1-14.
    15. Sebastian Kohl & Jan Schoenfelder & Andreas Fügener & Jens O. Brunner, 2019. "The use of Data Envelopment Analysis (DEA) in healthcare with a focus on hospitals," Health Care Management Science, Springer, vol. 22(2), pages 245-286, June.
    16. Simões, J. & Augusto, G.F. & Fronteira, I., 2017. "Introduction of freedom of choice for hospital outpatient care in Portugal: Implications and results of the 2016 reform," Health Policy, Elsevier, vol. 121(12), pages 1203-1207.
    17. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    18. Olesen, Ole B. & Petersen, Niels Christian, 2016. "Stochastic Data Envelopment Analysis—A review," European Journal of Operational Research, Elsevier, vol. 251(1), pages 2-21.
    19. Zhou, P. & Ang, B.W. & Poh, K.L., 2007. "A mathematical programming approach to constructing composite indicators," Ecological Economics, Elsevier, vol. 62(2), pages 291-297, April.
    20. Claude J. P. Bélisle & H. Edwin Romeijn & Robert L. Smith, 1993. "Hit-and-Run Algorithms for Generating Multivariate Distributions," Mathematics of Operations Research, INFORMS, vol. 18(2), pages 255-266, May.
    21. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    22. Diogo Cunha Ferreira & Alexandre Morais Nunes & Rui Cunha Marques, 2020. "Operational efficiency vs clinical safety, care appropriateness, timeliness, and access to health care," Journal of Productivity Analysis, Springer, vol. 53(3), pages 355-375, June.
    23. Alexandre Morais Nunes & Diogo Cunha Ferreira, 2019. "Reforms in the Portuguese health care sector: Challenges and proposals," International Journal of Health Planning and Management, Wiley Blackwell, vol. 34(1), pages 21-33, January.
    24. L Cherchye & W Moesen & N Rogge & T Van Puyenbroeck & M Saisana & A Saltelli & R Liska & S Tarantola, 2008. "Creating composite indicators with DEA and robustness analysis: the case of the Technology Achievement Index," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(2), pages 239-251, February.
    25. Gutacker, Nils & Siciliani, Luigi & Moscelli, Giuseppe & Gravelle, Hugh, 2016. "Choice of hospital: Which type of quality matters?," Journal of Health Economics, Elsevier, vol. 50(C), pages 230-246.
    26. Rajiv D. Banker, 1993. "Maximum Likelihood, Consistency and Data Envelopment Analysis: A Statistical Foundation," Management Science, INFORMS, vol. 39(10), pages 1265-1273, October.
    27. Takamura, Yoshiharu & Tone, Kaoru, 2003. "A comparative site evaluation study for relocating Japanese government agencies out of Tokyo," Socio-Economic Planning Sciences, Elsevier, vol. 37(2), pages 85-102, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rita Matos & Diogo Ferreira & Maria Isabel Pedro, 2021. "Economic Analysis of Portuguese Public Hospitals Through the Construction of Quality, Efficiency, Access, and Financial Related Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 157(1), pages 361-392, August.
    2. Dinesh R. Pai & Fatma Pakdil & Nasibeh Azadeh-Fard, 2024. "Applications of data envelopment analysis in acute care hospitals: a systematic literature review, 1984–2022," Health Care Management Science, Springer, vol. 27(2), pages 284-312, June.
    3. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    4. Ferreira, D.C. & Marques, R.C., 2019. "Do quality and access to hospital services impact on their technical efficiency?," Omega, Elsevier, vol. 86(C), pages 218-236.
    5. Annalina Sarra & Eugenia Nissi, 2020. "A Spatial Composite Indicator for Human and Ecosystem Well-Being in the Italian Urban Areas," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 148(2), pages 353-377, April.
    6. Salvatore Greco & Alessio Ishizaka & Menelaos Tasiou & Gianpiero Torrisi, 2019. "On the Methodological Framework of Composite Indices: A Review of the Issues of Weighting, Aggregation, and Robustness," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 141(1), pages 61-94, January.
    7. Pourmahmoud, Jafar & Bagheri, Narges, 2023. "Uncertain Malmquist productivity index: An application to evaluate healthcare systems during COVID-19 pandemic," Socio-Economic Planning Sciences, Elsevier, vol. 87(PA).
    8. Athanassoglou, Stergios, 2015. "Revisiting Worst-case DEA for Composite Indicators," Climate Change and Sustainable Development 198712, Fondazione Eni Enrico Mattei (FEEM).
    9. Fu-Chiang Yang & Rui-Hsin Kao & Yi-Tui Chen & Yueh-Fei Ho & Cheng-Chung Cho & Shi-Wei Huang, 2018. "A Common Weight Approach to Construct Composite Indicators: The Evaluation of Fourteen Emerging Markets," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 137(2), pages 463-479, June.
    10. Hatefi, S.M. & Torabi, S.A., 2010. "A common weight MCDA-DEA approach to construct composite indicators," Ecological Economics, Elsevier, vol. 70(1), pages 114-120, November.
    11. Paulo Caldas & Miguel Varela, 2023. "The Portuguese Public Hospitals Performance Evolution before and during the SARS-CoV-2 Pandemic (2017–2022)," Sustainability, MDPI, vol. 15(15), pages 1-31, July.
    12. Alireza Amirteimoori & Biresh K. Sahoo & Saber Mehdizadeh, 2023. "Data envelopment analysis for scale elasticity measurement in the stochastic case: with an application to Indian banking," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 9(1), pages 1-36, December.
    13. Oliveira, Renata & Zanella, Andreia & Camanho, Ana S., 2019. "The assessment of corporate social responsibility: The construction of an industry ranking and identification of potential for improvement," European Journal of Operational Research, Elsevier, vol. 278(2), pages 498-513.
    14. Alireza Amirteimoori & Tofigh Allahviranloo & Maryam Nematizadeh, 2024. "A firm-specific Malmquist productivity index model for stochastic data envelopment analysis: an application to commercial banks," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 10(1), pages 1-27, December.
    15. Genave, Anna & Blancard, Stéphane & Garabedian, Sabine, 2020. "An assessment of energy vulnerability in Small Island Developing States," Ecological Economics, Elsevier, vol. 171(C).
    16. Sahoo, Biresh K. & Acharya, Debashis, 2010. "An alternative approach to monetary aggregation in DEA," European Journal of Operational Research, Elsevier, vol. 204(3), pages 672-682, August.
    17. Rogge, Nicky, 2018. "Composite indicators as generalized benefit-of-the-doubt weighted averages," European Journal of Operational Research, Elsevier, vol. 267(1), pages 381-392.
    18. Stergios Athanassoglou, 2016. "Revisiting Worst-Case DEA for Composite Indicators," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 128(3), pages 1259-1272, September.
    19. Zanella, Andreia & Camanho, Ana S. & Dias, Teresa G., 2015. "Undesirable outputs and weighting schemes in composite indicators based on data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 245(2), pages 517-530.
    20. Van Puyenbroeck, Tom & Rogge, Nicky, 2017. "Geometric mean quantity index numbers with Benefit-of-the-Doubt weights," European Journal of Operational Research, Elsevier, vol. 256(3), pages 1004-1014.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:hcarem:v:28:y:2025:i:1:d:10.1007_s10729-024-09693-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.