IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0093977.html

Application of Balanced Scorecard in the Evaluation of a Complex Health System Intervention: 12 Months Post Intervention Findings from the BHOMA Intervention: A Cluster Randomised Trial in Zambia

Author

Listed:
  • Wilbroad Mutale
  • Jeffrey Stringer
  • Namwinga Chintu
  • Roma Chilengi
  • Margaret Tembo Mwanamwenge
  • Nkatya Kasese
  • Dina Balabanova
  • Neil Spicer
  • James Lewis
  • Helen Ayles

Abstract

Introduction: In many low income countries, the delivery of quality health services is hampered by health system-wide barriers which are often interlinked, however empirical evidence on how to assess the level and scope of these barriers is scarce. A balanced scorecard is a tool that allows for wider analysis of domains that are deemed important in achieving the overall vision of the health system. We present the quantitative results of the 12 months follow-up study applying the balanced scorecard approach in the BHOMA intervention with the aim of demonstrating the utility of the balanced scorecard in evaluating multiple building blocks in a trial setting. Methods: The BHOMA is a cluster randomised trial that aims to strengthen the health system in three rural districts in Zambia. The intervention aims to improve clinical care quality by implementing practical tools that establish clear clinical care standards through intensive clinic implementations. This paper reports the findings of the follow-up health facility survey that was conducted after 12 months of intervention implementation. Comparisons were made between those facilities in the intervention and control sites. STATA version 12 was used for analysis. Results: The study found significant mean differences between intervention(I) and control (C) sites in the following domains: Training domain (Mean I:C; 87.5.vs 61.1, mean difference 23.3, p = 0.031), adult clinical observation domain (mean I:C; 73.3 vs.58.0, mean difference 10.9, p = 0.02 ) and health information domain (mean I:C; 63.6 vs.56.1, mean difference 6.8, p = 0.01. There was no gender differences in adult service satisfaction. Governance and motivation scores did not differ between control and intervention sites. Conclusion: This study demonstrates the utility of the balanced scorecard in assessing multiple elements of the health system. Using system wide approaches and triangulating data collection methods seems to be key to successful evaluation of such complex health intervention. Trial number: ClinicalTrials.gov NCT01942278

Suggested Citation

  • Wilbroad Mutale & Jeffrey Stringer & Namwinga Chintu & Roma Chilengi & Margaret Tembo Mwanamwenge & Nkatya Kasese & Dina Balabanova & Neil Spicer & James Lewis & Helen Ayles, 2014. "Application of Balanced Scorecard in the Evaluation of a Complex Health System Intervention: 12 Months Post Intervention Findings from the BHOMA Intervention: A Cluster Randomised Trial in Zambia," PLOS ONE, Public Library of Science, vol. 9(4), pages 1-10, April.
    • Handle: RePEc:plo:pone00:0093977
    DOI: 10.1371/journal.pone.0093977
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0093977
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0093977&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0093977?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Anbrasi Edward & Binay Kumar & Faizullah Kakar & Ahmad Shah Salehi & Gilbert Burnham & David H Peters, 2011. "Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years' Evaluation in Afghanistan," PLOS Medicine, Public Library of Science, vol. 8(7), pages 1-9, July.
    2. Wilbroad Mutale & Peter Godfrey-Fausset & Margaret Tembo Mwanamwenge & Nkatya Kasese & Namwinga Chintu & Dina Balabanova & Neil Spicer & Helen Ayles, 2013. "Measuring Health System Strengthening: Application of the Balanced Scorecard Approach to Rank the Baseline Performance of Three Rural Districts in Zambia," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-11, March.
    3. Edward Anbrasi & Binay Kumar & Faizullah Kakar & Ahmed Shah Salehi & Gilbert Burnham & David H Peters, 2011. "Configuring Balanced Scorecards for Measuring Health System Performance: Evidence from 5 Years’ Evaluation in Afghanistan," Working Papers id:4362, eSocialSciences.
    4. Gauld, Robin & Al-wahaibi, Suhaila & Chisholm, Johanna & Crabbe, Rebecca & Kwon, Boomi & Oh, Timothy & Palepu, Raja & Rawcliffe, Nic & Sohn, Stephen, 2011. "Scorecards for health system performance assessment: The New Zealand example," Health Policy, Elsevier, vol. 103(2), pages 200-208.
    5. Julio Frenk, 2010. "The Global Health System: Strengthening National Health Systems as the Next Step for Global Progress," PLOS Medicine, Public Library of Science, vol. 7(1), pages 1-3, January.
    6. El-Jardali, Fadi & Saleh, Shadi & Ataya, Nour & Jamal, Diana, 2011. "Design, implementation and scaling up of the balanced scorecard for hospitals in Lebanon: Policy coherence and application lessons for low and middle income countries," Health Policy, Elsevier, vol. 103(2), pages 305-314.
    7. Jeffrey M Wooldridge, 2010. "Econometric Analysis of Cross Section and Panel Data," MIT Press Books, The MIT Press, edition 2, volume 1, number 0262232588, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wilbroad Mutale & Peter Godfrey-Fausset & Margaret Tembo Mwanamwenge & Nkatya Kasese & Namwinga Chintu & Dina Balabanova & Neil Spicer & Helen Ayles, 2013. "Measuring Health System Strengthening: Application of the Balanced Scorecard Approach to Rank the Baseline Performance of Three Rural Districts in Zambia," PLOS ONE, Public Library of Science, vol. 8(3), pages 1-11, March.
    2. Edward, Anbrasi & Branchini, Casey & Aitken, Iain & Roach, Melissa & Osei-Bonsu, Kojo & Arwal, Said Habib, 2015. "Toward universal coverage in Afghanistan: A multi-stakeholder assessment of capacity investments in the community health worker system," Social Science & Medicine, Elsevier, vol. 145(C), pages 173-183.
    3. Davis, Peter & Milne, Barry & Parker, Karl & Hider, Phil & Lay-Yee, Roy & Cumming, Jackie & Graham, Patrick, 2013. "Efficiency, effectiveness, equity (E3). Evaluating hospital performance in three dimensions," Health Policy, Elsevier, vol. 112(1), pages 19-27.
    4. Liu, Hu-Chen, 2013. "A theoretical framework for holistic hospital management in the Japanese healthcare context," Health Policy, Elsevier, vol. 113(1), pages 160-169.
    5. Carla Alexandra da Encarnação Filipe Amado & Olena Kalinichenko & Sérgio Pereira dos Santos, 2013. "Performance Assessment in Primary Health Care: A Systematic Literature Review," CEFAGE-UE Working Papers 2013_03, University of Evora, CEFAGE-UE (Portugal).
    6. Osmani, Ahmad Reshad & Okunade, Albert, 2025. "Impact of supply-side conditional cash transfers on household family planning decisions: Evidence from a field experiment in Afghanistan," Social Science & Medicine, Elsevier, vol. 383(C).
    7. Milena Dalton & Benjamin Sanderson & Leanne J Robinson & Caroline S E Homer & William Pomat & Margie Danchin & Stefanie Vaccher, 2023. "Impact of COVID-19 on routine childhood immunisations in low- and middle-income countries: A scoping review," PLOS Global Public Health, Public Library of Science, vol. 3(8), pages 1-17, August.
    8. Mengyuan Zhou, 2022. "Does the Source of Inheritance Matter in Bequest Attitudes? Evidence from Japan," Journal of Family and Economic Issues, Springer, vol. 43(4), pages 867-887, December.
    9. Campbell, Randall C. & Nagel, Gregory L., 2016. "Private information and limitations of Heckman's estimator in banking and corporate finance research," Journal of Empirical Finance, Elsevier, vol. 37(C), pages 186-195.
    10. Christopher Dick-Sagoe & Ernest Ngeh Tingum & Peter Asare-Nuamah & Denis N. Yuni & Nicholas Baidoo, 2025. "Central transfers and incentives to collect local revenue among the Central Region of Ghana’s local government officials: analysing the flypaper effect," Humanities and Social Sciences Communications, Palgrave Macmillan, vol. 12(1), pages 1-12, December.
    11. Ilona Babenko & Benjamin Bennett & John M Bizjak & Jeffrey L Coles & Jason J Sandvik, 2023. "Clawback Provisions and Firm Risk," The Review of Corporate Finance Studies, Society for Financial Studies, vol. 12(2), pages 191-239.
    12. Şahan, Duygu & Tuna, Okan, 2018. "Environmental innovation of transportation sector in OECD countries," Chapters from the Proceedings of the Hamburg International Conference of Logistics (HICL), in: Kersten, Wolfgang & Blecker, Thorsten & Ringle, Christian M. (ed.), The Road to a Digitalized Supply Chain Management: Smart and Digital Solutions for Supply Chain Management. Proceedings of the Hamburg International C, volume 25, pages 157-170, Hamburg University of Technology (TUHH), Institute of Business Logistics and General Management.
    13. Ruomeng Cui & Dennis J. Zhang & Achal Bassamboo, 2019. "Learning from Inventory Availability Information: Evidence from Field Experiments on Amazon," Management Science, INFORMS, vol. 65(3), pages 1216-1235, March.
    14. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    15. Shaikh M. S. U. Eskander & Sam Fankhauser, 2022. "Income Diversification and Income Inequality: Household Responses to the 2013 Floods in Pakistan," Sustainability, MDPI, vol. 14(1), pages 1-12, January.
    16. Meng, Chang & Ghafoori, Noorulhaq, 2024. "The economic impact of terrorism in South Asia," Socio-Economic Planning Sciences, Elsevier, vol. 96(C).
    17. Hwang, Yujung & Nguyen, Toan, 2025. "Is sex ratio a valid distribution factor in a collective model?," Labour Economics, Elsevier, vol. 92(C).
    18. Peter Harasztosi & Attila Lindner, 2019. "Who Pays for the Minimum Wage?," American Economic Review, American Economic Association, vol. 109(8), pages 2693-2727, August.
    19. Jinwon Kim & Jucheol Moon & Dongyun Yang, 2024. "Pigouvian Congestion Tolls and the Welfare Gain: Estimates for California Freeways," Working Papers 2402, Nam Duck-Woo Economic Research Institute, Sogang University (Former Research Institute for Market Economy).
    20. Cho, Seong-Hoon & Kim, Heeho & Roberts, Roland K. & Kim, Taeyoung & Lee, Daegoon, 2014. "Effects of changes in forestland ownership on deforestation and urbanization and the resulting effects on greenhouse gas emissions," Journal of Forest Economics, Elsevier, vol. 20(1), pages 93-109.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0093977. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.