IDEAS home Printed from https://ideas.repec.org/a/eee/hepoli/v124y2020i12p1387-1394.html
   My bibliography  Save this article

Measuring research in the big data era: The evolution of performance measurement systems in the Italian teaching hospitals

Author

Listed:
  • Horenberg, Frank
  • Lungu, Daniel Adrian
  • Nuti, Sabina

Abstract

In the healthcare system, Teaching Hospitals(THs) not only provide care, but also train healthcare professionals and carry out research activities. Research is a fundamental pillar of THs’ mission and relevant for the healthcare system monitored by Performance Evaluation Systems. Research activities can be measured using citation index services and this paper highlights differences between two services based on bibliometrics, describes opportunities and risks when performance indicators rely on data collected, controlled and validated by external services and discusses the possible impact on health policy at a system and provider level.

Suggested Citation

  • Horenberg, Frank & Lungu, Daniel Adrian & Nuti, Sabina, 2020. "Measuring research in the big data era: The evolution of performance measurement systems in the Italian teaching hospitals," Health Policy, Elsevier, vol. 124(12), pages 1387-1394.
    • Handle: RePEc:eee:hepoli:v:124:y:2020:i:12:p:1387-1394
    DOI: 10.1016/j.healthpol.2020.10.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0168851020302554
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.healthpol.2020.10.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Emilio Delgado López-Cózar & Nicolás Robinson-García & Daniel Torres-Salinas, 2014. "The Google scholar experiment: How to index false papers and manipulate bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 446-454, March.
    2. William M. Cockriel & James B. McDonald, 2018. "The influence of dispersion on journal impact measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 609-622, July.
    3. Todd Dewett & Angelo S. Denisi, 2004. "Exploring scholarly reputation: It's more than just productivity," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 249-272, June.
    4. Nuti, Sabina & Vola, Federico & Bonini, Anna & Vainieri, Milena, 2016. "Making governance work in the health care sector: evidence from a ‘natural experiment’ in Italy," Health Economics, Policy and Law, Cambridge University Press, vol. 11(1), pages 17-38, January.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2014. "How do you define and measure research productivity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1129-1144, November.
    6. Mutz, Rüdiger & Daniel, Hans-Dieter, 2012. "Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor," Journal of Informetrics, Elsevier, vol. 6(2), pages 169-176.
    7. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2016. "The museum of errors/horrors in Scopus," Journal of Informetrics, Elsevier, vol. 10(1), pages 174-182.
    8. Vinay Prasad & Jeffrey A Goldstein, 2014. "US News and World Report Cancer Hospital Rankings: Do They Reflect Measures of Research Productivity?," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-6, September.
    9. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    10. Gwyn Bevan & Deborah Wilson, 2013. "Does ‘naming and shaming’ work for schools and hospitals? Lessons from natural experiments following devolution in England and Wales," Public Money & Management, Taylor & Francis Journals, vol. 33(4), pages 245-252, July.
    11. Éric Archambault & David Campbell & Yves Gingras & Vincent Larivière, 2009. "Comparing bibliometric statistics obtained from the Web of Science and Scopus," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1320-1326, July.
    12. Vainieri, Milena & Vola, Federico & Gomez Soriano, Gregorio & Nuti, Sabina, 2016. "How to set challenging goals and conduct fair evaluation in regional public health systems. Insights from Valencia and Tuscany Regions," Health Policy, Elsevier, vol. 120(11), pages 1270-1278.
    13. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2016. "Empirical analysis and classification of database errors in Scopus and Web of Science," Journal of Informetrics, Elsevier, vol. 10(4), pages 933-953.
    14. Milena Vainieri & Daniel Adrian Lungu & Sabina Nuti, 2018. "Insights on the effectiveness of reward schemes from 10‐year longitudinal case studies in 2 Italian regions," International Journal of Health Planning and Management, Wiley Blackwell, vol. 33(2), pages 474-484, April.
    15. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    16. Bevan, Gwyn & Evans, Alice & Nuti, Sabina, 2019. "Reputations count: why benchmarking performance is improving health care across the world," Health Economics, Policy and Law, Cambridge University Press, vol. 14(2), pages 141-161, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D'Aniello, Luca & Spano, Maria & Cuccurullo, Corrado & Aria, Massimo, 2022. "Academic Health Centers’ configurations, scientific productivity, and impact: Insights from the Italian setting," Health Policy, Elsevier, vol. 126(12), pages 1317-1323.
    2. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    3. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    4. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    5. Shirley Ainsworth & Jane M. Russell, 2018. "Has hosting on science direct improved the visibility of Latin American scholarly journals? A preliminary analysis of data quality," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1463-1484, June.
    6. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    7. De Rosis, Sabina & Guidotti, Elisa & Zuccarino, Sara & Venturi, Giulia & Ferré, Francesca, 2020. "Waiting time information in the Italian NHS: A citizen perspective," Health Policy, Elsevier, vol. 124(8), pages 796-804.
    8. Nabil Amara & Réjean Landry & Norrin Halilem, 2015. "What can university administrators do to increase the publication and citation scores of their faculty members?," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 489-530, May.
    9. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    10. Mikhail Rogov & Céline Rozenblat, 2018. "Urban Resilience Discourse Analysis: Towards a Multi-Level Approach to Cities," Sustainability, MDPI, vol. 10(12), pages 1-21, November.
    11. Weishu Liu & Meiting Huang & Haifeng Wang, 2021. "Same journal but different numbers of published records indexed in Scopus and Web of Science Core Collection: causes, consequences, and solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4541-4550, May.
    12. Thelwall, Mike, 2018. "Dimensions: A competitor to Scopus and the Web of Science?," Journal of Informetrics, Elsevier, vol. 12(2), pages 430-435.
    13. Haunschild, Robin & Bornmann, Lutz, 2023. "Which papers cited which tweets? An exploratory analysis based on Scopus data," Journal of Informetrics, Elsevier, vol. 17(2).
    14. Chen, Meiqian & Guo, Zhaoxia & Dong, Yucheng & Chiclana, Francisco & Herrera-Viedma, Enrique, 2021. "Citations optimal growth path: A tool to analyze sensitivity to citations of h-like indexes," Journal of Informetrics, Elsevier, vol. 15(4).
    15. Liu, Weishu & Hu, Guangyuan & Tang, Li, 2018. "Missing author address information in Web of Science—An explorative study," Journal of Informetrics, Elsevier, vol. 12(3), pages 985-997.
    16. Nuti, Sabina & Ferré, Francesca & Seghieri, Chiara & Foresi, Elisa & Stukel, Therese A., 2020. "Managing the performance of general practitioners and specialists referral networks: A system for evaluating the heart failure pathway," Health Policy, Elsevier, vol. 124(1), pages 44-51.
    17. Zhesi Shen & Liying Yang & Zengru Di & Jinshan Wu, 2019. "Large enough sample size to rank two groups of data reliably according to their means," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 653-671, February.
    18. Igor Savchenko & Denis Kosyakov, 2022. "Lost in affiliation: apatride publications in international databases," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3471-3487, June.
    19. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    20. Junwen Zhu & Fang Liu & Weishu Liu, 2019. "The secrets behind Web of Science’s DOI search," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1745-1753, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:hepoli:v:124:y:2020:i:12:p:1387-1394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu or the person in charge (email available below). General contact details of provider: http://www.elsevier.com/locate/healthpol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.