IDEAS home Printed from https://ideas.repec.org/a/eee/hepoli/v124y2020i12p1387-1394.html
   My bibliography  Save this article

Measuring research in the big data era: The evolution of performance measurement systems in the Italian teaching hospitals

Author

Listed:
  • Horenberg, Frank
  • Lungu, Daniel Adrian
  • Nuti, Sabina

Abstract

In the healthcare system, Teaching Hospitals(THs) not only provide care, but also train healthcare professionals and carry out research activities. Research is a fundamental pillar of THs’ mission and relevant for the healthcare system monitored by Performance Evaluation Systems. Research activities can be measured using citation index services and this paper highlights differences between two services based on bibliometrics, describes opportunities and risks when performance indicators rely on data collected, controlled and validated by external services and discusses the possible impact on health policy at a system and provider level.

Suggested Citation

  • Horenberg, Frank & Lungu, Daniel Adrian & Nuti, Sabina, 2020. "Measuring research in the big data era: The evolution of performance measurement systems in the Italian teaching hospitals," Health Policy, Elsevier, vol. 124(12), pages 1387-1394.
    • Handle: RePEc:eee:hepoli:v:124:y:2020:i:12:p:1387-1394
    DOI: 10.1016/j.healthpol.2020.10.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0168851020302554
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.healthpol.2020.10.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Emilio Delgado López-Cózar & Nicolás Robinson-García & Daniel Torres-Salinas, 2014. "The Google scholar experiment: How to index false papers and manipulate bibliometric indicators," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 446-454, March.
    2. William M. Cockriel & James B. McDonald, 2018. "The influence of dispersion on journal impact measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(1), pages 609-622, July.
    3. Todd Dewett & Angelo S. Denisi, 2004. "Exploring scholarly reputation: It's more than just productivity," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 249-272, June.
    4. Nuti, Sabina & Vola, Federico & Bonini, Anna & Vainieri, Milena, 2016. "Making governance work in the health care sector: evidence from a ‘natural experiment’ in Italy," Health Economics, Policy and Law, Cambridge University Press, vol. 11(1), pages 17-38, January.
    5. Giovanni Abramo & Ciriaco Andrea D’Angelo, 2014. "How do you define and measure research productivity?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1129-1144, November.
    6. Mutz, Rüdiger & Daniel, Hans-Dieter, 2012. "Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor," Journal of Informetrics, Elsevier, vol. 6(2), pages 169-176.
    7. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2016. "The museum of errors/horrors in Scopus," Journal of Informetrics, Elsevier, vol. 10(1), pages 174-182.
    8. Vinay Prasad & Jeffrey A Goldstein, 2014. "US News and World Report Cancer Hospital Rankings: Do They Reflect Measures of Research Productivity?," PLOS ONE, Public Library of Science, vol. 9(9), pages 1-6, September.
    9. Christoph Bartneck & Servaas Kokkelmans, 2011. "Detecting h-index manipulation through self-citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 87(1), pages 85-98, April.
    10. Gwyn Bevan & Deborah Wilson, 2013. "Does ‘naming and shaming’ work for schools and hospitals? Lessons from natural experiments following devolution in England and Wales," Public Money & Management, Taylor & Francis Journals, vol. 33(4), pages 245-252, July.
    11. Éric Archambault & David Campbell & Yves Gingras & Vincent Larivière, 2009. "Comparing bibliometric statistics obtained from the Web of Science and Scopus," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(7), pages 1320-1326, July.
    12. Vainieri, Milena & Vola, Federico & Gomez Soriano, Gregorio & Nuti, Sabina, 2016. "How to set challenging goals and conduct fair evaluation in regional public health systems. Insights from Valencia and Tuscany Regions," Health Policy, Elsevier, vol. 120(11), pages 1270-1278.
    13. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2016. "Empirical analysis and classification of database errors in Scopus and Web of Science," Journal of Informetrics, Elsevier, vol. 10(4), pages 933-953.
    14. Milena Vainieri & Daniel Adrian Lungu & Sabina Nuti, 2018. "Insights on the effectiveness of reward schemes from 10‐year longitudinal case studies in 2 Italian regions," International Journal of Health Planning and Management, Wiley Blackwell, vol. 33(2), pages 474-484, April.
    15. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    16. Bevan, Gwyn & Evans, Alice & Nuti, Sabina, 2019. "Reputations count: why benchmarking performance is improving health care across the world," Health Economics, Policy and Law, Cambridge University Press, vol. 14(2), pages 141-161, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D'Aniello, Luca & Spano, Maria & Cuccurullo, Corrado & Aria, Massimo, 2022. "Academic Health Centers’ configurations, scientific productivity, and impact: Insights from the Italian setting," Health Policy, Elsevier, vol. 126(12), pages 1317-1323.
    2. Eugenio Petrovich, 2022. "Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2195-2233, May.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    2. Sergio Copiello, 2019. "The open access citation premium may depend on the openness and inclusiveness of the indexing database, but the relationship is controversial because it is ambiguous where the open access boundary lie," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(2), pages 995-1018, November.
    3. Raminta Pranckutė, 2021. "Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World," Publications, MDPI, vol. 9(1), pages 1-59, March.
    4. Thelwall, Mike, 2018. "Microsoft Academic automatic document searches: Accuracy for journal articles and suitability for citation analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 1-9.
    5. De Rosis, Sabina & Guidotti, Elisa & Zuccarino, Sara & Venturi, Giulia & Ferré, Francesca, 2020. "Waiting time information in the Italian NHS: A citizen perspective," Health Policy, Elsevier, vol. 124(8), pages 796-804.
    6. Nabil Amara & Réjean Landry & Norrin Halilem, 2015. "What can university administrators do to increase the publication and citation scores of their faculty members?," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(2), pages 489-530, May.
    7. Shirley Ainsworth & Jane M. Russell, 2018. "Has hosting on science direct improved the visibility of Latin American scholarly journals? A preliminary analysis of data quality," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1463-1484, June.
    8. Houqiang Yu & Xueting Cao & Tingting Xiao & Zhenyi Yang, 2020. "How accurate are policy document mentions? A first look at the role of altmetrics database," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1517-1540, November.
    9. Chen, Meiqian & Guo, Zhaoxia & Dong, Yucheng & Chiclana, Francisco & Herrera-Viedma, Enrique, 2021. "Citations optimal growth path: A tool to analyze sensitivity to citations of h-like indexes," Journal of Informetrics, Elsevier, vol. 15(4).
    10. Liu, Weishu & Hu, Guangyuan & Tang, Li, 2018. "Missing author address information in Web of Science—An explorative study," Journal of Informetrics, Elsevier, vol. 12(3), pages 985-997.
    11. Brito, Ricardo & Rodríguez-Navarro, Alonso, 2018. "Research assessment by percentile-based double rank analysis," Journal of Informetrics, Elsevier, vol. 12(1), pages 315-329.
    12. Junwen Zhu & Fang Liu & Weishu Liu, 2019. "The secrets behind Web of Science’s DOI search," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(3), pages 1745-1753, June.
    13. Sergio Copiello, 2020. "The alleged citation advantage of video abstracts may be a matter of self-citations and self-selection bias. Comment on “The impact of video abstract on citation counts” by Zong et al," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(1), pages 751-757, January.
    14. Giovanni Abramo & Corrado Costa & Ciriaco Andrea D’Angelo, 2015. "A multivariate stochastic model to assess research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(2), pages 1755-1772, February.
    15. Franceschini, Fiorenzo & Maisano, Domenico & Mastrogiacomo, Luca, 2016. "Empirical analysis and classification of database errors in Scopus and Web of Science," Journal of Informetrics, Elsevier, vol. 10(4), pages 933-953.
    16. Asma Hammami & Nabil Semmar, 2022. "The simplex simulation as a tool to reveal publication strategies and citation factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 319-350, January.
    17. Enrique Orduna-Malea & Selenay Aytac & Clara Y. Tran, 2019. "Universities through the eyes of bibliographic databases: a retroactive growth comparison of Google Scholar, Scopus and Web of Science," Scientometrics, Springer;Akadémiai Kiadó, vol. 121(1), pages 433-450, October.
    18. Koh Yamamoto & Takuo Yasunaga, 2022. "A percentile rank score of group productivity: an evaluation of publication productivity for researchers from various fields," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(4), pages 1737-1754, April.
    19. Vîiu, Gabriel-Alexandru, 2016. "A theoretical evaluation of Hirsch-type bibliometric indicators confronted with extreme self-citation," Journal of Informetrics, Elsevier, vol. 10(2), pages 552-566.
    20. Mikhail Rogov & Céline Rozenblat, 2018. "Urban Resilience Discourse Analysis: Towards a Multi-Level Approach to Cities," Sustainability, MDPI, vol. 10(12), pages 1-21, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:hepoli:v:124:y:2020:i:12:p:1387-1394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu or the person in charge (email available below). General contact details of provider: http://www.elsevier.com/locate/healthpol .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.