IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v15y2021i4s1751157721000596.html
   My bibliography  Save this article

System for evaluating the reliability and novelty of medical scientific papers

Author

Listed:
  • Martín de Diego, Isaac
  • González-Fernández, César
  • Fernández-Isabel, Alberto
  • Fernández, Rubén R.
  • Cabezas, Javier

Abstract

As society develops, the number of published research articles raises. Besides, the pressure to publish has been increased because the competitiveness between researchers working on similar topics. Although this increment is desirable, it leads to multiple issues. At a reader level, insurmountable barriers to keep up with the state of the art appear. From a publisher perspective, it is a very demanding task to determine which research articles are worth publishing. Automatizing these tasks appears as a core solution. In the case of readers, a previous evaluation of articles would simplify the filtering process. As for publishers, they could perform preliminary selections or estimate the reviewing effort. This paper presents Medical Evaluator System for Scientific Interoperability (MESSI) system to overcome all these issues. It is able to evaluate the novelty and reliability of health-related texts. The novelty calculation is based on previously acquired knowledge after processing more than 500,000 papers. The reliability estimation is based on the reputations of similar articles calculated based on previously defined metrics. Multiple experiments have been addressed to illustrate the viability of the proposal. The obtained results show a good performance that encourage to continue evolving the system.

Suggested Citation

  • Martín de Diego, Isaac & González-Fernández, César & Fernández-Isabel, Alberto & Fernández, Rubén R. & Cabezas, Javier, 2021. "System for evaluating the reliability and novelty of medical scientific papers," Journal of Informetrics, Elsevier, vol. 15(4).
    • Handle: RePEc:eee:infome:v:15:y:2021:i:4:s1751157721000596
    DOI: 10.1016/j.joi.2021.101188
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157721000596
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2021.101188?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Yundong Xie & Qiang Wu & Xingchen Li, 2019. "Editorial team scholarly index (ETSI): an alternative indicator for evaluating academic journal reputation," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(3), pages 1333-1349, September.
    2. Yoshiko Okubo, 1997. "Bibliometric Indicators and Analysis of Research Systems: Methods and Examples," OECD Science, Technology and Industry Working Papers 1997/1, OECD Publishing.
    3. Franceschet, Massimo, 2010. "The difference between popularity and prestige in the sciences and in the social sciences: A bibliometric analysis," Journal of Informetrics, Elsevier, vol. 4(1), pages 55-63.
    4. Guerrero-Bote, Vicente P. & Moya-Anegón, Félix, 2012. "A further step forward in measuring journals’ scientific prestige: The SJR2 indicator," Journal of Informetrics, Elsevier, vol. 6(4), pages 674-688.
    5. Lutz Bornmann & Rüdiger Mutz, 2015. "Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2215-2222, November.
    6. Vahe Tshitoyan & John Dagdelen & Leigh Weston & Alexander Dunn & Ziqin Rong & Olga Kononova & Kristin A. Persson & Gerbrand Ceder & Anubhav Jain, 2019. "Unsupervised word embeddings capture latent knowledge from materials science literature," Nature, Nature, vol. 571(7763), pages 95-98, July.
    7. Diana Hicks & Julia Melkers, 2013. "Bibliometrics as a tool for research evaluation," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 11, pages 323-349, Edward Elgar Publishing.
    8. Eleni Fragkiadaki & Georgios Evangelidis & Nikolaos Samaras & Dimitris A. Dervos, 2011. "f-Value: measuring an article’s scientific impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(3), pages 671-686, March.
    9. Bas Hofstra & Vivek V. Kulkarni & Sebastian Munoz-Najar Galvez & Bryan He & Dan Jurafsky & Daniel A. McFarland, 2020. "The Diversity–Innovation Paradox in Science," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 117(17), pages 9284-9291, April.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Walters, William H., 2017. "Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?," Journal of Informetrics, Elsevier, vol. 11(3), pages 730-744.
    2. Wu, Lingfei & Kittur, Aniket & Youn, Hyejin & Milojević, Staša & Leahey, Erin & Fiore, Stephen M. & Ahn, Yong-Yeol, 2022. "Metrics and mechanisms: Measuring the unmeasurable in the science of science," Journal of Informetrics, Elsevier, vol. 16(2).
    3. Lin, Yiling & Evans, James A. & Wu, Lingfei, 2022. "New directions in science emerge from disconnection and discord," Journal of Informetrics, Elsevier, vol. 16(1).
    4. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    5. Zhang, Baolong & Wang, Hao & Deng, Sanhong & Su, Xinning, 2020. "Measurement and analysis of Chinese journal discriminative capacity," Journal of Informetrics, Elsevier, vol. 14(1).
    6. Saarela, Mirka & Kärkkäinen, Tommi & Lahtonen, Tommi & Rossi, Tuomo, 2016. "Expert-based versus citation-based ranking of scholarly and scientific publication channels," Journal of Informetrics, Elsevier, vol. 10(3), pages 693-718.
    7. Liwei Cai & Jiahao Tian & Jiaying Liu & Xiaomei Bai & Ivan Lee & Xiangjie Kong & Feng Xia, 2019. "Scholarly impact assessment: a survey of citation weighting solutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 453-478, February.
    8. Eleni Fragkiadaki & Georgios Evangelidis, 2014. "Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 99(2), pages 261-288, May.
    9. Noushin Islam & Malindu Sandanayake & Shobha Muthukumaran & Dimuth Navaratna, 2024. "Review on Sustainable Construction and Demolition Waste Management—Challenges and Research Prospects," Sustainability, MDPI, vol. 16(8), pages 1-30, April.
    10. Chao Min & Qingyu Chen & Erjia Yan & Yi Bu & Jianjun Sun, 2021. "Citation cascade and the evolution of topic relevance," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(1), pages 110-127, January.
    11. Ananthan Nambiar & Tobias Rubel & James McCaull & Jon deVries & Mark Bedau, 2021. "Dropping diversity of products of large US firms: Models and measures," Papers 2110.08367, arXiv.org.
    12. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2014. "How to improve the prediction based on citation impact percentiles for years shortly after the publication date?," Journal of Informetrics, Elsevier, vol. 8(1), pages 175-180.
    13. Ramona Weinrich, 2019. "Opportunities for the Adoption of Health-Based Sustainable Dietary Patterns: A Review on Consumer Research of Meat Substitutes," Sustainability, MDPI, vol. 11(15), pages 1-15, July.
    14. Aleš LEBEDA & Věra Kroftová & Václav KŮDELA & Marcela BRAUNOVÁ, 2014. "Fifty-year anniversary of Plant Protection Science," Plant Protection Science, Czech Academy of Agricultural Sciences, vol. 50(2), pages 53-63.
    15. Jason Youn & Navneet Rai & Ilias Tagkopoulos, 2022. "Knowledge integration and decision support for accelerated discovery of antibiotic resistance genes," Nature Communications, Nature, vol. 13(1), pages 1-11, December.
    16. Piers Steel & Sjoerd Beugelsdijk & Herman Aguinis, 2021. "The anatomy of an award-winning meta-analysis: Recommendations for authors, reviewers, and readers of meta-analytic reviews," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 52(1), pages 23-44, February.
    17. Dunaiski, Marcel & Geldenhuys, Jaco & Visser, Willem, 2019. "On the interplay between normalisation, bias, and performance of paper impact metrics," Journal of Informetrics, Elsevier, vol. 13(1), pages 270-290.
    18. Dejian Yu & Wanru Wang & Shuai Zhang & Wenyu Zhang & Rongyu Liu, 2017. "A multiple-link, mutually reinforced journal-ranking model to measure the prestige of journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(1), pages 521-542, April.
    19. Augusteijn, Hilde Elisabeth Maria & van Aert, Robbie Cornelis Maria & van Assen, Marcel A. L. M., 2021. "Posterior Probabilities of Effect Sizes and Heterogeneity in Meta-Analysis: An Intuitive Approach of Dealing with Publication Bias," OSF Preprints avkgj, Center for Open Science.
    20. Thor, Andreas & Marx, Werner & Leydesdorff, Loet & Bornmann, Lutz, 2016. "Introducing CitedReferencesExplorer (CRExplorer): A program for reference publication year spectroscopy with cited references standardization," Journal of Informetrics, Elsevier, vol. 10(2), pages 503-515.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:15:y:2021:i:4:s1751157721000596. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.