IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v16y2022i4s1751157722000852.html
   My bibliography  Save this article

Predicting the clinical citation count of biomedical papers using multilayer perceptron neural network

Author

Listed:
  • Li, Xin
  • Tang, Xuli
  • Cheng, Qikai

Abstract

The number of clinical citations received from clinical guidelines or clinical trials has been considered as one of the most appropriate indicators for quantifying the clinical impact of biomedical papers. Therefore, the early prediction of clinical citation count of biomedical papers is critical to scientific activities in biomedicine, such as research evaluation, resource allocation, and clinical translation. In this study, we designed a four-layer multilayer perceptron neural network (MPNN) model to predict the clinical citation count of biomedical papers in the future by using 9,822,620 biomedical papers published from 1985 to 2005. We extracted ninety-one paper features from three dimensions as the input of the model, including twenty-one features in the paper dimension, thirty-five in the reference dimension, and thirty-five in the citing paper dimension. In each dimension, the features can be classified into three categories, i.e., the citation-related features, the clinical translation-related features, and the topic-related features. Besides, in the paper dimension, we also considered the features that have previously been demonstrated to be related to the citation counts of research papers. The results showed that the proposed MPNN model outperformed the other five baseline models, and the features in the reference dimension were the most important. In all the three dimensions, the citation-related and topic-related features were more important than the clinical translation-related features for the prediction. It also turned out that the features helpful in predicting the citation count of papers are not important for predicting the clinical citation count of biomedical papers. Furthermore, we explored the MPNN model based on different categories of biomedical papers. The results showed that the clinical translation-related features were more important for the prediction of clinical citation count of basic papers rather than those papers closer to clinical science. This study provided a novel dimension (i.e., the reference dimension) for the research community and could be applied to other related research tasks, such as the research assessment for translational programs. In addition, the findings in this study could be useful for biomedical authors (especially for those in basic science) to get more attention from clinical research.

Suggested Citation

  • Li, Xin & Tang, Xuli & Cheng, Qikai, 2022. "Predicting the clinical citation count of biomedical papers using multilayer perceptron neural network," Journal of Informetrics, Elsevier, vol. 16(4).
    • Handle: RePEc:eee:infome:v:16:y:2022:i:4:s1751157722000852
    DOI: 10.1016/j.joi.2022.101333
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1751157722000852
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2022.101333?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Sergio Jimenez & Youlin Avila & George Dueñas & Alexander Gelbukh, 2020. "Automatic prediction of citability of scientific articles by stylometry of their titles and abstracts," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 3187-3232, December.
    2. Tang, Xuli & Li, Xin & Ding, Ying & Song, Min & Bu, Yi, 2020. "The pace of artificial intelligence innovations: Speed, talent, and trial-and-error," Journal of Informetrics, Elsevier, vol. 14(4).
    3. Mike Thelwall & Nabeil Maflahi, 2016. "Guideline references and academic citations as evidence of the clinical value of health research," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 67(4), pages 960-966, April.
    4. Loet Leydesdorff & Caroline S. Wagner & Lutz Bornmann, 2018. "Betweenness and diversity in journal citation networks as measures of interdisciplinarity—A tribute to Eugene Garfield," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(2), pages 567-592, February.
    5. Ruan, Xuanmin & Zhu, Yuanyang & Li, Jiang & Cheng, Ying, 2020. "Predicting the citation counts of individual papers via a BP neural network," Journal of Informetrics, Elsevier, vol. 14(3).
    6. Ke, Qing, 2020. "The citation disadvantage of clinical research," Journal of Informetrics, Elsevier, vol. 14(1).
    7. B Ian Hutchins & Matthew T Davis & Rebecca A Meseroll & George M Santangelo, 2019. "Predicting translational progress in biomedical research," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-25, October.
    8. Tehmina Amjad & Nafeesa Shahid & Ali Daud & Asma Khatoon, 2022. "Citation burst prediction in a bibliometric network," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(5), pages 2773-2790, May.
    9. Qi Yu & Qi Wang & Yafei Zhang & Chongyan Chen & Hyeyoung Ryu & Namu Park & Jae-Eun Baek & Keyuan Li & Yifei Wu & Daifeng Li & Jian Xu & Meijun Liu & Jeremy J. Yang & Chenwei Zhang & Chao Lu & Peng Zha, 2021. "Analyzing knowledge entities about COVID-19 using entitymetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(5), pages 4491-4509, May.
    10. Li, Xin & Tang, Xuli, 2021. "Characterizing interdisciplinarity in drug research: A translational science perspective," Journal of Informetrics, Elsevier, vol. 15(4).
    11. Narin, Francis & Rozek, Richard P., 1988. "Bibliometric analysis of U.S. pharmaceutical industry research performance," Research Policy, Elsevier, vol. 17(3), pages 139-154, June.
    12. Grant Lewison & Guillermo Paraje, 2004. "The classification of biomedical journals by research level," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 145-157, June.
    13. Mike Thelwall & Kayvan Kousha, 2016. "Are citations from clinical trials evidence of higher impact research? An analysis of ClinicalTrials.gov," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1341-1351, November.
    14. Magnus Eriksson & Annika Billhult & Tommy Billhult & Elena Pallari & Grant Lewison, 2020. "A new database of the references on international clinical practice guidelines: a facility for the evaluation of clinical research," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1221-1235, February.
    15. Abrishami, Ali & Aliakbary, Sadegh, 2019. "Predicting citation counts based on deep neural network learning techniques," Journal of Informetrics, Elsevier, vol. 13(2), pages 485-499.
    16. Yeon Hak Kim & Aaron D. Levine & Eric J. Nehl & John P. Walsh, 2020. "A bibliometric measure of translational science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2349-2382, December.
    17. Zhentao Liang & Jin Mao & Kun Lu & Gang Li, 2021. "Finding citations for PubMed: a large-scale comparison between five freely available bibliographic data sources," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(12), pages 9519-9542, December.
    18. Boyack, Kevin W. & Patek, Michael & Ungar, Lyle H. & Yoon, Patrick & Klavans, Richard, 2014. "Classification of individual articles from all of science by research level," Journal of Informetrics, Elsevier, vol. 8(1), pages 1-12.
    19. Natsuo Onodera & Fuyuki Yoshikane, 2015. "Factors affecting citation rates of research articles," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(4), pages 739-764, April.
    20. Fengqing Zhang & Erjia Yan & Xin Niu & Yongjun Zhu, 2018. "Joint modeling of the association between NIH funding and its three primary outcomes: patents, publications, and citation impact," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 591-602, October.
    21. Vincent Larivière & Yves Gingras & Cassidy R. Sugimoto & Andrew Tsou, 2015. "Team size matters: Collaboration and scientific impact since 1900," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(7), pages 1323-1332, July.
    22. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    23. Francis Narin & Gabriel Pinski & Helen Hofer Gee, 1976. "Structure of the Biomedical Literature," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 27(1), pages 25-45, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Xin Li & Xuli Tang & Wei Lu, 2023. "Tracking biomedical articles along the translational continuum: a measure based on biomedical knowledge representation," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1295-1319, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dongyu Zang & Chunli Liu, 2023. "Exploring the clinical translation intensity of papers published by the world’s top scientists in basic medicine," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2371-2416, April.
    2. Xin Li & Xuli Tang & Wei Lu, 2023. "Tracking biomedical articles along the translational continuum: a measure based on biomedical knowledge representation," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1295-1319, February.
    3. Li, Xin & Tang, Xuli, 2021. "Characterizing interdisciplinarity in drug research: A translational science perspective," Journal of Informetrics, Elsevier, vol. 15(4).
    4. Wanjun Xia & Tianrui Li & Chongshou Li, 2023. "A review of scientific impact prediction: tasks, features and methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 543-585, January.
    5. Shengzhi Huang & Jiajia Qian & Yong Huang & Wei Lu & Yi Bu & Jinqing Yang & Qikai Cheng, 2022. "Disclosing the relationship between citation structure and future impact of a publication," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(7), pages 1025-1042, July.
    6. Belén Álvarez-Bornstein & Adrián A. Díaz-Faes & María Bordons, 2019. "What characterises funded biomedical research? Evidence from a basic and a clinical domain," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 805-825, May.
    7. Boyack, Kevin W. & Patek, Michael & Ungar, Lyle H. & Yoon, Patrick & Klavans, Richard, 2014. "Classification of individual articles from all of science by research level," Journal of Informetrics, Elsevier, vol. 8(1), pages 1-12.
    8. Martorell Cunil, Onofre & Otero González, Luis & Durán Santomil, Pablo & Mulet Forteza, Carlos, 2023. "How to accomplish a highly cited paper in the tourism, leisure and hospitality field," Journal of Business Research, Elsevier, vol. 157(C).
    9. Anckaert, Paul-Emmanuel & Cassiman, David & Cassiman, Bruno, 2020. "Fostering practice-oriented and use-inspired science in biomedical research," Research Policy, Elsevier, vol. 49(2).
    10. Rons, Nadine, 2018. "Bibliometric approximation of a scientific specialty by combining key sources, title words, authors and references," Journal of Informetrics, Elsevier, vol. 12(1), pages 113-132.
    11. Mike Thelwall & Kayvan Kousha & Mahshid Abdoli, 2017. "Is medical research informing professional practice more highly cited? Evidence from AHFS DI Essentials in drugs.com," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 509-527, July.
    12. Du, Jian & Li, Peixin & Guo, Qianying & Tang, Xiaoli, 2019. "Measuring the knowledge translation and convergence in pharmaceutical innovation by funding-science-technology-innovation linkages analysis," Journal of Informetrics, Elsevier, vol. 13(1), pages 132-148.
    13. Akella, Akhil Pandey & Alhoori, Hamed & Kondamudi, Pavan Ravikanth & Freeman, Cole & Zhou, Haiming, 2021. "Early indicators of scientific impact: Predicting citations with altmetrics," Journal of Informetrics, Elsevier, vol. 15(2).
    14. Mingyue Sun & Tingcan Ma & Lewei Zhou & Mingliang Yue, 2023. "Analysis of the relationships among paper citation and its influencing factors: a Bayesian network-based approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(5), pages 3017-3033, May.
    15. Gao, Qiang & Liang, Zhentao & Wang, Ping & Hou, Jingrui & Chen, Xiuxiu & Liu, Manman, 2021. "Potential index: Revealing the future impact of research topics based on current knowledge networks," Journal of Informetrics, Elsevier, vol. 15(3).
    16. Diana Hicks & Julia Melkers & Kimberley R. Isett, 2019. "A characterization of professional media and its links to research," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 827-843, May.
    17. Anqi Ma & Yu Liu & Xiujuan Xu & Tao Dong, 2021. "A deep-learning based citation count prediction model with paper metadata semantic features," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 6803-6823, August.
    18. Lawson, Cornelia & Salter, Ammon, 2023. "Exploring the effect of overlapping institutional applications on panel decision-making," Research Policy, Elsevier, vol. 52(9).
    19. Sato, Ryoma & Yamada, Makoto & Kashima, Hisashi, 2022. "Poincare: Recommending Publication Venues via Treatment Effect Estimation," Journal of Informetrics, Elsevier, vol. 16(2).
    20. Xuli Tang & Xin Li & Feicheng Ma, 2022. "Internationalizing AI: evolution and impact of distance factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 181-205, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:16:y:2022:i:4:s1751157722000852. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.