IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v127y2022i6d10.1007_s11192-022-04389-4.html
   My bibliography  Save this article

Reviewer recommendation method for scientific research proposals: a case for NSFC

Author

Listed:
  • Xiaoyu Liu

    (Beijing Electronic Science & Technology Institute)

  • Xuefeng Wang

    (Beijing Institute of Technology)

  • Donghua Zhu

    (Beijing Institute of Technology)

Abstract

Peer review is one of the important procedures to determine which research proposals are to be funded and to evaluate the quality of scientific research. How to find suitable reviewers for scientific research proposals is an important task for funding agencies. Traditional methods for reviewer recommendation focus on the relevance of the proposal and knowledge of candidate reviewers by mainly matching the keywords or disciplines. However, the sparsity of keyword space and the broadness of disciplines lead to inaccurate reviewer recommendations. To overcome these limitations, this paper introduces a reviewer recommendation method (RRM) for scientific research proposals. This research applies word embedding to construct vector representation for terms, which provides a semantic and syntactic measurement. Further, we develop representation models for reviewers’ knowledge and proposals, and recommend reviewers by matching two representation models incorporating ranking fusions. The proposed method is implemented and tested by recommending reviewers for scientific research proposals of the National Natural Science Foundation of China. This research invites reviewers to provide feedback, which works as the benchmark for evaluation. We construct three evaluation metrics, Precision, Strict-precision, and Recall. The results show that the proposed reviewer recommendation method highly improves the accuracy. Research results can provide feasible options for the decision-making of the committee, and improve the efficiency of funding agencies.

Suggested Citation

  • Xiaoyu Liu & Xuefeng Wang & Donghua Zhu, 2022. "Reviewer recommendation method for scientific research proposals: a case for NSFC," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(6), pages 3343-3366, June.
    • Handle: RePEc:spr:scient:v:127:y:2022:i:6:d:10.1007_s11192-022-04389-4
    DOI: 10.1007/s11192-022-04389-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-022-04389-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-022-04389-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hendy Abdoul & Christophe Perrey & Philippe Amiel & Florence Tubach & Serge Gottot & Isabelle Durand-Zaleski & Corinne Alberti, 2012. "Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices," PLOS ONE, Public Library of Science, vol. 7(9), pages 1-15, September.
    2. Zhang, Yi & Lu, Jie & Liu, Feng & Liu, Qian & Porter, Alan & Chen, Hongshu & Zhang, Guangquan, 2018. "Does deep learning help topic extraction? A kernel k-means clustering method with word embedding," Journal of Informetrics, Elsevier, vol. 12(4), pages 1099-1117.
    3. Wade D. Cook & Boaz Golany & Moshe Kress & Michal Penn & Tal Raviv, 2005. "Optimal Allocation of Proposals to Reviewers to Facilitate Effective Ranking," Management Science, INFORMS, vol. 51(4), pages 655-661, April.
    4. Shu Zhao & Dong Zhang & Zhen Duan & Jie Chen & Yan-ping Zhang & Jie Tang, 2018. "A novel classification method for paper-reviewer recommendation," Scientometrics, Springer;Akadémiai Kiadó, vol. 115(3), pages 1293-1313, June.
    5. David Liben‐Nowell & Jon Kleinberg, 2007. "The link‐prediction problem for social networks," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(7), pages 1019-1031, May.
    6. Xiaoyu Liu & Alan L. Porter, 2020. "A 3-dimensional analysis for evaluating technology emergence indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(1), pages 27-55, July.
    7. Kai Hu & Huayi Wu & Kunlun Qi & Jingmin Yu & Siluo Yang & Tianxing Yu & Jie Zheng & Bo Liu, 2018. "A domain keyword analysis approach extending Term Frequency-Keyword Active Index with Google Word2Vec model," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1031-1068, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Lu Huang & Xiang Chen & Yi Zhang & Yihe Zhu & Suyi Li & Xingxing Ni, 2021. "Dynamic network analytics for recommending scientific collaborators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(11), pages 8789-8814, November.
    2. Lu Huang & Xiang Chen & Yi Zhang & Changtian Wang & Xiaoli Cao & Jiarun Liu, 2022. "Identification of topic evolution: network analytics with piecewise linear representation and word embedding," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5353-5383, September.
    3. Xiaowen Xi & Jiaqi Wei & Ying Guo & Weiyu Duan, 2022. "Academic collaborations: a recommender framework spanning research interests and network topology," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6787-6808, November.
    4. Lu Huang & Yijie Cai & Erdong Zhao & Shengting Zhang & Yue Shu & Jiao Fan, 2022. "Measuring the interdisciplinarity of Information and Library Science interactions using citation analysis and semantic analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(11), pages 6733-6761, November.
    5. Byungun Yoon & Songhee Kim & Sunhye Kim & Hyeonju Seol, 2022. "Doc2vec-based link prediction approach using SAO structures: application to patent network," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5385-5414, September.
    6. Woo, Seokkyun & Youtie, Jan & Ott, Ingrid & Scheu, Fenja, 2021. "Understanding the long-term emergence of autonomous vehicles technologies," Technological Forecasting and Social Change, Elsevier, vol. 170(C).
    7. Huang, Lu & Chen, Xiang & Ni, Xingxing & Liu, Jiarun & Cao, Xiaoli & Wang, Changtian, 2021. "Tracking the dynamics of co-word networks for emerging topic identification," Technological Forecasting and Social Change, Elsevier, vol. 170(C).
    8. Jeong, Yujin & Park, Inchae & Yoon, Byungun, 2019. "Identifying emerging Research and Business Development (R&BD) areas based on topic modeling and visualization with intellectual property right data," Technological Forecasting and Social Change, Elsevier, vol. 146(C), pages 655-672.
    9. Yifei Zhou & Shaoyong Li & Yaping Liu, 2020. "Graph-based Method for App Usage Prediction with Attributed Heterogeneous Network Embedding," Future Internet, MDPI, vol. 12(3), pages 1-16, March.
    10. Karimi, Fatemeh & Lotfi, Shahriar & Izadkhah, Habib, 2021. "Community-guided link prediction in multiplex networks," Journal of Informetrics, Elsevier, vol. 15(4).
    11. Fernandez Martinez, Roberto & Lostado Lorza, Ruben & Santos Delgado, Ana Alexandra & Piedra, Nelson, 2021. "Use of classification trees and rule-based models to optimize the funding assignment to research projects: A case study of UTPL," Journal of Informetrics, Elsevier, vol. 15(1).
    12. Xu, Hua & Wang, Minggang & Jiang, Shumin & Yang, Weiguo, 2020. "Carbon price forecasting with complex network and extreme learning machine," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 545(C).
    13. Andreas Spitz & Anna Gimmler & Thorsten Stoeck & Katharina Anna Zweig & Emőke-Ágnes Horvát, 2016. "Assessing Low-Intensity Relationships in Complex Networks," PLOS ONE, Public Library of Science, vol. 11(4), pages 1-17, April.
    14. Paul Donner, 2021. "Validation of the Astro dataset clustering solutions with external data," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1619-1645, February.
    15. Karol Król & Dariusz Zdonek, 2023. "Cultural Heritage Topics in Online Queries: A Comparison between English- and Polish-Speaking Internet Users," Sustainability, MDPI, vol. 15(6), pages 1-20, March.
    16. Xiang Zhu & Yunqiu Zhang, 2020. "Co-word analysis method based on meta-path of subject knowledge network," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 753-766, May.
    17. Xiaoli Wang & Yun Liu & Lingdi Chen & Yifan Zhang, 2022. "Correlation Monitoring Method and model of Science-Technology-Industry in the AI Field: A Case of the Neural Network," SAGE Open, , vol. 12(4), pages 21582440221, December.
    18. Qiaoran Yang & Zhiliang Dong & Yichi Zhang & Man Li & Ziyi Liang & Chao Ding, 2021. "Who Will Establish New Trade Relations? Looking for Potential Relationship in International Nickel Trade," Sustainability, MDPI, vol. 13(21), pages 1-15, October.
    19. Nora Connor & Albert Barberán & Aaron Clauset, 2017. "Using null models to infer microbial co-occurrence networks," PLOS ONE, Public Library of Science, vol. 12(5), pages 1-23, May.
    20. Aslan, Serpil & Kaya, Buket & Kaya, Mehmet, 2019. "Predicting potential links by using strengthened projections in evolving bipartite networks," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 525(C), pages 998-1011.

    More about this item

    Keywords

    Reviewer recommendation; Knowledge representation; Word embedding; Scientific research proposal selection; Peer review;
    All these keywords.

    JEL classification:

    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:127:y:2022:i:6:d:10.1007_s11192-022-04389-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.