IDEAS home Printed from https://ideas.repec.org/a/gam/jsusta/v13y2021i13p7147-d582138.html
   My bibliography  Save this article

Data-Driven Public R&D Project Performance Evaluation: Results from China

Author

Listed:
  • Hongbo Li

    (School of Management, Shanghai University, Shanghai 200044, China)

  • Bowen Yao

    (School of Management, Shanghai University, Shanghai 200044, China)

  • Xin Yan

    (School of Management, Shanghai University, Shanghai 200044, China)

Abstract

In public R&D projects, to improve the decision-making process and ensure the sustainability of public investment, it is indispensable to effectively evaluate the project performance. Currently, public R&D project management departments and various academic databases have accumulated a large number of project-related data. In view of this, we propose a data-driven performance evaluation framework for public R&D projects. In our framework, we collect structured and unstructured data related to completed projects from multiple websites. Then, these data are cleaned and fused to form a unified dataset. We train a project performance evaluation model by extracting the project performance information implicit in the dataset based on multi-classification supervised learning algorithms. When facing a new project that needs to be evaluated, its performance can be automatically predicted by inputting the characteristic information of the project into our performance evaluation model. Our framework is validated based on the project data of the National Natural Science Foundation of China (NSFC) in terms of four performance measures (i.e., Accuracy, Recall, Precision, F 1 score). In addition, we provide a case study that applies our framework to evaluate the project performance in the logistics and supply chain area of NSFC. In conclusion, this paper contributes to the body of knowledge in sustainability by developing a data-driven method that equips the decision-maker with an automated project performance evaluation tool to make sustainable project decisions.

Suggested Citation

  • Hongbo Li & Bowen Yao & Xin Yan, 2021. "Data-Driven Public R&D Project Performance Evaluation: Results from China," Sustainability, MDPI, vol. 13(13), pages 1-14, June.
    • Handle: RePEc:gam:jsusta:v:13:y:2021:i:13:p:7147-:d:582138
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2071-1050/13/13/7147/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2071-1050/13/13/7147/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Ji-ping Gao & Cheng Su & Hai-yan Wang & Li-hua Zhai & Yun-tao Pan, 2019. "Research fund evaluation based on academic publication output analysis: the case of Chinese research fund evaluation," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 959-972, May.
    2. Johnes, Jill & Yu, Li, 2008. "Measuring the research performance of Chinese higher education institutions using data envelopment analysis," China Economic Review, Elsevier, vol. 19(4), pages 679-696, December.
    3. Woo Sung Kim & Kunsu Park & Sang Hoon Lee & Hongyoung Kim, 2018. "R&D Investments and Firm Value: Evidence from China," Sustainability, MDPI, vol. 10(11), pages 1-17, November.
    4. Hsu, Fang-Ming & Hsueh, Chao-Chih, 2009. "Measuring relative efficiency of government-sponsored R&D projects: A three-stage approach," Evaluation and Program Planning, Elsevier, vol. 32(2), pages 178-186, May.
    5. Karasakal, Esra & Aker, Pınar, 2017. "A multicriteria sorting approach based on data envelopment analysis for R&D project selection problem," Omega, Elsevier, vol. 73(C), pages 79-92.
    6. Jaehun Park & Joonyoung Kim & Si-Il Sung, 2017. "Performance Evaluation of Research and Business Development: A Case Study of Korean Public Organizations," Sustainability, MDPI, vol. 9(12), pages 1-16, December.
    7. Eilat, Harel & Golany, Boaz & Shtub, Avraham, 2008. "R&D project evaluation: An integrated DEA and balanced scorecard approach," Omega, Elsevier, vol. 36(5), pages 895-912, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Hongbo Li & Rui Chen & Xianchao Zhang, 2022. "Uncertain Public R&D Project Portfolio Selection Considering Sectoral Balancing and Project Failure," Sustainability, MDPI, vol. 14(23), pages 1-13, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Dalton Garcia Borges de Souza & Erivelton Antonio dos Santos & Nei Yoshihiro Soma & Carlos Eduardo Sanches da Silva, 2021. "MCDM-Based R&D Project Selection: A Systematic Literature Review," Sustainability, MDPI, vol. 13(21), pages 1-34, October.
    2. Sungmin Park, 2015. "The R&D logic model: Does it really work? An empirical verification using successive binary logistic regression models," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1399-1439, December.
    3. Kun Chen & Xian-tong Ren & Guo-liang Yang & Hai-bo Qin, 2022. "The other side of the coin: The declining of Chinese social science," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 127-143, January.
    4. Lu, Wen-Min & Kweh, Qian Long & Nourani, Mohammad & Huang, Feng-Wen, 2016. "Evaluating the efficiency of dual-use technology development programs from the R&D and socio-economic perspectives," Omega, Elsevier, vol. 62(C), pages 82-92.
    5. Li, Yan-Lai & Tang, Jia-Fu & Chin, Kwai-Sang & Jiang, Yu-Shi & Han, Yi & Pu, Yun, 2011. "Estimating the final priority ratings of engineering characteristics in mature-period product improvement by MDBA and AHP," International Journal of Production Economics, Elsevier, vol. 131(2), pages 575-586, June.
    6. Salimi, Negin & Rezaei, Jafar, 2018. "Evaluating firms’ R&D performance using best worst method," Evaluation and Program Planning, Elsevier, vol. 66(C), pages 147-155.
    7. Fu, Tsu-Tan & See, Kok Fong, 2022. "An integrated analysis of quality and productivity growth in China’s and Taiwan’s higher education institutions," Economic Analysis and Policy, Elsevier, vol. 74(C), pages 234-249.
    8. Xianmei Wang & Hanhui Hu, 2017. "Sustainability in Chinese Higher Educational Institutions’ Social Science Research: A Performance Interface toward Efficiency," Sustainability, MDPI, vol. 9(11), pages 1-18, October.
    9. Rafael Lizarralde & Jaione Ganzarain & Mikel Zubizarreta, 2020. "Assessment and Selection of Technologies for the Sustainable Development of an R&D Center," Sustainability, MDPI, vol. 12(23), pages 1-23, December.
    10. George E. Halkos & Nickolaos G. Tzeremes & Stavros A. Kourtzidis, 2010. "An application of statistical interference in DEA models: An analysis of public owned university departments' efficiency," EERI Research Paper Series EERI_RP_2010_17, Economics and Econometrics Research Institute (EERI), Brussels.
    11. Zhang, Daqun & Banker, Rajiv D. & Li, Xiaoxuan & Liu, Wenbin, 2011. "Performance impact of research policy at the Chinese Academy of Sciences," Research Policy, Elsevier, vol. 40(6), pages 875-885, July.
    12. Partovi, Fariborz Y., 2011. "Corporate philanthropic selection using data envelopment analysis," Omega, Elsevier, vol. 39(5), pages 522-527, October.
    13. Wang, Derek D., 2019. "Performance-based resource allocation for higher education institutions in China," Socio-Economic Planning Sciences, Elsevier, vol. 65(C), pages 66-75.
    14. Qianqian Jin & Hongshu Chen & Ximeng Wang & Tingting Ma & Fei Xiong, 2022. "Exploring funding patterns with word embedding-enhanced organization–topic networks: a case study on big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(9), pages 5415-5440, September.
    15. Zhang, Ganggang & Wu, Jie & Zhu, Qingyuan, 2020. "Performance evaluation and enrollment quota allocation for higher education institutions in China," Evaluation and Program Planning, Elsevier, vol. 81(C).
    16. Chia-Hua Cheng & James J. H. Liou & Chui-Yu Chiu, 2017. "A Consistent Fuzzy Preference Relations Based ANP Model for R&D Project Selection," Sustainability, MDPI, vol. 9(8), pages 1-17, August.
    17. Khaled Belahcène & Vincent Mousseau & Wassila Ouerdane & Marc Pirlot & Olivier Sobrie, 2023. "Multiple criteria sorting models and methods—Part I: survey of the literature," 4OR, Springer, vol. 21(1), pages 1-46, March.
    18. Ma, Zhanxin & See, Kok Fong & Yu, Ming-Miin & Zhao, Chunying, 2021. "Research efficiency analysis of China's university faculty members: A modified meta-frontier DEA approach," Socio-Economic Planning Sciences, Elsevier, vol. 76(C).
    19. Marius Sorin Dincă & Gheorghiţa Dincă & Maria Letiţia Andronic & Anna Maria Pasztori, 2021. "Assessment of the European Union’s Educational Efficiency," Sustainability, MDPI, vol. 13(6), pages 1-29, March.
    20. Kristof De Witte & Laura López-Torres, 2017. "Efficiency in education: a review of literature and a way forward," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 68(4), pages 339-363, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jsusta:v:13:y:2021:i:13:p:7147-:d:582138. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.