IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v107y2016i2d10.1007_s11192-016-1852-2.html
   My bibliography  Save this article

iSEER: an intelligent automatic computer system for scientific evaluation of researchers

Author

Listed:
  • Ashkan Ebadi

    (Concordia University)

  • Andrea Schiffauerova

    (Concordia University
    Masdar Institute of Science and Technology)

Abstract

Funding is one of the crucial drivers of scientific activities. The increasing number of researchers and the limited financial resources have caused a tight competition among scientists to secure research funding. On the other side, it is now even harder for funding allocation organizations to select the most proper researchers. Number of publications and citation counts based indicators are the most common methods in the literature for analyzing the performance of researchers. However, the mentioned indicators are highly correlated with the career age and reputation of the researchers, since they accumulate over time. This makes it almost impossible to evaluate the performance of a researcher based on quantity and impact of his/her articles at the time of the publication. This article proposes an intelligent machine learning framework for scientific evaluation of researchers (iSEER). iSEER may help decision makers to better allocate the available funding to the distinguished scientists through providing fair comparative results, regardless of the career age of the researchers. Our results show that iSEER performs well in predicting the performance of the researchers with high accuracy, as well as classifying them based on collaboration patterns, research performance, and efficiency.

Suggested Citation

  • Ashkan Ebadi & Andrea Schiffauerova, 2016. "iSEER: an intelligent automatic computer system for scientific evaluation of researchers," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(2), pages 477-498, May.
    • Handle: RePEc:spr:scient:v:107:y:2016:i:2:d:10.1007_s11192-016-1852-2
    DOI: 10.1007/s11192-016-1852-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-016-1852-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-016-1852-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Payne A. Abigail & Siow Aloysius, 2003. "Does Federal Research Funding Increase University Research Output?," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 3(1), pages 1-24, May.
    2. Ebadi, Ashkan & Schiffauerova, Andrea, 2015. "How to become an important player in scientific collaboration networks?," Journal of Informetrics, Elsevier, vol. 9(4), pages 809-825.
    3. T. J. Phelan, 1999. "A compendium of issues for citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 45(1), pages 117-136, May.
    4. Yoshiko Okubo, 1997. "Bibliometric Indicators and Analysis of Research Systems: Methods and Examples," OECD Science, Technology and Industry Working Papers 1997/1, OECD Publishing.
    5. Cowan, R. & Jonard, N., 2003. "The dynamics of collective invention," Journal of Economic Behavior & Organization, Elsevier, vol. 52(4), pages 513-532, December.
    6. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of research grant funding on scientific productivity," Journal of Public Economics, Elsevier, vol. 95(9), pages 1168-1177.
    7. Abbasi, Alireza & Altmann, Jörn & Hossain, Liaquat, 2011. "Identifying the effects of co-authorship networks on the performance of scholars: A correlation and regression analysis of performance measures and social network analysis measures," Journal of Informetrics, Elsevier, vol. 5(4), pages 594-607.
    8. Bell, John G & Seater, John J, 1978. "Publishing Performance: Departmental and Individual," Economic Inquiry, Western Economic Association International, vol. 16(4), pages 599-615, October.
    9. Diana Hicks & Hiroyuki Tomizawa & Yoshiko Saitoh & Shinichi Kobayashi, 2004. "Bibliometric techniques in the evaluation of federally funded research in the United States," Research Evaluation, Oxford University Press, vol. 13(2), pages 76-86, August.
    10. Katz, J. Sylvan & Martin, Ben R., 1997. "What is research collaboration?," Research Policy, Elsevier, vol. 26(1), pages 1-18, March.
    11. Per O. Seglen, 1992. "The skewness of science," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 43(9), pages 628-638, October.
    12. Lawrence D. Fu & Constantin F. Aliferis, 2010. "Using content-based and bibliometric features for machine learning models to predict citation counts in the biomedical literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 85(1), pages 257-270, October.
    13. Ashkan Ebadi & Andrea Schiffauerova, 2013. "Impact of Funding on Scientific Output and Collaboration: A Survey of Literature," Journal of Information & Knowledge Management (JIKM), World Scientific Publishing Co. Pte. Ltd., vol. 12(04), pages 1-16.
    14. Beaudry, Catherine & Allaoui, Sedki, 2012. "Impact of public and private research funding on scientific production: The case of nanotechnology," Research Policy, Elsevier, vol. 41(9), pages 1589-1606.
    15. Lawrence D. Fu & Yindalon Aphinyanaphongs & Constantin F. Aliferis, 2013. "Computer models for identifying instrumental citations in the biomedical literature," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(3), pages 871-882, December.
    16. Tijssen, Robert J. W., 2004. "Is the commercialisation of scientific research affecting the production of public knowledge?: Global trends in the output of corporate research articles," Research Policy, Elsevier, vol. 33(5), pages 709-733, July.
    17. Anthony F. J. van Raan, 2005. "Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods," Scientometrics, Springer;Akadémiai Kiadó, vol. 62(1), pages 133-143, January.
    18. Hamidreza Eslami & Ashkan Ebadi & Andrea Schiffauerova, 2013. "Effect of collaboration network structure on knowledge creation and technological performance: the case of biotechnology in Canada," Scientometrics, Springer;Akadémiai Kiadó, vol. 97(1), pages 99-119, October.
    19. Svein Kyvik & Terje Bruen Olsen, 2008. "Does the aging of tenured academic staff affect the research performance of universities?," Scientometrics, Springer;Akadémiai Kiadó, vol. 76(3), pages 439-455, September.
    20. Paul R. McAllister & Francis Narin, 1983. "Characterization of the research papers of U.S. medical schools," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 34(2), pages 123-131, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Anahita Hajibabaei & Andrea Schiffauerova & Ashkan Ebadi, 2023. "Women and key positions in scientific collaboration networks: analyzing central scientists’ profiles in the artificial intelligence ecosystem through a gender lens," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1219-1240, February.
    2. Parreiras, R.O. & Kokshenev, I. & Carvalho, M.O.M. & Willer, A.C.M. & Dellezzopolles, C.F. & Nacif, D.B. & Santana, J.A., 2019. "A flexible multicriteria decision-making methodology to support the strategic management of Science, Technology and Innovation research funding programs," European Journal of Operational Research, Elsevier, vol. 272(2), pages 725-739.
    3. Ebadi, Ashkan & Tremblay, Stéphane & Goutte, Cyril & Schiffauerova, Andrea, 2020. "Application of machine learning techniques to assess the trends and alignment of the funded research output," Journal of Informetrics, Elsevier, vol. 14(2).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ebadi, Ashkan & Schiffauerova, Andrea, 2015. "How to become an important player in scientific collaboration networks?," Journal of Informetrics, Elsevier, vol. 9(4), pages 809-825.
    2. Ashkan Ebadi & Andrea Schiffauerova, 2015. "How to Receive More Funding for Your Research? Get Connected to the Right People!," PLOS ONE, Public Library of Science, vol. 10(7), pages 1-19, July.
    3. Ashkan Ebadi & Andrea Schiffauerova, 2016. "How to boost scientific production? A statistical analysis of research funding and other influencing factors," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 1093-1116, March.
    4. Kazuki Nakajima & Kazuyuki Shudo & Naoki Masuda, 2023. "Higher-order rich-club phenomenon in collaborative research grant networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(4), pages 2429-2446, April.
    5. Annita Nugent & Ho Fai Chan & Uwe Dulleck, 2022. "Government funding of university-industry collaboration: exploring the impact of targeted funding on university patent activity," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(1), pages 29-73, January.
    6. Anahita Hajibabaei & Andrea Schiffauerova & Ashkan Ebadi, 2023. "Women and key positions in scientific collaboration networks: analyzing central scientists’ profiles in the artificial intelligence ecosystem through a gender lens," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(2), pages 1219-1240, February.
    7. Alona Zharova & Wolfgang K. Härdle & Stefan Lessmann, 2017. "Is Scientific Performance a Function of Funds?," SFB 649 Discussion Papers SFB649DP2017-028, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany.
    8. Nuha Zamzami & Andrea Schiffauerova, 2017. "The impact of individual collaborative activities on knowledge creation and transmission," Scientometrics, Springer;Akadémiai Kiadó, vol. 111(3), pages 1385-1413, June.
    9. Barbara S. Lancho-Barrantes & Hector G. Ceballos-Cancino & Francisco J. Cantu-Ortiz, 2021. "Comparing the efficiency of countries to assimilate and apply research investment," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(4), pages 1347-1369, August.
    10. Beaudry, Catherine & Allaoui, Sedki, 2012. "Impact of public and private research funding on scientific production: The case of nanotechnology," Research Policy, Elsevier, vol. 41(9), pages 1589-1606.
    11. Belén Álvarez-Bornstein & Adrián A. Díaz-Faes & María Bordons, 2019. "What characterises funded biomedical research? Evidence from a basic and a clinical domain," Scientometrics, Springer;Akadémiai Kiadó, vol. 119(2), pages 805-825, May.
    12. Corsini, Alberto & Pezzoni, Michele, 2023. "Does grant funding foster research impact? Evidence from France," Journal of Informetrics, Elsevier, vol. 17(4).
    13. Álvarez-Bornstein, Belén & Bordons, María, 2021. "Is funding related to higher research impact? Exploring its relationship and the mediating role of collaboration in several disciplines," Journal of Informetrics, Elsevier, vol. 15(1).
    14. Hugo Confraria & Fernando Vargas, 2019. "Scientific systems in Latin America: performance, networks, and collaborations with industry," The Journal of Technology Transfer, Springer, vol. 44(3), pages 874-915, June.
    15. Jing Tu, 2019. "What connections lead to good scientific performance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(2), pages 587-604, February.
    16. Daniela Filippo & Pablo Sastrón-Toledo, 2023. "Influence of research on open science in the public policy sphere," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(3), pages 1995-2017, March.
    17. Yu, Nannan & Dong, Yueyan & de Jong, Martin, 2022. "A helping hand from the government? How public research funding affects academic output in less-prestigious universities in China," Research Policy, Elsevier, vol. 51(10).
    18. Guillermo Armando Ronda-Pupo & J. Sylvan Katz, 2017. "The scaling relationship between degree centrality of countries and their citation-based performance on Management Information Systems," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1285-1299, September.
    19. Rachel Heyard & Hanna Hottenrott, 2021. "The value of research funding for knowledge creation and dissemination: A study of SNSF Research Grants," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-16, December.
    20. Alberto Corsini & Michele Pezzoni, 2022. "Does grant funding foster research impact? Evidence from France," SciencePo Working papers Main hal-03912647, HAL.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:107:y:2016:i:2:d:10.1007_s11192-016-1852-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.