IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v102y2015i1d10.1007_s11192-014-1420-6.html
   My bibliography  Save this article

Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan

Author

Listed:
  • Kuang-hua Chen

    (National Taiwan University)

  • Muh-chyun Tang

    (National Taiwan University)

  • Chun-mei Wang

    (National Taiwan University)

  • Jieh Hsiang

    (National Taiwan University)

Abstract

Research output and impact metrics derived from commercial citation databases such as Web of Science and Scopus have become the de facto indicators of scholarly performance across different disciplines and regions. However, it has been pointed out that the existing metrics are largely inadequate to reflect scholars’ overall peer-mediated performance, especially in the social sciences and humanities (SSH) where publication channels are more diverse. In this paper alternative metrics exploring a variety of formal and informal communication channels were proposed, with the aim of better reflecting SSH scholarship. Data for a group of SSH scholars in Taiwan on these metrics were collected. Principal component analysis revealed four underlying dimensions represented by the 18 metrics. Multiple-regression analyses were then performed to examine how well each of these dimensions predicted the academic standing of the scholars, measured by the number of public grants awarded and prestigious research awards received. Differences in the significance of the predictors were found between the social sciences and humanities. The results suggest the need to consider disciplinary differences when evaluating scholarly performance.

Suggested Citation

  • Kuang-hua Chen & Muh-chyun Tang & Chun-mei Wang & Jieh Hsiang, 2015. "Exploring alternative metrics of scholarly performance in the social sciences and humanities in Taiwan," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 97-112, January.
    • Handle: RePEc:spr:scient:v:102:y:2015:i:1:d:10.1007_s11192-014-1420-6
    DOI: 10.1007/s11192-014-1420-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-014-1420-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-014-1420-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Norris, Michael & Oppenheim, Charles, 2007. "Comparing alternatives to the Web of Science for coverage of the social sciences’ literature," Journal of Informetrics, Elsevier, vol. 1(2), pages 161-169.
    2. Anton J. Nederhof, 2006. "Bibliometric monitoring of research performance in the Social Sciences and the Humanities: A Review," Scientometrics, Springer;Akadémiai Kiadó, vol. 66(1), pages 81-100, January.
    3. Mu‐hsuan Huang & Yu‐wei Chang, 2008. "Characteristics of research output in social sciences and humanities: From a research evaluation perspective," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(11), pages 1819-1828, September.
    4. Johan Bollen & Herbert Van de Sompel & Aric Hagberg & Ryan Chute, 2009. "A Principal Component Analysis of 39 Scientific Impact Measures," PLOS ONE, Public Library of Science, vol. 4(6), pages 1-11, June.
    5. Xuemei Li & Mike Thelwall & Dean Giustini, 2012. "Validating online reference managers for scholarly impact measurement," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 461-471, May.
    6. Pardeep Sud & Mike Thelwall, 2014. "Evaluating altmetrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(2), pages 1131-1143, February.
    7. Ülle Must, 2012. "Alone or together: examples from history research," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(2), pages 527-537, May.
    8. Kayvan Kousha & Mike Thelwall, 2008. "Assessing the impact of disciplinary research on teaching: An automatic analysis of online syllabuses," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(13), pages 2060-2069, November.
    9. Heather Piwowar, 2013. "Value all research products," Nature, Nature, vol. 493(7431), pages 159-159, January.
    10. Lokman I. Meho & Kiduk Yang, 2007. "Impact of data sources on citation counts and rankings of LIS faculty: Web of science versus scopus and google scholar," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 58(13), pages 2105-2125, November.
    11. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    12. Haustein, Stefanie & Siebenlist, Tobias, 2011. "Applying social bookmarking data to evaluate journal usage," Journal of Informetrics, Elsevier, vol. 5(3), pages 446-457.
    13. Ludo Waltman & Rodrigo Costas, 2014. "F1000 Recommendations as a Potential New Data Source for Research Evaluation: A Comparison With Citations," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(3), pages 433-445, March.
    14. Mike Thelwall & Gareth Harries, 2004. "Do the Web sites of higher rated scholars have significantly more online impact?," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 55(2), pages 149-159, January.
    15. Hadas Shema & Judit Bar-Ilan & Mike Thelwall, 2014. "Do blog citations correlate with a higher number of future citations? Research blogs as a potential source for alternative metrics," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(5), pages 1018-1027, May.
    16. Isidro F. Aguillo & Begoña Granadino & José L. Ortega & José A. Prieto, 2006. "Scientific research activity and communication measured with cybermetrics indicators," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 57(10), pages 1296-1302, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mousumi Karmakar & Sumit Kumar Banshal & Vivek Kumar Singh, 2020. "Does presence of social media plugins in a journal website result in higher social media attention of its research publications?," Scientometrics, Springer;Akadémiai Kiadó, vol. 124(3), pages 2103-2143, September.
    2. Abdelghani Maddi & Aouatif de La Laurencie, 2018. "La dynamique des SHS françaises dans le Web of Science : un manque de représentativité ou de visibilité internationale ?," CEPN Working Papers hal-01922266, HAL.
    3. Thelwall, Mike & Nevill, Tamara, 2018. "Could scientists use Altmetric.com scores to predict longer term citation counts?," Journal of Informetrics, Elsevier, vol. 12(1), pages 237-248.
    4. Jámbor, Attila & Popp, József & Balogh, Péter & Kovács, Sándor, 2015. "Hálózatosodás az agrárgazdaságtanban. Szerzői és hivatkozási kapcsolatok a Kelet-Közép-Európáról szóló szakirodalomban [Cliques in agricultural economics: co-authorship and co-citation networks in ," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(5), pages 525-543.
    5. Banshal, Sumit Kumar & Gupta, Solanki & Lathabai, Hiran H & Singh, Vivek Kumar, 2022. "Power Laws in altmetrics: An empirical analysis," Journal of Informetrics, Elsevier, vol. 16(3).
    6. Abdelghani Maddi & Aouatif De La Laurencie, 2018. "La dynamique des SHS françaises dans le Web of Science," CEPN Working Papers 2018-05, Centre d'Economie de l'Université de Paris Nord.
    7. József Popp & Péter Balogh & Judit Oláh & Sebastian Kot & Mónika Harangi Rákos & Péter Lengyel, 2018. "Social Network Analysis of Scientific Articles Published by Food Policy," Sustainability, MDPI, vol. 10(3), pages 1-20, February.
    8. Daniela De Filippo & Fernanda Morillo & Borja González-Albo, 2023. "Measuring the Impact and Influence of Scientific Activity in the Humanities and Social Sciences," Publications, MDPI, vol. 11(2), pages 1-17, June.
    9. Ospina-Delgado, Julieth & Zorio-Grima, Ana, 2016. "Innovation at universities: A fuzzy-set approach for MOOC-intensiveness," Journal of Business Research, Elsevier, vol. 69(4), pages 1325-1328.
    10. Siluo Yang & Xin Xing & Fan Qi & Maria Cláudia Cabrini Grácio, 2021. "Comparison of academic book impact from a disciplinary perspective: an analysis of citations and altmetric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(2), pages 1101-1123, February.
    11. Yang, Siluo & Zheng, Mengxue & Yu, Yonghao & Wolfram, Dietmar, 2021. "Are Altmetric.com scores effective for research impact evaluation in the social sciences and humanities?," Journal of Informetrics, Elsevier, vol. 15(1).
    12. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ehsan Mohammadi & Mike Thelwall & Stefanie Haustein & Vincent Larivière, 2015. "Who reads research articles? An altmetrics analysis of Mendeley user categories," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(9), pages 1832-1846, September.
    2. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    3. Mojisola Erdt & Aarthy Nagarajan & Sei-Ching Joanna Sin & Yin-Leng Theng, 2016. "Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(2), pages 1117-1166, November.
    4. Zohreh Zahedi & Rodrigo Costas & Paul Wouters, 2014. "How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(2), pages 1491-1513, November.
    5. Hou, Jianhua & Yang, Xiucai, 2020. "Social media-based sleeping beauties: Defining, identifying and features," Journal of Informetrics, Elsevier, vol. 14(2).
    6. Amalia Mas-Bleda & Mike Thelwall & Kayvan Kousha & Isidro F. Aguillo, 2014. "Do highly cited researchers successfully use the social web?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 337-356, October.
    7. Sergio Copiello, 2020. "Other than detecting impact in advance, alternative metrics could act as early warning signs of retractions: tentative findings of a study into the papers retracted by PLoS ONE," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2449-2469, December.
    8. Mike Thelwall, 2012. "Journal impact evaluation: a webometric perspective," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 429-441, August.
    9. Bornmann, Lutz, 2014. "Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics," Journal of Informetrics, Elsevier, vol. 8(4), pages 895-903.
    10. Maryam Moshtagh & Tahereh Jowkar & Maryam Yaghtin & Hajar Sotudeh, 2023. "The moderating effect of altmetrics on the correlations between single and multi-faceted university ranking systems: the case of THE and QS vs. Nature Index and Leiden," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(1), pages 761-781, January.
    11. Kousha, Kayvan & Thelwall, Mike & Rezaie, Somayeh, 2010. "Using the Web for research evaluation: The Integrated Online Impact indicator," Journal of Informetrics, Elsevier, vol. 4(1), pages 124-135.
    12. Jianhua Hou & Xiucai Yang & Yang Zhang, 2023. "The effect of social media knowledge cascade: an analysis of scientific papers diffusion," Scientometrics, Springer;Akadémiai Kiadó, vol. 128(9), pages 5169-5195, September.
    13. Isidro F. Aguillo, 2020. "Altmetrics of the Open Access Institutional Repositories: a webometrics approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(3), pages 1181-1192, June.
    14. Yu Liu & Dan Lin & Xiujuan Xu & Shimin Shan & Quan Z. Sheng, 2018. "Multi-views on Nature Index of Chinese academic institutions," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 823-837, March.
    15. Saeideh Ebrahimy & Jafar Mehrad & Fatemeh Setareh & Massoud Hosseinchari, 2016. "Path analysis of the relationship between visibility and citation: the mediating roles of save, discussion, and recommendation metrics," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1497-1510, December.
    16. Ekaterina L. Dyachenko, 2014. "Internationalization of academic journals: Is there still a gap between social and natural sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 241-255, October.
    17. Andrea Bonaccorsi & Cinzia Daraio & Stefano Fantoni & Viola Folli & Marco Leonetti & Giancarlo Ruocco, 2017. "Do social sciences and humanities behave like life and hard sciences?," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(1), pages 607-653, July.
    18. Amalia Mas-Bleda & Mike Thelwall, 2016. "Can alternative indicators overcome language biases in citation counts? A comparison of Spanish and UK research," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 2007-2030, December.
    19. Ashraf Maleki, 2022. "Why does library holding format really matter for book impact assessment?: Modelling the relationship between citations and altmetrics with print and electronic holdings," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(2), pages 1129-1160, February.
    20. Saeed-Ul Hassan & Mubashir Imran & Uzair Gillani & Naif Radi Aljohani & Timothy D. Bowman & Fereshteh Didegah, 2017. "Measuring social media activity of scientific literature: an exhaustive comparison of scopus and novel altmetrics big data," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 1037-1057, November.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:102:y:2015:i:1:d:10.1007_s11192-014-1420-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.