IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i1d10.1007_s11192-020-03758-1.html
   My bibliography  Save this article

A comprehensive quality assessment framework for scientific events

Author

Listed:
  • Sahar Vahdati

    (University of Oxford
    Institute for Applied Informatics (InfAI))

  • Said Fathalla

    (University of Bonn
    University of Alexandria)

  • Christoph Lange

    (RWTH Aachen University
    Fraunhofer FIT)

  • Andreas Behrend

    (Institute for Telecommunications (INT))

  • Aysegul Say

    (University of Bonn)

  • Zeynep Say

    (University of Bonn)

  • Sören Auer

    (Leibniz University of Hannover
    TIB Leibniz Information Centre for Science and Technology)

Abstract

Systematic assessment of scientific events has become increasingly important for research communities. A range of metrics (e.g., citations, h-index) have been developed by different research communities to make such assessments effectual. However, most of the metrics for assessing the quality of less formal publication venues and events have not yet deeply investigated. It is also rather challenging to develop respective metrics because each research community has its own formal and informal rules of communication and quality standards. In this article, we develop a comprehensive framework of assessment metrics for evaluating scientific events and involved stakeholders. The resulting quality metrics are determined with respect to three general categories—events, persons, and bibliometrics. Our assessment methodology is empirically applied to several series of computer science events, such as conferences and workshops, using publicly available data for determining quality metrics. We show that the metrics’ values coincide with the intuitive agreement of the community on its “top conferences”. Our results demonstrate that highly-ranked events share similar profiles, including the provision of outstanding reviews, visiting diverse locations, having reputed people involved, and renowned sponsors.

Suggested Citation

  • Sahar Vahdati & Said Fathalla & Christoph Lange & Andreas Behrend & Aysegul Say & Zeynep Say & Sören Auer, 2021. "A comprehensive quality assessment framework for scientific events," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 641-682, January.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03758-1
    DOI: 10.1007/s11192-020-03758-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03758-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03758-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Markus Wolf & Hans-Dieter Daniel, 2012. "Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 843-856, June.
    2. Hu, Zhigang & Chen, Chaomei & Liu, Zeyuan, 2013. "Where are citations located in the body of scientific articles? A study of the distributions of citation locations," Journal of Informetrics, Elsevier, vol. 7(4), pages 887-896.
    3. Goldberg, Marvin E & Hartwick, Jon, 1990. "The Effects of Advertiser Reputation and Extremity of Advertising Claim on Advertising Effectiveness," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 17(2), pages 172-179, September.
    4. Said Fathalla & Sahar Vahdati & Christoph Lange & Sören Auer, 2020. "Scholarly event characteristics in four fields of science: a metrics-based analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 677-705, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yaxue Ma & Zhichao Ba & Yuxiang Zhao & Jin Mao & Gang Li, 2021. "Understanding and predicting the dissemination of scientific papers on social media: a two-step simultaneous equation modeling–artificial neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7051-7085, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hren, Darko & Pina, David G. & Norman, Christopher R. & Marušić, Ana, 2022. "What makes or breaks competitive research proposals? A mixed-methods analysis of research grant evaluation reports," Journal of Informetrics, Elsevier, vol. 16(2).
    2. repec:dau:papers:123456789/5853 is not listed on IDEAS
    3. Kaveh Moghaddam & Thomas Weber & Pouya Seifzadeh & Sara Azarpanah, 2021. "Internal Reputation of the Firm: CEO Retention and Firm Market Performance," Corporate Reputation Review, Palgrave Macmillan, vol. 24(4), pages 205-221, November.
    4. Naif Radi Aljohani & Ayman Fayoumi & Saeed-Ul Hassan, 2021. "An in-text citation classification predictive model for a scholarly search system," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5509-5529, July.
    5. Lariviere, Bart & Van den Poel, Dirk, 2007. "Banking behaviour after the lifecycle event of "moving in together": An exploratory study of the role of marketing investments," European Journal of Operational Research, Elsevier, vol. 183(1), pages 345-369, November.
    6. Karpinska-Krakowiak, Malgorzata, 2021. "Women are more likely to buy unknown brands than men: The effects of gender and known versus unknown brands on purchase intentions," Journal of Retailing and Consumer Services, Elsevier, vol. 58(C).
    7. Shih-Ping Jeng, 2008. "Effects of corporate reputations, relationships and competing suppliers' marketing programmes on customers' cross-buying intentions," The Service Industries Journal, Taylor & Francis Journals, vol. 28(1), pages 15-26, January.
    8. Mo Ran Yi, 2023. "Corporate Reputation and Users’ Behavioral Intentions: Is Reputation the Master Key That Moves Consumers?," SAGE Open, , vol. 13(1), pages 21582440231, February.
    9. Elgaaied-Gambier, Leila & Monnot, Elisa & Reniou, Fanny, 2018. "Using descriptive norm appeals effectively to promote green behavior," Journal of Business Research, Elsevier, vol. 82(C), pages 179-191.
    10. Qianjin Zong & Yafen Xie & Jiechun Liang, 2020. "Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 607-623, October.
    11. Przemys?aw Jeziorski & Sridhar Moorthy, 2018. "Advertiser Prominence Effects in Search Advertising," Management Science, INFORMS, vol. 64(3), pages 1365-1383, March.
    12. Harald Van Heerde & Kristiaan Helsen & Marnik G. Dekimpe, 2007. "The Impact of a Product-Harm Crisis on Marketing Effectiveness," Marketing Science, INFORMS, vol. 26(2), pages 230-245, 03-04.
    13. Cheng Yi & Zhenhui (Jack) Jiang & Xiuping Li & Xianghua Lu, 2019. "Leveraging User-Generated Content for Product Promotion: The Effects of Firm-Highlighted Reviews," Information Systems Research, INFORMS, vol. 30(3), pages 711-725, September.
    14. Hamid R. Jamali & Majid Nabavi & Saeid Asadi, 2018. "How video articles are cited, the case of JoVE: Journal of Visualized Experiments," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(3), pages 1821-1839, December.
    15. John P A Ioannidis & Kevin Boyack & Paul F Wouters, 2016. "Citation Metrics: A Primer on How (Not) to Normalize," PLOS Biology, Public Library of Science, vol. 14(9), pages 1-7, September.
    16. Tahamtan, Iman & Bornmann, Lutz, 2018. "Core elements in the process of citing publications: Conceptual overview of the literature," Journal of Informetrics, Elsevier, vol. 12(1), pages 203-216.
    17. CholMyong Pak & Guang Yu & Weibin Wang, 2018. "A study on the citation situation within the citing paper: citation distribution of references according to mention frequency," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 905-918, March.
    18. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    19. Newell, Stephen J. & Goldsmith, Ronald E., 2001. "The development of a scale to measure perceived corporate credibility," Journal of Business Research, Elsevier, vol. 52(3), pages 235-247, June.
    20. Yuhei Inoue & Aubrey Kent, 2014. "A Conceptual Framework for Understanding the Effects of Corporate Social Marketing on Consumer Behavior," Journal of Business Ethics, Springer, vol. 121(4), pages 621-633, June.
    21. Arthur Lackner & Said Fathalla & Mojtaba Nayyeri & Andreas Behrend & Rainer Manthey & Sören Auer & Jens Lehmann & Sahar Vahdati, 2021. "Analysing the evolution of computer science events leveraging a scholarly knowledge graph: a scientometrics study of top-ranked events in the past decade," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8129-8151, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03758-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.