IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v126y2021i1d10.1007_s11192-020-03758-1.html
   My bibliography  Save this article

A comprehensive quality assessment framework for scientific events

Author

Listed:
  • Sahar Vahdati

    (University of Oxford
    Institute for Applied Informatics (InfAI))

  • Said Fathalla

    (University of Bonn
    University of Alexandria)

  • Christoph Lange

    (RWTH Aachen University
    Fraunhofer FIT)

  • Andreas Behrend

    (Institute for Telecommunications (INT))

  • Aysegul Say

    (University of Bonn)

  • Zeynep Say

    (University of Bonn)

  • Sören Auer

    (Leibniz University of Hannover
    TIB Leibniz Information Centre for Science and Technology)

Abstract

Systematic assessment of scientific events has become increasingly important for research communities. A range of metrics (e.g., citations, h-index) have been developed by different research communities to make such assessments effectual. However, most of the metrics for assessing the quality of less formal publication venues and events have not yet deeply investigated. It is also rather challenging to develop respective metrics because each research community has its own formal and informal rules of communication and quality standards. In this article, we develop a comprehensive framework of assessment metrics for evaluating scientific events and involved stakeholders. The resulting quality metrics are determined with respect to three general categories—events, persons, and bibliometrics. Our assessment methodology is empirically applied to several series of computer science events, such as conferences and workshops, using publicly available data for determining quality metrics. We show that the metrics’ values coincide with the intuitive agreement of the community on its “top conferences”. Our results demonstrate that highly-ranked events share similar profiles, including the provision of outstanding reviews, visiting diverse locations, having reputed people involved, and renowned sponsors.

Suggested Citation

  • Sahar Vahdati & Said Fathalla & Christoph Lange & Andreas Behrend & Aysegul Say & Zeynep Say & Sören Auer, 2021. "A comprehensive quality assessment framework for scientific events," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(1), pages 641-682, January.
    • Handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03758-1
    DOI: 10.1007/s11192-020-03758-1
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-020-03758-1
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-020-03758-1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Lutz Bornmann & Markus Wolf & Hans-Dieter Daniel, 2012. "Closed versus open reviewing of journal manuscripts: how far do comments differ in language use?," Scientometrics, Springer;Akadémiai Kiadó, vol. 91(3), pages 843-856, June.
    2. Said Fathalla & Sahar Vahdati & Christoph Lange & Sören Auer, 2020. "Scholarly event characteristics in four fields of science: a metrics-based analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 677-705, May.
    3. Goldberg, Marvin E & Hartwick, Jon, 1990. "The Effects of Advertiser Reputation and Extremity of Advertising Claim on Advertising Effectiveness," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 17(2), pages 172-179, September.
    4. Hu, Zhigang & Chen, Chaomei & Liu, Zeyuan, 2013. "Where are citations located in the body of scientific articles? A study of the distributions of citation locations," Journal of Informetrics, Elsevier, vol. 7(4), pages 887-896.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yaxue Ma & Zhichao Ba & Yuxiang Zhao & Jin Mao & Gang Li, 2021. "Understanding and predicting the dissemination of scientific papers on social media: a two-step simultaneous equation modeling–artificial neural network approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(8), pages 7051-7085, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hren, Darko & Pina, David G. & Norman, Christopher R. & Marušić, Ana, 2022. "What makes or breaks competitive research proposals? A mixed-methods analysis of research grant evaluation reports," Journal of Informetrics, Elsevier, vol. 16(2).
    2. Sascha Raithel & Manfred Schwaiger, 2015. "The effects of corporate reputation perceptions of the general public on shareholder value," Strategic Management Journal, Wiley Blackwell, vol. 36(6), pages 945-956, June.
    3. repec:dau:papers:123456789/5853 is not listed on IDEAS
    4. Linder, Christian & Seidenstricker, Sven, 2018. "How does a component from a supplier with high reputation for product innovation improve the perception of a final offering? A process perspective," European Management Journal, Elsevier, vol. 36(2), pages 288-299.
    5. Saeed-Ul Hassan & Iqra Safder & Anam Akram & Faisal Kamiran, 2018. "A novel machine-learning approach to measuring scientific knowledge flows using citation context analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 116(2), pages 973-996, August.
    6. Kaveh Moghaddam & Thomas Weber & Pouya Seifzadeh & Sara Azarpanah, 2021. "Internal Reputation of the Firm: CEO Retention and Firm Market Performance," Corporate Reputation Review, Palgrave Macmillan, vol. 24(4), pages 205-221, November.
    7. Naif Radi Aljohani & Ayman Fayoumi & Saeed-Ul Hassan, 2021. "An in-text citation classification predictive model for a scholarly search system," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(7), pages 5509-5529, July.
    8. Christian Linder & Sven Seidenstricker, 2017. "Does The Supplier Affects Consumers’ Product Performance Expectations? An Analysis About The Innovativeness Perception," International Journal of Innovation Management (ijim), World Scientific Publishing Co. Pte. Ltd., vol. 21(01), pages 1-27, January.
    9. Lariviere, Bart & Van den Poel, Dirk, 2007. "Banking behaviour after the lifecycle event of "moving in together": An exploratory study of the role of marketing investments," European Journal of Operational Research, Elsevier, vol. 183(1), pages 345-369, November.
    10. Dietmar Wolfram & Peiling Wang & Adam Hembree & Hyoungjoo Park, 2020. "Open peer review: promoting transparency in open science," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(2), pages 1033-1051, November.
    11. Karpinska-Krakowiak, Malgorzata, 2021. "Women are more likely to buy unknown brands than men: The effects of gender and known versus unknown brands on purchase intentions," Journal of Retailing and Consumer Services, Elsevier, vol. 58(C).
    12. Shih-Ping Jeng, 2008. "Effects of corporate reputations, relationships and competing suppliers' marketing programmes on customers' cross-buying intentions," The Service Industries Journal, Taylor & Francis Journals, vol. 28(1), pages 15-26, January.
    13. Chao Lu & Ying Ding & Chengzhi Zhang, 2017. "Understanding the impact change of a highly cited article: a content-based citation analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 927-945, August.
    14. Mo Ran Yi, 2023. "Corporate Reputation and Users’ Behavioral Intentions: Is Reputation the Master Key That Moves Consumers?," SAGE Open, , vol. 13(1), pages 21582440231, February.
    15. Nguyen Minh Ha & Le Quoc Phong & Tran Kiem Viet Thang & Huynh Luong Tam & Tran Hai Dang & Nguyen Thi Mai Hoa, 2023. "Mediating role of corporate reputation in the relationship between corporate social responsibility and repurchase intention: A case study of McDonald’s fast food restaurant chain," HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE - ECONOMICS AND BUSINESS ADMINISTRATION, HO CHI MINH CITY OPEN UNIVERSITY JOURNAL OF SCIENCE, HO CHI MINH CITY OPEN UNIVERSITY, vol. 13(2), pages 19-38.
    16. Elgaaied-Gambier, Leila & Monnot, Elisa & Reniou, Fanny, 2018. "Using descriptive norm appeals effectively to promote green behavior," Journal of Business Research, Elsevier, vol. 82(C), pages 179-191.
    17. Qianjin Zong & Yafen Xie & Jiechun Liang, 2020. "Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(1), pages 607-623, October.
    18. Lundmark, Erik & Milanov, Hana & Seigner, Benedikt David Christian, 2022. "Can it be measured? A quantitative assessment of critiques of the entrepreneurship literature," Journal of Business Venturing Insights, Elsevier, vol. 17(C).
    19. Liyue Chen & Jielan Ding & Vincent Larivière, 2022. "Measuring the citation context of national self‐references," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 671-686, May.
    20. Przemys?aw Jeziorski & Sridhar Moorthy, 2018. "Advertiser Prominence Effects in Search Advertising," Management Science, INFORMS, vol. 64(3), pages 1365-1383, March.
    21. Harald Van Heerde & Kristiaan Helsen & Marnik G. Dekimpe, 2007. "The Impact of a Product-Harm Crisis on Marketing Effectiveness," Marketing Science, INFORMS, vol. 26(2), pages 230-245, 03-04.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:126:y:2021:i:1:d:10.1007_s11192-020-03758-1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.