IDEAS home Printed from https://ideas.repec.org/a/bla/jinfst/v73y2022i5p655-670.html
   My bibliography  Save this article

Classification and analysis of PubPeer comments: How a web journal club is used

Author

Listed:
  • José Luis Ortega

Abstract

This study explores the use of PubPeer by the scholarly community, to understand the issues discussed in an online journal club, the disciplines most commented on, and the characteristics of the most prolific users. A sample of 39,985 posts about 24,779 publications were extracted from PubPeer in 2019 and 2020. These comments were divided into seven categories according to their degree of seriousness (Positive review, Critical review, Lack of information, Honest errors, Methodological flaws, Publishing fraud, and Manipulation). The results show that more than two‐thirds of comments are posted to report some type of misconduct, mainly about image manipulation. These comments generate most discussion and take longer to be posted. By discipline, Health Sciences and Life Sciences are the most discussed research areas. The results also reveal “super commenters,” users who access the platform to systematically review publications. The study ends by discussing how various disciplines use the site for different purposes.

Suggested Citation

  • José Luis Ortega, 2022. "Classification and analysis of PubPeer comments: How a web journal club is used," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(5), pages 655-670, May.
    • Handle: RePEc:bla:jinfst:v:73:y:2022:i:5:p:655-670
    DOI: 10.1002/asi.24568
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/asi.24568
    Download Restriction: no
    File URL: https://libkey.io/10.1002/asi.24568?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    2. S. P. J. M. Horbach & W. Halffman, 2019. "The ability of different peer review procedures to flag problematic publications," Scientometrics, Springer;Akadémiai Kiadó, vol. 118(1), pages 339-373, January.
    3. José Luis Ortega, 2017. "Are peer-review activities related to reviewer bibliometric performance? A scientometric analysis of Publons," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(2), pages 947-962, August.
    4. Ed Yong & Heidi Ledford & Richard Van Noorden, 2013. "Research ethics: 3 ways to blow the whistle," Nature, Nature, vol. 503(7477), pages 454-457, November.
    5. Elizabeth L. Pier & Markus Brauer & Amarette Filut & Anna Kaatz & Joshua Raclaw & Mitchell J. Nathan & Cecilia E. Ford & Molly Carnes, 2018. "Low agreement among reviewers evaluating the same NIH grant applications," Proceedings of the National Academy of Sciences, Proceedings of the National Academy of Sciences, vol. 115(12), pages 2952-2957, March.
    6. Charles W. Fox, 2017. "Difficulty of recruiting reviewers predicts review scores and editorial decisions at six journals of ecology and evolution," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(1), pages 465-477, October.
    7. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Agrawal, Shiv Ratan & Mittal, Divya, 2022. "Optimizing customer engagement content strategy in retail and E-tail: Available on online product review videos," Journal of Retailing and Consumer Services, Elsevier, vol. 67(C).
    2. Martin Haselmayer & Marcelo Jenny, 2017. "Sentiment analysis of political communication: combining a dictionary approach with crowdcoding," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2623-2646, November.
    3. Balázs Győrffy & Andrea Magda Nagy & Péter Herman & Ádám Török, 2018. "Factors influencing the scientific performance of Momentum grant holders: an evaluation of the first 117 research groups," Scientometrics, Springer;Akadémiai Kiadó, vol. 117(1), pages 409-426, October.
    4. Young Bin Kim & Sang Hyeok Lee & Shin Jin Kang & Myung Jin Choi & Jung Lee & Chang Hun Kim, 2015. "Virtual World Currency Value Fluctuation Prediction System Based on User Sentiment Analysis," PLOS ONE, Public Library of Science, vol. 10(8), pages 1-18, August.
    5. Singh, Amit & Jenamani, Mamata & Thakkar, Jitesh J. & Rana, Nripendra P., 2022. "Quantifying the effect of eWOM embedded consumer perceptions on sales: An integrated aspect-level sentiment analysis and panel data modeling approach," Journal of Business Research, Elsevier, vol. 138(C), pages 52-64.
    6. Ping-Yu Hsu & Hong-Tsuen Lei & Shih-Hsiang Huang & Teng Hao Liao & Yao-Chung Lo & Chin-Chun Lo, 2019. "Effects of sentiment on recommendations in social network," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(2), pages 253-262, June.
    7. Fatma Najar & Nizar Bouguila, 2023. "On smoothing and scaling language model for sentiment based information retrieval," Advances in Data Analysis and Classification, Springer;German Classification Society - Gesellschaft für Klassifikation (GfKl);Japanese Classification Society (JCS);Classification and Data Analysis Group of the Italian Statistical Society (CLADAG);International Federation of Classification Societies (IFCS), vol. 17(3), pages 725-744, September.
    8. Luis J. Callarisa-Fiol & Miguel Ángel Moliner-Tena & Rosa Rodríguez-Artola & Javier Sánchez-García, 2023. "Entrepreneurship innovation using social robots in tourism: a social listening study," Review of Managerial Science, Springer, vol. 17(8), pages 2945-2971, November.
    9. Jifeng Mu & Jonathan Z. Zhang, 2021. "Seller marketing capability, brand reputation, and consumer journeys on e-commerce platforms," Journal of the Academy of Marketing Science, Springer, vol. 49(5), pages 994-1020, September.
    10. Annamalai, Balamurugan & Yoshida, Masayuki & Varshney, Sanjeev & Pathak, Atul Arun & Venugopal, Pingali, 2021. "Social media content strategy for sport clubs to drive fan engagement," Journal of Retailing and Consumer Services, Elsevier, vol. 62(C).
    11. P. D. Mahendhiran & S. Kannimuthu, 2018. "Deep Learning Techniques for Polarity Classification in Multimodal Sentiment Analysis," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 17(03), pages 883-910, May.
    12. Simon Albrecht & Bernhard Lutz & Dirk Neumann, 2020. "The behavior of blockchain ventures on Twitter as a determinant for funding success," Electronic Markets, Springer;IIM University of St. Gallen, vol. 30(2), pages 241-257, June.
    13. Yankang Su & Zbigniew J. Kabala, 2023. "Public Perception of ChatGPT and Transfer Learning for Tweets Sentiment Analysis Using Wolfram Mathematica," Data, MDPI, vol. 8(12), pages 1-27, November.
    14. Yawar Abbas & M. S. I. Malik, 2023. "Defective products identification framework using online reviews," Electronic Commerce Research, Springer, vol. 23(2), pages 899-920, June.
    15. Li, Xinwei & Xu, Mao & Zeng, Wenjuan & Tse, Ying Kei & Chan, Hing Kai, 2023. "Exploring customer concerns on service quality under the COVID-19 crisis: A social media analytics study from the retail industry," Journal of Retailing and Consumer Services, Elsevier, vol. 70(C).
    16. Xiong, Xi & Li, Yuanyuan & Qiao, Shaojie & Han, Nan & Wu, Yue & Peng, Jing & Li, Binyong, 2018. "An emotional contagion model for heterogeneous social media with multiple behaviors," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 490(C), pages 185-202.
    17. Kristina Lerman & Luciano G. Marin & Megha Arora & Lucas H. Costa Lima & Emilio Ferrara & David Garcia, 2018. "Language, demographics, emotions, and the structure of online social networks," Journal of Computational Social Science, Springer, vol. 1(1), pages 209-225, January.
    18. Ghasem Javadi & Mohammad Taleai, 2020. "Integration of User Generated Geo-contents and Official Data to Assess Quality of Life in Intra-national Level," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 152(1), pages 205-235, November.
    19. Sykora, Martin & Elayan, Suzanne & Hodgkinson, Ian R. & Jackson, Thomas W. & West, Andrew, 2022. "The power of emotions: Leveraging user generated content for customer experience management," Journal of Business Research, Elsevier, vol. 144(C), pages 997-1006.
    20. Sunita Goel & Ozlem Uzuner, 2016. "Do Sentiments Matter in Fraud Detection? Estimating Semantic Orientation of Annual Reports," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 23(3), pages 215-239, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jinfst:v:73:y:2022:i:5:p:655-670. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.asis.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.