IDEAS home Printed from https://ideas.repec.org/a/eee/infome/v15y2021i3s175115772100047x.html
   My bibliography  Save this article

A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field

Author

Listed:
  • Chen, Kun
  • Ren, Xian-tong
  • Yang, Guo-liang

Abstract

This paper aims to investigate academic journal evaluation through the integer DEA (Data envelopment analysis) method. By including expert-based and citation-based journal ranking indicators such as the ABS Ranking, UT/DALLAS list, IF (impact factor), H-index and Eigenfactor score, and considering the first two sequencing expert-based journal ranking indicators and H-index are integer data, this study constructed an integer DEA model to evaluate academic journals in the field of management science and operations research. An indicator is also introduced as input to punish mismanagement of journals by judging whether the self-citation rate is too high. Based on the calculation of our model, four main conclusions are reached. First, the integer DEA model could be a useful method to amend the deviation caused by ignoring integer data and integrate indicators based on expert and citations. Second, by introducing expert-based journal ranking indicators, better results can be obtained when identifying the most important journals; and by introducing citation-based journal ranking indicators, the subjective bias caused by experts’ difficulty in familiarizing with all journals can also be avoid to some extent. Third, by introducing the Cost indicator measured by excessive self-citation rate, we identify the cost of mismanagement by journal editorial board and punish them in DEA score. Last but not the least, we proposed that CRS is more suitable in academic journal evaluation than VRS because there is no effect of increasing or decreasing returns to scale in the process of journal publication and citation. In summary, our integer DEA model is feasible and could be used by universities to form suitable journal ranking list.

Suggested Citation

  • Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    • Handle: RePEc:eee:infome:v:15:y:2021:i:3:s175115772100047x
    DOI: 10.1016/j.joi.2021.101176
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S175115772100047X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.joi.2021.101176?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gregoriou, Greg N. & Sedzro, Komlan & Zhu, Joe, 2005. "Hedge fund performance appraisal using data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 164(2), pages 555-571, July.
    2. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(2), pages 370-379, June.
    3. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    4. Loet Leydesdorff, 2012. "Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers," Scientometrics, Springer;Akadémiai Kiadó, vol. 92(2), pages 355-365, August.
    5. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    6. Henk F. Moed, 2011. "The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(1), pages 211-213, January.
    7. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    8. Joe Zhu, 2014. "Data Envelopment Analysis," International Series in Operations Research & Management Science, in: Quantitative Models for Performance Evaluation and Benchmarking, edition 3, chapter 1, pages 1-9, Springer.
    9. Henk F. Moed, 2011. "The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(1), pages 211-213, January.
    10. Simar, Leopold & Wilson, Paul W., 2002. "Non-parametric tests of returns to scale," European Journal of Operational Research, Elsevier, vol. 139(1), pages 115-132, May.
    11. Amado, Carla A.F. & Santos, Sérgio P. & Sequeira, João F.C., 2013. "Using Data Envelopment Analysis to support the design of process improvement interventions in electricity distribution," European Journal of Operational Research, Elsevier, vol. 228(1), pages 226-235.
    12. Yang, Guo-liang & Yang, Jian-bo & Liu, Wen-bin & Li, Xiao-xuan, 2013. "Cross-efficiency aggregation in DEA models using the evidential-reasoning approach," European Journal of Operational Research, Elsevier, vol. 231(2), pages 393-404.
    13. James H. Fowler & Dag W. Aksnes, 2007. "Does self-citation pay?," Scientometrics, Springer;Akadémiai Kiadó, vol. 72(3), pages 427-437, September.
    14. Gianfranco Ennas & Battista Biggio & Maria Chiara Di Guardo, 2015. "Data-driven journal meta-ranking in business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1911-1929, December.
    15. Maria Bordons & M. T. Fernández & Isabel Gómez, 2002. "Advantages and limitations in the use of impact factor measures for the assessment of research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 195-206, February.
    16. Waltman, Ludo, 2016. "A review of the literature on citation impact indicators," Journal of Informetrics, Elsevier, vol. 10(2), pages 365-391.
    17. Wilhite, Allen & Fong, Eric A. & Wilhite, Seth, 2019. "The influence of editorial decisions and the academic network on self-citations and journal impact factors," Research Policy, Elsevier, vol. 48(6), pages 1513-1522.
    18. Ortega, Francisco J. & Gavilan, Jose M., 2013. "The measurement of production efficiency in scientific journals through stochastic frontier analysis models: Application to quantitative economics journals," Journal of Informetrics, Elsevier, vol. 7(4), pages 959-965.
    19. Kuosmanen, Timo & Matin, Reza Kazemi, 2009. "Theory of integer-valued data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 192(2), pages 658-667, January.
    20. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(1), pages 157-171, March.
    21. Frandsen, Tove Faber, 2007. "Journal self-citations—Analysing the JIF mechanism," Journal of Informetrics, Elsevier, vol. 1(1), pages 47-58.
    22. Lee, Hakyeon & Shin, Juneseuk, 2014. "Measuring journal performance for multidisciplinary research: An efficiency perspective," Journal of Informetrics, Elsevier, vol. 8(1), pages 77-88.
    23. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    24. Stephen P. Harter & Thomas E. Nisonger, 1997. "ISI's impact factor as misnomer: A proposed new measure to assess journal impact," Journal of the American Society for Information Science, Association for Information Science & Technology, vol. 48(12), pages 1146-1148, December.
    25. Martin Szomszor & David A. Pendlebury & Jonathan Adams, 2020. "How much is too much? The difference between research influence and self-citation excess," Scientometrics, Springer;Akadémiai Kiadó, vol. 123(2), pages 1119-1147, May.
    26. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    27. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    28. Seiford, Lawrence M. & Zhu, Joe, 2002. "Modeling undesirable factors in efficiency evaluation," European Journal of Operational Research, Elsevier, vol. 142(1), pages 16-20, October.
    29. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2015. "Towards a consolidation of worldwide journal rankings – A classification using random forests and aggregate rating via data envelopment analysis," Omega, Elsevier, vol. 51(C), pages 11-23.
    30. Mei Xue & Patrick T. Harker, 2002. "Note: Ranking DMUs with Infeasible Super-Efficiency DEA Models," Management Science, INFORMS, vol. 48(5), pages 705-710, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    2. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    3. Liu, John S. & Lu, Louis Y.Y. & Lu, Wen-Min, 2016. "Research fronts in data envelopment analysis," Omega, Elsevier, vol. 58(C), pages 33-45.
    4. Sahoo, Biresh K. & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2017. "Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis," Omega, Elsevier, vol. 66(PA), pages 118-139.
    5. Huang, Beijia & Zhang, Long & Ma, Linmao & Bai, Wuliyasu & Ren, Jingzheng, 2021. "Multi-criteria decision analysis of China’s energy security from 2008 to 2017 based on Fuzzy BWM-DEA-AR model and Malmquist Productivity Index," Energy, Elsevier, vol. 228(C).
    6. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    7. Kaffash, Sepideh & Azizi, Roza & Huang, Ying & Zhu, Joe, 2020. "A survey of data envelopment analysis applications in the insurance industry 1993–2018," European Journal of Operational Research, Elsevier, vol. 284(3), pages 801-813.
    8. Taleb, Mushtaq & Khalid, Ruzelan & Ramli, Razamin & Ghasemi, Mohammad Reza & Ignatius, Joshua, 2022. "An integrated bi-objective data envelopment analysis model for measuring returns to scale," European Journal of Operational Research, Elsevier, vol. 296(3), pages 967-979.
    9. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    10. Jie Wu & Zhixiang Zhou, 2015. "A mixed-objective integer DEA model," Annals of Operations Research, Springer, vol. 228(1), pages 81-95, May.
    11. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2016. "Revisiting the standing of international business journals in the competitive landscape," Journal of World Business, Elsevier, vol. 51(4), pages 487-498.
    12. Andreas Eder & Bernhard Mahlberg & Bernhard Stürmer, 2021. "Measuring and explaining productivity growth of renewable energy producers: An empirical study of Austrian biogas plants," Empirica, Springer;Austrian Institute for Economic Research;Austrian Economic Association, vol. 48(1), pages 37-63, February.
    13. Halkos, George & Petrou, Kleoniki Natalia, 2018. "Assessment of national waste generation in EU Member States’ efficiency," MPRA Paper 84590, University Library of Munich, Germany.
    14. Martina Halaskova & Beata Gavurova & Kristina Kocisova, 2020. "Research and Development Efficiency in Public and Private Sectors: An Empirical Analysis of EU Countries by Using DEA Methodology," Sustainability, MDPI, vol. 12(17), pages 1-22, August.
    15. Benita, Francisco & Urzúa, Carlos M., 2018. "Efficient creativity in Mexican metropolitan areas," Economic Modelling, Elsevier, vol. 71(C), pages 25-33.
    16. Yang, Guoliang & Ahlgren, Per & Yang, Liying & Rousseau, Ronald & Ding, Jielan, 2016. "Using multi-level frontiers in DEA models to grade countries/territories," Journal of Informetrics, Elsevier, vol. 10(1), pages 238-253.
    17. Ke Wang & Yi-Ming Wei & Zhimin Huang, 2017. "Environmental efficiency and abatement efficiency measurements of China¡¯s thermal power industry: A data envelopment analysis based materials balance approach," CEEP-BIT Working Papers 108, Center for Energy and Environmental Policy Research (CEEP), Beijing Institute of Technology.
    18. Sahoo, Biresh & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2015. "Research Productivity in Management Schools of India: A Directional Benefit-of-Doubt Model Analysis," MPRA Paper 67046, University Library of Munich, Germany.
    19. Halkos, George & Tzeremes, Nickolaos, 2011. "Does the Kyoto Protocol Agreement matters? An environmental efficiency analysis," MPRA Paper 30652, University Library of Munich, Germany.
    20. Johan Lyhagen & Per Ahlgren, 2020. "Uncertainty and the ranking of economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2545-2560, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:infome:v:15:y:2021:i:3:s175115772100047x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/joi .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.