IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v130y2025i1d10.1007_s11192-024-05205-x.html
   My bibliography  Save this article

Analysis of citation impact of ORMS journals by DEA models

Author

Listed:
  • Josef Jablonský

    (Prague University of Economics and Business)

Abstract

Performance and citation impact of scientific journals are measured by traditional metrics such as impact factor, article influence score, journal citation indicator, and others. While the impact factor is based on the total number of citations and does not reflect the quality of journals cited, the article influence score considers the past importance of the citing journals. This paper aims the analyze the possibility of measuring the performance of journals by data envelopment analysis (DEA) models and propose a new DEA based citation performance metrics for ranking of a set of journals. We applied traditional radial and slack-based measure DEA models with weight restrictions where the outputs of the models are the citation counts from Q1 to Q4 categories, and other journals. This basic model is extended by considering the impact factor of the journals from the previous year as one of the inputs of the model. The results of the study are illustrated in the set of 80 journals from the Web of Science category Operations Research and Management Science (ORMS). The dataset for the study was obtained from the Journal Citation Reports in the period from 2017 until 2022. The relative efficiency scores and the ranking of journals obtained by the models are compared with traditional metrics, Academic Journal Guide 2021 classification and the results of the study (Chen et al., Journal of Informetrics, 15(3), 2021) that applies DEA models for the classification of ORMS journals in the same year as our study.

Suggested Citation

  • Josef Jablonský, 2025. "Analysis of citation impact of ORMS journals by DEA models," Scientometrics, Springer;Akadémiai Kiadó, vol. 130(1), pages 343-365, January.
    • Handle: RePEc:spr:scient:v:130:y:2025:i:1:d:10.1007_s11192-024-05205-x
    DOI: 10.1007/s11192-024-05205-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-024-05205-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-024-05205-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Per Andersen & Niels Christian Petersen, 1993. "A Procedure for Ranking Efficient Units in Data Envelopment Analysis," Management Science, INFORMS, vol. 39(10), pages 1261-1264, October.
    2. F. Hosseinzadeh Lotfi & G. R. Jahanshahloo & M. Khodabakhshi & M. Rostamy-Malkhlifeh & Z. Moghaddas & M. Vaez-Ghasemi, 2013. "A Review of Ranking Models in Data Envelopment Analysis," Journal of Applied Mathematics, Hindawi, vol. 2013, pages 1-20, July.
    3. Torres-Salinas, Daniel & Valderrama-Baca, Pilar & Arroyo-Machado, Wenceslao, 2022. "Is there a need for a new journal metric? Correlations between JCR Impact Factor metrics and the Journal Citation Indicator—JCI," Journal of Informetrics, Elsevier, vol. 16(3).
    4. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    5. Josef Jablonsky, 2018. "Ranking of countries in sporting events using two-stage data envelopment analysis models: a case of Summer Olympic Games 2016," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 26(4), pages 951-966, December.
    6. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    7. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    8. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    9. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    10. S. Alonso & F. J. Cabrerizo & E. Herrera-Viedma & F. Herrera, 2010. "hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 391-400, February.
    11. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    12. Josef Jablonsky, 2012. "Multicriteria approaches for ranking of efficient units in DEA models," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 20(3), pages 435-449, September.
    13. F. Hosseinzadeh Lotfi & G. R. Jahanshahloo & M. Khodabakhshi & M. Rostamy-Malkhlifeh & Z. Moghaddas & M. Vaez-Ghasemi, 2013. "A Review of Ranking Models in Data Envelopment Analysis," Journal of Applied Mathematics, John Wiley & Sons, vol. 2013(1).
    14. Anne‐Wil Harzing & Ron van der Wal, 2009. "A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 41-46, January.
    15. Lee, Hakyeon & Shin, Juneseuk, 2014. "Measuring journal performance for multidisciplinary research: An efficiency perspective," Journal of Informetrics, Elsevier, vol. 8(1), pages 77-88.
    16. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    17. Tone, Kaoru, 2002. "A slacks-based measure of super-efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 143(1), pages 32-41, November.
    18. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    19. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    20. Tibor Braun & Wolfgang Glänzel & András Schubert, 2006. "A Hirsch-type index for journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 169-173, October.
    21. Tone, Kaoru, 2001. "A slacks-based measure of efficiency in data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 130(3), pages 498-509, May.
    22. Mojtaba Ghiyasi, 2019. "Full ranking of efficient and inefficient DMUs with the same measure of efficiency in DEA," International Journal of Business Performance and Supply Chain Modelling, Inderscience Enterprises Ltd, vol. 10(3), pages 236-252.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    2. Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    3. Vicente J. Bolós & Rafael Benítez & Vicente Coll-Serrano, 2023. "Continuous models combining slacks-based measures of efficiency and super-efficiency," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 31(2), pages 363-391, June.
    4. Marcel Clermont & Julia Schaefer, 2019. "Identification of Outliers in Data Envelopment Analysis," Schmalenbach Business Review, Springer;Schmalenbach-Gesellschaft, vol. 71(4), pages 475-496, October.
    5. Yung-ho Chiu & Chin-wei Huang & Chung-te Ting, 2012. "A non-radial measure of different systems for Taiwanese tourist hotels’ efficiency assessment," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 20(1), pages 45-63, March.
    6. Tran, Trung Hieu & Mao, Yong & Nathanail, Paul & Siebers, Peer-Olaf & Robinson, Darren, 2019. "Integrating slacks-based measure of efficiency and super-efficiency in data envelopment analysis," Omega, Elsevier, vol. 85(C), pages 156-165.
    7. Du, Juan & Chen, Chien-Ming & Chen, Yao & Cook, Wade D. & Zhu, Joe, 2012. "Additive super-efficiency in integer-valued data envelopment analysis," European Journal of Operational Research, Elsevier, vol. 218(1), pages 186-192.
    8. Ruomeng Zhou & Yunsheng Zhang, 2023. "Measurement of Urban Green Total Factor Productivity and Analysis of Its Temporal and Spatial Evolution in China," Sustainability, MDPI, vol. 15(12), pages 1-32, June.
    9. Victoria Wojcik & Harald Dyckhoff & Marcel Clermont, 2019. "Is data envelopment analysis a suitable tool for performance measurement and benchmarking in non-production contexts?," Business Research, Springer;German Academic Association for Business Research, vol. 12(2), pages 559-595, December.
    10. Vladimír Holý & Karel Šafr, 2018. "Are economically advanced countries more efficient in basic and applied research?," Central European Journal of Operations Research, Springer;Slovak Society for Operations Research;Hungarian Operational Research Society;Czech Society for Operations Research;Österr. Gesellschaft für Operations Research (ÖGOR);Slovenian Society Informatika - Section for Operational Research;Croatian Operational Research Society, vol. 26(4), pages 933-950, December.
    11. Necmi Kemal Avkiran, 2017. "An illustration of multiple-stakeholder perspective using a survey across Australia, China and Japan," Annals of Operations Research, Springer, vol. 248(1), pages 93-121, January.
    12. Fang, Tao & Fang, Debin & Yu, Bolin, 2022. "Carbon emission efficiency of thermal power generation in China: Empirical evidence from the micro-perspective of power plants," Energy Policy, Elsevier, vol. 165(C).
    13. Rashidi, Kamran & Cullinane, Kevin, 2019. "Evaluating the sustainability of national logistics performance using Data Envelopment Analysis," Transport Policy, Elsevier, vol. 74(C), pages 35-46.
    14. Mushtaq Taleb & Ruzelan Khalid & Ali Emrouznejad & Razamin Ramli, 2023. "Environmental efficiency under weak disposability: an improved super efficiency data envelopment analysis model with application for assessment of port operations considering NetZero," Environment, Development and Sustainability: A Multidisciplinary Approach to the Theory and Practice of Sustainable Development, Springer, vol. 25(7), pages 6627-6656, July.
    15. Chiu, Yung-Ho & Chen, Yu-Chuan, 2009. "The analysis of Taiwanese bank efficiency: Incorporating both external environment risk and internal risk," Economic Modelling, Elsevier, vol. 26(2), pages 456-463, March.
    16. Jamal Ouenniche & Kaoru Tone, 2017. "An out-of-sample evaluation framework for DEA with application in bankruptcy prediction," Annals of Operations Research, Springer, vol. 254(1), pages 235-250, July.
    17. Lihong Zhao & Yuge Zhang & Fuzhu Li & Yuki Yi Gong & Hideyuki Hao Sun & Sing Lui So & Zehua Chen, 2025. "Spatial Network Evolution of Water Resources Green Efficiency in Yellow River Basin Urban Agglomerations," Sustainability, MDPI, vol. 17(3), pages 1-31, January.
    18. Avkiran, Necmi K., 2011. "Association of DEA super-efficiency estimates with financial ratios: Investigating the case for Chinese banks," Omega, Elsevier, vol. 39(3), pages 323-334, June.
    19. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    20. Yung-Ho Chiu & Yu-Chuan Chen & Xue-Jie Bai, 2011. "Efficiency and risk in Taiwan banking: SBM super-DEA estimation," Applied Economics, Taylor & Francis Journals, vol. 43(5), pages 587-602.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:130:y:2025:i:1:d:10.1007_s11192-024-05205-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.