IDEAS home Printed from https://ideas.repec.org/a/eee/jomega/v70y2017icp135-147.html
   My bibliography  Save this article

A data envelopment analysis approach for ranking journals

Author

Listed:
  • Rosenthal, Edward C.
  • Weiss, Howard J.

Abstract

We create a data envelopment analysis (DEA) model to rank business journals, using data from the Thomson Reuters Journal Citation Reports® (JCR). As opposed to previous models that ranked journals in only one field and mostly relied on survey data, this model is used to rank 358 business journals from five different JCR categories according to such citation-based factors as the number of articles, the number of citations, impact factor, five-year impact factor, immediacy index, eigenfactor score, and article influence score. We compute relative efficiencies of the journals and thereby create plausible journal rankings that largely, but not completely, corroborate three widely used business publication journal ranking lists. In addition, we show how the different characteristics of the JCR data impact the DEA ranking model. Finally, we identify journals that are not on the business publication lists but consistently perform very well relative to those benchmark journals, and should possibly be included in the business publication ranking lists. We also identify journals whose inclusion in widely used business publication rankings cannot be justified by our methodology.

Suggested Citation

  • Rosenthal, Edward C. & Weiss, Howard J., 2017. "A data envelopment analysis approach for ranking journals," Omega, Elsevier, vol. 70(C), pages 135-147.
    • Handle: RePEc:eee:jomega:v:70:y:2017:i:c:p:135-147
    DOI: 10.1016/j.omega.2016.09.006
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0305048316306272
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.omega.2016.09.006?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. S Lozano & G Villa & F Guerrero & P Cortés, 2002. "Measuring the performance of nations at the Summer Olympics using data envelopment analysis," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 53(5), pages 501-511, May.
    2. Wolfgang Glänzel & Henk F. Moed, 2002. "Journal impact measures in bibliometric research," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 171-193, February.
    3. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(2), pages 370-379, June.
    4. Charnes, A. & Cooper, W. W. & Rhodes, E., 1978. "Measuring the efficiency of decision making units," European Journal of Operational Research, Elsevier, vol. 2(6), pages 429-444, November.
    5. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.
    6. Laura Vana & Ronald Hochreiter & Kurt Hornik, 2016. "Computing a journal meta-ranking using paired comparisons and adaptive lasso estimators," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 229-251, January.
    7. Josephine E. Olson, 2005. "Top-25-Business-School Professors Rate Journals in Operations Management and Related Fields," Interfaces, INFORMS, vol. 35(4), pages 323-338, August.
    8. Lins, Marcos P. Estellita & Gomes, Eliane G. & Soares de Mello, Joao Carlos C. B. & Soares de Mello, Adelino Jose R., 2003. "Olympic ranking based on a zero sum gains DEA model," European Journal of Operational Research, Elsevier, vol. 148(2), pages 312-322, July.
    9. Henk F. Moed, 2011. "The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(1), pages 211-213, January.
    10. Michael Dummett, 1998. "The Borda count and agenda manipulation," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 15(2), pages 289-296.
    11. Michael F. Gorman & John J. Kanet, 2005. "Evaluating Operations Management--Related Journals via the Author Affiliation Index," Manufacturing & Service Operations Management, INFORMS, vol. 7(1), pages 3-19, February.
    12. Llamazares, Bonifacio & Pea, Teresa, 2009. "Preference aggregation and DEA: An analysis of the methods proposed to discriminate efficient candidates," European Journal of Operational Research, Elsevier, vol. 197(2), pages 714-721, September.
    13. Murthi, B. P. S. & Choi, Yoon K. & Desai, Preyas, 1997. "Efficiency of mutual funds and portfolio performance measurement: A non-parametric approach," European Journal of Operational Research, Elsevier, vol. 98(2), pages 408-418, April.
    14. Henk F. Moed, 2011. "The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(1), pages 211-213, January.
    15. Wade D. Cook & Tal Raviv & Alan J. Richardson, 2010. "Aggregating Incomplete Lists of Journal Rankings: An Application to Academic Accounting Journals," Accounting Perspectives, John Wiley & Sons, vol. 9(3), pages 217-235, September.
    16. Martin Eling, 2006. "Performance measurement of hedge funds using data envelopment analysis," Financial Markets and Portfolio Management, Springer;Swiss Society for Financial Market Research, vol. 20(4), pages 442-471, December.
    17. S. Alonso & F. J. Cabrerizo & E. Herrera-Viedma & F. Herrera, 2010. "hg-index: a new index to characterize the scientific output of researchers based on the h- and g-indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 82(2), pages 391-400, February.
    18. Stefano Benati & Silvana Stefani, 2011. "The Academic Journal Ranking Problem: A Fuzzy-Clustering Approach," Journal of Classification, Springer;The Classification Society, vol. 28(1), pages 7-20, April.
    19. Gianfranco Ennas & Battista Biggio & Maria Chiara Di Guardo, 2015. "Data-driven journal meta-ranking in business and management," Scientometrics, Springer;Akadémiai Kiadó, vol. 105(3), pages 1911-1929, December.
    20. Maria Bordons & M. T. Fernández & Isabel Gómez, 2002. "Advantages and limitations in the use of impact factor measures for the assessment of research performance," Scientometrics, Springer;Akadémiai Kiadó, vol. 53(2), pages 195-206, February.
    21. Zhou Xu & Brenda Cheang & Andrew Lim & Qi Wen, 2011. "Evaluating OR/MS Journals via PageRank," Interfaces, INFORMS, vol. 41(4), pages 375-388, August.
    22. Chen, Carl R. & Huang, Ying, 2007. "Author Affiliation Index, finance journal ranking, and the pattern of authorship," Journal of Corporate Finance, Elsevier, vol. 13(5), pages 1008-1026, December.
    23. Meredith, Jack R. & Steward, Michelle D. & Lewis, Bruce R., 2011. "Knowledge dissemination in operations management: Published perceptions versus academic reality," Omega, Elsevier, vol. 39(4), pages 435-446, August.
    24. Journal of Economics Bibliography, 2016. "New Economics Journals," Journal of Economics Bibliography, KSP Journals, vol. 3(1), pages 157-171, March.
    25. Satterthwaite, Mark Allen, 1975. "Strategy-proofness and Arrow's conditions: Existence and correspondence theorems for voting procedures and social welfare functions," Journal of Economic Theory, Elsevier, vol. 10(2), pages 187-217, April.
    26. Alexander, John C & Mabry, Rodney H, 1994. "Relative Significance of Journals, Authors, and Articles Cited in Financial Research," Journal of Finance, American Finance Association, vol. 49(2), pages 697-712, June.
    27. Llamazares, Bonifacio & Peña, Teresa, 2013. "Aggregating preferences rankings with variable weights," European Journal of Operational Research, Elsevier, vol. 230(2), pages 348-355.
    28. Moussa, Salim & Touzani, Mourad, 2010. "Ranking marketing journals using the Google Scholar-based hg-index," Journal of Informetrics, Elsevier, vol. 4(1), pages 107-117.
    29. Keith D. Brouthers & Ram Mudambi & David M. Reeb, 2012. "The blockbuster hypothesis: influencing the boundaries of knowledge," Scientometrics, Springer;Akadémiai Kiadó, vol. 90(3), pages 959-982, March.
    30. Wade D. Cook & Moshe Kress, 1990. "A Data Envelopment Model for Aggregating Preference Rankings," Management Science, INFORMS, vol. 36(11), pages 1302-1310, November.
    31. Anne‐Wil Harzing & Ron van der Wal, 2009. "A Google Scholar h‐index for journals: An alternative metric to measure journal impact in economics and business," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 60(1), pages 41-46, January.
    32. Frank L DuBois & David Reeb, 2000. "Ranking the International Business Journals," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 31(4), pages 689-704, December.
    33. Loet Leydesdorff, 2008. "Caveats for the use of citation indicators in research and journal evaluations," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 59(2), pages 278-287, January.
    34. Timothy J. Coelli & D.S. Prasada Rao & Christopher J. O’Donnell & George E. Battese, 2005. "An Introduction to Efficiency and Productivity Analysis," Springer Books, Springer, edition 0, number 978-0-387-25895-9, September.
    35. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    36. R. D. Banker & A. Charnes & W. W. Cooper, 1984. "Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis," Management Science, INFORMS, vol. 30(9), pages 1078-1092, September.
    37. Sarkis, Joseph, 2000. "A comparative analysis of DEA as a discrete alternative multiple criteria decision tool," European Journal of Operational Research, Elsevier, vol. 123(3), pages 543-557, June.
    38. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2015. "Towards a consolidation of worldwide journal rankings – A classification using random forests and aggregate rating via data envelopment analysis," Omega, Elsevier, vol. 51(C), pages 11-23.
    39. Daniel Tyteca, 1997. "Linear Programming Models for the Measurement of Environmental Performance of Firms—Concepts and Empirical Results," Journal of Productivity Analysis, Springer, vol. 8(2), pages 183-197, May.
    40. Bonner, Sarah E. & Hesford, James W. & Van der Stede, Wim A. & Young, S. Mark, 2006. "The most influential journals in academic accounting," Accounting, Organizations and Society, Elsevier, vol. 31(7), pages 663-685, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Aparicio, Juan & Zofío, José L., 2021. "Economic cross-efficiency," Omega, Elsevier, vol. 100(C).
      • Aparicio, J. & Zofío, J.L., 2019. "Economic Cross-Efficiency," ERIM Report Series Research in Management ERS-2019-001-LIS, Erasmus Research Institute of Management (ERIM), ERIM is the joint research institute of the Rotterdam School of Management, Erasmus University and the Erasmus School of Economics (ESE) at Erasmus University Rotterdam.
    2. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chen, Kun & Ren, Xian-tong & Yang, Guo-liang, 2021. "A novel approach for assessing academic journals: Application of integer DEA model for management science and operations research field," Journal of Informetrics, Elsevier, vol. 15(3).
    2. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2016. "Revisiting the standing of international business journals in the competitive landscape," Journal of World Business, Elsevier, vol. 51(4), pages 487-498.
    3. Tüselmann, Heinz & Sinkovics, Rudolf R. & Pishchulov, Grigory, 2015. "Towards a consolidation of worldwide journal rankings – A classification using random forests and aggregate rating via data envelopment analysis," Omega, Elsevier, vol. 51(C), pages 11-23.
    4. Laura Vana & Ronald Hochreiter & Kurt Hornik, 2016. "Computing a journal meta-ranking using paired comparisons and adaptive lasso estimators," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(1), pages 229-251, January.
    5. Brenda Cheang & Samuel Kai Wah Chu & Chongshou Li & Andrew Lim, 2014. "OR/MS journals evaluation based on a refined PageRank method: an updated and more comprehensive review," Scientometrics, Springer;Akadémiai Kiadó, vol. 100(2), pages 339-361, August.
    6. Brenda Cheang & Chongshou Li & Andrew Lim & Zhenzhen Zhang, 2015. "Identifying patterns and structural influences in the scientific communication of business knowledge," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(1), pages 159-189, April.
    7. Sahoo, Biresh K. & Singh, Ramadhar & Mishra, Bineet & Sankaran, Krithiga, 2017. "Research productivity in management schools of India during 1968-2015: A directional benefit-of-doubt model analysis," Omega, Elsevier, vol. 66(PA), pages 118-139.
    8. Pishchulov, Grigory & Trautrims, Alexander & Chesney, Thomas & Gold, Stefan & Schwab, Leila, 2019. "The Voting Analytic Hierarchy Process revisited: A revised method with application to sustainable supplier selection," International Journal of Production Economics, Elsevier, vol. 211(C), pages 166-179.
    9. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    10. Mingkun Wei, 2020. "Research on impact evaluation of open access journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 122(2), pages 1027-1049, February.
    11. Johan Lyhagen & Per Ahlgren, 2020. "Uncertainty and the ranking of economics journals," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2545-2560, December.
    12. Domingo Docampo & Vicente Safón, 2021. "Journal ratings: a paper affiliation methodology," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 8063-8090, September.
    13. Yang, Min & Li, Yongjun & Chen, Ya & Liang, Liang, 2014. "An equilibrium efficiency frontier data envelopment analysis approach for evaluating decision-making units with fixed-sum outputs," European Journal of Operational Research, Elsevier, vol. 239(2), pages 479-489.
    14. Tarnaud, Albane Christine & Leleu, Hervé, 2018. "Portfolio analysis with DEA: Prior to choosing a model," Omega, Elsevier, vol. 75(C), pages 57-76.
    15. Alexandre Marinho & Simone de Souza Cardoso & Vivian Vicente de Almeida, 2009. "Avaliação da Eficiência Técnica dos Países nos Jogos Olímpicos de Pequim – 2008," Discussion Papers 1394, Instituto de Pesquisa Econômica Aplicada - IPEA.
    16. Yang, Min & Li, Yong Jun & Liang, Liang, 2015. "A generalized equilibrium efficient frontier data envelopment analysis approach for evaluating DMUs with fixed-sum outputs," European Journal of Operational Research, Elsevier, vol. 246(1), pages 209-217.
    17. Petridis, Konstantinos & Malesios, Chrisovalantis & Arabatzis, Garyfallos & Thanassoulis, Emmanuel, 2013. "Efficiency analysis of forestry journals: Suggestions for improving journals’ quality," Journal of Informetrics, Elsevier, vol. 7(2), pages 505-521.
    18. De Witte, Kristof & Schiltz, Fritz, 2018. "Measuring and explaining organizational effectiveness of school districts: Evidence from a robust and conditional Benefit-of-the-Doubt approach," European Journal of Operational Research, Elsevier, vol. 267(3), pages 1172-1181.
    19. Ioannis E. Tsolas, 2020. "The Determinants of the Performance of Precious Metal Mutual Funds," JRFM, MDPI, vol. 13(11), pages 1-10, November.
    20. Mingers, John & Yang, Liying, 2017. "Evaluating journal quality: A review of journal citation indicators and ranking in business and management," European Journal of Operational Research, Elsevier, vol. 257(1), pages 323-337.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:jomega:v:70:y:2017:i:c:p:135-147. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/375/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.