IDEAS home Printed from https://ideas.repec.org/a/ods/journl/v7y2018i1p24-37.html
   My bibliography  Save this article

Efficiency vs Effectiveness: Alternative Metrics for Research Performance

Author

Listed:
  • Anatoliy G. Goncharuk

    (International Humanitarian University, Ukraine)

Abstract

This paper raised the problem of adequacy of the current performance metrics for a research. The author has looked at this problem from the publisher perspective, and tried answer the question how effective the publications are and how to adequately evaluate their performance. He reviewed the advantages and disadvantages of the existing metrics and concluded that the evaluation of research performance is a complex task which cannot adequately be expressed by a single index. The author developed the metrics applying a non-parametric technique DEA that calculates the relative efficiency on a relatively small number of publicly available data. The bidimensional tool for research (publication) performance analysis – "Efficiency-Effectiveness" matrix is suggested. The practicality of suggested metrics is demonstrated on a case of the Journal of Applied Management and Investments (JAMI). It helped to find the root causes of the low efficiency and low effectiveness of most of the articles of JAMI. Among them: a language variety, restricted access for readers to some volumes (issues), non-indexation in the leading international scientific databases, and practical orientation of many publications that usually are not high-cited. The identified problems should be the reasons for revising the editorial policy aimed at increasing the efficiency and effectiveness of the journal's publications. The author concluded that suggested metrics can’t find all problems of the publisher (journal), but they can allocate a group of problematic publications that burden it. The results of this study should be useful for publisher and chief editors of the research journals that want to improve the editorial policy and to increase the efficiency and quality of the publications (issues).

Suggested Citation

  • Anatoliy G. Goncharuk, 2018. "Efficiency vs Effectiveness: Alternative Metrics for Research Performance," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 7(1), pages 24-37, February.
    • Handle: RePEc:ods:journl:v:7:y:2018:i:1:p:24-37
    as

    Download full text from publisher

    File URL: http://www.jami.org.ua/Papers/JAMI_7_1_2018_24-37.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Jon H. Hanf, 2014. "Literature Survey on New Participative Pricing Mechanisms for Wine Tastings," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 3(4), pages 201-213.
    2. Natalia O. Lazareva, 2015. "Evaluating the Efficiency of Wineries in Ukraine: A Three-Criteria Approach," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 4(4), pages 239-242, November.
    3. Mihir Dash, 2013. "A Study of the Impact of Asset-Liability Management on the Profitability of Banks in India," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 2(4), pages 230-234.
    4. Jon H. Hanf & Victoria Marquardt, 2012. "Armenian Wine Business And Its Interplay With Foreign Influence," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 1(4), pages 406-417.
    5. Anatoliy G. Goncharuk, 2014. "Measuring Enterprise Performance to Achieve Managerial Goals," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 3(1), pages 8-14.
    6. Jon H. Hanf & Taras Gagalyuk & Erik Schweickert, 2013. "Problems of Goal Alignment for German Wine Cooperatives," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 2(4), pages 244-252.
    7. Sushma Vegesna & Mihir Dash, 2014. "Efficiency of Public and Private Sector Banks in India," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 3(3), pages 183-187.
    8. Anatoliy G. Goncharuk, 2013. "About The Influence Of High Gas Price On An Efficiency," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 2(1), pages 58-67.
    9. Leo Egghe, 2006. "Theory and practise of the g-index," Scientometrics, Springer;Akadémiai Kiadó, vol. 69(1), pages 131-152, October.
    10. Anatoliy G. Goncharuk, 2016. "Evaluating the Efficiency of Higher Education," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(2), pages 82-91, May.
    11. Lampe, Hannes W. & Hilgers, Dennis, 2015. "Trajectories of efficiency measurement: A bibliometric analysis of DEA and SFA," European Journal of Operational Research, Elsevier, vol. 240(1), pages 1-21.
    12. Anatoliy G. Goncharuk, 2016. "Banking Sector Challenges in Research," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(1), pages 34-39, February.
    13. Narasinha Sawaikar, 2014. "Samsung Smartphones and Core Competency," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 3(4), pages 251-256.
    14. Anatoliy G. Goncharuk, 2012. "What investments are most effective during the financial crisis?," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 1(1), pages 5-7.
    15. Anatoliy G. Goncharuk, 2017. "Socioeconomic Criteria of Healthcare Efficiency: An International Comparison," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 6(2), pages 89-95, May.
    16. Costas, Rodrigo & Bordons, María, 2007. "The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level," Journal of Informetrics, Elsevier, vol. 1(3), pages 193-203.
    17. Alonso, S. & Cabrerizo, F.J. & Herrera-Viedma, E. & Herrera, F., 2009. "h-Index: A review focused in its variants, computation and standardization for different scientific fields," Journal of Informetrics, Elsevier, vol. 3(4), pages 273-289.
    18. Manoj Kumar & Madhu Anand, 2013. "Assessing Financial Health Of A Firm Using Altman’S Original And Revised Z-Score Models: A Case Of Kingfisher Airlines Ltd (India)," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 2(1), pages 36-48.
    19. Kumbhakar,Subal C. & Lovell,C. A. Knox, 2003. "Stochastic Frontier Analysis," Cambridge Books, Cambridge University Press, number 9780521666633.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Yahia, F.B. & Essid, H., 2019. "Determinants of Tunisian Schools’ Efficiency: A DEA-Tobit Approach," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 8(1), pages 44-56, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anatoliy G. Goncharuk, 2017. "The Challenges of Wine Business in Research," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 6(4), pages 253-259, November.
    2. Jon H. Hanf, 2016. "Vertical Integration in the Azerbaijani Wine Business," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(2), pages 92-99, May.
    3. Anatoliy G. Goncharuk, 2016. "Banking Sector Challenges in Research," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(1), pages 34-39, February.
    4. Natalia O. Lazareva, 2015. "Evaluating the Efficiency of Wineries in Ukraine: A Three-Criteria Approach," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 4(4), pages 239-242, November.
    5. Anatoliy G. Goncharuk, 2017. "Wine Value Chains: Challenges and Prospects," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 6(1), pages 11-27, February.
    6. Anatoliy G. Goncharuk, 2016. "Evaluating the Efficiency of Higher Education," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(2), pages 82-91, May.
    7. Sourour Ramzi & Mohamed Ayadi, 2016. "Assessment of Universities Efficiency Using Data Envelopment Analysis: Weights Restrictions and Super-Efficiency Measure," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(1), pages 40-58, February.
    8. Gaviria-Marin, Magaly & Merigó, José M. & Baier-Fuentes, Hugo, 2019. "Knowledge management: A global examination based on bibliometric analysis," Technological Forecasting and Social Change, Elsevier, vol. 140(C), pages 194-220.
    9. Anatoliy G. Goncharuk, 2017. "Socioeconomic Criteria of Healthcare Efficiency: An International Comparison," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 6(2), pages 89-95, May.
    10. Shahryar Rahnamayan & Sedigheh Mahdavi & Kalyanmoy Deb & Azam Asilian Bidgoli, 2020. "Ranking Multi-Metric Scientific Achievements Using a Concept of Pareto Optimality," Mathematics, MDPI, vol. 8(6), pages 1-46, June.
    11. Bornmann, Lutz & Marx, Werner, 2012. "HistCite analysis of papers constituting the h index research front," Journal of Informetrics, Elsevier, vol. 6(2), pages 285-288.
    12. Yu Liu & Wei Zuo & Ying Gao & Yanhong Qiao, 2013. "Comprehensive geometrical interpretation of h-type indices," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 605-615, August.
    13. Saroj Upadhyay & Sudeshna Gupta, 2016. "Taking a Relook into the Decision Making Process in Indian Banking Sector with Respect to Non-Performing Asset Management," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 5(3), pages 193-202, August.
    14. Kuan, Chung-Huei & Huang, Mu-Hsuan & Chen, Dar-Zen, 2011. "Ranking patent assignee performance by h-index and shape descriptors," Journal of Informetrics, Elsevier, vol. 5(2), pages 303-312.
    15. Lorna Wildgaard & Jesper W. Schneider & Birger Larsen, 2014. "A review of the characteristics of 108 author-level bibliometric indicators," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 125-158, October.
    16. Fiorenzo Franceschini & Domenico Maisano, 2011. "Bibliometric positioning of scientific manufacturing journals: a comparative analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 86(2), pages 463-485, February.
    17. Anatoliy G. Goncharuk, 2015. "Food Business and Food Security Challenges in Research," Journal of Applied Management and Investments, Department of Business Administration and Corporate Security, International Humanitarian University, vol. 4(4), pages 223-230, November.
    18. Chen, Meiqian & Guo, Zhaoxia & Dong, Yucheng & Chiclana, Francisco & Herrera-Viedma, Enrique, 2021. "Citations optimal growth path: A tool to analyze sensitivity to citations of h-like indexes," Journal of Informetrics, Elsevier, vol. 15(4).
    19. R. Álvarez & E. Cahué & J. Clemente-Gallardo & A. Ferrer & D. Íñiguez & X. Mellado & A. Rivero & G. Ruiz & F. Sanz & E. Serrano & A. Tarancón & Y. Vergara, 2015. "Analysis of academic productivity based on Complex Networks," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(3), pages 651-672, September.
    20. Maor Weinberger & Maayan Zhitomirsky-Geffet, 2021. "Diversity of success: measuring the scholarly performance diversity of tenured professors in the Israeli academia," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(4), pages 2931-2970, April.

    More about this item

    Keywords

    research journal; publications; performance; efficiency; effectiveness;
    All these keywords.

    JEL classification:

    • C14 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Semiparametric and Nonparametric Methods: General
    • L82 - Industrial Organization - - Industry Studies: Services - - - Entertainment; Media
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ods:journl:v:7:y:2018:i:1:p:24-37. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Anatoliy G. Goncharuk (email available below). General contact details of provider: https://edirc.repec.org/data/dmonaua.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.