IDEAS home Printed from https://ideas.repec.org/a/mul/jb33yl/doi10.1428-35915y2011i3p369-402.html
   My bibliography  Save this article

On the Evaluation of Economic Research: The Case of Italy

Author

Listed:
  • Marcella Corsi
  • Carlo D'Ippoliti
  • Federico Lucidi

Abstract

The Italian case can be considered as an internationally relevant example to suggest a critical reflection on the evaluation criteria adopted in research assessment exercises, pointing at the need of clear and shared guidelines based on transparency and accountability and aiming at preserving (or even encouraging) the pluralism of ideas. Our findings support the view that if research institutions are encouraged to engage only in those lines of research that are likely to receive the highest rating according to the adopted evaluation criteria, a convergence process is to be expected within Economics, resulting in a disregard of heterodox schools and historical methods in favour of mainstream approaches and quantitative methods. In our view, a proper fine-tuning of the assessment methodology is needed, before subsequent rankings can be used as a guide for the allocation of public financing among research institutions. In the case of Economics, this means overcoming the limits of commonly adopted peer review approaches, through the development of proper evaluation designs and the integration of qualitative appraisals with quantitative indicators. In order to preserve pluralism and originality of research, we propose a simple quantitative index based on field-normalization.

Suggested Citation

  • Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2011. "On the Evaluation of Economic Research: The Case of Italy," Economia politica, Società editrice il Mulino, issue 3, pages 369-402.
  • Handle: RePEc:mul:jb33yl:doi:10.1428/35915:y:2011:i:3:p:369-402
    as

    Download full text from publisher

    File URL: https://www.rivisteweb.it/download/article/10.1428/35915
    Download Restriction: Access to full text is restricted to subscribers

    File URL: https://www.rivisteweb.it/doi/10.1428/35915
    Download Restriction: no
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    2. Aldo Geuna, 2001. "The Changing Rationale for European University Research Funding: Are There Negative Unintended Consequences?," Journal of Economic Issues, Taylor & Francis Journals, vol. 35(3), pages 607-632, September.
    3. Frederic S. Lee, 2007. "The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities," Cambridge Journal of Economics, Oxford University Press, vol. 31(2), pages 309-325, March.
    4. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2011. "On the Evaluation of Economic Research: The Case of Italy," Economia politica, Società editrice il Mulino, issue 3, pages 369-402.
    5. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    6. Bruno S. Frey & Margit Osterloh, 2006. "Evaluations: Hidden Costs, Questionable Benefits, and Superior Alternatives," IEW - Working Papers 302, Institute for Empirical Research in Economics - University of Zurich.
    7. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism In Economics And The Evaluation Of Economic Research In Italy," DULBEA Working Papers 10-05, ULB -- Universite Libre de Bruxelles.
    8. Michel Lubrano & Luc Bauwens & Alan Kirman & Camelia Protopopescu, 2003. "Ranking Economics Departments in Europe: A Statistical Approach," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1367-1401, December.
    9. David Colander, 2009. "Can European Economics Compete with U.S. Economics? And Should It"," Middlebury College Working Paper Series 0902, Middlebury College, Department of Economics.
    10. Luigi Pasinetti & Alessandro Roncaglia, 2006. "Le scienze umane in Italia: il caso dell'economia politica," Rivista italiana degli economisti, Società editrice il Mulino, issue 3, pages 463-463.
    11. Kalaitzidakis, Pantelis & Mamuneas, Theofanis P. & Stengos, Thanasis, 1999. "European economics: An analysis based on publications in the core journals," European Economic Review, Elsevier, vol. 43(4-6), pages 1150-1168, April.
    12. Marco Lippi & Franco Peracchi, 2007. "Il primo esercizio italiano di valutazione della ricerca: una prima valutazione," Rivista italiana degli economisti, Società editrice il Mulino, issue 2, pages 267-276.
    13. Thed N. Van Leeuwen & Martijn S. Visser & Henk F. Moed & Ton J. Nederhof & Anthony F. J. Van Raan, 2003. "The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence," Scientometrics, Springer;Akadémiai Kiadó, vol. 57(2), pages 257-280, June.
    14. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy," American Journal of Economics and Sociology, Wiley Blackwell, vol. 69(5), pages 1495-1529, November.
    15. Hodgson, Geoffrey M & Rothman, Harry, 1999. "The Editors and Authors of Economics Journals: A Case of Institutional Oligopoly?," Economic Journal, Royal Economic Society, vol. 109(453), pages 165-186, February.
    16. Silvia Ferrini, 2007. "L’Impact Factor: luci e ombre," Department of Economic Policy, Finance and Development (DEPFID) University of Siena 002, Department of Economic Policy, Finance and Development (DEPFID), University of Siena.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2011. "On the Evaluation of Economic Research: The Case of Italy," Economia politica, Società editrice il Mulino, issue 3, pages 369-402.
    2. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    3. Corsi, Marcella & D’Ippoliti, Carlo & Zacchia, Giulia, 2019. "Diversity of backgrounds and ideas: The case of research evaluation in economics," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    4. Marcella Corsi, 2017. "Note Bibliografiche: Rochon L.-P., Rossi S. (2017): A Modern Guide to Rethinking Economics; Jo T.-H., Chester L., D'Ippoliti C. (2017), The Routledge Handbook of Heterodox Economics. Theorizing, Analy," Moneta e Credito, Economia civile, vol. 70(280), pages 389-392.
    5. Carlo D'Ippoliti, 2021. "“Many‐Citedness”: Citations Measure More Than Just Scientific Quality," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1271-1301, December.
    6. Anania, Giovanni & Caruso, Annarosa, 2012. "Two New Simple Bibliometric Indexes to Better Evaluate Research in Economics," 2012 First Congress, June 4-5, 2012, Trento, Italy 124116, Italian Association of Agricultural and Applied Economics (AIEAA).
    7. Paulo Guimarães & Mariana Barbosa, 2022. "The state of Portuguese research in economics: 20 years after," Portuguese Economic Journal, Springer;Instituto Superior de Economia e Gestao, vol. 21(3), pages 283-309, September.
    8. Sandra Rousseau & Ronald Rousseau, 2021. "Bibliometric Techniques And Their Use In Business And Economics Research," Journal of Economic Surveys, Wiley Blackwell, vol. 35(5), pages 1428-1451, December.
    9. Carlo D’Ippoliti & Giulia Zacchia, 2017. "On the Efficiency of Italian Universities: A Comment," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 3(1), pages 113-123, March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy," American Journal of Economics and Sociology, Wiley Blackwell, vol. 69(5), pages 1495-1529, November.
    2. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    3. William A. Jackson, 2018. "Strategic Pluralism and Monism in Heterodox Economics," Review of Radical Political Economics, Union for Radical Political Economics, vol. 50(2), pages 237-251, June.
    4. Buehling, Kilian, 2021. "Changing research topic trends as an effect of publication rankings – The case of German economists and the Handelsblatt Ranking," Journal of Informetrics, Elsevier, vol. 15(3).
    5. Anania, Giovanni & Caruso, Annarosa, 2012. "Two New Simple Bibliometric Indexes to Better Evaluate Research in Economics," 2012 First Congress, June 4-5, 2012, Trento, Italy 124116, Italian Association of Agricultural and Applied Economics (AIEAA).
    6. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    7. Jan Ours & Frederic Vermeulen, 2007. "Ranking Dutch Economists," De Economist, Springer, vol. 155(4), pages 469-487, December.
    8. Matthias Aistleitner & Jakob Kapeller & Stefan Steinerberger, 2018. "Citation Patterns in Economics and Beyond," Working Papers Series 85, Institute for New Economic Thinking.
    9. Libman, A., 2010. "Economics in Germany – from National to Global," Journal of the New Economic Association, New Economic Association, issue 8, pages 155-158.
    10. James B. Davies & Martin G. Kocher & Matthias Sutter, 2008. "Economics research in Canada: a long‐run assessment of journal publications," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(1), pages 22-45, February.
    11. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    12. David L. Anderson & John Tressler, 2008. "Research Output in New Zealand Economics Department 2000-2006," Working Papers in Economics 08/05, University of Waikato.
    13. Birolo, Adriano & Rosselli, Annalisa, 2009. "Research standards for the Italian young academics: what has changed over the last thirty years?," MPRA Paper 21368, University Library of Munich, Germany.
    14. David Anderson & John Tressler, 2008. "Research output in New Zealand economics departments 2000-2006: A stock approach," New Zealand Economic Papers, Taylor & Francis Journals, vol. 42(2), pages 155-189.
    15. Johansson, Dan & Karlsson, Johan & Malm, Arvid, 2020. "Family business—A missing link in economics?," Journal of Family Business Strategy, Elsevier, vol. 11(1).
    16. Ferenc Moksony & Rita Hegedűs & Melinda Császár, 2014. "Rankings, research styles, and publication cultures: a study of American sociology departments," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1715-1729, December.
    17. Jakob Kapeller & Matthias Aistleitner & Stefan Steinerberger, 2017. "Citation Patterns in Economics and Beyond: Assessing the Peculiarities of Economics from Two Scientometric Perspectives," ICAE Working Papers 60, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.
    18. Justus Haucap & Johannes Muck, 2015. "What drives the relevance and reputation of economics journals? An update from a survey among economists," Scientometrics, Springer;Akadémiai Kiadó, vol. 103(3), pages 849-877, June.
    19. Stephen Bazen & Patrick Moyes, 2012. "Elitism and stochastic dominance," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 39(1), pages 207-251, June.
    20. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).

    More about this item

    Keywords

    A11; A14; B20; B40; B50;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • B20 - Schools of Economic Thought and Methodology - - History of Economic Thought since 1925 - - - General
    • B40 - Schools of Economic Thought and Methodology - - Economic Methodology - - - General
    • B50 - Schools of Economic Thought and Methodology - - Current Heterodox Approaches - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mul:jb33yl:doi:10.1428/35915:y:2011:i:3:p:369-402. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://www.rivisteweb.it/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.