IDEAS home Printed from https://ideas.repec.org/a/bla/ajecsc/v69y2010i5p1495-1529.html
   My bibliography  Save this article

Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy

Author

Listed:
  • Marcella Corsi
  • Carlo D'Ippoliti
  • Federico Lucidi

Abstract

We analyze Italy's recent research evaluation exercise (VTR) as a salient example in discussing some internationally relevant issues emerging from the evaluation of research in economics. We claim that evaluation and its criteria, together with its linkage to research institutions' financing, are likely to affect the direction of research in a problematic way. As the Italian case documents, it is specifically economists who adopt unorthodox paradigms or pursue less diffused topics of research that should be concerned about research evaluation and its criteria. After outlining the recent practice of economic research in Italy and highlighting the relevant scope for pluralism that traditionally characterizes it, we analyze the publications submitted for evaluation to the VTR. By comparing these publications to all the entries in the EconLit database authored by economists located in Italy, we find a risk that the adopted ranking criteria may lead to disregarding historical methods in favor of quantitative and econometric methods, and heterodox schools in favor of mainstream approaches. Finally, by summarizing the current debate in Italy, we claim that evaluation should not be refused by heterodox economists, but rather that a reflection on the criteria of evaluation should be put forward at an international level in order to establish fair competition among research paradigms, thus, preserving pluralism in the discipline.

Suggested Citation

  • Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2010. "Pluralism at Risk? Heterodox Economic Approaches and the Evaluation of Economic Research in Italy," American Journal of Economics and Sociology, Wiley Blackwell, vol. 69(5), pages 1495-1529, November.
  • Handle: RePEc:bla:ajecsc:v:69:y:2010:i:5:p:1495-1529
    DOI: 10.1111/j.1536-7150.2010.00754.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1536-7150.2010.00754.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1536-7150.2010.00754.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Andrew J. Oswald, 2007. "An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers," Economica, London School of Economics and Political Science, vol. 74(293), pages 21-31, February.
    2. Luigi Pasinetti & Alessandro Roncaglia, 2006. "Le scienze umane in Italia: il caso dell'economia politica," Rivista italiana degli economisti, Società editrice il Mulino, issue 3, pages 463-463.
    3. Maria Cristina Marcuzzo & Giulia Zacchia, 2007. "L'ECONLIT e gli strumenti per la valutazione della ricerca economica in Italia," Rivista italiana degli economisti, Società editrice il Mulino, issue 2, pages 277-306.
    4. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    5. Kalaitzidakis, Pantelis & Mamuneas, Theofanis P. & Stengos, Thanasis, 1999. "European economics: An analysis based on publications in the core journals," European Economic Review, Elsevier, vol. 43(4-6), pages 1150-1168, April.
    6. Aldo Geuna, 2001. "The Changing Rationale for European University Research Funding: Are There Negative Unintended Consequences?," Journal of Economic Issues, Taylor & Francis Journals, vol. 35(3), pages 607-632, September.
    7. J. Peter Neary & James A. Mirrlees & Jean Tirole, 2003. "Evaluating Economics Research in Europe: An Introduction," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1239-1249, December.
    8. Frey, Bruno S, 2003. "Publishing as Prostitution?--Choosing between One's Own Ideas and Academic Success," Public Choice, Springer, vol. 116(1-2), pages 205-223, July.
    9. Frederic S. Lee, 2007. "The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities," Cambridge Journal of Economics, Oxford University Press, vol. 31(2), pages 309-325, March.
    10. Kostas Axarloglou & Vasilis Theoharakis, 2003. "Diversity in Economics: An Analysis of Journal Quality Perceptions," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1402-1423, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Carlo D’Ippoliti & Giulia Zacchia, 2017. "On the Efficiency of Italian Universities: A Comment," Italian Economic Journal: A Continuation of Rivista Italiana degli Economisti and Giornale degli Economisti, Springer;Società Italiana degli Economisti (Italian Economic Association), vol. 3(1), pages 113-123, March.
    2. Halkos, George & Tzeremes, Nickolaos, 2012. "Ranking agricultural, environmental and natural resource economics journals: A note," MPRA Paper 36233, University Library of Munich, Germany.
    3. Buehling, Kilian, 2021. "Changing research topic trends as an effect of publication rankings – The case of German economists and the Handelsblatt Ranking," Journal of Informetrics, Elsevier, vol. 15(3).
    4. Jakob Kapeller & Matthias Aistleitner & Stefan Steinerberger, 2017. "Citation Patterns in Economics and Beyond: Assessing the Peculiarities of Economics from Two Scientometric Perspectives," ICAE Working Papers 60, Johannes Kepler University, Institute for Comprehensive Analysis of the Economy.
    5. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2011. "On the Evaluation of Economic Research: The Case of Italy," Economia politica, Società editrice il Mulino, issue 3, pages 369-402.
    6. Arne Heise, 2018. "Reclaiming the University: transforming economics as a discipline," The Journal of Philosophical Economics, Bucharest Academy of Economic Studies, The Journal of Philosophical Economics, vol. 11(2), pages 37-66, May.
    7. Corsi, Marcella & D’Ippoliti, Carlo & Zacchia, Giulia, 2019. "Diversity of backgrounds and ideas: The case of research evaluation in economics," Research Policy, Elsevier, vol. 48(9), pages 1-1.
    8. Anania, Giovanni & Caruso, Annarosa, 2012. "Two New Simple Bibliometric Indexes to Better Evaluate Research in Economics," 2012 First Congress, June 4-5, 2012, Trento, Italy 124116, Italian Association of Agricultural and Applied Economics (AIEAA).
    9. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    10. Matthias Aistleitner & Jakob Kapeller & Stefan Steinerberger, 2018. "Citation Patterns in Economics and Beyond," Working Papers Series 85, Institute for New Economic Thinking.
    11. Giovanni Anania & Annarosa Caruso, 2013. "Two simple new bibliometric indexes to better evaluate research in disciplines where publications typically receive less citations," Scientometrics, Springer;Akadémiai Kiadó, vol. 96(2), pages 617-631, August.
    12. Zacchia, Giulia, 2016. "Segregation or homologation? Gender differences in recent Italian economic thought," MPRA Paper 72279, University Library of Munich, Germany.
    13. Michael C. Calver & Maggie Lilith & Christopher R. Dickman, 2013. "A ‘perverse incentive’ from bibliometrics: could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(1), pages 243-255, April.
    14. Ferenc Moksony & Rita Hegedűs & Melinda Császár, 2014. "Rankings, research styles, and publication cultures: a study of American sociology departments," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(3), pages 1715-1729, December.
    15. Hicks, Diana, 2012. "Performance-based university research funding systems," Research Policy, Elsevier, vol. 41(2), pages 251-261.
    16. Marcella Corsi & Giulia Zacchia, 2014. "Women Economists in Italy: A Bibliometric Analysis of their Scientific Production in the Past Decade," Working Papers CEB 14-008, ULB -- Universite Libre de Bruxelles.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcella Corsi & Carlo D'Ippoliti & Federico Lucidi, 2011. "On the Evaluation of Economic Research: The Case of Italy," Economia politica, Società editrice il Mulino, issue 3, pages 369-402.
    2. Jan Ours & Frederic Vermeulen, 2007. "Ranking Dutch Economists," De Economist, Springer, vol. 155(4), pages 469-487, December.
    3. Oswald, Andrew J., 2015. "The Objective Measurement of World-Leading Research," IZA Discussion Papers 8829, Institute of Labor Economics (IZA).
    4. Oswald, Andrew J., 2009. "World-Leading Research and its Measurement," Economic Research Papers 271312, University of Warwick - Department of Economics.
    5. Andrew J. Oswald, 2010. "A suggested method for the measurement of world-leading research (illustrated with data on economics)," Scientometrics, Springer;Akadémiai Kiadó, vol. 84(1), pages 99-113, July.
    6. Oswald, Andrew J., 2008. "Can We Test for Bias in Scientific Peer-Review?," IZA Discussion Papers 3665, Institute of Labor Economics (IZA).
    7. Javier Ruiz-Castillo, 2008. "Economics research in Spain during the 1990s: a literature review," Spanish Economic Review, Springer;Spanish Economic Association, vol. 10(3), pages 221-249, September.
    8. Mirucki, Jean & Nicot, Bernadette & Poshyvak, Maria, 2007. "What Can EconLit Reveal Us About Ukraine's Scholarly Production?," MPRA Paper 27717, University Library of Munich, Germany.
    9. Stanley, T.D. & Doucouliagos, Chris & Jarrell, Stephen B., 2008. "Meta-regression analysis as the socio-economics of economics research," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 37(1), pages 276-292, February.
    10. Rafols, Ismael & Leydesdorff, Loet & O’Hare, Alice & Nightingale, Paul & Stirling, Andy, 2012. "How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management," Research Policy, Elsevier, vol. 41(7), pages 1262-1282.
    11. Cloos, Janis & Greiff, Matthias & Rusch, Hannes, 2020. "Geographical Concentration and Editorial Favoritism within the Field of Laboratory Experimental Economics (RM/19/029-revised-)," Research Memorandum 014, Maastricht University, Graduate School of Business and Economics (GSBE).
    12. Margit Osterloh & Bruno S. Frey, 2009. "Research governance in academia: are there alternatives to academic rankings?," IEW - Working Papers 423, Institute for Empirical Research in Economics - University of Zurich.
    13. Buehling, Kilian, 2021. "Changing research topic trends as an effect of publication rankings – The case of German economists and the Handelsblatt Ranking," Journal of Informetrics, Elsevier, vol. 15(3).
    14. Ali, Showkat & Carden, Giles & Culling, Benjamin & Hunter, Rosalind & Oswald, Andrew J & Owen, Nicola & Ralsmark, Hilda & Snodgrass, Natalie, 2007. "Elite Scientists and the Global Brain Drain," The Warwick Economics Research Paper Series (TWERPS) 825, University of Warwick, Department of Economics.
    15. Daniel Sgroi & Andrew J. Oswald, 2013. "How Should Peer‐review Panels Behave?," Economic Journal, Royal Economic Society, vol. 0, pages 255-278, August.
    16. Jan Ours, 2009. "Will You Still Need Me: When I’m 64?," De Economist, Springer, vol. 157(4), pages 441-460, December.
    17. Rodgers, Joan R. & Valadkhani, Abbas, 2005. "Ranking of Australian Economics Departments Based on Their Total and Per Academic Staff Research Output," Economics Working Papers wp05-18, School of Economics, University of Wollongong, NSW, Australia.
    18. Berlemann, Michael & Haucap, Justus, 2015. "Which factors drive the decision to opt out of individual research rankings? An empirical study of academic resistance to change," Research Policy, Elsevier, vol. 44(5), pages 1108-1115.
    19. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    20. Damien Besancenot & Kim Huynh & Joao Faria, 2012. "Search and research: the influence of editorial boards on journals’ quality," Theory and Decision, Springer, vol. 73(4), pages 687-702, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:ajecsc:v:69:y:2010:i:5:p:1495-1529. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: . General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0002-9246 .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0002-9246 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.