IDEAS home Printed from https://ideas.repec.org/a/spr/scient/v104y2015i1d10.1007_s11192-015-1581-y.html
   My bibliography  Save this article

The inner quality of an article: Will time tell?

Author

Listed:
  • Ho Fai Chan

    (Queensland University of Technology)

  • Malka Guillot

    (Ecole Nationale Statistique et de l’Administration Economique (ENSAE))

  • Lionel Page

    (Queensland University of Technology)

  • Benno Torgler

    (Queensland University of Technology
    Center for Research in Economics, Management and the Arts (CREMA))

Abstract

In this paper, we assess whether quality survives the test of time in academia by comparing up to 80 years of academic journal article citations from two top journals, Econometrica and the American Economic Review. The research setting under analysis is analogous to a controlled real world experiment in that it involves a homogeneous task (trying to publish in top journals) by individuals with a homogenous job profile (academics) in a specific research environment (economics and econometrics). Comparing articles published concurrently in the same outlet at the same time (same issue) indicates that symbolic capital or power due to institutional affiliation or connection does seem to boost citation success at the beginning, giving those educated at or affiliated with leading universities an initial comparative advantage. Such advantage, however, does not hold in the long run: at a later stage, the publications of other researchers become as or even more successful.

Suggested Citation

  • Ho Fai Chan & Malka Guillot & Lionel Page & Benno Torgler, 2015. "The inner quality of an article: Will time tell?," Scientometrics, Springer;Akadémiai Kiadó, vol. 104(1), pages 19-41, July.
    • Handle: RePEc:spr:scient:v:104:y:2015:i:1:d:10.1007_s11192-015-1581-y
    DOI: 10.1007/s11192-015-1581-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11192-015-1581-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11192-015-1581-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Markus Schaffner & Benno Torgler & Stephen Whyte, 2016. "External Influence as an Indicator of Scholarly Importance," CESifo Economic Studies, CESifo, vol. 62(1), pages 170-195.
    2. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2011. "An updated ranking of academic journals in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 44(4), pages 1525-1538, November.
    3. Kodrzycki Yolanda K. & Yu Pingkang, 2006. "New Approaches to Ranking Economics Journals," The B.E. Journal of Economic Analysis & Policy, De Gruyter, vol. 5(1), pages 1-44, August.
    4. Benno Torgler & Marco Piatti, 2011. "A Century of American Economic Review," Working Papers 2011.27, Fondazione Eni Enrico Mattei.
    5. Rabah Amir & Malgorzata Knauff, 2008. "Ranking Economics Departments Worldwide on the Basis of PhD Placement," The Review of Economics and Statistics, MIT Press, vol. 90(1), pages 185-190, August.
    6. Christopher Watts & Nigel Gilbert, 2011. "Does cumulative advantage affect collective learning in science? An agent-based simulation," Scientometrics, Springer;Akadémiai Kiadó, vol. 89(1), pages 437-463, October.
    7. Benno Torgler & Marco Piatti, 2013. "A Century of American Economic Review: Insights on Critical Factors in Journal Publishing," Palgrave Macmillan Books, Palgrave Macmillan, number 978-1-137-33305-6, October.
    8. Pierre Azoulay & Toby Stuart & Yanbo Wang, 2014. "Matthew: Effect or Fable?," Management Science, INFORMS, vol. 60(1), pages 92-109, January.
    9. Wolfgang Glänzel & Balázs Schlemmer & Bart Thijs, 2003. "Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon," Scientometrics, Springer;Akadémiai Kiadó, vol. 58(3), pages 571-586, November.
    10. S. Redner, 1998. "How popular is your paper? An empirical study of the citation distribution," The European Physical Journal B: Condensed Matter and Complex Systems, Springer;EDP Sciences, vol. 4(2), pages 131-134, July.
    11. David W. Johnston & Marco Piatti & Benno Torgler, 2013. "Citation success over time: theory or empirics?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1023-1029, June.
    12. Norio Ohba & Kumiko Nakao, 2012. "Sleeping beauties in ophthalmology," Scientometrics, Springer;Akadémiai Kiadó, vol. 93(2), pages 253-264, November.
    13. Ponomarev, Ilya V. & Williams, Duane E. & Hackett, Charles J. & Schnell, Joshua D. & Haak, Laurel L., 2014. "Predicting highly cited papers: A Method for Early Detection of Candidate Breakthroughs," Technological Forecasting and Social Change, Elsevier, vol. 81(C), pages 49-55.
    14. John Hudson, 2007. "Be known by the company you keep: Citations — quality or chance?," Scientometrics, Springer;Akadémiai Kiadó, vol. 71(2), pages 231-238, May.
    15. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    16. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.
    17. Wei Huang, 2015. "DO ABCs GET MORE CITATIONS THAN XYZs?," Economic Inquiry, Western Economic Association International, vol. 53(1), pages 773-789, January.
    18. Ho Fai Chan & Bruno S. Frey & Jana Gallus & Markus Schaffner & Benno Torgler & Stephen Whyte, 2014. "Do the best scholars attract the highest speaking fees? An exploration of internal and external influence," Scientometrics, Springer;Akadémiai Kiadó, vol. 101(1), pages 793-817, October.
    19. J Mingers, 2008. "Exploring the dynamics of journal citations: Modelling with s-curves," Journal of the Operational Research Society, Palgrave Macmillan;The OR Society, vol. 59(8), pages 1013-1025, August.
    20. Young-Ho Eom & Santo Fortunato, 2011. "Characterizing and Modeling Citation Dynamics," PLOS ONE, Public Library of Science, vol. 6(9), pages 1-7, September.
    21. Stigler, George J & Stigler, Stephen M & Friedland, Claire, 1995. "The Journals of Economics," Journal of Political Economy, University of Chicago Press, vol. 103(2), pages 331-359, April.
    22. Samuel Bjork & Avner Offer & Gabriel Söderberg, 2014. "Time series citation data: the Nobel Prize in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 98(1), pages 185-196, January.
    23. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    24. Victor Ginsburgh, 2003. "Economics of Art and Culture: Invited Papers at the 12th international conference of the Association of Cultural Economics International," ULB Institutional Repository 2013/152418, ULB -- Universite Libre de Bruxelles.
    25. Jonathan M. Levitt & Mike Thelwall, 2008. "Patterns of annual citation of highly cited articles and the prediction of their citation ranking: A comparison across subjects," Scientometrics, Springer;Akadémiai Kiadó, vol. 77(1), pages 41-60, October.
    26. Kostas Axarloglou & Vasilis Theoharakis, 2003. "Diversity in Economics: An Analysis of Journal Quality Perceptions," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1402-1423, December.
    27. Lorenzo Ductor, 2015. "Does Co-authorship Lead to Higher Academic Productivity?," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 77(3), pages 385-407, June.
    28. Roth, Camille & Wu, Jiang & Lozano, Sergi, 2012. "Assessing impact and quality from local dynamics of citation networks," Journal of Informetrics, Elsevier, vol. 6(1), pages 111-120.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Franklin G. Mixon & Benno Torgler & Kamal P. Upadhyaya, 2017. "Scholarly impact and the timing of major awards in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1837-1852, September.
    2. Naomi Moy & Ho Fai Chan & Benno Torgler, 2018. "How much is too much? The effects of information quantity on crowdfunding performance," PLOS ONE, Public Library of Science, vol. 13(3), pages 1-15, March.
    3. Saikou Y. Diallo & Christopher J. Lynch & Ross Gore & Jose J. Padilla, 2016. "Identifying key papers within a journal via network centrality measures," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(3), pages 1005-1020, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David W. Johnston & Marco Piatti & Benno Torgler, 2013. "Citation success over time: theory or empirics?," Scientometrics, Springer;Akadémiai Kiadó, vol. 95(3), pages 1023-1029, June.
    2. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    3. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    4. Chan, Ho Fai & Frey, Bruno S. & Gallus, Jana & Torgler, Benno, 2014. "Academic honors and performance," Labour Economics, Elsevier, vol. 31(C), pages 188-204.
    5. João R. Faria & Franklin G. Mixon & Kamal P. Upadhyaya, 2016. "Human capital, collegiality, and stardom in economics: empirical analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 106(3), pages 917-943, March.
    6. Ho F. Chan & Franklin G. Mixon & Benno Torgler, 2018. "Relation of early career performance and recognition to the probability of winning the Nobel Prize in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 114(3), pages 1069-1086, March.
    7. Ho Fai Chan & Benno Torgler, 2020. "Gender differences in performance of top cited scientists by field and country," Scientometrics, Springer;Akadémiai Kiadó, vol. 125(3), pages 2421-2447, December.
    8. Chan, Ho Fai & Bodiuzzman, Sohel Md & Torgler, Benno, 2020. "The power of social cues in the battle for attention: Evidence from an online platform for scholarly commentary," Journal of Informetrics, Elsevier, vol. 14(4).
    9. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    10. Ho Fai Chan & Benno Torgler, 2015. "The implications of educational and methodological background for the career success of Nobel laureates: an investigation of major awards," Scientometrics, Springer;Akadémiai Kiadó, vol. 102(1), pages 847-863, January.
    11. David L. Anderson & John Tressler, 2013. "The Relevance of the “h-” and “g-” Index to Economics in the Context of A Nation-Wide Research Evaluation Scheme: The New Zealand Case," Economic Papers, The Economic Society of Australia, vol. 32(1), pages 81-94, March.
    12. John Gibson & David L. Anderson & John Tressler, 2014. "Which Journal Rankings Best Explain Academic Salaries? Evidence From The University Of California," Economic Inquiry, Western Economic Association International, vol. 52(4), pages 1322-1340, October.
    13. Ana Rute Cardoso & Paulo Guimarães & Klaus F. Zimmermann, 2010. "Trends in Economic Research: An International Perspective," Kyklos, Wiley Blackwell, vol. 63(4), pages 479-494, November.
    14. Johannes König & David I. Stern & Richard S.J. Tol, 2022. "Confidence Intervals for Recursive Journal Impact Factors," Tinbergen Institute Discussion Papers 22-038/III, Tinbergen Institute.
    15. Ham, John C. & Wright, Julian & Ye, Ziqiu, 2023. "Documenting and Explaining the Dramatic Rise of the New Society Journals in Economics," IZA Discussion Papers 16337, Institute of Labor Economics (IZA).
    16. Lachance, Christian & Larivière, Vincent, 2014. "On the citation lifecycle of papers with delayed recognition," Journal of Informetrics, Elsevier, vol. 8(4), pages 863-872.
    17. Franklin G. Mixon & Benno Torgler & Kamal P. Upadhyaya, 2017. "Scholarly impact and the timing of major awards in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 112(3), pages 1837-1852, September.
    18. Konstantinos Chatzimichael & Pantelis Kalaitzidakis & Vangelis Tzouvelekas, 2017. "Measuring the publishing productivity of economics departments in Europe," Scientometrics, Springer;Akadémiai Kiadó, vol. 113(2), pages 889-908, November.
    19. James J. Heckman & Sidharth Moktan, 2020. "Publishing and promotion in economics - The tyranny of the Top Five," Vox eBook Chapters, in: Sebastian Galliani & Ugo Panizza (ed.), Publishing and Measuring Success in Economics, edition 1, volume 1, chapter 1, pages 23-32, Centre for Economic Policy Research.
    20. George Emm Halkos & Nickolaos G. Tzeremes, 2011. "Measuring economic journals’ citation efficiency: a data envelopment analysis approach," Scientometrics, Springer;Akadémiai Kiadó, vol. 88(3), pages 979-1001, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:scient:v:104:y:2015:i:1:d:10.1007_s11192-015-1581-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.