IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v178y2015i4p1025-1049.html
   My bibliography  Save this article

Ranking scientific journals via latent class models for polytomous item response data

Author

Listed:
  • Francesco Bartolucci
  • Valentino Dardanoni
  • Franco Peracchi

Abstract

type="main" xml:id="rssa12106-abs-0001"> We propose a model-based strategy for ranking scientific journals starting from a set of observed bibliometric indicators that represent imperfect measures of the unobserved ‘value’ of a journal. After discretizing the available indicators, we estimate an extended latent class model for polytomous item response data and use the estimated model to cluster journals. We illustrate our approach by using the data from the Italian research evaluation exercise that was carried out for the period 2004–2010, focusing on the set of journals that are considered relevant for the subarea statistics and financial mathematics. Using four bibliometric indicators (IF, IF5, AIS and the h-index), some of which are not available for all journals, and the information contained in a set of covariates, we derive a complete ordering of these journals. We show that the methodology proposed is relatively simple to implement, even when the aim is to cluster journals into a small number of ordered groups of a fixed size. We also analyse the robustness of the obtained ranking with respect to different discretization rules.

Suggested Citation

  • Francesco Bartolucci & Valentino Dardanoni & Franco Peracchi, 2015. "Ranking scientific journals via latent class models for polytomous item response data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 178(4), pages 1025-1049, October.
  • Handle: RePEc:bla:jorssa:v:178:y:2015:i:4:p:1025-1049
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/rssa.2015.178.issue-4
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Christian Zimmermann, 2013. "Academic Rankings with RePEc," Econometrics, MDPI, Open Access Journal, vol. 1(3), pages 1-32, December.
    2. Bertocchi, Graziella & Gambardella, Alfonso & Jappelli, Tullio & Nappi, Carmela A. & Peracchi, Franco, 2015. "Bibliometric evaluation vs. informed peer review: Evidence from Italy," Research Policy, Elsevier, vol. 44(2), pages 451-466.
    3. Chia-Lin Chang & Esfandiar Maasoumi & Michael McAleer, 2016. "Robust Ranking of Journal Quality: An Application to Economics," Econometric Reviews, Taylor & Francis Journals, vol. 35(1), pages 50-97, January.
    4. Chang, C-L. & McAleer, M.J. & Oxley, L., 2010. "Journal Impect Factor Versus Eigenfactor and Article Influence," Econometric Institute Research Papers EI 2010-67, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    5. Francesco Bartolucci, 2007. "A class of multidimensional IRT models for testing unidimensionality and clustering items," Psychometrika, Springer;The Psychometric Society, vol. 72(2), pages 141-157, June.
    6. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    7. Gilles Celeux & Gilda Soromenho, 1996. "An entropy criterion for assessing the number of clusters in a mixture model," Journal of Classification, Springer;The Classification Society, vol. 13(2), pages 195-212, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.
    2. Battistin, Erich & Ovidi, Marco, 2017. "Rising Stars," IZA Discussion Papers 11198, Institute of Labor Economics (IZA).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Francesco Bartolucci & Valentino Dardanoni & Franco Peracchi, 2013. "Ranking Scientific Journals via Latent Class Models for Polytomous Item Response," EIEF Working Papers Series 1313, Einaudi Institute for Economics and Finance (EIEF), revised May 2013.
    2. Lutz Bornmann & Alexander Butz & Klaus Wohlrabe, 2018. "What are the top five journals in economics? A new meta-ranking," Applied Economics, Taylor & Francis Journals, vol. 50(6), pages 659-675, February.
    3. Hsieh, Chih-Sheng & König, Michael & Liu, Xiaodong & Zimmermann, Christian, 2018. "Superstar Economists: Coauthorship networks and research output," CEPR Discussion Papers 13239, C.E.P.R. Discussion Papers.
    4. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    5. McAleer, M.J. & Oláh, J. & Popp, J., 2018. "Pros and Cons of the Impact Factor in a Rapidly Changing Digital World," Econometric Institute Research Papers EI2018-11, Erasmus University Rotterdam, Erasmus School of Economics (ESE), Econometric Institute.
    6. Chia-Lin Chang & Michael McAleer, 2013. "Ranking Leading Econometrics Journals Using Citations Data from ISI and RePEc," Econometrics, MDPI, Open Access Journal, vol. 1(3), pages 1-19, November.
    7. Antonio García-Romero & Daniel Santín & Gabriela Sicilia, 2016. "Another brick in the wall: a new ranking of academic journals in Economics using FDH," Scientometrics, Springer;Akadémiai Kiadó, vol. 107(1), pages 91-101, April.
    8. László Kóczy & Alexandru Nichifor, 2013. "The intellectual influence of economic journals: quality versus quantity," Economic Theory, Springer;Society for the Advancement of Economic Theory (SAET), vol. 52(3), pages 863-884, April.
    9. Lutz Bornmann & Klaus Wohlrabe, 2019. "Normalisation of citation impact in economics," Scientometrics, Springer;Akadémiai Kiadó, vol. 120(2), pages 841-884, August.
    10. William C. Horrace & Christopher F. Parmeter, 2017. "Accounting for Multiplicity in Inference on Economics Journal Rankings," Southern Economic Journal, John Wiley & Sons, vol. 84(1), pages 337-347, July.
    11. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    12. Battistin, Erich & Ovidi, Marco, 2017. "Rising Stars," IZA Discussion Papers 11198, Institute of Labor Economics (IZA).
    13. Wohlrabe, Klaus, 2016. "Taking the Temperature: A Meta-Ranking of Economics Journals," MPRA Paper 68933, University Library of Munich, Germany.
    14. Sascha Baghestanian & Sergey V. Popov, 2017. "Alma Mat(t)er(s): Determinants of Early Career Success in Economics," Economics Working Papers 17-02, Queen's Management School, Queen's University Belfast.
    15. Gianfranco Di Vaio & Jacob Louis Weisdorf, 2010. "Ranking economic history journals: a citation-based impact-adjusted analysis," Cliometrica, Journal of Historical Economics and Econometric History, Association Française de Cliométrie (AFC), vol. 4(1), pages 1-17, January.
    16. Javier Ruiz-Castillo, 2008. "Economics research in Spain during the 1990s: a literature review," Spanish Economic Review, Springer;Spanish Economic Association, vol. 10(3), pages 221-249, September.
    17. Scrucca, Luca, 2016. "Identifying connected components in Gaussian finite mixture models for clustering," Computational Statistics & Data Analysis, Elsevier, vol. 93(C), pages 5-17.
    18. Klaus Wohlrabe, 2018. "Einige Anmerkungen zum FAZ-Ökonomenranking 2018," ifo Schnelldienst, ifo Institute - Leibniz Institute for Economic Research at the University of Munich, vol. 71(20), pages 29-33, October.
    19. Marianna Virtanen & Jussi Vahtera & Jenny Head & Rosemary Dray-Spira & Annaleena Okuloff & Adam G Tabak & Marcel Goldberg & Jenni Ervasti & Markus Jokela & Archana Singh-Manoux & Jaana Pentti & Marie , 2015. "Work Disability among Employees with Diabetes: Latent Class Analysis of Risk Factors in Three Prospective Cohort Studies," PLOS ONE, Public Library of Science, vol. 10(11), pages 1-14, November.
    20. Adrian Bruhin & Ernst Fehr & Daniel Schunk, 2019. "The many Faces of Human Sociality: Uncovering the Distribution and Stability of Social Preferences," Journal of the European Economic Association, European Economic Association, vol. 17(4), pages 1025-1069.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:178:y:2015:i:4:p:1025-1049. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley Content Delivery). General contact details of provider: http://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.