IDEAS home Printed from https://ideas.repec.org/p/mia/wpaper/2016-08.html
   My bibliography  Save this paper

Accounting for Multiplicity in Inference on Economics Journal Rankings

Author

Listed:
  • William Horrace

    (Syracuse University)

  • Christopher Parmeter

    (University of Miami)

Abstract

Nearly all journal rankings are constructed as deterministic, yet they are clearly stochastic. To remedy this, Stern (2013) calculates standard errors and performs inference on the ranks of economics journal based on impact factors. However, this inference is essentially a series of univariate tests that do not control for the overall error rate of the inferential exercise. Using multiple comparison and ranking procedures, we reevaluate the inference while controlling for the multiplicity (the implied multivariate inference) in the rank statistic. The results are compared and di_erences highlighted.

Suggested Citation

  • William Horrace & Christopher Parmeter, 2016. "Accounting for Multiplicity in Inference on Economics Journal Rankings," Working Papers 2016-08, University of Miami, Department of Economics.
  • Handle: RePEc:mia:wpaper:2016-08
    as

    Download full text from publisher

    File URL: http://bus.miami.edu/_assets/files/repec/WP2016-08.pdf
    File Function: First version, 2016
    Download Restriction: no

    Other versions of this item:

    References listed on IDEAS

    as
    1. Chia-Lin Chang & Esfandiar Maasoumi & Michael McAleer, 2016. "Robust Ranking of Journal Quality: An Application to Economics," Econometric Reviews, Taylor & Francis Journals, vol. 35(1), pages 50-97, January.
    2. William Horrace & Seth Richards-Shubik & Ian Wright, 2015. "Expected efficiency ranks from parametric stochastic frontier models," Empirical Economics, Springer, vol. 48(2), pages 829-848, March.
    3. Baltagi, Badi H., 2007. "Worldwide Econometrics Rankings: 1989 2005," Econometric Theory, Cambridge University Press, vol. 23(05), pages 952-1012, October.
    4. William C. Horrace, 2005. "On the ranking uncertainty of labor market wage gaps," Journal of Population Economics, Springer;European Society for Population Economics, vol. 18(1), pages 181-187, September.
    5. Ignacio Palacios-Huerta & Oscar Volij, 2004. "The Measurement of Intellectual Influence," Econometrica, Econometric Society, vol. 72(3), pages 963-977, May.
    6. Hong Li & Wei Ning, 2012. "Multiple comparisons with a control under heteroscedasticity," Journal of Applied Statistics, Taylor & Francis Journals, vol. 39(10), pages 2275-2283, June.
    7. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    8. Kalaitzidakis, P. & Mamuneas, T.P. & Stengos, T., 2003. "Rankings of Academic Journals and Institutions," Working Papers 2003-8, University of Guelph, Department of Economics and Finance.
    9. Alfonso Flores-Lagunes & William C. Horrace & Kurt E. Schnier, 2007. "Identifying technically efficient fishing vessels: a non-empty, minimal subset approach," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(4), pages 729-745.
    10. William C. Horrace & Peter Schmidt, 2000. "Multiple comparisons with the best, with economic applications," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(1), pages 1-26.
    11. William Horrace & Joseph Marchand & Timothy Smeeding, 2008. "Ranking inequality: Applications of multivariate subset selection," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 6(1), pages 5-32, March.
    12. Pierre-Philippe Combes & Laurent Linnemer, 2010. "Inferring Missing Citations: A Quantitative Multi-Criteria Ranking of all Journals in Economics," Working Papers halshs-00520325, HAL.
    13. David I. Stern, 2013. "Uncertainty Measures for Economics Journal Impact Factors," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 173-189, March.
    14. Pantelis Kalaitzidakis & Theofanis P. Mamuneas & Thanasis Stengos, 2003. "Rankings of Academic Journals and Institutions in Economics," Journal of the European Economic Association, MIT Press, vol. 1(6), pages 1346-1366, December.
    15. William C. Horrace, 2006. "Selection Procedures for Economics," Applied Economics Quarterly (formerly: Konjunkturpolitik), Duncker & Humblot, Berlin, vol. 52(4), pages 357-374.
    Full references (including those not matched with items on IDEAS)

    More about this item

    Keywords

    Ranking and Selection; Subset Selection; Multiple Comparisons; Journal Impact Factors Publication Status: Under Review;

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • C12 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General - - - Hypothesis Testing: General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:mia:wpaper:2016-08. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Christopher Parmeter). General contact details of provider: http://edirc.repec.org/data/demiaus.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.