IDEAS home Printed from https://ideas.repec.org/a/bla/jorssa/v174y2011i4p857-879.html
   My bibliography  Save this article

A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?

Author

Listed:
  • Lutz Bornmann
  • Rüdiger Mutz
  • Werner Marx
  • Hermann Schier
  • Hans‐Dieter Daniel

Abstract

No abstract is available for this item.

Suggested Citation

  • Lutz Bornmann & Rüdiger Mutz & Werner Marx & Hermann Schier & Hans‐Dieter Daniel, 2011. "A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high profile journal select manuscripts that are highly cited after publication?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 857-879, October.
  • Handle: RePEc:bla:jorssa:v:174:y:2011:i:4:p:857-879 DOI: j.1467-985X.2011.00689.x
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1111/j.1467-985X.2011.00689.x
    Download Restriction: Access to full text is restricted to subscribers.

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Robin Shealy & William Stout, 1993. "A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF," Psychometrika, Springer;The Psychometric Society, vol. 58(2), pages 159-194, June.
    2. Meng, Chun-Lo & Schmidt, Peter, 1985. "On the Cost of Partial Observability in the Bivariate Probit Model," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 26(1), pages 71-85, February.
    3. Kristensen, Nicolai & Johansson, Edvard, 2008. "New evidence on cross-country differences in job satisfaction using anchoring vignettes," Labour Economics, Elsevier, vol. 15(1), pages 96-117, February.
    4. Greene,William H. & Hensher,David A., 2010. "Modeling Ordered Choices," Cambridge Books, Cambridge University Press, number 9780521194204, December.
    5. Nigel Rice & Silvana Robone & Peter Smith, 2011. "Analysis of the validity of the vignette approach to correct for heterogeneity in reporting health system responsiveness," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 12(2), pages 141-162, April.
    6. repec:cup:apsrev:v:97:y:2003:i:04:p:567-583_00 is not listed on IDEAS
    7. Alfonso Miranda & Sophia Rabe-Hesketh, 2005. "Maximum Likelihood Estimation of Endogenous Switching And Sample Selection Models for Binary, Count, And Ordinal Variables," Keele Economics Research Papers KERP 2005/14, Centre for Economic Research, Keele University.
    8. Arie Kapteyn & James P. Smith & Arthur van Soest, 2007. "Vignettes and Self-Reports of Work Disability in the United States and the Netherlands," American Economic Review, American Economic Association, vol. 97(1), pages 461-473, March.
    9. Giuseppe De Luca & Valeria Perotti, 2011. "Estimation of ordered response models with sample selection," Stata Journal, StataCorp LP, vol. 11(2), pages 213-239, June.
    10. Arthur van Soest & Tatiana Andreyeva & Arie Kapteyn & James P. Smith, 2011. "Self-Reported Disability and Reference Groups," NBER Chapters,in: Investigations in the Economics of Aging, pages 237-264 National Bureau of Economic Research, Inc.
    11. John Bound, 1991. "Self-Reported Versus Objective Measures of Health in Retirement Models," Journal of Human Resources, University of Wisconsin Press, vol. 26(1), pages 106-138.
    12. Giuseppe De Luca & Franco Peracchi, 2012. "Estimating Engel curves under unit and item nonresponse," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 27(7), pages 1076-1099, November.
    13. John Fitzgerald & Peter Gottschalk & Robert Moffitt, 1998. "An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics," Journal of Human Resources, University of Wisconsin Press, vol. 33(2), pages 251-299.
    14. Teresa Bago d’Uva & Maarten Lindeboom & Owen O’Donnell & Eddy van Doorslaer, 2011. "Slipping Anchor?: Testing the Vignettes Approach to Identification and Correction of Reporting Heterogeneity," Journal of Human Resources, University of Wisconsin Press, vol. 46(4), pages 875-906.
    15. Heckman, James, 2013. "Sample selection bias as a specification error," Applied Econometrics, Publishing House "SINERGIA PRESS", pages 129-137.
    16. Keane, Michael P, 1992. "A Note on Identification in the Multinomial Probit Model," Journal of Business & Economic Statistics, American Statistical Association, vol. 10(2), pages 193-200, April.
    17. repec:cup:apsrev:v:98:y:2004:i:01:p:191-207_00 is not listed on IDEAS
    18. Viola Angelini & Danilo Cavapozzi & Luca Corazzini & Omar Paccagnella, 2014. "Do Danes and Italians Rate Life Satisfaction in the Same Way? Using Vignettes to Correct for Individual-Specific Scale Biases," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 76(5), pages 643-666, October.
    19. Heckman, James J, 1978. "Dummy Endogenous Variables in a Simultaneous Equation System," Econometrica, Econometric Society, vol. 46(4), pages 931-959, July.
    20. Datta Gupta, Nabanita & Kristensen, Nicolai & Pozzoli, Dario, 2010. "External validation of the use of vignettes in cross-country health studies," Economic Modelling, Elsevier, vol. 27(4), pages 854-865, July.
    21. Francis Vella, 1998. "Estimating Models with Sample Selection Bias: A Survey," Journal of Human Resources, University of Wisconsin Press, vol. 33(1), pages 127-169.
    22. Teresa Bago d'Uva & Eddy Van Doorslaer & Maarten Lindeboom & Owen O'Donnell, 2008. "Does reporting heterogeneity bias the measurement of health disparities?," Health Economics, John Wiley & Sons, Ltd., vol. 17(3), pages 351-375.
    23. Arthur van Soest & Liam Delaney & Colm Harmon & Arie Kapteyn & James P. Smith, 2011. "Validating the use of anchoring vignettes for the correction of response scale differences in subjective questions," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(3), pages 575-595, July.
    24. Rebecca R. Andridge & Roderick J. A. Little, 2010. "A Review of Hot Deck Imputation for Survey Non-response," International Statistical Review, International Statistical Institute, vol. 78(1), pages 40-64, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Cova, Tânia F.G.G. & Jarmelo, Susana & Formosinho, Sebastião J. & de Melo, J. Sérgio Seixas & Pais, Alberto A.C.C., 2015. "Unsupervised characterization of research institutions with task-force estimation," Journal of Informetrics, Elsevier, vol. 9(1), pages 59-68.
    2. Abramo, Giovanni & D’Angelo, Ciriaco Andrea & Grilli, Leonardo, 2015. "Funnel plots for visualizing uncertainty in the research performance of institutions," Journal of Informetrics, Elsevier, vol. 9(4), pages 954-961.
    3. Wiltrud Kuhlisch & Magnus Roos & Jörg Rothe & Joachim Rudolph & Björn Scheuermann & Dietrich Stoyan, 2016. "A statistical approach to calibrating the scores of biased reviewers of scientific papers," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 79(1), pages 37-57, January.
    4. Bornmann, Lutz & Leydesdorff, Loet, 2012. "Which are the best performing regions in information science in terms of highly cited papers? Some improvements of our previous mapping approaches," Journal of Informetrics, Elsevier, vol. 6(2), pages 336-345.
    5. Mutz, Rüdiger & Daniel, Hans-Dieter, 2012. "Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor," Journal of Informetrics, Elsevier, vol. 6(2), pages 169-176.
    6. Wiltrud Kuhlisch & Magnus Roos & Jörg Rothe & Joachim Rudolph & Björn Scheuermann & Dietrich Stoyan, 2016. "A statistical approach to calibrating the scores of biased reviewers of scientific papers," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 79(1), pages 37-57, January.
    7. repec:spr:scient:v:89:y:2011:i:3:d:10.1007_s11192-011-0472-0 is not listed on IDEAS
    8. Lutz Bornmann, 2015. "Interrater reliability and convergent validity of F1000Prime peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(12), pages 2415-2426, December.
    9. Bornmann, Lutz & Leydesdorff, Loet, 2013. "The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000," Journal of Informetrics, Elsevier, vol. 7(2), pages 286-291.
    10. Bornmann, Lutz & Marx, Werner, 2013. "The proposal of a broadening of perspective in evaluative bibliometrics by complementing the times cited with a cited reference analysis," Journal of Informetrics, Elsevier, vol. 7(1), pages 84-88.
    11. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2016. "Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully," Journal of Informetrics, Elsevier, vol. 10(1), pages 312-327.
    12. A. I. M. Jakaria Rahman & Raf Guns & Loet Leydesdorff & Tim C. E. Engels, 2016. "Measuring the match between evaluators and evaluees: cognitive distances between panel members and research groups at the journal level," Scientometrics, Springer;Akadémiai Kiadó, vol. 109(3), pages 1639-1663, December.
    13. Bornmann, Lutz & Schier, Hermann & Marx, Werner & Daniel, Hans-Dieter, 2012. "What factors determine citation counts of publications in chemistry besides their quality?," Journal of Informetrics, Elsevier, vol. 6(1), pages 11-18.
    14. Bornmann, Lutz, 2013. "The problem of citation impact assessments for recent publication years in institutional evaluations," Journal of Informetrics, Elsevier, vol. 7(3), pages 722-729.
    15. JingJing Zhang & Jiancheng Guan, 2017. "Scientific relatedness and intellectual base: a citation analysis of un-cited and highly-cited papers in the solar energy field," Scientometrics, Springer;Akadémiai Kiadó, vol. 110(1), pages 141-162, January.
    16. Bornmann, Lutz & Leydesdorff, Loet & Wang, Jian, 2013. "Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P1," Journal of Informetrics, Elsevier, vol. 7(4), pages 933-944.
    17. Bornmann, Lutz & Leydesdorff, Loet & Mutz, Rüdiger, 2013. "The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits," Journal of Informetrics, Elsevier, vol. 7(1), pages 158-165.
    18. Raffaele Miniaci & Michele Pezzoni, 2015. "Is Publication in the Hands of Outstanding Scientists? A Study on the Determinants of Editorial Boards Membership in Economics," GREDEG Working Papers 2015-17, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), University of Nice Sophia Antipolis.
    19. repec:spr:scient:v:91:y:2012:i:3:d:10.1007_s11192-012-0647-3 is not listed on IDEAS
    20. Rüdiger Mutz & Lutz Bornmann & Hans-Dieter Daniel, 2015. "Testing for the fairness and predictive validity of research funding decisions: A multilevel multiple imputation for missing data approach using ex-ante and ex-post peer evaluation data from the Austr," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 66(11), pages 2321-2339, November.
    21. Bornmann, Lutz & Williams, Richard, 2013. "How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects," Journal of Informetrics, Elsevier, vol. 7(2), pages 562-574.
    22. Carole J. Lee & Cassidy R. Sugimoto & Guo Zhang & Blaise Cronin, 2013. "Bias in peer review," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(1), pages 2-17, January.
    23. repec:eee:infome:v:11:y:2017:i:3:p:704-712 is not listed on IDEAS
    24. Eisend, Martin & Schmidt, Susanne, 2014. "The influence of knowledge-based resources and business scholars’ internationalization strategies on research performance," Research Policy, Elsevier, vol. 43(1), pages 48-59.
    25. Bornmann, Lutz & Stefaner, Moritz & de Moya Anegón, Felix & Mutz, Rüdiger, 2014. "What is the effect of country-specific characteristics on the research performance of scientific institutions? Using multi-level statistical models to rank and map universities and research-focused in," Journal of Informetrics, Elsevier, vol. 8(3), pages 581-593.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssa:v:174:y:2011:i:4:p:857-879. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (Wiley-Blackwell Digital Licensing) or (Christopher F. Baum). General contact details of provider: http://edirc.repec.org/data/rssssea.html .

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service hosted by the Research Division of the Federal Reserve Bank of St. Louis . RePEc uses bibliographic data supplied by the respective publishers.