IDEAS home Printed from
MyIDEAS: Login to save this article or follow this journal

Relative Performance Measurement of Researchers: The Impact of Data Source Selection

  • Matthias Meyer
  • Rüdiger W. Waldkirch
  • Michael A. Zaggl
Registered author(s):

    We investigate the impact of choosing between four data sources on the relative performance measurement of scholars’ research output. Our results show that rankings can differ considerably and that the observed correlations are often even lower than those reported in previous studies. We show that data source selection generally has a higher effect than does measure selection and that performance measures based on Google Scholar might provide information that complements the more traditional data sources. We find greater differences for accounting, which suggests that decision makers must pay more careful atten-tion when using rankings in this field.

    If you experience problems downloading a file, check if you have the proper application to view it first. In case of further problems read the IDEAS help page. Note that these files are not on the IDEAS site. Please be patient as the files may be large.

    File URL:
    Download Restriction: no

    Article provided by LMU Munich School of Management in its journal Schmalenbach Business Review.

    Volume (Year): 64 (2012)
    Issue (Month): 4 (October)
    Pages: 308-330

    in new window

    Handle: RePEc:sbr:abstra:v:64:y:2012:i:4:p:308-330
    Contact details of provider: Postal: Geschwister-Scholl-Platz 1, 80539 Muenchen
    Phone: 0049 89 2180 2166
    Fax: 0049 89 2180 6327
    Web page:

    More information through EDIRC

    References listed on IDEAS
    Please report citation or reference errors to , or , if you are the registered author of the cited work, log in to your RePEc Author Service profile, click on "citations" and make appropriate adjustments.:

    as in new window
    1. Henrekson, Magnus & Waldenström, Daniel, 2007. "How Should Research Performance be Measured? A Study of Swedish Economists," Working Paper Series 712, Research Institute of Industrial Economics, revised 08 Nov 2007.
    2. Bernard Raffournier & Alain Schatt, 2010. "Is European Accounting Research Fairly Reflected in Academic Journals? An Investigation of Possible Non-mainstream and Language Barrier Biases," European Accounting Review, Taylor & Francis Journals, vol. 19(1), pages 161-190.
    3. Soenke Albers, 2011. "Discussion of “Quantitative and Qualitative Rankings of Scholars”: Esteem Indicators: Membership in Editorial Boards or Honorary Doctorates," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 92-98, January.
    4. Robert Hofmeister & Heinrich W. Ursprung, 2008. "Das Handelsblatt Ökonomen-Ranking 2007: Eine kritische Beurteilung," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 9(3), pages 254-266, 08.
    5. Bruno S. Frey & Katja Rost, 2008. "Do Rankings Reflect Research Quality?," CREMA Working Paper Series 2008-22, Center for Research in Economics, Management and the Arts (CREMA).
    6. Uschi Backes-Gellner, 2011. "Discussion of “Quantitative and Qualitative Rankings of Scholars”: Rankings upon Rankings – and no End in Sight –," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 99-108, January.
    7. Dilger, Alexander, 2009. "Rankings von Zeitschriften und Personen in der BWL," IÖB-Diskussionspapiere 5/09, University of Münster, Institute for Economic Education.
    8. Werner Reinartz, 2011. "Discussion of “Quantitative and Qualitative Rankings of Scholars”: Feeling Good or Feeling Right?," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 109-114, January.
    9. Bruno S. Frey, 2007. "Evaluierungen, Evaluierungen H Evaluitis," Perspektiven der Wirtschaftspolitik, Verein für Socialpolitik, vol. 8(3), pages 207-220, 08.
    10. Holmstrom, Bengt & Milgrom, Paul, 1991. "Multitask Principal-Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design," Journal of Law, Economics and Organization, Oxford University Press, vol. 7(0), pages 24-52, Special I.
    11. Bruno S. Frey & Margit Osterloh, 2006. "Evaluations: Hidden Costs, Questionable Benefits, and Superior Alternatives," CREMA Working Paper Series 2006-23, Center for Research in Economics, Management and the Arts (CREMA), revised Oct 2006.
    12. Katja Rost & Bruno S. Frey, 2011. "Quantitative and Qualitative Rankings of Scholars," Schmalenbach Business Review (sbr), LMU Munich School of Management, vol. 63(1), pages 63-91, January.
    13. Matthias Krapf, 2010. "Research evaluation and journal quality weights: Much ado about nothing?," Working Paper Series of the Department of Economics, University of Konstanz 2010-02, Department of Economics, University of Konstanz.
    14. Daniel B. Klein & Eric Chiang, 2004. "The Social Science Citation Index: A Black Box—with an Ideological Bias?," Econ Journal Watch, Econ Journal Watch, vol. 1(1), pages 134-165, April.
    15. Sönke Albers, 2009. "Misleading Rankings of Research in Business," German Economic Review, Verein für Socialpolitik, vol. 10, pages 352-363, 08.
    Full references (including those not matched with items on IDEAS)

    This item is not listed on Wikipedia, on a reading list or among the top items on IDEAS.

    When requesting a correction, please mention this item's handle: RePEc:sbr:abstra:v:64:y:2012:i:4:p:308-330. See general information about how to correct material in RePEc.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: (sbr)

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If references are entirely missing, you can add them using this form.

    If the full references list an item that is present in RePEc, but the system did not link to it, you can help with this form.

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your profile, as there may be some citations waiting for confirmation.

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    This information is provided to you by IDEAS at the Research Division of the Federal Reserve Bank of St. Louis using RePEc data.