How much do journal titles tell us about the academic interest and relevance of economic research? An empirical analysis

Unlike in other disciplines, research output in economics is commonly measured based on the journal titles in which an author has published. Here, I examine how much output measures based on journal titles tell us about the academic interest and relevance of economic papers as measured by citation frequency. Using data from the 2008 Handelsblatt ranking of economists in German speaking countries and interdisciplinary citation data from the Web of Science, I find that researcher scores based on journal titles explain only about one fourth of the variation (variance) in article citations. When the top 10 (20) percent of the researchers according to journal title scores are excluded, the percentage of explained variation in citation frequency drops to 5 (3) percent. These findings empirically confirm the hypothesis that the measures of research output in economics promote narrow and complacent work that is of interest to few, even among an academic audience. They suggest that responsible hiring committees and funding institutions should re-examine existing standards in evaluation and abandon the heavy reliance on journal titles as a measure of individual research output.