Methodological quality in clinical trials and bibliometric indicators: no evidence of correlations

Citation frequencies and journal impact factors (JIFs) are being used more and more to assess the quality of research and allocate research resources. If these bibliometric indicators are not an adequate predictor of research quality, there could be severe negative consequences for research. To analyse to which extent citation frequencies and journal impact factors correlate with the methodological quality of clinical research articles included in an SBU systematic review of antibiotic prophylaxis in surgery. All 212 eligible original articles were extracted from the SBU systematic review “Antibiotic Prophylaxis in Surgery” and categorized according to their methodological rigourness as high, moderate or low quality articles. Median of citation frequencies and JIFs were compared between the methodological quality groups using Kruskal–Wallis non-parametric test. An in-depth study of low-quality studies with higher citation frequencies/JIFs was also conducted. No significant differences were found in median citation frequencies (p = 0.453) or JIFs (p = 0.185) between the three quality groups. Studies that had high citation frequencies/JIFs but were assessed as low-quality lacked control groups, had high dropout rates or low internal validity. This study of antibiotic prophylaxis in surgery does not support the hypothesis that bibliometric indicators are a valid instrument for assessing methodological quality in clinical trials. This is a worrying observation, since bibliometric indicators have a major influence on research funding. However, further studies in other areas are needed.