Following the Crowd: Literature Support and the Capabilities of Autonomous Research Agents

Machine-learning models often perform poorly when asked to generalize beyond the support of the training distribution. This paper asks whether the same limitation shapes the research capabilities of autonomous large language model (LLM) agents: do they perform better when generating papers that follow research paradigms already well represented in the literature? I study this question using evidence from the Autonomous Policy Evaluation (APE) project, an open platform developed by the Social Catalyst Lab at the University of Zurich in which LLM agents generate empirical economic policy papers and compete in a tournament-style evaluation against human-written benchmarks. I construct a measure of literature support by locating each paper abstract in the semantic space of economics using a comprehensive corpus of English-language economics abstracts from OpenAlex. This measure captures whether a paper lies in a crowded or sparse region of the discipline’s existing research landscape. I then test whether literature support predicts tournament performance. I find that literature support shows a statistically significant positive association with performance for AI-generated papers, but not for human-written papers. Because outcomes are assigned by an LLM judge, this relationship could partly reflect evaluation bias toward more familiar topics. However, the absence of a comparable pattern among human papers suggests that the result is not purely judge-side. The evidence is more consistent with a production-side interpretation: autonomous research agents perform better when operating in areas that are more densely represented in the existing literature and, plausibly, in model training data. The findings shed some light on both the promise and the limits of agentic LLM systems as producers of scientific research.