Uncertainty and Individual Discretion in Allocating Research Funds

There is a long-standing tradition in public research funding agencies of distributing funds via peer review, which aggregates evaluations of proposed research ideas from a group of external experts. Despite complaints that this process is biased against novel ideas, there is poor understanding of an alternative system that may overcome this bias: the use of individual discretion. Here, we conduct the first quantitative study of how individual discretion affects a research funding portfolio. Using internal project selection data from the Advanced Research Projects Agency-Energy (ARPA-E), we describe how a portfolio of projects selected by individual discretion differs from a portfolio of projects selected by traditional peer review. We show that ARPA-E program directors tend to fund proposals with greater disagreement among experts, and they also appear to prefer proposals described in reviewer comments as “creative.” These choices do not result in a significant tradeoff with short-term project performance, and they enable ARPA-E to fund more uncertain and creative research ideas, which supports the agency’s mission of pursuing novel ideas for transformational energy technology.