The effect of public funding on research output: the New Zealand Marsden Fund

We estimate the impact of participating in the NZ Marsden Fund on research output trajectories, by comparing the subsequent performance of funded researchers to those who submitted proposals but were not funded. We control for selection bias using the evaluations of the proposals generated by the grant selection process. We carry out the analysis in two data frames. First we consider the researcher teams behind 1263 second-round proposals submitted 2003-2008, and look at the post-proposal publication and citation performance of the team as a whole, as a function of pre-proposal performance, the ranking of the proposal by the panel, and the funding. This estimation does not deal with individual researchers’ multiple proposals and funding over time. To disentangle these effects, we consider the 1500 New Zealand researchers who appeared on any of these proposals, and estimate a model predicting annual individual performance as a function of previous performance, recent proposal activity, ranking of any recent proposals, and funding received through recent proposals. Overall, we find that funding is associated with a 6-15% increase in publications and a 22-26% increase in citation-weighted papers for research teams. For individuals, funding is associated with a 3-5% increase in annual publications, and a 5-8% increase in citation-weighted papers for 5 years after grant; however, the lag structure and persistence of this effect post-grant is difficult to pin down. Surprisingly, we find no systematic evidence that the evaluation of proposals by the Marsden system is predictive of subsequent success. We conclude that the Marsden Fund is modestly successful in increasing scientific performance, but that the selection process does not appear to be effective in discriminating among second-round proposals in terms of their likely success.