Assessing the quality of “Furfine-based” algorithms

To conduct academic research on the federal funds (fed funds) market, historically one of the most important financial markets in the U.S., some empirical economists have used market level measures published by the Markets Group at the Federal Reserve Bank of New York (FRBNY). To obtain more disaggregate data, some researchers have relied on a separate source of information: individual transactions inferred indirectly from an algorithm based on the work of Furfine (1999). To date, however, the accuracy of this algorithm has not been formally established. In this paper, we conduct a test aimed at assessing the ability of the algorithm to identify correctly individual overnight fed funds transactions conducted by two banks, which are among the most active in the fed funds market. The results are discouraging. We estimate the average type I and type II errors from 2007 to 2011 to be 81% and 23%, respectively. Furthermore, we argue that these errors i) apply to almost half of the algorithm's output, ii) introduce systematic biases, and iii) may not subside when the algorithm's output is aggregated. Our results therefore raise serious concerns about the appropriateness of using the algorithm's output to study the fed funds market. Because the FRBNY Markets Group relies on a different source of data than the algorithm output, our results have no bearing on their understanding of the fed funds market and their calculation of market level measures, including the effective fed funds rate.