E-governance, Accountability, and Leakage in Public Programs: Experimental Evidence from a Financial Management Reform in India - A Reproducibility and Replicability Assessment of Banerjee et al. (2020)

Banerjee, Duflo, Imbert, Mathew, and Pande (2020) study a "just-in-time" payment reform in India, first using a randomized controlled trial in Bihar and then examining the reform's subsequent nationwide, non-randomized rollout. The RCT finds lower programme expenditure, longer payment delays, more days worked, and fewer "ghost workers," while the difference-in-differences analysis of the national scale-up finds lower labour expenditures after implementation. This paper replicates these findings. The computational reproduction is largely successful, reproducing most tables of the original paper. However, three errors are identified: two minor, and one that shifts the significance of the ghost-worker outcome from the 5% to the 10% level. Robustness checks show that two of the five selected outcomes, programme expenditure and payment delays, are very robust. The remaining three outcomes are sensitive even to relatively modest analytical variations, several of which were already considered in the original paper or are straightforward alternatives to the published specifications. For number of days worked and the ghost-worker outcome, most alternative specifications are insignificant at the 5% level, and several remain insignificant even at the 10% level. Likewise, the national scale-up outcome, labour expenditure, is only partially robust, with most alternative specifications insignificant. These findings leave the evidence for some direct effects of the reform intact, but weaken the evidence for reduced leakage and for the reform's effectiveness when implemented at national scale.