A Replication of “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” (American Journal of Agricultural Economics, 2019)

This study replicates Brown, Lambert, and Wojan (2019) (BLW) and their bootstrapping procedure for calculating ex post power. At the current time there is no generally accepted way of calculating ex post power. BLW provide a novel method for doing this though they provide little justification for their method or evidence of its reliability. My replication makes three contributions. First, it confirms that the data and code provided with their paper is sufficient to reproduce their results. Second, it performs two robustness checks to determine if slight alterations to their procedure affect their results. I determine that including a constant term in their procedure does not affect the results. On the other hand, using a different bootstrapping procedure produces somewhat different results. However, without any ground truth to use as a benchmark, one cannot say which bootstrapping procedure is better. My third contribution is that I use Monte Carlo experiments to assess the performance of BLW’s method. My experimental results indicate that their method is unbiased and produces a relatively narrow range of estimates. This suggests that BLW’s method may provide a reliable method for researchers to calculate ex post power, though further investigation needs to be done.