Does economic policy matter? A note on the narrative approach and exact inference

This note examines uncertainty in time-series inference from rare episodes, focusing on the narrative approach. A small number of randomly drawn episodes may falsely suggest policy effects because they are associated with macroeconomic shocks that do not cancel out in inference. We illustrate this using Fisher-style exact inference. Applying our test to Romer and Romer’s (2023) analysis, we find substantial uncertainty. Although the unemployment rate’s peak response to an identified monetary contraction exceeds the 95-percent confidence bands of the counterfactual distribution based on randomly drawn months—suggesting systematic policy effects—this finding is reversed once additional controls are included.