A Note on the Theme of Too Many Instruments

The “difference” and “system” generalized method of moments (GMM) estimators for dynamic panel models are growing steadily in popularity. The estimators are designed for panels with short time dimensions (T), and by default they generate instruments sets whose number grows quadratically in T. The dangers associated with having many instruments relative to observations are documented in the applied literature. The instruments can overfit endogenous variables, failing to expunge their endogenous components and biasing coefficient estimates. Meanwhile they can vitiate the Hansen J test for joint validity of those instruments, as well as the difference-in-Sargan/Hansen test for subsets of instruments. The weakness of these specification tests is a particular concern for system GMM, whose distinctive instruments are only valid under a non-trivial assumption. Judging by current practice, many researchers do not fully appreciate that popular implementations of these estimators can by default generate results that simultaneously are invalid yet appear valid. The potential for type I errors—false positives—is therefore substantial, especially after amplification by publication bias. This paper explains the risks and illustrates them with reference to two early applications of the estimators to economic growth, Forbes (2000) on income inequality and Levine, Loayza, and Beck (LLB, 2000) on financial sector development. Endogenous causation proves hard to rule out in both papers. Going forward, for results from these GMM estimators to be credible, researchers must report the instrument count and aggressively test estimates and specification test results for robustness to reductions in that count.