Robustness? Range Tests for Equality and Equivalence Across Specifications

Applied economists routinely compare estimates across specifications, observe that they are "similar," and conclude that their results are "robust." This common procedure makes an implicit inferential claim about the range of estimates, but usually does not account for their joint sampling distribution when the estimates are generated from the same data. I formalize informal practice with two bootstrap statistics that account for the dependence among estimates. The minimum equivalence bound is the smallest tolerance within which the estimates can be judged equivalent. The range-based equality p-value tests whether the estimates are statistically distinguishable. Together they distinguish failure to detect differences from affirmative evidence of agreement. Simulations across a range of designs show approximately correct size and demonstrate that common visual rules often provide false comfort. Applications to five prominent papers using common identification strategies validate some conventional robustness claims while revealing cases in which apparent agreement reflects imprecision rather than stability. In a pre-registered survey of CEPR and NBER affiliates, expert judgments align with the framework in obvious cases but diverge in intermediate cases, where the verdict depends on the joint sampling distribution rather than on the visible spread of point estimates alone. I suggest that the minimum equivalence bound the equality p-value be reported whenever multiple specifications are presented as evidence of robustness.