On the Variability of Best Critical Regions and the Curvature of Exponential Models

The Neyman-Pearson Lemma usually implies that the best critical region (BCR) for testing a given hypothesis varies with the alternative. For this reason, most commonly used tests are justified either by replacing the objective of maximum power by a weaker objective such as maximum slope of the power function at the null, or by resorting to some other testing principle, such as a likelihood ratio test. However, BCRs that vary with the alternative may sometimes in fact vary very little, and in this 'paper I suggest a measure of the variability of such regions. Some examples confirm the conjecture that there are cases where the BCRs vary very little. Section 4 of the paper applies this idea to the family of one-parameter curved exponential models, and relates it to Efron's [41 notion that inference in highly curved models is likely to be more difficult that it is in uncurved or linear exponential models. Finally, we discuss testing tactics in problems where it can be shown that the BCRs for the problem vary very little.