Unusually Small F-Statistic in Analysis of Variance and Regression Analysis: A Warning in Design of Experiments and Regression

All textbooks and articles dealing with classical tests in the context of linear models stress the implications of a significantly large F-ratio since it indicates that the mean square for whatever effect is being evaluated contains significantly more than just error variation. In general, though, with one minor exception, all texts and articles, to the authors' knowledge, ignore the implications of an F-ratio that is significantly smaller than one would expect due to chance alone. Why this is so difficult to explain since such an occurrence is similar to a range value falling below the lower limit on a control chart for variation or a p-value falling below the lower limit on a control chart for proportion defective. In both of those cases the small value represents an unusual and significant occurrence and, if valid, a process change that indicates an improvement. Therefore, it behooves the quality manager to determine what that change is in order to have it continue. In the case of a significantly small F-ratio some problem may be indicated that requires the designer of the experiment to identify it, and to take “corrective action.” While graphical procedures are available for helping to identify some of the possible problems that are discussed they are somewhat subjective when deciding if one is looking at an actual effect; e.g., interaction, or whether the result is merely due to random variation. A significantly small F-ratio can be used to support conclusions based on the graphical procedures by providing a level of statistical significance as well as serving as a warning flag or warning that problems may exist in the design and/or analysis.