Using theory to define a computationally tractable specification space in confirmatory factor modeling

Researchers constructing measurement models must decide how to proceed when an initial specification fits poorly. Common approaches include search algorithms that optimize fit, and piecemeal changes to the item list or the error specification. The former approach may yield a good-fitting model that is inconsistent with theory, or may fail to identify the best-fitting model due to local optimization issues. The latter suffers from poor reproducibility and may also fail to identify the optimal model. We outline a new approach that defines a computationally tractable specification space based on theory. We use the example of a hypothesized latent variable with 25 candidate indicators divided across five content areas. Using Stata’s –tuples- command, we identify all combinations of indicators containing >=1 indicator per content area. In our example, this yields 7,294 models. We estimate each model on a derivation dataset and select candidate models with fit statistics that are acceptable, or could be rendered acceptable by permitting correlated errors. Eight models fit these criteria. We evaluate modification indices, re-specify if there is theoretical justification for correlated errors, and select a final model based on fit statistics. In contrast to other methods, this approach is easily replicable and may result in a model that is consistent with theory and has acceptable fit.