Bias and Overconfidence in Parametric Models of Interactive Processes

We assess the ability of logit, probit and numerous other parametric models to test a hypothesis that two variables interact in influencing the probability that some event will occur [Pr(Y)] in what we believe is a very common situation: when one's theory is insufficiently strong to dictate a specific functional form for the data generating process. Using Monte Carlo analysis, we find that many models yield overconfident inferences by generating 95% confidence intervals for estimates of the strength of interaction that are far too narrow, but that some logit and probit models produce approximately accurate intervals. Yet all models we study generate point estimates for the strength of interaction with large enough average error to often distort substantive conclusions. We propose an approach to make the most effective use of logit and probit in the situation of specification uncertainty, but argue that nonparametric models may ultimately prove to be superior.