Logistic Regression in Cases of Separation by Means of Penalized Maximum Likelihood Estimation

Users of –logit- or –logistic- occasionally encounter instances in which one or more predictors perfectly predict one or both outcomes (a condition called separation), or in which some outcomes are completely determined (quasicomplete separation). Finite maximum likelihood estimates do not exist under conditions of separation. Exact logistic regression with –exlogistic- can serve as an alternative in these circumstances, but is sometimes not feasible. In the 1990s, David Firth proposed a type of penalization for reducing bias of maximum likelihood estimates in generalized linear models by means of modifying the score equations. Firth’s method has the interpretation of penalized maximum likelihood when the canonical link function is used, such as in logistic regression. In this decade, Georg Heinze and colleagues have explored this technique as a solution to the problem of separation in logistic regression. A Stata implementation, -firthlogit-, which maximizes the log penalized likelihood using –ml-, is described here. Model fitting and predictions, inference with penalized likelihood ratio tests, and construction of profile penalized likelihood confidence intervals is illustrated using examples where –logit- and –logistic- either balk or do not give finite maximum likelihood estimates, and where exact logistic regression is problematic because of memory requirements or degenerate conditional distributions.