Likelihood Statistical Inference

Maximum-likelihood estimation permits elegant testing of the null hypothesis that the unknown parameters satisfy a linear constraint. The context is asymptotic statistical inference when the sample size is large. The Wald test checks directly how closely the maximum-likelihood estimate satisfies the constraint. The Wilks likelihood-ratio test maximizes the likelihood subject to the constraint and checks how much it reduces the likelihood. Rao's score test looks at the derivative of the log-likelihood at the constrained maximum-likelihood estimate and checks whether this derivative is close to zero. The Aitchison and Silvey Lagrange-multiplier test checks whether the Lagrange multiplier in the constrained maximum-likelihood is close to zero. Asymptotically, these four tests are equivalent. We model the estimation in Euclidean space, in which the inner product embodies the negative of the matrix of second derivatives, in a coordinate-free formulation simplifying the mathematics. Because the sample size is large, approximate the log-likelihood by a simple quadratic in the maximum-likelihood estimate minus the vector of parameters. Each test reduces to a least-squares linear regression. The constrained maximum-likelihood estimate reduces to a least-squares linear regression, and its Lagrange-multiplier dual reduces to another least-squares linear regression. The formulas for the variance of the parameter estimates and the Lagrange multiplier are simple and show immediately the chi-square distribution of the test statistics.