Minimization of a Second-Degree Polynomial (in n Variables) Subject to Linear Constraints

The subject of this chapter is the minimization of a quadratic form, or, more generally, a second-degree polynomial, in some number, say n, of variables that may be subject to linear constraints. Special cases of this minimization problem are encountered in various areas of statistics and in many related disciplines. In particular, they are encountered in estimating the parameters of a linear statistical model. One approach to the estimation goes by the acronym BLUE (for best linear unbiased estimation); in this approach, consideration is restricted to estimators that are linear (i.e., that are expressible as linear combinations of the data) and that are unbiased (i.e., whose “expected values” equal the parameters), and the estimator of each parameter is chosen to have minimum variance among all estimators that are linear and unbiased. The minimization problem encompassed in this approach can be formulated as one of minimizing a quadratic form (in the coefficients of the linear combination) subject to linear constraints—the constraints arise from the restriction to unbiased estimators. Another approach is to regard the estimation of the parameters as a least squares Problem—the least squares problem was considered (from a geometrical perspective) in Chapter 12. This approach consists of minimizing the sum of the squared deviations between the data points and the linear combinations of the parameters (in the model) that correspond to those points. The sum of the squared deviations is a second-degree polynomial (in the parameters). In some cases, the parameters of the model may be subject to linear constraints—knowledge of the process that gave rise to the data may suggest such constraints (along with the other aspects of the model). In the presence of such constraints, the minimization problem encompassed in the least squares appraoch is one of minimizing a second-degree polynomial (in the parameters) subject to linear constraints (on the parameters).