Newton-Type Methods for Nonlinear Least Squares Using Restricted Second Order Information

Summary In the paper, a special approximated Newton method for minimizing a sum of squares $$f\left( \mathfrak{x} \right) = \frac{1} {2}\left\| {f\left( \mathfrak{x} \right)} \right\|^2 = \frac{1} {2}\Sigma _{i = 1}^m \left[ {F_i \left( \mathfrak{x} \right)} \right]^2 $$ is introduced. In this Restricted Newton method, the Hessian $$H = G + S$$ of f where $$G = \left( {F'} \right)^T F',S = F \circ F''$$ , is approximated by $$A_{RN} = G + B$$ where $$B = Z_2 Z_2^T SZ_2 Z_2^T $$ is the restriction of the second order term S on the subspace im Z2 spanned by the eigenvectors of the Gauss-Newton matrix G which belong to the q smallest eigenvalues of G. Some properties of this approximation are derived, and a related trust region method is tested on hand of some test functions from the literature.