Least Squares Estimators and Residuals Analysis

The main objective of this chapter is to introduce tools for analyzing the impact of the noise acting on the data sets, on the one hand, on the least squares estimation results introduced in the former chapter and, on the other hand, on the modeling of the residuals. Thus, this new chapter is devoted to the effective description of (i) these disturbances by resorting to statistics and (ii) their impacts as far as estimated model parameter accuracy and precision are concerned. After reading this chapter, the reader will know that Adding statistical assumptions on the observations is essential to explain the errors in the estimated parameters. In order to extract statistical properties of the residuals from one (short) realization, guaranteeing the stationar,ty and ergodicity of the residuals sequences is paramount. Contrary to the regularized linear least squares estimator, the linear least squares estimator is unbiased. Regularization can reduce the estimator variance considerably when compared to the ordinary least squares solution. Thanks to the strong link between regularized least squares and nonlinear optimization techniques like the Levenberg–Marquardt algorithm, consistency properties of the linear least squares are shared by the nonlinear least squares estimator asymptotically. Asymptotic consistency of the least squares estimators help the user determine confidence interval for the estimated parameters as well as the estimated and predicted model outputs. When possible, bootstrap solutions can be implemented to assess the accuracy of the asymptotic statistical properties of the least squares estimators. Once the deterministic components of a time series is available, the remaining part can be described by linear difference equations involving constant coefficients and zero mean white noise sequences only thanks to the Wold decomposition theorem.