Bounds for the Least Squares Residual Using Scaled Total Least Squares

The standard approaches to solving overdetermined linear systems Ax ≈ b construct minimal corrections to the data to make the corrected system compatible. In ordinary least squares (LS) the correction is restricted to the right hand side b, while in scaled total least squares (Scaled TLS) [10, 9] corrections to both b and A are allowed, and their relative sizes are determined by a real positive parameter γ. As γ → 0, the Scaled TLS solution approaches the LS solution. Fundamentals of the Scaled TLS problem are analyzed in our paper [9] and in the contribution in this book entitled Unifying least squares,total least squares and data least squares. This contribution is based on the paper [7]. It presents a theoretical analysis of the relationship between the sizes of the LS and Scaled TLS corrections (called the LS and Scaled TLS distances) in terms of γ. We give new upper and lower bounds on the LS distance in terms of the Scaled TLS distance, compare these to existing bounds, and examine the tightness of the new bounds. This work can be applied to the analysis of iterative methods which minimize the residual norm [8, 5].