A multi-precision quadratic regularization method for unconstrained optimization with rounding error analysis

We propose a multi-precision extension of the Quadratic Regularization (R2) algorithm that enables it to take advantage of low-precision computations, and by extension to decrease energy consumption during the solve. The lower the precision in which computations occur, the larger the errors induced in the objective value and gradient, as well as in all other computations that occur in the course of the iterations. The Multi-Precision R2 (MPR2) algorithm monitors the accumulation of rounding errors with two aims: to provide guarantees on the result of all computations, and to permit evaluations of the objective and its gradient in the lowest precision possible while preserving convergence properties. MPR2’s numerical results show that single precision offers enough accuracy for more than half of objective evaluations and most of gradient evaluations during the algorithm’s execution. However, MPR2 fails to converge on several problems of the test set for which double precision does not offer enough precision to ensure the convergence conditions before reaching a first order critical point. That is why we propose a practical version of MPR2 with relaxed conditions which converges for almost as many problems as R2 and potentially enables saving about 50% time and 60% energy for objective evaluation and 50% time and 70% energy for gradient evaluation.