Line Search Descent Methods For Unconstrained Minimization

Over the last 40 years many powerful direct search algorithms have been developed for the unconstrained minimization of general functions. These algorithms require an initial estimate to the optimum point, denoted by $$\mathbf{x}^0$$ . With this estimate as starting point, the algorithm generates a sequence of estimates $$\mathbf{x}^0$$ , $$\mathbf{x}^1$$ , $$\mathbf{x}^2$$ , $$\dots $$ , by successively searching directly from each point in a direction of descent to determine the next point. The process is terminated if either no further progress is made, or if a point $$\mathbf{x}^k$$ is reached (for smooth functions) at which the first necessary condition in, i.e. $${\varvec{\nabla }} f(\mathbf{x})=\mathbf{0}$$ is sufficiently accurately satisfied, in which case $$\mathbf{x}^*\cong \mathbf{x}^k$$ . It is usually, although not always, required that the function value at the new iterate $$\mathbf{x}^{i+1}$$ be lower than that at $$\mathbf{x}^i$$ .