Gradient Extremals, Talwegs, Valleys, and Directional Alignment for Generic Gradient Descent

Gradient extremals are loci along which the gradient is an eigenvector of the Hessian. These objects provide a natural geometric framework connecting several notions, notably valleys and talwegs, which we analyze from a variational viewpoint in the generic case. We then show that trajectories of the gradient flow and of its discrete counterpart exhibit directional alignment with the tangent spaces to gradient extremals, and generically to the talweg. Under non-resonance assumptions, and in contrast with the quadratic case, alignment rates are governed either by the first spectral gap or by the smallest eigenvalue of the Hessian at the limit point. Nonlinearities and the step length may both distort these rates in a complex manner. We further prove a volume concentration phenomenon emphasizing the structuring role of gradient extremals: for large times, the images of sets of initial conditions concentrate inside valleys and asymptotically around talwegs.