Non-equilibrium spectral thermodynamics of pruning: Phase transitions in sparse neural networks

Sparse neural networks often match the performance of dense models until a sharp pruning level is crossed, after which trainability and generalization collapse abruptly—a hallmark of a non-equilibrium phase transition in a complex system. Existing explanations based on static connectivity (e.g., expansion or Ramanujan properties) overlook the stochastic, time-dependent nature of learning dynamics. We recast pruning+training as a random dynamical system driven by a sequence of stochastic update operators and analyze its Lyapunov spectrum and associated spectral entropy as physically meaningful order parameters. This viewpoint yields a mechanism-level account of when sparsity preserves stability and when it fails.