Hybrid Random Concentrated Optimization Without Convexity Assumption

We propose a new random method to minimize deterministic continuous functions over subsetsSof high-dimensional spaceR K without assuming convexity. Our procedure alternates between a Global Search(GS) regime to identify candidates and a Concentrated Search (CS) regime to improve an eligible candidate in the constraint setS. Beyond the alternation between those completely different regimes, the originality of our approach lies in leveraging high dimensionality. We demonstrate rigorous concentration properties under theCSregime. In parallel, we also show thatGSreaches any point inSin finite time. Finally, we demonstrate the relevance of our new method by giving two concrete applications. The first deals with the reduction of theℓ1−norm of a LASSO solution. Secondly, we compress a neural network by pruning weights while maintaining performance; our approach achieves significant weight reduction with minimal performance loss, offering an effective solution for network optimization.