Title: Gradient Norm Regularizer Seeks Flat Minima and Improves Generalization. Abstract: The heavy overparameterization of current deep neural networks requires model generalization guarantees. Recently, flat minima are proven to be effective for improving generalization and sharpness-aware minimization (SAM) achieves state-of-the-art performance. Yet we show that SAM fails to measure flatness/sharpness when there are multiple minima within the perturbation radius. We present a novel regularizer named Gradient Norm Regularizer (GNR) to seek minima with uniformly small curvature across all directions and measure sharpness even when multiple minima are within the perturbation radius. We show that GNR bounds both the maximum eigenvalue of Hessian at local minima and the regularization function of SAM. We present experimental results showing that GNR improves the generalization of models trained with current optimizers such as SGD and AdamW on various datasets and networks. Furthermore, we show that GNR can help SAM find flatter minima and achieve better generalization.   