Title: Critical Learning Periods Augmented Model Poisoning Attacks to Byzantine-Robust Federated Learning. Abstract: Existing attacks in federated learning (FL) control a set of malicious clients and share a fixed number of malicious gradients with the central server in each training round, to achieve a desired tradeoff between attack impact and resilience against defenses.  In this paper, we show that such a tradeoff is not fundamental and an adaptive attack budget not only improves the impact of attack $\mathcal{A}$ but makes it more resilient to defenses.  Inspired by recent findings on critical learning periods (CLP), where small gradient errors have irrecoverable impact on model accuracy, we advocate CLP augmented model poisoning attacks $\mathcal{A}$-CLP, which merely augment attack $\mathcal{A}$ with an adaptive attack budget scheme. $\mathcal{A}$-CLP inspects the changes in federated gradient norms to identify CLP and adaptively adjusts the number of malicious clients that share their malicious gradients with the central server in each round, leading to dramatically improved attack impact compared to $\mathcal{A}$ itself by up to 6.85$\times$, with a smaller attack budget and hence improved resilience of $\mathcal{A}$ by up to 2$\times$.  Based on understandings on $\mathcal{A}$-CLP, we further relax the inner attack subroutine $\mathcal{A}$ in $\mathcal{A}$-CLP, and propose SimAttack-CLP, a lightweight CLP augmented similarity-based attack, which is more flexible and impactful. 