Title: $z$-SignFedAvg: A Unified  Stochastic Sign-based Compression for Federated Learning. Abstract: Federated Learning (FL) is a promising privacy-preserving distributed learning paradigm but suffers from high communication cost when training large-scale machine learning models.  Sign-based methods,  such as SignSGD \citep{bernstein2018signsgd},  have been proposed as a biased gradient compression technique for reducing the communication cost. However,  sign-based algorithms could diverge under heterogeneous data, which thus motivated the development of advanced techniques, such as the error-feedback method and stochastic sign-based compression, to fix this issue.
Nevertheless, these methods still suffer from slower convergence rates. Besides, none of them allows multiple local SGD updates like FedAvg \citep{mcmahan2017communication}.  In this paper,  we propose a novel noisy perturbation scheme with a general symmetric noise distribution for sign-based compression, which not only allows one to flexibly control the tradeoff between gradient bias and convergence performance, but also provides a unified viewpoint to existing stochastic sign-based methods.  More importantly,  we propose the very first sign-based FedAvg algorithm ($z$-SignFedAvg). Theoretically,  we show that $z$-SignFedAvg achieves a faster convergence rate than existing sign-based methods and,  under the uniformly distributed noise, can enjoy the same convergence rate as its uncompressed counterpart. Last but not the least,  we remark that adding random noise to the local gradients has a double benefit: it protects the clients' privacy by, e.g., the Differential Privacy. Extensive experiments are conducted to demonstrate that the $z$-SignFedAvg can achieve competitive empirical performance on real datasets. 