1.The main part can be more concise (especially for the introduction part)and including empirical results.
2.Given the new introduced hyper-parameters, it is still not clear whether this new proposed method is empirically useful. How to choose hyper-parameters in a more practical training setting?
3.The empirical evaluations can not well supported their theoretical analysis. As the authors claim running experiments with 24 A100 GPUs, all methods should be compared in a relatively large scaled training task. Only small linear regression experiment results are reported, where communication is not really an issue.
The paper discusses a new variant on a technique in distributed training. As far as I’m concerned, there is no serious issue or limitation that would impact society.
1.The main part can be more concise (especially for the introduction part)and including empirical results.