1.It’s suggested that authors give a comprehensive survey on adaptive sparse training method. Although authors claim “Previous works have only managed to solve one, or perhaps two of these challenges”, can authors give a comprehensive comparison of existing methods?
2.Considering different clients train different submodels, the server also maintains a full model. So can the sparsity of clients be different to apply for heterogeneous hardware?
3.Can authors further explain why clients should achieves consensus on the clients’ sparse model masks when server always maintain a full model.
4.What’s the definition of the model plasticity?
5.In experimental section, authors only compared with two baselines, there’re several works also focus on the same questions, for example [1,2,3], so it’s suggested to add more experimental to show the effectiveness of proposed method.
6.Considering the model architecture, authors only show the effectiveness on convolutional network, what’s the performance on other architecture, for example Transformer?
[1]Stripelis, Dimitris, et al. "Federated progressive sparsification (purge, merge, tune)+." arXiv preprint arXiv:2204.12430 (2022).
[2]Wang, Yangyang, et al. "Theoretical convergence guaranteed resource-adaptive federated learning with mixed heterogeneity." Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 2023.
[3]Zhou, Hanhan, et al. "Every parameter matters: Ensuring the convergence of federated learning with dynamic heterogeneous models reduction." Advances in Neural Information Processing Systems 36 (2024).
5.In experimental section, authors only compared with two baselines, there’re several works also focus on the same questions, for example [1,2,3], so it’s suggested to add more experimental to show the effectiveness of proposed method.