This paper mostly combines standard algorithms (random walk, Adam without momentum, SAM), although this is not a problem, the theoretically analysis needs to be improved. Meanwhile, the experimental part lacks new insights except for some expected results.
Major comments:
1.	Theorem 3.7 looks like a direct combination of theoretical results obtained by Adam without momentum and SAM. Furthermore, the proof in Appendix does not consider the convergence guarantee that could be achieved by random walk method. That is, the Markov chain is not considered. Note that the last equation in Page 13 is almost the same as the convergence result (Theorem 4.3, Triastcyn et al., 2022)) except it does not have the compression part. The proof also follows exactly as Triastcyn et al. (2022). The perturbed model is not used, means that sharp awareness minimization is not analyzed, which makes me question the soundness of Theorem 3.7.
2.	Since SAM is integrated to prevent potential overfitting, the experiment should present this effect compared with its counterpart that does not have the perturbed model. The lack of this experiment comparison would question the necessity of incorporating SAM in the proposed Localized framework.
3.	The simulation only shows the loss performance of the proposed algorithms and the benchmarks, however, in practical, we would be more interested to see the classification accuracy.
4.	The proposed algorithm is compared with FedAvg, however, for FedAvg, not all agents are communicating all the time, which does not make sense in the setting that FedAvg does not need to consider communication. That means, I suppose that if all agents in FedAvg communicate all the time, the performance of FedAvg might be much better than all the other methods, since there exists a coordinator, although the communication cost would be very high. The figures presented, however, show that Localized SAM is always better than FedAvg in the random sample setting in both performance and communication, which is not a fair comparison.
Minor comments:
1.	In Page 2, first paragraph, Localized SAM is introduced first and then “sharpness-aware minimization (SAM (Foret et al., 2021))” is repeated again. It would be better to revise it.
2.	Page 2, second paragraph in Related Work, the Walkman algorithm (Mao et al., 2020) is solved by ADMM, with two versions, one is to solve a local optimization problem, the other is to solve a gradient approximation. Therefore, it is not accurate to say that “However, these works are all based on the simple SGD for decentralized optimization.”
3.	Section 3, first paragraph, in “It can often have faster convergence and better generalization than the SGD-based Algorithm 1, as will be demonstrated empirically in Section 4.1.” The “it” does not have a clear reference.
4.	In Section 3.1, you introduced $\boldsymbol{u}_k$, which was not defined previously and did not show up after Algorithm 3.
5.	Figure 6 seems to be reused from your previous LoL optimizer work.
2. Page 2, second paragraph in Related Work, the Walkman algorithm (Mao et al., 2020) is solved by ADMM, with two versions, one is to solve a local optimization problem, the other is to solve a gradient approximation. Therefore, it is not accurate to say that “However, these works are all based on the simple SGD for decentralized optimization.” 3. Section 3, first paragraph, in “It can often have faster convergence and better generalization than the SGD-based Algorithm 1, as will be demonstrated empirically in Section 4.1.” The “it” does not have a clear reference.