<p>I would say behavior such as this is to be expected for stochastic optimization in general. The inherent variance causes you to oscillate somewhere around good solution. The magnitude of the variance and properties of the optimization objective control how much this oscillates when looking at a accuracy metric.</p>
<p>For plain SGD, decreasing learning rate decreases the variance and slows down convergence.</p>
<p>For optimization methods for federated learning, the story is a bit more complicated, but decreasing the <em>client</em> learning rate, or decreasing the number of local steps (while keeping other things the same) can have a similar effect, typically including slowing down convergence. More details can be found in <a href="https://arxiv.org/abs/2007.00878" rel="nofollow noreferrer">https://arxiv.org/abs/2007.00878</a> mentioned also in the other answer. Potentially decreasing the client learning rate across rounds could also work. The details can differ also based on what exactly is the optimization method you are using.</p>
TFF : test accuracy fluctuate
<p>I train a ResNet50 model with TFF, I use test accuracy on test data for evaluation, but I find many fluctuations as shown in the figure below, So please how can I avoid this fluctuation ?</p>
<p><a href="https://i.stack.imgur.com/JDtmg.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/JDtmg.png" alt="enter image description here" /></a></p>