of CGD can not be recovered by simply changing the stepsize $\eta$ used by the other methods.\
While introducing the competitive term is enough to fix the cycling behaviour of GDA, OGDA and ConOpt (for small enough $\eta$) add the additional consensus term to the update rule, with opposite signs.\
In the second experiment (where convergence is desired), OGDA converges in a smaller parameter range than GDA and SGA, while only diverging slightly faster in the third experiment (where divergence is desired).\
ConOpt, on the other hand, converges faster than GDA in the second experiment, for $\alpha = 1.0$ however, it diverges faster for the remaining values of $\alpha$ and, what is more problematic, it converges to a spurious solution in the third experiment for $\alpha = 1.0$.\
Based on these findings, the consensus term with either sign does not seem to systematically improve the performance of the algorithm, which is why we suggest to only use the competitive term (that is, use LOLA/LCGD, or CGD, or SGA).

![The first 50 iterations of GDA, LCGD, ConOpt, OGDA, and CGD with parameters $\eta = 0.2$ and $\gamma = 1.0$. The objective function is $f(x,y) = \alpha x^{\top}y$ for, from left to 