loss function (here $\gamma \geq 0$ is a hyperparameter). As we see in Figure \[fig:ingredients\], the resulting gradient field has two additional Hessian corrections. [@balduzzi2018mechanics; @letcher2019differentiable] observe that any game can be written as the sum of a *potential game* (that is easily solved by GDA), and a *Hamiltonian game* (that is easily solved by ConOpt). Based on this insight, they propose *symplectic gradient adjustment* that applies (in its simplest form) ConOpt only using the skew-symmetric part of the Hessian, thus alleviating the problematic tendency of ConOpt to converge to spurious solutions. The same algorithm was independently discovered by [@gemp2018global], who also provide a detailed analysis in the case of linear-quadratic GANs.\
[@daskalakis2017training] proposed to modify GDA as $$\begin{aligned}
    \label{eqn:updateOGDA}
    \Delta x &= - \left( \nabla_x f(x_{k},y_{k}) 
        + \left( \nabla_x f(x_{k},y_{k}) - \nabla_x f(x_{k-1},y_{k-1}) \right) \right) \\
    \Delta y &= - \left( \nabla_y g(x_{k},y_{k}) 
        + \left( \nabla_y g(x_{k},y_{k}) - \nabla_y g(x_{k-1},y_{k-1}) \right) \right),\end{aligned}$$ which we will refer to as optimistic gradient descent ascent (OGDA). By interpreting the differences appearing in the update rule as finite difference approximations to Hessian vector products, we see that (to leading order) OGDA corresponds to yet another second order correction of GDA (see Figure \[fig:ingredients\]). 