is the gradient vector at iteration $t$, $\boldsymbol{g} = \dot{F}(\boldsymbol{\theta})$, and $\boldsymbol{A}_t$ a “learning rate” matrix. Common choices for the learning rate in SEM software are the observed information at step $t$, i.e. $\boldsymbol{A}_t := \ddot{F}(\boldsymbol{\theta})^{-1}$, which gives Newton-Raphson optimization; the expected information, $\boldsymbol{A}_t := \left(\boldsymbol{\Delta}^T \mathbf{V} \boldsymbol{\Delta}\right)^{-1}$, which gives Fisher scoring; or an approximation to the inverse observed Hessian used in quasi-Newton methods such as BFGS. In the machine learning literature, other gradient-baed methods have been developed, with a baseline choice being $\boldsymbol{A}_t:=\gamma$, a scalar constant learning rate [for a short overview of various optimization methods, see @goodfellow_deep_2016 Ch. 8].

Convergence of these optimization algorithms is achieved when the gradient vector equals zero. However, when the gradients are linearly dependent, convergence will never be achieved, since the norm of the gradient will not decrease along the line $\mathbf{n} \cdot \boldsymbol{g} = \mathbf{0}$ for some $\mathbf{n} \neq \mathbf{0}$, by the definition of linear dependence. Here, $\mathbf{n}$ is a non-trivial (nonzero) basis for the “nullspace” of the gradient. Note that the same holds for non-gradient based optimization methods; for example, proof of convergence of the Nelder-Mead algorithm requires the absence of linear dependencies in the gradient as well [@lagarias_convergence_1998 e.g.]. For 