factor analysis (CFA) model is then $$\textbf{y} = \Lambda \boldsymbol{\eta} + \boldsymbol{\epsilon},$$ where the “common factors” $\boldsymbol{\eta} \in \mathbb{R}^{q^*}$ and “residuals” $\boldsymbol{\epsilon} \in \mathbb{R}^q$ are unobserved vectors of latent variables. Generally, there are fewer factors than observed variables, $q^*<q$, and we assume the factors and residuals are uncorrelated, $\mathbb{E}(\boldsymbol{\eta} \boldsymbol{\epsilon})  = \mathbf{0}$, and both latent variable vectors have constant variance matrices, say, $\text{Var}(\boldsymbol{\eta}) = \boldsymbol{\Phi}$, and $\text{Var}(\boldsymbol{\epsilon}) = \boldsymbol{\Psi}$.

The parameters of interest of the factor model are the loading matrix $\Lambda$, the factor variance matrix $\boldsymbol{\Phi}$, and the residual variance matrix $\boldsymbol{\Psi}$. Generally, these matrices are sparsely parameterized, so that only certain elements are free parameters of the model to be estimated. We collect these free parameters into a single parameter vector, $\boldsymbol{\theta} = (\boldsymbol{\lambda}^T, \boldsymbol{\phi}^T, \boldsymbol{\psi}^T)^T$, say, of length $p$. Under the above assumptions of linearity and homoskedasticity, the implied variance of the observed variable vector is then $$\label{eq:sigma}
    \text{Var}(\mathbf{y} | \boldsymbol{\theta}) = \Sigma(\boldsymbol{\theta}) = \Lambda \boldsymbol{\Phi} \Lambda^T + \boldsymbol{\Psi}.$$ The parameters of interest $\boldsymbol{\theta}$ can be estimated in a sample through maximum-likelihood with $\Sigma(\boldsymbol{\theta})$ as the covariance matrix, or, equivalently, by minimizing the weighted least squares loss, $$\hat{\boldsymbol{\theta}}_n = 
        \arg\min_{\boldsymbol{\theta} \in \Theta} F,
\label{eq:objective}$$ where the loss 