essentially conducting “multivariate regression” on $\bm{P}$ after dimension reduction with $\bm{W}$ as the projection matrix.

So the question becomes, what matrix serves as a good projection matrix and why a pre-trained word embedding would be a good choice? When doing dimension reduction, we either strive to preserve the most variance (PCA) [@PCA] or achieve the least reconstruction error (SVD) [@SVD]. Suppose we have access to $\bm{P}$, we could easily perform PCA or SVD. Concretely, one could perform SVD on $\bm{M} \in  \mathcal{R}^{N \times V}$, and get $  \bm{M_m} = \bm{U_m} \bm{\Sigma_m} \bm{V_m^T},$ where $\bm{U_m} \in \mathcal{R}^{N \times m}$, $\bm{\Sigma_m} \in \mathcal{R}^{m \times m}$ and $\bm{V_m} \in \mathcal{R}^{V \times m}$. In PCA, $\bm{M}$ is centered $\bm{P}$ and in SVD, $\bm{M}$ is the raw $\bm{P}$. $\bm{M_m}$ is the matrix of rank $m$ that best approximates the original matrix $\bm{M}$. $\bm{V_m^T}$ is the optimal projection matrix $\bm{W}$ in either case. In practice, though we could not get the full $\bm{P}$, we could use a simplified definition of context and get an approximated $\bm{P}$.

Interestingly, using a simplified definition of context is a common practice in learning word embedding [@levy2014neural]. SVD is also a recognized method to construct word embedding. When $\bm{M}$ is the 