<s>[INST] Can't understand this pseudo-inverse relation. In the Answer to a different Question, a curious matrix relation came up:
M is symmetric and non-singular, G is non-symmetric and singular.
Theorem:
When $M$ is positive/negative definite, or more generally, if $G$ and $(G^\dagger G) M (G^\dagger G)$ have the same ranks then
$$G^{T}\left(GMG^{T}\right)^{\dagger}G=\left((G^\dagger G)M(G^\dagger G)\right)^\dagger$$ 
I have no idea how to prove this or why the rank relationships asserted would be true.
An example:
$M=\left(
\begin{array}{ccc}
 1. & 2. & 3. \\
 2. & 4. & 5. \\
 3. & 5. & 6. \\
\end{array}
\right)$   $G=\left(
\begin{array}{ccc}
 1 & 0 & 0 \\
 0 & 2 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right)$   $GMG^T=\left(
\begin{array}{ccc}
 1. & 4. & 0. \\
 4. & 16. & 0. \\
 0. & 0. & 0. \\
\end{array}
\right)$
$GMG^T$ has a rank of 1 and the relation does not hold. But if we add a small number times the Identity Matrix to M, $GMG^T$ will have rank 2 and the relation will hold.
What is going on there? Can anyone help with an explanation or proof?  And under exactly what circumstances is this true or not true?
 [/INST] Primary matrices
The matrix 
$$
  \mathbf{M} = 
\left[
\begin{array}{ccc}
 1 & 2 & 3 \\
 2 & 4 & 5 \\
 3 & 5 & 6 \\
\end{array}
\right]
$$
has rank $\rho_{M} = 3$. The matrix 
$$
  \mathbf{G} = 
\left[
\begin{array}{ccc}
 1 & 0 & 0 \\
 2 & 0 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right]
$$
has rank $\rho_{G} = 2$. 
To make manipulation easier, introduce the submatrices
$$
m =
\left[
\begin{array}{ccc}
 1 & 2 \\
 2 & 4 
\end{array}
\right], \quad
  g = 
\left[
\begin{array}{ccc}
 1 & 0 \\
 2 & 0 
\end{array}
\right].
$$
We see a stencil matrix
$$
%
  \mathbf{G} =
%
\left[
\begin{array}{ccc}
 g & \mathbf{0} \\
 \mathbf{0} & 0 
\end{array}
\right], \quad
%
  \mathbf{G}^{\dagger} =
%
\left[
\begin{array}{ccc}
 g^{-1} & \mathbf{0} \\
 \mathbf{0} & 0 
\end{array}
\right], \quad
%
%
$$
Product matrices
The product matrix 
$$
  \mathbf{G} \mathbf{M} \mathbf{G}^{\mathrm{T}} = 
%
\left[
\begin{array}{cc}
 gmg^{\mathrm{T}} & \mathbf{0} \\
 \mathbf{0} & 0 \\
\end{array}
\right]
%
  =
%
\left[
\begin{array}{rrr}
 1 & 2 & 0 \\
 2 & 4 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right]
$$
has rank $\rho_{GMG^{T}} = 1$.
The pseudoinverse is 
$$
  \left( \mathbf{G} \mathbf{M} \mathbf{G}^{\mathrm{T}} \right)^{\dagger} = 
%
\frac{1}{25}
\left[
\begin{array}{rrr}
 1 & 2 & 0 \\
 2 & 4 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right]
%
  =
%
\frac{1}{25}
\mathbf{G} \mathbf{M} \mathbf{G}^{\mathrm{T}}.
$$
The final product matrix is
$$
\mathbf{G}^{\dagger}\mathbf{G} \mathbf{M} \mathbf{G}^{\dagger}\mathbf{G} =
\left[
\begin{array}{ccc}
 1 & 0 & 0 \\
 0 & 0 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right].
$$
LHS
The left hand side of the identity is
$$
  \mathbf{G}^{\mathrm{T}} \left( \mathbf{G} \mathbf{M} \mathbf{G}^{\mathrm{T}} \right)^{\dagger} \mathbf{G} =
%
\left[
\begin{array}{rrr}
 1 & 0 & 0 \\
 0 & 0 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right]
$$
RHS
The right hand side of the identity is
$$
  \left( \mathbf{G}^{\dagger}\mathbf{G} \mathbf{M} \mathbf{G}^{\dagger}\mathbf{G} \right)^{\dagger} =
\left[
\begin{array}{rrr}
 1 & 0 & 0 \\
 0 & 0 & 0 \\
 0 & 0 & 0 \\
\end{array}
\right]
$$
The equality holds in this case:
$$
 \mathbf{G}^{\mathrm{T}} \left( \mathbf{G} \mathbf{M} \mathbf{G}^{\mathrm{T}} \right)^{\dagger} \mathbf{G}
=
\left( \mathbf{G}^{\dagger}\mathbf{G} \mathbf{M} \mathbf{G}^{\dagger}\mathbf{G} \right)^{\dagger}
$$
Postscript Asymmetric $\mathbf{G}$ used after comments by @Dohleman.
</s>