<s>[INST] Positive semidefinite ordering for covariance matrices Suppose that X and Z are matrices with the same number of rows. Let
$$ D = \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1} - \left[\begin{array}{cc} (X' X)^{-1} & 0 \\ 0 & 0 \end{array} \right],$$ where all inverses are assumed to exist and the zeros represent zero matrices of suitable dimensions. How can we prove that $D$ is positive semidefinite?
 [/INST] $U = \left(\begin{matrix} X^{T}X & X^TZ \\ Z^TX & Z^TZ \end{matrix}\right) = \left( \begin{matrix} X^T \\ Z^T\end{matrix}\right) \left(\begin{matrix} X & Z \end{matrix} \right) $ is clearly positive semidefinite and since we are given it is invertible it must be positive definite.
Consequently for any invertible matrix $P$ the matrix $PUP^T$ is also positive definite and in particular for $P=\left( \begin{matrix} I & 0 \\ -Z^TX(X^TX)^{-1} & I \end{matrix} \right)$ the matrix $$PUP^T  = \left( \begin{matrix} X^TX & 0 \\ 0 & Z^TZ - Z^TX(X^TX)^{-1}X^TZ\end{matrix}\right)$$ is positive definite and hence both $X^TX$ and $Z^TZ - Z^TX(X^TX)^{-1}X^TZ = W$(say) must be symmetric positive definite matrices.
Inverting the above we have $(P^{-1})^{T}   \left(\begin{matrix} X^{T}X & X^TZ \\ Z^TX & Z^TZ \end{matrix}\right)^{-1} P^{-1} = \left( \begin{matrix} (X^TX)^{-1} & 0 \\ 0 & 0\end{matrix}\right) + \left(\begin{matrix} 0 & 0 \\ 0 & W^{-1}\end{matrix}\right). $ 
Since $P^{-1} = \left( \begin{matrix} I & 0 \\ Z^TX(X^TX)^{-1} & I \end{matrix} \right) $ we also have $(P^{-1})^T  \left( \begin{matrix} (X^TX)^{-1} & 0 \\ 0 & 0\end{matrix}\right) P^{-1} =  \left( \begin{matrix} (X^TX)^{-1} & 0 \\ 0 & 0\end{matrix}\right) $.
So we have the identity
$$ (P^{-1})^{T}\left(\left(\begin{matrix} X^{T}X & X^TZ \\ Z^TX & Z^TZ \end{matrix}\right)^{-1} -  \left( \begin{matrix} (X^TX)^{-1} & 0 \\ 0 & 0\end{matrix}\right) \right) P^{-1} = \left(\begin{matrix} 0 & 0 \\ 0 & W^{-1}\end{matrix}\right)$$
that is
$$
\left(\begin{matrix} X^{T}X & X^TZ \\ Z^TX & Z^TZ \end{matrix}\right)^{-1} -  \left( \begin{matrix} (X^TX)^{-1} & 0 \\ 0 & 0\end{matrix}\right) = P^T \left( \begin{matrix} 0 & 0 \\ 0 & W^{-1} \end{matrix} \right) P. $$
Since $W$ is symmetric positive definite matrix, so is $W^{-1}$ and the right hand side is positive semi-definite.
</s>