<s>[INST] Compacting a Matrix Suppose you have a matrix that is zero-valued everywhere except the diagonal.
As an example, take the identity matrix, $I$.  For this example, let's say you are using the $4\times 4$ version.
Is there an operation that can produce a $2\times 2$ matrix consisting of only the non-zero values?  If not, is there a notation to express this intent?
Example:
$$    I = \pmatrix{
    i_{11} &0    &0    &0\\
    0    &i_{22} &0    &0\\
    0    &0    &i_{33} &0\\
    0    &0    &0    &i_{44}}
$$
$$    I_2 =\pmatrix{ 
    i_{11} &i_{22}\\
    i_{33} &i_{44}}$$
 [/INST] Here is a solution that you can find by solving a linear system. First we will obtain a $4\times 4$ matrix that looks like your target matrix ($I_2$) but padded. We let
$$A = \begin{bmatrix}a_1 & 0 & 0 & 0\\0 & a_2 & 0 & 0\\0 & 0 & a_3 & 0\\0 & 0 & 0 & a_4\end{bmatrix} \quad{}\text{and}\quad X = \begin{bmatrix}
 x_{1,1} & x_{1,2} & x_{1,3} & x_{1,4} \\
 x_{2,1} & x_{2,2} & x_{2,3} & x_{2,4} \\
 x_{3,1} & x_{3,2} & x_{3,3} & x_{3,4} \\
 x_{4,1} & x_{4,2} & x_{4,3} & x_{4,4}
\end{bmatrix}$$
When you multiply these two two matrices you obtain
$$AX = 

\begin{bmatrix}
 a_1 x_{1,1} & a_1 x_{1,2} & a_1 x_{1,3} & a_1 x_{1,4} \\
 a_2 x_{2,1} & a_2 x_{2,2} & a_2 x_{2,3} & a_2 x_{2,4} \\
 a_3 x_{3,1} & a_3 x_{3,2} & a_3 x_{3,3} & a_3 x_{3,4} \\
 a_4 x_{4,1} & a_4 x_{4,2} & a_4 x_{4,3} & a_4 x_{4,4}
\end{bmatrix}

$$
You want all the entries of this matrix to be $0$ except the four entries in the upper left corner (which we want to be $a_1,a_2,a_3$ and $a_4$). Solving the equations formed leads to 
$$ X =
\begin{bmatrix}
 1 & a_2/a_1 & 0 & 0 \\
 a_3/a_2 & a_4/a_2 & 0 & 0 \\
 0 & 0 & 0 & 0 \\
 0 & 0 & 0 & 0
\end{bmatrix}
$$
and when you multiply $A$ and $X$ together you obtain
$$AX = \begin{bmatrix}
 a_1 & a_2 & 0 & 0 \\
 a_3 & a_4 & 0 & 0 \\
 0 & 0 & 0 & 0 \\
 0 & 0 & 0 & 0
\end{bmatrix}$$
Note that the solution assumes that $a_1 \neq 0$ and $a_2 \neq 0$. If one or more of these is zero, we cannot solve the system. In your case it is okay because you say that the entries on the diagonal are non-zero. As a final step we need to remove two of the columns and two of the rows. Let $B$ be given by
$$B = 
\begin{bmatrix}
 1 & 0 \\
 0 & 1 \\
 0 & 0 \\
 0 & 0
\end{bmatrix}.$$
Then $B^T A X B = I_2$. Note that the matrix $X$ depends on the entries of matrix $A$. I am not sure if there is a matrix that will work on all such diagonal matrices.
</s>