<s>[INST] Why is the sum of outer products equal to the matrix product of a matrix and its transpose , so $A^TA = \sum_{i=1}^n a_i a_i^T$? Why is the sum of outer products equal to the matrix product of a matrix and its transpose? So $A^TA = \sum_{i=1}^n a_i a_i^T$, where $A = [ a_0, a_1 , ... , a_n ] $, $a_i \in \mathbb{R}^k$. An answer or a link to the answer would be greatly appriciated. It seems so simple but I just cannot figure it out.
It is essentially this question of which I do not understand the answer:
Matrix $A^T A$ as sum of outer products
 [/INST] I think you got the order wrong.  $ A A^T = \sum_{i=1}^n a_i a_i^T $.  What you should do is to is to look at a sequence of examples:
$$ \left( \begin{array}{c c c}
1 \\
2 \\
3
\end{array} \right) 
 \left( \begin{array}{c c c}
1 \\
2 \\
3
\end{array} \right) ^T = 
$$
$$ \left( \begin{array}{c c c}
1 & -1\\
2 & -2\\
3 & -3
\end{array} \right) 
\left( \begin{array}{c c c}
1 & -1\\
2 & -2\\
3 & -3
\end{array} \right)  ^T = 
$$
Maybe you will see the pattern.
Then, look at 
$$ \left( \begin{array}{c c c}
a_{1,1} \\
a_{2,1} \\
a_{3,1}
\end{array} \right) 
\left( \begin{array}{c c c}
a_{1,1} \\
a_{2,1} \\
a_{3,1}
\end{array} \right) ^T = 
$$
$$ \left( \begin{array}{c c c}
a_{1,1} & a_{1,2}  \\
a_{2,1} & a_{2,2}\\
a_{3,1}& a_{3,2}
\end{array} \right) 
\left( \begin{array}{c c c}
a_{1,1} & a_{1,2}  \\
a_{2,1} & a_{2,2}\\
a_{3,1}& a_{3,2}
\end{array} \right) ^T = 
$$
and see if you get the pattern from that.
</s>