<s>[INST] Given matrix $A$, find a matrix $S$ such that result of $S^{-1} \circ A \circ S$ is a diagonal matrix 
Given is matrix $A=\begin{pmatrix} 3 & 0 & 7\\  0 & 1 & 0\\  7 & 0 & 3
\end{pmatrix}$, find a matrix $S$ such that $D=S^{-1} \circ A \circ S$
  where $D$ is a diagonal matrix.

So I'm not sure how this is supposed to work, I tried to create some linear system but soon realized there are too many unknowns! Then I thought about the zero matrix, if it's actually a diagonal matrix. This should be the case because a diagonal matrix is a matrix where all non-diagonal entries are zero and this is satisfied in a zero matrix.
But the bad thing is when we want the inverse of that zero matrix, we have a problem because division by zero doesn't work : /
So I can't use that matrix. But how else do you solve this problem?
 [/INST] Find first the eigenvalues of $A$, these are the roots of the characteristic polynomial of $A$, 
$$
(x-10)(x-1)(x+4)\ .
$$
So the eigenvalues are $10$, $1$, $-4$. We subtract them from the diagonal and search for eigenvectors. Here are my choices:
$$
\begin{aligned}
\begin{bmatrix}
3-10 & 0 & 7\\
0& 1-10 & 0\\
7 & 0 & 3-10
\end{bmatrix}
\begin{bmatrix}
1\\0\\1
\end{bmatrix}
&=
\begin{bmatrix}
0\\ 0\\ 0
\end{bmatrix}
\ ,\\
\begin{bmatrix}
3-1 & 0 & 7\\
0& 1-1 & 0\\
7 & 0 & 3-1
\end{bmatrix}
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
&=
\begin{bmatrix}
0\\ 0\\ 0
\end{bmatrix}
\ ,
\\
\begin{bmatrix}
3-(-4) & 0 & 7\\
0& 1-(-4) & 0\\
7 & 0 & 3-(-4)
\end{bmatrix}
\begin{bmatrix}
1\\ 0\\ -1
\end{bmatrix}
&=
\begin{bmatrix}
0\\ 0\\ 0
\end{bmatrix}
\ ,\\
\end{aligned}
$$
Equivalently,
$$
\begin{aligned}
\begin{bmatrix}
3 & 0 & 7\\
0& 1 & 0\\
7 & 0 & 3
\end{bmatrix}
\begin{bmatrix}
1\\0\\1
\end{bmatrix}
&=
\begin{bmatrix}
1\\0\\1
\end{bmatrix}
\cdot[10]
\ ,\\
\begin{bmatrix}
3 & 0 & 7\\
0& 1 & 0\\
7 & 0 & 3
\end{bmatrix}
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
&=
\begin{bmatrix}
0\\1\\0
\end{bmatrix}
\cdot
[1]
\ ,
\\
\begin{bmatrix}
3 & 0 & 7\\
0& 1 & 0\\
7 & 0 & 3
\end{bmatrix}
\begin{bmatrix}
1\\ 0\\ -1
\end{bmatrix}
&=
\begin{bmatrix}
1\\ 0\\ -1
\end{bmatrix}
[-4]
\ ,\\
\end{aligned}
$$
equalities of products of matrices of the shape 
$(3\times 3)\cdot(3\times 1)$, and
respectively  $(3\times 1)\cdot(1\times 1)$.
Now consider the matrix built from the eigenvectors, taken as columns.
We have the block matrix computation, subsumming the above three equalities:
$$
\underbrace{
\begin{bmatrix}
3 & 0 & 7\\
0& 1 & 0\\
7 & 0 & 3
\end{bmatrix}}_A
\underbrace{
\begin{bmatrix}
1 & 0 & 1\\
0 & 1 & 0\\
1& 0& -1
\end{bmatrix}}_S
=
\underbrace{
\begin{bmatrix}
1 & 0 & 1\\
0 & 1 & 0\\
1& 0& -1
\end{bmatrix}}_S
\underbrace{
\begin{bmatrix}
10 & & \\
&1 & \\
& & -4
\end{bmatrix}}_D
\ .
$$
We have $AS=SD$, so the above choices answer the request.
</s>