<s>[INST] Prove that $\cot^2{(\pi/7)} + \cot^2{(2\pi/7)} + \cot^2{(3\pi/7)} = 5$ Prove that $\cot^2{(\pi/7)} + \cot^2{(2\pi/7)} + \cot^2{(3\pi/7)} = 5$ .
I am sure this is derived from using roots of unity and Euler's complex number function, but I am very uncomfortable in these areas so some help would be great. It is evident that $(a + b + c)^2 - 2(ab + ac + bc) = a^2 + b^2 + c^2$ .
So, using a polynomial of degree 3 and the coefficients on the $x^2$ and $x$ terms will get where we need to be.
 [/INST] The following approach is intimately related to Ofir's; you can think of this as the linear algebraic variation of his route.
Consider the perturbed Toeplitz tridiagonal matrix
$$\begin{pmatrix}2&-1&&&\\-1&2&-1&&\\&-1&\ddots&\ddots&\\&&\ddots&2&-1\\&&&-1&1\end{pmatrix}$$
which has the characteristic polynomial
$$(-1)^n\left((x-1)U_{n-1}\left(\frac{x-2}{2}\right)-U_{n-2}\left(\frac{x-2}{2}\right)\right)$$
(where $U_n(x)$ is the Chebyshev polynomial of the second kind) and the eigenvalues
$$\mu_k=4\cos^2\left(\frac{k\pi}{2n+1}\right),\qquad k=1\dots n$$
We find then that the matrix $\mathbf H=4\mathbf I-\mathbf T$ has the eigenvalues
$$\eta_k=4\sin^2\left(\frac{k\pi}{2n+1}\right),\qquad k=1\dots n$$
Now, take the case $n=3$:
$$\mathbf H=\begin{pmatrix}2&1&\\1&2&1\\&1&3\end{pmatrix}$$
We have
$$\mathbf U=\mathbf H^{-1}=\frac17\begin{pmatrix}5&-3&1\\-3&6&-2\\1&-2&3\end{pmatrix}$$
The eigenvalues of $\mathbf U$ are
$$\xi_k=\frac14\csc^2\left(\frac{k\pi}{7}\right),\qquad k=1\dots 3$$
which means the eigenvalues of $\mathbf W=4\mathbf U-\mathbf I$ are
$$\lambda_k=\cot^2\left(\frac{k\pi}{7}\right),\qquad k=1\dots 3$$
Now, with
$$\mathbf W=\frac17\begin{pmatrix}13&-12&4\\-12&17&-8\\4&-8&5\end{pmatrix}$$
you can see that the trace of $\mathbf W$ (which is also the sum of the eigenvalues of $\mathbf W$) is $5$.
</s>