<s>[INST] How to calculate sum, if the recurrence is given? Let  $a_n=4a_{n-1}-a_{n-2}$, for $n\geq 3$. Prove that $\sum\limits_{n=1}^\infty \cot^{-1}{a_n^2}=\frac{\pi}{12}$, if $a_1=2$, $a_2=8$.
This recurrent formula is easy to solve. Characteristic equation $x^2-4x+1=0$ has two roots, $2\pm \sqrt{3}$. Therefore general solution is $a_n=a(2+\sqrt{3})^n+b(2-\sqrt{3})^n$, where a and b can be obtained from $a_1=2$ and $a_2=8$. But, I have no idea what to do next. How to calculate sum $\sum\limits_{n=1}^\infty \cot^{-1}{a_n^2}$?
Any help is welcome. Thanks in advance.
 [/INST] Lemma: $\frac{a_{n+1}^2+a_n^2}{1+a_na_{n+1}}=4$
solving the recurrence,
$a_n=\frac{1}{\sqrt{3}}(a^n-1/a^n)$ where $a=2+\sqrt{3}$
$a_{n+1}^2+a_n^2=\frac{1+a^2}{a}(a^{2n+1}+1/a^{2n+1})-4 =4(a^{2n+1}+1/a^{2n+1}-1)$
$1+a_n a_{n+1}=3+(a^{2n+1}+1/a^{2n+1})-(a+1/a)=a^{2n+1}+1/a^{2n+1}-1$
using the lemma,
$\frac{a_{n+1}^2+a_n^2}{1+a_na_{n+1}}=4=\frac{a_{n+2}+a_n}{a_{n+1}}$
$a_{n+2}+a_n=\frac{a_{n+1}^3+a_n^2 a_{n+1}}{1+a_na_{n+1}}$
$a_{n+2}=\frac{a_{n+1}^3-a_n}{1+a_na_{n+1}}$
then continue with Martin R's soln.
</s>