<s>[INST] $f(a)=b,f(b)=c,f(c)=a$, find $\frac{1}{a}+\frac{1}{b}+\frac{1}{c}$ 

Let $f(x)=x^2-x-2$. It is given that $a, b,c \in \mathbb{R}$ such that
$$f(a)=b,f(b)=c,f(c)=a$$ Then find the value of $\frac{1}{a}+\frac{1}{b}+\frac{1}{c}$.


My attempt:
We have:
$$\begin{array}{l}
a^{2}-a-2=b \\
b^{2}-b-2=c \\
c^{2}-c-2=a
\end{array}$$
Subtracting pair wise we get:
$$(a-b)(a+b)=a-c$$
$$(b-c)(b+c)=b-a$$
$$(c-a)(c+a)=c-b$$
Trivially $a=b=c$ satisfies. So we get $a=b=c=\sqrt{3}+1, 1-\sqrt{3}$
Thus we get
$$\frac{1}{a}+\frac{1}{b}+\frac{1}{c}=\frac{3\sqrt{3}+3}{2}, \frac{-3-3\sqrt{3}}{2}$$
Now if $a\ne b \ne c$
Then we have:
$$(a+b)(b+c)(c+a)=-1$$
Letting $p=a+b+c$ we have
$$(p-a)(p-b)(p-c)=-1$$ $\implies$
$$p(ab+bc+ca)-abc+1=0$$
Any hint here?
 [/INST] From the condition, we need $f\circ f\circ f(a)=a$.  Now
$$f(x) = x^2-x-2=(x-2)(x+1)$$
$$f\circ f(x) = (x^2-x-4)(x^2-x-1)=x^4-2x^3-4x^2+5x+4$$
$$\implies f\circ f\circ f(x)=(x^4-2x^3-4x^2+4x+2)(x^4-2x^3-4x^2+4x+5)$$
$$\implies f\circ f\circ f(a)-a=a^8-4a^7-4a^6+26a^5+3a^4-54a^3-3a^2+34a+10 \\=(a^2-2a-2)(a^3-3a-1)(a^3-2a^2-3a+5)$$
It is easy to note all roots are real (using Descartes and evaluation at $a=-1, 0, 1$), so we have $8$ possibilities for $a$, which fall into two cases.
Case 1: The first/quadratic factor gives $a=1\pm\sqrt3$, each of which is also a solution to $f(a)=a$, so as you have already noted, we have two values for $\frac1a+\frac1b+\frac1c=\frac3a$ from these, viz. $\frac12(\pm3\sqrt3-3)$.
Case 2:  Now we have no more roots of $f(a)=a$, hence the rest of the roots (six roots from the two cubic factors), must contribute to two possible sets of $\{a, b, c\}$.
Taking the first cubic factor,
$\alpha^3-3\alpha-1=0 \implies f(\alpha)^3-3f(\alpha)-1 = (\alpha^2-\alpha-2)^3-3(\alpha^2-\alpha-2)-1 \\=(\alpha^3-3\alpha-1)(\alpha^3-3\alpha^2+3)=0$
i.e. if $\alpha$ is a root of the first cubic, so is $f(\alpha)$.  Thus the roots of that cubic contribute one set of $\{a, b, c\}$.  Now using Vieta, sum of the reciprocal of roots is $$\frac1a+\frac1b+\frac1c=\frac{ab+bc+ca}{abc}=-3$$
Similarly the roots of the last cubic form the last set of possible $a, b, c$, hence have a sum or reciprocals $=\frac35$.
Summarising the value of $\frac1a+\frac1b+\frac1c \in \{\frac12(\pm3\sqrt3-3), -3, \frac35\}$, four possibilities.
</s>