<s>[INST] Need help proving this integration If $a>b>0$, prove that :

$$\int_0^{2\pi} \frac{\sin^2\theta}{a+b\cos\theta}\ d\theta = \frac{2\pi}{b^2} \left(a-\sqrt{a^2-b^2} \right) $$

 [/INST] $$I = \int_0^{2 \pi} \dfrac{\sin^2(x)}{a+b \cos(x)} dx \implies aI = \int_0^{2 \pi} \dfrac{\sin^2(x)}{1+\dfrac{b \cos(x)}a}dx$$
\begin{align}
aI & = \sum_{k=0}^{\infty}\left(\dfrac{(-b)^k}{a^k} \int_0^{2 \pi}\sin^2(x) \cos^k(x) dx \right) = \sum_{k=0}^{\infty}\left(\dfrac{b^{2k}}{a^{2k}} \int_0^{2 \pi}\sin^2(x) \cos^{2k}(x) dx \right)
\end{align}
Note that we have thrown away the odd terms since for $k$ odd, the integral $\displaystyle \int_0^{2 \pi}\sin^2(x) \cos^k(x) dx$ is zero.
\begin{align}
\dfrac{\displaystyle \int_0^{2 \pi}\sin^2(x) \cos^{2k}(x) dx}4 & = \int_0^{\pi/2}\sin^2(x) \cos^{2k}(x) dx\\
& 
\int_0^{\pi/2}\cos^{2k}(x) dx - \int_0^{\pi/2}\cos^{2k+2}(x) dx\\
& = \dfrac{2k-1}{2k} \dfrac{2k-3}{2k-2} \cdots \dfrac12 \dfrac{\pi}2 - \dfrac{2k+1}{2k+2} \dfrac{2k-1}{2k} \dfrac{2k-3}{2k-2} \cdots \dfrac12 \dfrac{\pi}2\\
& = \dfrac1{2k+2} \dfrac{2k-1}{2k} \dfrac{2k-3}{2k-2} \cdots \dfrac12 \dfrac{\pi}2\\
& = \dfrac{\pi}{2^{k+2}} \dfrac{(2k-1)(2k-3)\cdots3 \cdot 1}{(k+1)!} = \dfrac{\pi}{2^{2k+2}} \dfrac{(2k)!}{k! (k+1)!}
\end{align}
Hence,
$$\dfrac{aI}{\pi} = \sum_{k=0}^{\infty} \left(\dfrac{b}{2a} \right)^{2k} \underbrace{\dfrac{(2k)!}{k! (k+1)!}}_{\text{Catalan numbers}}$$
Now $$\sum_{k=0}^{\infty} \dfrac{\dbinom{2k}k x^{k}}{k+1} = \dfrac{1-\sqrt{1-4x}}{2x} \,\,\,\,\,\, \forall \vert x \vert < \dfrac14$$ This is the generating function for the Catalan numbers. Hence, in our case, we get that
$$\sum_{k=0}^{\infty} \left(\dfrac{b}{2a} \right)^{2k} \underbrace{\dfrac{(2k)!}{k! (k+1)!}}_{\text{Catalan numbers}} = \dfrac{1-\sqrt{1-4 \cdot \left(\dfrac{b}{2a} \right)^2}}{2 \cdot \left(\dfrac{b}{2a} \right)^2} \,\,\,\,\,\,\,\,\forall \dfrac{b}{2a} < \dfrac12$$
Hence,
$$\dfrac{aI}{\pi} = \dfrac{1-\sqrt{1-\left(\dfrac{b}a\right)^2}}{\dfrac{b^2}{2a^2}} = \dfrac{a-\sqrt{a^2-b^2}}{\dfrac{b^2}{2a}}$$
Hence,
$$I = \dfrac{2\pi}{b^2} (a-\sqrt{a^2-b^2})$$
</s>