<s>[INST] A difficult integral evaluation problem How do I compute the integration for $a>0$, 
$$
\int_0^\pi \frac{x\sin x}{1-2a\cos x+a^2}dx?
$$
I want to find a complex function and integrate by the residue theorem.
 [/INST] I'll first consider the case $0<a<1$.
Let $\displaystyle f(z) = \frac{z}{a-e^{-iz}}$ and integrate around a rectangle with vertices at $\pm \pi$ and $\pm \pi + iR$.
The function $f(z)$ has poles where $a-e^{-iz}=e^{\ln a + 2 \pi i n}-e^{-iz}= 0$.  
That is, when $z= i \ln a - 2 \pi n$.
If $ 0<a< 1$, all of those points are in the lower half-plane.
Letting $ R \to \infty$ and going counterclockwise around the contour,
$$ \int_{- \pi}^{0} \frac{z}{a-e^{-ix}} \ dx + \int_{0}^{\pi} \frac{x}{a-e^{-ix}} \ dx + i \int_{0}^{\infty}f(\pi + i t) \ dt + \lim_{R \to \infty} \int_{-\pi}^{\pi} f(t + iR) \ dt $$
$$+ \ i \int_{\infty}^{0} f(-\pi+it) \ dy =0$$
Combing the first two integrals,
$$ \begin{align} \int_{- \pi}^{0} \frac{x}{a-e^{-ix}} \ dx + \int_{0}^{\pi} \frac{x}{a-e^{-ix}} \ dx  &= -\int_{0}^{\pi} \frac{u}{a-e^{iu}} \ du + \int_{0}^{\pi} \frac{x}{a-e^{-ix}} \ dx \\ &= \int_{0}^{\pi} \frac{-x(a-e^{-ix})+ x(a-e^{ix}) }{(a-e^{ix})(a-e^{-ix})} \ dx \\ &= - 2 i \int_{0}^{\pi} \frac{x \sin x}{1-2a \cos x +a^{2}} \ dx \end{align}$$
Combining the third and fifth integrals,
$$ \begin{align} i \int_{0}^{\infty} f(\pi + it) \ dt - i \int_{0}^{\infty} f(-\pi + it) \ dt &= i \int_{0}^{\infty} \frac{\pi + it}{a-e^{-i(\pi + it)}} \ dt - i \int_{0}^{\infty} \frac{- \pi + it}{a-e^{-i( -\pi + it)}} \ dy \\ &= i \int_{0}^{\infty} \frac{\pi + it}{a +e^{t}} \ dt - i \int_{0}^{\infty} \frac{- \pi + it}{a+ e^{t}} \ dt \\ &= 2 \pi i \int_{0}^{\infty} \frac{1}{a+e^{t}} \ dt \\ &= 2 \pi i \int_{0}^{\infty} \frac{e^{-t}}{1+ae^{-t}} \ dt \\ &= - \frac{2 \pi i}{a} \ln (1+ae^{-t}) \Big|^{\infty}_{0} \\ &= \frac{2 \pi i}{a} \ln(1+a) \end{align}$$
And the fourth integral vanishes.
So we have
$$ - 2i \int_{0}^{\pi} \frac{x \sin x}{1-2a \cos x +a^{2}} \ dx + \frac{2 \pi i}{a} \ln(1+a) = 0$$
or 
$$ \int_{0}^{\pi } \frac{x \sin x}{1-2a \cos x +a^{2}} \ dx = \frac{\pi \ln (1+a)}{a}$$
The case for when $a >1$ is similar.  
The only difference is that there is now a pole inside of the contour at $z=i \ln a$ with residue $ \displaystyle \frac{\ln a}{a}$.
And therefore,
$$- 2i \int_{0}^{\pi} \frac{x \sin x}{1-2a \cos x +a^{2}} \ dx + \frac{2 \pi i}{a} \ln(1+a) = 2 \pi i \frac{\ln a}{a} $$
or
$$ \int_{0}^{\pi} \frac{x \sin x}{1-2a \cos x+a^{2}} \ dx = \frac{\pi \ln (1+a)}{a} - \frac{\pi \ln a}{a} = \frac{\pi \ln \left(\frac{1+a}{a} \right)}{a}$$
</s>