<s>[INST] Integral using contour integration Here is the integral I want to evaluate:
$$\int_{0}^{2\pi} \frac{dx}{a+b \cos x }, \quad a>b >0$$
Apparently there are limitations as to what values $a, b$ are supposed to take but let us not concern about this. Since using the sub $u =\tan \frac{x}{2}$ (the Weiersstrass sub) results that the integral is $0$ (as it should, since it is not $1-1$ function in this interval) I got down down the way of contour integration. Hence:
$$\begin{aligned}
\int_{0}^{2\pi}\frac{dx}{a+ b\cos x} &\overset{x=i \ln u}{=\! =\! =\!} \oint \limits_{|z|=1} \frac{dz}{iz \left [ a + \frac{b}{2}\left ( z+z^{-1} \right ) \right ]} \\ 
 &= \frac{1}{i} \oint \limits_{|z|=1} \frac{dz}{za + \frac{bz^2}{2}+\frac{b}{2}}\\ 
 &=\frac{2}{i} \oint \limits_{|z|=1} \frac{dz}{bz^2 +b +2za} \\ 
 &= \frac{2}{i} 2\pi i \sum_\text{residues} f(z)\\ 
 &= \frac{4\pi}{1+ \sqrt{1-8ab}+2a}
\end{aligned}$$
However, judging by intuition this must not be the result. Because this one restricts the integral too much. What i mean is, that for $a=6, b=3$ we have that:
$$\int_0^{2\pi} \frac{dx}{6+3\cos x}= \frac{2\pi}{3\sqrt{3}}$$
My formula cannot derive the result because then radical would be negative. What am I doing wrong here? 
 [/INST] The issue I see with your formula is that scaling $a$ and $b$ by $\lambda$ fails to scale the result by $\lambda^{-1}$. Anyway, I do not know how you computed the residue(s), but you did that wrong somehow.
Also, the substitution is $x=i^{-1}\ln z$, not $x=i\ln z$. Anyway, we get
$$\frac{2}{i}\oint\limits_{|z|=1}\frac{dz}{bz^2+2zaz+b}=\frac{2}{ib}\oint\limits_{|z|=1}\frac{dz}{z^2+\frac{2a}{b}z+1}=\frac{2}{ib}\oint\limits_{|z|=1}\frac{dz}{(z+\frac{a}{b})^2-(\frac{a^2}{b^2}-1)} $$
$$=\frac{2}{ib}\oint\limits_{|z|=1}\frac{dz}{\left(z+c+\sqrt{c^2-1}\right)\left(z+c-\sqrt{c^2-1}\right)}\qquad {\rm where}~~c:=\frac{a}{b}>1. $$
The poles are $-c\pm\sqrt{c^2-1}$. Notice
$$\left(-c-\sqrt{c^2-1}\right) < -1 <  \left(-c+\sqrt{c^2-1}\right)  < 0 $$
so the relevant pole is $-c+\sqrt{c^2-1}$. Plug that into $(z+c+\sqrt{c^2-1})^{-1}$ to obtain
$$\frac{2}{ib}(2\pi i)\frac{1}{2\sqrt{c^2-1}}=\frac{2\pi}{\sqrt{a^2-b^2}} .$$
</s>