<s>[INST] How to evaluate the integral $\int_0^{\pi}\frac{a^n\sin^2x+b^n\cos^2x}{a^{2n}\sin^2x+b^{2n}\cos^2x}dx$? 
Evaluate the integral $$\int_0^{\pi}\frac{a^n\sin^2x+b^n\cos^2x}{a^{2n}\sin^2x+b^{2n}\cos^2x}dx.$$

I have no idea. 
$$\int_0^{\pi}\dfrac{a^n\sin^2x+b^n\cos^2x}{a^{2n}\sin^2x+b^{2n}\cos^2x}dx=\int_0^{\pi/2}\dfrac{a^n\sin^2x+b^n\cos^2x}{a^{2n}\sin^2x+b^{2n}\cos^2x}dx=\int_0^{\pi/2}\dfrac{a^n\tan^2x+b^n}{a^{2n}\tan^2x+b^{2n}}dx$$
I try the substitution $\tan{x}=t$, but it doesn’t work. 
 [/INST] Your approach does work if you would do it correctly. Assume $a\neq 0$ and $b\neq 0$, why? Otherwise the integral is boring. Also assume that $|a|\neq |b|$ for the same reason. Let:
\begin{align}
I=\int^{\pi}_0 \frac{a^n\sin^2(x)+b^n\cos^2(x)}{a^{2n}\sin^2(x)+b^{2b}\cos^2(x)}dx = 2\int^{\pi/2}_0 \frac{a^n\sin^2(x)+b^n\cos^2(x)}{a^{2n}\sin^2(x)+b^{2b}\cos^2(x)}dx
\end{align}
Set $\tan(x) =t$ and get $\sin(x)=\frac{t^2}{1+t^2}$ and $\cos(x)=\frac{1}{1+t^2}$ and $dx = \frac{dt}{1+t^2}$ so that the original integral $I$ becomes:
\begin{align}
I = 2\int^\infty_0 \frac{a^nt^2+b^n}{(1+t^2)(a^{2n}t^2+b^{2n})}dt = \frac{1}{a^{2n}} \int^{\infty}_{-\infty} \frac{a^nt^2+b^n}{(1+t^2)(t^2+\frac{b^{2n}}{a^{2n}})} dt
\end{align}
Using a semi cirkel contour in the upper half plane and using the Residue Theorem we get:
\begin{align}
I = \frac{1}{a^{2n}} 2\pi i \left( \text{Res}_{z=i} \frac{a^nz^2+b^n}{(1+z^2)(z^2+\frac{b^{2n}}{a^{2n}})}  + \text{Res}_{z=i|b|^n/|a|^n} \frac{a^nz^2+b^n}{(1+z^2)(z^2+\frac{b^{2n}}{a^{2n}})}\right)
\end{align}
Calculating these residues:
\begin{align}
\text{Res}_{z=i} \frac{a^nz^2+b^n}{(1+z^2)(z^2+\frac{b^{2n}}{a^{2n}})}   = \frac{-a^n+b^n}{2i (-1+\frac{b^{2n}}{a^{2n}})}
\end{align}
And the other one:
\begin{align}
\text{Res}_{z=i|b|^n/|a|^n} \frac{a^nz^2+b^n}{(1+z^2)(z^2+\frac{b^{2n}}{a^{2n}})} = \frac{-a^n\frac{b^{2n}}{a^{2n}}+b^n}{2i\frac{|b|^n}{|a|^n}(1-\frac{b^{2n}}{a^{2n}})}
\end{align}
If we put everything together we get:
\begin{align}
I = \frac{\pi}{a^{2n}}\left( \frac{-a^n+b^n}{ (-1+\frac{b^{2n}}{a^{2n}})} + \frac{-a^n\frac{b^{2n}}{a^{2n}}+b^n}{\frac{|b|^n}{|a|^n}(1-\frac{b^{2n}}{a^{2n}})}\right)
\end{align}
Simplifying a bit gives us:
\begin{align}
I = \frac{\pi}{(b^{2n}-a^{2n})} \left( b^n-a^n+\frac{|a|^n}{|b|^n}\left( \frac{b^{2n}}{a^{n}}-b^n\right)\right)
\end{align}
</s>