<s>[INST] Trigonometric Integral $\int _0^{\pi/4}\:\frac{dx}{\cos^4x-\cos^2x\sin^2x+\sin^4x}$ Q. $$\int _0^{\pi/4}\:\frac{1}{\cos^4x-\cos^2x\sin^2x+\sin^4x}$$
My method:
$$\int_0^{ \pi /4} \frac{dx}{(\cos^2x+\sin^2x)^2-3\cos^2x \sin^2x}=\int_0^{ \pi /4} \frac{dx}{1-3\cos^2x\sin^2x} $$
Dividing numerator and denominator by $\cos^2x$ we have:
$$\int_0^{\pi /4}\frac{\sec^2x}{\sec^2x-3\tan^2x}dx=\int_0^{\pi /4}\frac{\sec^2x}{1-2\tan^2x}dx=\int _0^1 \frac{dt}{1-2t^2}=\int _0^1 \frac{1}{2}\frac{dt}{\frac{1}{2}-t^2}=\frac{1}{2\sqrt{2}}\log\left|\frac{1-\sqrt{2}t}{1+\sqrt{2}t}\right|_0^1=\frac{1}{2\sqrt{2}}\log\left|\frac{1-\sqrt{2}}{1+\sqrt{2}}\right|$$
But when we do the same integration by dividing the initial term by $\sec^4x$ and solving it yields an answer $$\frac{\pi }{2}$$
Am I wrong somewhere?
 [/INST] When you divide the numerator and the denominator by $\cos^2 x$, you should get $$\displaystyle \int^{\pi/4}_0 \dfrac{\sec^2 x dx}{\sec^2 x - 3\sin^2 x}$$ instead of $$\displaystyle \int^{\pi/4}_0 \dfrac{\sec^2 x dx}{\sec^2 x - 3 \tan^2 x}$$
because 
$$\begin{array}{rcll}
\displaystyle \int^{\pi/4}_0 \dfrac{dx}{1 - 3\sin^2 x \cos^2 x} &=& \displaystyle\int^{\pi/4}_0 \dfrac{\frac{dx}{\cos^2 x}}{\frac{1 - 3\sin^2 x\cos^2 x}{\cos^2 x}}\\
&=&\displaystyle\int^{\pi/4}_0 \dfrac{\sec^2 x dx}{\frac{1}{\cos^2 x} - \frac{3\sin^2 x\cos^2 x}{\cos^2 x}}\\
&=& \displaystyle\int^{\pi/4}_0 \dfrac{\sec^2 x dx}{\sec^2 x - 3 \sin^2 x}
\end{array}$$
Denote the original integral as $I$.
For me, at this point:$$\displaystyle \int^{\pi/4}_0 \dfrac{dx}{1 - 3\sin^2 x \cos^2 x}$$
I would use double angle formula to write $3 \sin^2 x \cos^2 x$ in terms of $2x$.
From $\sin 2x = 2 \sin x \cos x$, square both sides:$$\sin^2 2x = 4\sin^2 x \cos^2 x$$Multiply by $\dfrac{3}{4}$:$$\dfrac{3}{4}\sin^2 2x = 3\sin^2 x\cos^2 x$$
The integral becomes $$\displaystyle \int^{\pi/4}_0 \dfrac{dx}{1 - \frac{3}{4}\sin^2 2x}$$
Next, let $u = 2x$,$\;du = 2dx$.$$\begin{array}{rcll}
I&=&\displaystyle \int^{\pi/4}_0 \dfrac{dx}{1 - \frac{3}{4}\sin^2 2x}\\ &=& \displaystyle \int^{\pi/2}_0 \dfrac{\frac{du}{2}}{1 - \frac{3}{4}\sin^2 u}\\
&=&2 \displaystyle \int^{\pi/2}_0 \dfrac{du}{4 - 3\sin^2 u}
\end{array}$$
Using the fact that $\sin^2 u = \dfrac{1 - \cos 2u}{2}$,
$$\begin{array}{rcll}
I&=&2\displaystyle\int^{\pi/2}_0 \dfrac{du}{4 - 3\sin^2 u}
\\&=& 2\displaystyle \int^{\pi/2}_0 \dfrac{du}{4 - 3 \left(\frac{1 - \cos 2u}{2}\right)}
\\&=& 4\displaystyle \int^{\pi/2}_0 \dfrac{du}{8 - 3(1 - \cos 2u)}
\\&=& 4\displaystyle \int^{\pi/2}_0 \dfrac{du}{5 + 3\cos 2u}
\end{array}$$
Next, noticing that $\cos 2u = \dfrac{1 - \tan^2 u}{1 + \tan^2 u}$,
$$\begin{array}{rcll}
I&=& 4\displaystyle \int^{\pi/2}_0 \dfrac{du}{5 + 3\cos 2u}
\\&=& 4 \displaystyle \int^{\pi/2}_0 \dfrac{du}{5 + 3\left(\frac{1 - \tan^2 u}{1 + \tan^2 u}\right)}
\\&=& 4 \displaystyle \int^{\pi/2}_0 \dfrac{\sec^2 u du}{5(1 + \tan^2 u) + 3\left(1 - \tan^2 u\right)}
\\&=& 4 \displaystyle \int^{\pi/2}_0 \dfrac{\sec^2 u du}{8 + 2 \tan^2 u}
\\&=& 2 \displaystyle \int^{\pi/2}_0 \dfrac{d\left(\tan u\right)}{4 + \tan^2 u}
\end{array}$$
Using the substitution $t = \tan u$,
$$\begin{array}{rcll}
I&=& 2 \displaystyle \int^{\pi/2}_0 \dfrac{d\left(\tan u\right)}{4 + \tan^2 u}
\\&=& 2 \displaystyle \lim_{\varepsilon \to 0^{+}}\int^{1/\varepsilon}_0 \dfrac{dt}{4 + t^2}
\\&=& 2 \displaystyle \lim_{\varepsilon \to 0^{+}}\left[\dfrac{1}{2} \arctan\left(\dfrac{t}{2}\right)\right]^{1/\varepsilon}_0
\\&=& 2 \displaystyle \lim_{\varepsilon \to 0^{+}} \left(\dfrac{1}{2}\arctan\left(\dfrac{1}{2\varepsilon}\right)\right)
\\&=& \displaystyle \lim_{\varepsilon \to 0^{+}} \arctan\left(\dfrac{1}{2\varepsilon}\right)
\\&=& \color{red}{\boxed{ \dfrac{\pi}{2} }}
\end{array}$$
</s>