<s>[INST] integrate $\int \frac{\sin^3x \cos^2x}{1+\sin^2x}dx$ 
$$\int \frac{\sin^3x \cos^2x}{1+\sin^2x}dx$$

$$\int \frac{\sin^3x \cos^2x}{1+\sin^2x}dx=\int \frac{\sin^2x \cos^2x  \sin x }{1+\sin^2x}dx=\int \frac{(1-\cos^2x )\cos^2x  \sin x }{1+1-\cos^2x}dx=\int \frac{(1-\cos^2x )\cos^2x  \sin x }{2-\cos^2x}dx$$
$u=cosx$ 
$du=-sinxdx$
$$-\int \frac{(1-u^2 )u^2}{2-u^2}du=\int \frac{u^4-u^2 }{2-u^2}du$$
Polynomial division give us 
$$\int-u^2-1+\frac{2}{u^2-2}du=-\frac{u^3}{3}-u=2\int \frac{1}{u^2-2}du$$
Using partial fraction we get to:
$$\frac{1}{u^2-2}=\frac{1}{(u+\sqrt{2})(u^2-\sqrt{2})}=\frac{A}{u^2+\sqrt{2}}+\frac{B}{u^2-\sqrt{2}}$$
So we get to:
$$1=(A+B)u^2+\sqrt{2}(A-B)$$
So $A=B$ but $0*\sqrt{2}\neq 1$, where did it go wrong?
 [/INST] Consider
$$
\frac{t^2-t}{2-t}=(-1-t)+\frac{2}{2-t}
$$
so your integral is
$$
\int\left(-\frac{2}{u^2-2}-u^2-1\right)\,du
$$
and your reduction is (almost) good, but a sign went wrong. Now your computation of partial fractions is really messed up:
$$
\frac{2}{u^2-2}=\frac{A}{u-\sqrt{2}}+\frac{B}{u+\sqrt{2}}
$$
so you get
$$
\begin{cases}
A+B=0\\[6px]
A\sqrt{2}-B\sqrt{2}=2
\end{cases}
$$
so $A=1/\sqrt{2}$ and $B=-1/\sqrt{2}$.
</s>