<s>[INST] Integrate $x^3\sqrt{x^2-9}\,dx$ with trig substitution $\newcommand{\arcsec}{\operatorname{arcsec}}$This expression can be integrated without trig substitution, but I wanted to try using it anyways and got a different answer.
Starting with finding the indefinite integral $$\int x^3\sqrt{x^2-9} \, dx$$
I then substituted $x = 3\sec\theta$, $\theta = \arcsec(\frac{x}{3})$, and $d\theta = \frac{3}{\sqrt{x^2-9} |x|} \, dx$.
Plugging these values into the original integral,
$$\int 27\sec^3\theta\sqrt{9\sec^2\theta - 9}\frac{3}{\sqrt{9\sec^2\theta-9} |3\sec\theta|} \, d\theta$$
$$\int \frac{81\sec^3\theta3\tan\theta}{3\tan\theta |3\sec\theta|} \, d\theta$$
$$\int \frac{81\sec^3\theta}{|3\sec\theta|} \, d\theta$$
$$\int 27\sec^2\theta \, d\theta$$
$$27\tan\theta + C$$
Now substituting $x$ back in and simplifying:
$$27\tan(\arcsec(\frac{x}{3})) + C$$
$$9\operatorname{sgn}(x)\sqrt{x^2-9} + C$$
This does not seem close at all to the solution I found by $u$-substitution,
$$\frac{(x^2-9)^\frac{3}{2}(x^2+6)}{5} + C$$
I am relatively new to integration so I think I made a mistake. What did I do wrong? Any help appreciated.
 [/INST] sometimes its easier to break it up into other expressions first:
$$I=\int x^3\sqrt{x^2-9}$$
$x=3u\Rightarrow dx=3du$
$$I=\int(3u)^3\sqrt{3^2(u^2-1)}(3du)$$
$$I=3^5\int u^3\sqrt{u^2-1}du$$
now we want to look for an identity that matches this, we know:

*

*$\cosh^2\theta-\sinh^2\theta=1$

*$\cos^2\theta+\sin^2\theta=1$
so we want to choose one of these that can be rearranged to our form, the nicest would be: $\cosh^2\theta-1=\sinh^2\theta$ so:
$u=\cosh(v)\Rightarrow du=\sinh(v)dv$ now replace everything:
$$I=3^5\int\cosh^3v\sqrt{\cosh^2v-1}\sinh v dv$$
$$I=3^5\int\cosh^3v\sinh^2vdv$$
</s>