<s>[INST] Integrate via substitution: $x^2\sqrt{x^2+1}\;dx$ I need to integrate the following using substitution:
$$
\int x^2\sqrt{x^2+1}\;dx
$$
My textbook has a similar example:
$$
\int \sqrt{x^2+1}\;x^5\;dx
$$
They integrate by splitting $x^5$ into $x^4\cdot x$ and then substituting with $u=x^2+1$:
$$
\int \sqrt{x^2+1}\;x^4\cdot x\;dx\\
=\frac{1}{2}\int \sqrt{u}\;(u-1)^2\;du\\
=\frac{1}{2}\int u^{\frac{1}{2}}(u^2-2u+1)\;du\\
=\frac{1}{2}\int u^{\frac{5}{2}}-2u^{\frac{3}{2}}+u^{\frac{1}{2}}\;du\\
=\frac{1}{7}u^{\frac{7}{2}}-\frac{2}{5}u^{\frac{5}{2}}+\frac{1}{3}u^{\frac{3}{2}}+C
$$
So far so good. But when I try this method on the given integral, I get the following:
$$
\int x^2\sqrt{x^2+1}\;dx\\
=\frac{1}{2}\int \sqrt{x^2+1}\;x\cdot x\;dx\\
=\frac{1}{2}\int \sqrt{u}\;\sqrt{u-1}\;du\;(u=x^2+1)\\
=\frac{1}{2}\int u^{\frac{1}{2}}(u-1)^\frac{1}{2}\;du
$$
Here is where it falls down. I can't expand the $(u-1)^\frac{1}{2}$ factor like the $(u-1)^2$ factor above was, because it results in an infinite series. I couldn't prove, but I think any even exponent for the $x$ factor outside the square root will cause an infinite series to result. Odd exponents for $x$ will work, since it will cause the $(u-1)$ term to have a positive integer exponent.
How should I proceed? I don't necessarily want an answer. I just want to know if I'm missing something obvious or if it is indeed above first year calculus level and probably a typo on the question.
 [/INST] As suggested in the comments we can use the substitution $x=\mbox{sinh}(u)$ and some other well known trig identities:
$1)$ $dx=(\mbox{sinh}(u))'du=\mbox{cosh}(u)du$
$2)$ $\mbox{cosh}^{2}(u)-\mbox{sinh}^{2}(u)=1$
$3)$ $\mbox{cosh}(u)=\frac{e^{u}+e^{-u}}{2}$
$4)$ $\mbox{arcsinh}(u)=\mbox{log}(u+\sqrt{1+u^{2}})$
$5)$ $\int \mbox{sinh}^{n}(u)du=\frac{\mbox{cosh}(u)\mbox{sinh}^{n-1}(u)}{n}-\frac{n-1}{n}\int \mbox{sinh}^{n-2}(u)du$
You can prove the last identity using integration by parts. So your integral becomes:
$\begin{align} \int x^{2}\sqrt{x^{2}+1}\;dx &=\int\mbox{sinh}^2(u)\sqrt{\mbox{sinh}^2(u)+1}\:\mbox{cosh}(u)\;du\\
&=\int \mbox{sinh}^{2}(u)\mbox{cosh}^{2}(u)\;du\\
&=\int \mbox{sinh}^{2}(u)(1+\mbox{sinh}^{2}(u))\;du\\
&=\int \mbox{sinh}^{2}(u)\;du\:+\:\int \mbox{sinh}^{4}(u)\;du\\
&=\int \mbox{sinh}^{2}(u)\;du\:+\:\begin{bmatrix}\frac{\mbox{cosh}(u)\mbox{sinh}^{3}(u)}{4}\;-\;\frac{3}{4}\int \mbox{sinh}^{2}(u)\;du\end{bmatrix}\\
&=\frac{1}{4}\int \mbox{sinh}^{2}(u)\;du\:+\:\frac{\mbox{cosh}(u)\mbox{sinh}^{3}(u)}{4}\\
&=\frac{1}{4}\begin{bmatrix} \frac{\mbox{cosh}(u)\mbox{sinh}(u)}{2}-\frac{1}{2} \int du \end{bmatrix}\:+\:\frac{\mbox{cosh}(u)\mbox{sinh}^{3}(u)}{4}\\
&=\bigstar
\end{align}$
Let's compute something useful: $u=\mbox{arcsinh}(x)=\mbox{log}(x+\sqrt{1+x^{2}})$.
So we have that:
$\mbox{sinh}(u)=\mbox{sinh}(\mbox{arcsinh}(x))=x$
$\begin{align} \mbox{cosh}(u)
&=\mbox{cosh}(\mbox{arcsinh}(x))\\
&=\frac{x+\sqrt{1+x^{2}}+\frac{1}{x+\sqrt{1+x^{2}}}}{2}\\
&=\frac{x^{2}+x\sqrt{1+x^{2}}+1}{x+\sqrt{1+x^{2}}}\\
&=\frac{x^{2}+x\sqrt{1+x^{2}}+1}{x+\sqrt{1+x^{2}}} \cdot \frac{\sqrt{1+x^{2}}-x}{\sqrt{1+x^{2}}-x}\\
&=...=\sqrt{1+x^{2}}
\end{align}$
So the integral becomes:
$\begin{align} \bigstar
&=\frac{\mbox{cosh}(u)\mbox{sinh}(u)-\mbox{arcsinh}(u)}{8}\:+\:\frac{\mbox{cosh}(u)\mbox{sinh}^{3}(u)}{4}+C\\
&=\frac{x\sqrt{1+x^{2}}\:-\:\mbox{log}(x+\sqrt{1+x^{2}})}{8}\:+\:\frac{x^{3}\sqrt{1+x^{2}}}{4}+C\\
&=\frac{(2x^{3}+x)\sqrt{1+x^{2}}\:-\:\mbox{log}(x+\sqrt{1+x^{2}})}{8}+C
\end{align}$
</s>