<s>[INST] How to find $\int \frac{x\ln(x+\sqrt{1+x^2})}{\sqrt{1+x^2}}\mathrm dx$ $$I=\int x.\frac{\ln(x+\sqrt{1+x^2})}{\sqrt{1+x^2}}\mathrm dx$$

Try 1:
Put $z= \ln(x+\sqrt{1+x^2})$, $\mathrm dz=1/\sqrt{1+x^2}\mathrm dx$
$$I=\int \underbrace{x}_{\mathbb u}\underbrace{z}_{\mathbb v}\mathrm dz=x\int zdz-\int (z^2/2)\mathrm dz\tag{Wrong}$$

Try 2:
Put 
$z= x+\sqrt{1+x^2}$
$$\implies x-z
=\sqrt{1+x^2}\implies x^2+z^2-2xz
=1+x^2\implies x
=\frac{z^2-1}{2z}$$ 
$$\mathrm dz
=\left(1+\frac{x}{\sqrt{1+x^2}}\right)\mathrm dx
=\frac{z\mathrm dx}{x-z}=\frac{-2z^2\mathrm dx}{1+z^2}$$
$$I
=\int\frac{(z^2-1)\ln z}{2z}.\frac{(1+z^2)\mathrm dz}{-2z^2}$$
$$=\int\frac{(z^4-1)\mathrm dz}{4z^3}
=\frac14\int\left(z-\frac1{z^3}\right)\mathrm dz
=z^2/2+2/z^2+C\tag{Wrong}$$

Try 3:
Put 
$z
=\sqrt{1+x^2},\mathrm dx
=x/\sqrt{1+x^2}\mathrm dx$
$$I
=\int \ln(x+z)\mathrm dz
=\int \ln(z+\sqrt{z^2-1})\mathrm dz$$
Don't know how to solve this integral.
[Note that if I take $u=z+\sqrt{z^2-1}$, it is $=\sqrt{1+x^2}+\sqrt{1+x^2-1}=x+\sqrt{1+x^2}$; same as first try.]

What's wrong in try 1 & 2, how to further solve try 3 and the best method to solve this question?

Update: Sorry, I don't know hyperbolic/inverse hyperbolic trigonometry.
 [/INST] $$
\int \ln(z+\sqrt{z^2-1})dz = \int \mathrm{arcosh}(z) dz\,\,\,z>1.
$$
the equality holds because for real x, $z = \sqrt{x^2+1}>1$.
then using the fact
$$
\int \mathrm{arcosh}(z) dz = z\mathrm{arcosh}(z) +\sqrt{z^2-1} + C
$$
then sub back in your substitution. 
$\textbf{brief derivation of the arcosh relation}$
$$
y = \cosh(x) \implies \mathrm{arcosh}(y) = x
$$
now 
$$
\cosh(x) = \frac{\mathrm{e}^{x}+\mathrm{e}^{-x}}{2} = y \tag{*}
$$
I can re-write the expression in Eq.(*) with $\mathrm{e}^{x} = t$ as
$$
\cosh(x) = \frac{t+1/t}{2} = \frac{1}{t}\frac{t^2+1}{2} = y 
$$
so solving for $t$ we find
$$
t^2-2yt+1 = 0 \implies t = \frac{2y\pm\sqrt{4y^2-4}}{2} = y\pm\sqrt{y^2-1}
$$
therefore
$$
\mathrm{e}^{x} = t \implies x = \ln\left(y\pm\sqrt{y^2-1}\right) = \mathrm{arcosh}(y)
$$
since we are looking for real valued function then we can ignore the minus sign.
At least in my head anyway.
</s>