<s>[INST] How find this integral $\int\frac{\sqrt{x-1}\arctan{(x\sqrt{x-1})}}{x}dx$ Find this integral
$$\int\dfrac{\sqrt{x-1}\arctan{(x\sqrt{x-1})}}{x}dx$$
My try: let 
$$\arctan{(x\sqrt{x-1})}=t$$
and that's very ugly,Thank you
 [/INST] $\because$ according to http://integrals.wolfram.com/index.jsp?expr=%28x-1%29%5E%281%2F2%29%2Fx&random=false, $\int\dfrac{\sqrt{x-1}}{x}dx=2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1}+C$
$\therefore\int\dfrac{\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})}{x}dx$
$=\int\tan^{-1}(x\sqrt{x-1})~d(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})$
$=(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})~d(\tan^{-1}(x\sqrt{x-1}))$
$\because$ according to http://www.wolframalpha.com/input/?i=d%2Fdx%28arctan%28x%28x-1%29%5E%281%2F2%29%29%29, $\dfrac{d}{dx}(\tan^{-1}(x\sqrt{x-1}))=\dfrac{3x-2}{2\sqrt{x-1}(x^3-x^2+1)}$
$\therefore(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})~d(\tan^{-1}(x\sqrt{x-1}))$
$=2\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-2\tan^{-1}\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-\int(2\sqrt{x-1}-2\tan^{-1}\sqrt{x-1})\dfrac{3x-2}{2\sqrt{x-1}(x^3-x^2+1)}dx$
$=2\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-2\tan^{-1}\sqrt{x-1}\tan^{-1}(x\sqrt{x-1})-\int\dfrac{3x-2}{x^3-x^2+1}dx+\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$
$\int\dfrac{3x-2}{x^3-x^2+1}dx$ is just an integral of rational functions and should have close form.
For $\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$ ,
Let $u=\sqrt{x-1}$ ,
Then $x=u^2+1$
$dx=2u~du$
$\therefore\int\dfrac{(3x-2)\tan^{-1}\sqrt{x-1}}{\sqrt{x-1}(x^3-x^2+1)}dx$
$=\int\dfrac{2(3u^2+1)\tan^{-1}u}{(u^2+1)^3-(u^2+1)^2+1}du$
Then you can separate it to the terms of $\int\dfrac{\tan^{-1}u}{au+b}du$ or $\int\dfrac{\tan^{-1}u}{pu^2+qu+r}du$ by partial fraction. According to http://pi.physik.uni-bonn.de/~dieckman/IntegralsIndefinite/IndefInt.html, they relate to the polylogarithm function.
</s>