<s>[INST] Evaluate : $I=\int_0^{\infty}\frac{\ln (1+ax+x^{2})}{1+x^2}\,dx$ Does the following integral have a closed form :
$$I=\displaystyle\int\limits_0^{\infty}\frac{\ln (1+ax+x^{2})}{1+x^2}dx$$
Where $|a|≤1$ , 
I was trying using Feynman's trick.
Define
$$I(b)=\displaystyle\int\limits_0^{\infty}\frac{\ln (b(1+x^{2})+ax)}{1+x^2}dx$$
Differentiating with respect to $b$ we get : 
$$I'(b)=\displaystyle\int\limits_0^{\infty}\frac{1}{b+ax+bx^{2}}dx$$
$$=2\left(\frac{π}{2\sqrt{4b^{2}-a^{2}}}-\frac{\arctan \left(\frac{a}{\sqrt{4b^{2}-a^{2}}}\right)}{\sqrt{4b^{2}-a^{2}}}\right)$$
Known : 
$$\displaystyle\int \frac{1}{\sqrt{4b^{2}-a^{2}}}db=\frac{\log (2 x + \sqrt{-a^2 + 4 x^{2}})}{2}$$ 
my problem in this integral : 
$$\displaystyle\int\limits_0^{1}\frac{\arctan \left(\frac{a}{\sqrt{4b^{2}-a^{2}}}\right)}{\sqrt{4b^{2}-a^{2}}}db=?$$
Of course here 
$$I=I(1)=I(0)+\int\limits_0^{1}I'(b)db$$
$$I(0)=\frac{π\ln a}{2}$$
I already waiting your hints or solution.
 [/INST] Parameterising  $2\sin a$  instead of $a$ yields the integral
$$
I(a)=\int_{0}^{\infty} \frac{\ln \left(1+2 x \sin a+x^{2}\right)}{1+x^{2}} d x,
$$
where $a\in (0, \frac{\pi}{2}). $
Differentiating $I(a)$ w.r.t. $a$ yields
$$
\begin{aligned}
I^{\prime}(a) &=\int_{0}^{\infty} \frac{2 x \cos a}{\left(1+x^{2}\right)\left(1+2 x \sin a+x^{2}\right)} d x \\
&=\cot a\int_{0}^{\infty}\left(\frac{1}{1+x^{2}}-\frac{1}{1+2 x \sin a+x^{2}}\right) d x \\
&=\cot a\left[\tan ^{-1} x-\frac{1}{\cos a} \tan ^{-1}\left(\frac{x+\sin a}{\cos a}\right)\right]_{0}^{\infty} \\
&=\cot a\left[\frac{\pi}{2}-\frac{1}{\cos a}\left(\frac{\pi}{2}-a\right)\right]
\end{aligned}
$$
Integrating $I’(a)$ back to $I(a)$, we have
$$
\begin{aligned}
I\left(a\right)- \underbrace{I(0)}_{=\pi\ln 2}  &=\int_{0}^{a} \cot x\left[\frac{\pi}{2}-\frac{1}{\cos x}\left(\frac{\pi}{2}-x\right)\right] d x \\
&=\frac{\pi}{2} \underbrace{ \int_{0}^{a}\left(\cot x-\frac{1}{\sin x}\right) d x}_{= 2 \ln \left(\cos \frac{a}{2}\right)} + \underbrace{\int_{0}^{a} \frac{x}{\sin x} d x}_{K}
\end{aligned}
$$
$$
\begin{aligned}
K &=\int_{0}^{a} \frac{x}{\sin x} d x=\int_{0}^{a} x\, d\left[\ln \left(\tan \frac{x}{2}\right)\right] \\
&=\left[x\ln \left(\tan \frac{x}{2}\right)\right]_{0}^{a}-\int_{0}^{a} \ln \left(\tan \frac{x}{2}\right) d x \\
&=a \ln \left(\tan \frac{a}{2}\right)-2 \int_{0}^{\frac{a}{2}} \ln (\tan x) d x 
\end{aligned}
$$
Now we can conclude that, for any $a\in (0, \frac{\pi}{2}]$,
$$
\boxed{I=I\left(a\right)=\pi \ln 2+\pi \ln \left(\cos \frac{a}{2}\right)+a \ln \left(\tan \frac{a}{2}\right) -2 \int_{0}^{\frac{a}{2}} \ln (\tan x) d x}
$$
Similarly, for any $a\in (-\frac{\pi}{2},0)$,
$$
\boxed{I=I\left(a\right)=\pi \ln 2+\pi \ln \left(\cos \frac{a}{2}\right)-a \ln \left(\tan \frac{-a}{2}\right) +2 \int_{0}^{\frac{-a}{2}} \ln (\tan x) d x}
$$
For the last integral, we may evaluate it by expressing it as a series of sine as below:
$$
\int_{0}^{x} \ln (\tan \theta) d \theta=-\sum_{k=0}^{\infty} \frac{\sin ((4 k+2) x)}{(2 k+1)^{2}}
$$
from the web.
Example 1
$$\begin{aligned}\quad \int_{0}^{\infty} \frac{\ln \left(1-x+x^{2}\right)}{1+x^{2}} d x&= 
I\left(-\frac{\pi}{6}\right)\\&= \frac{\pi}{2}[\ln (2+\sqrt{3})]+\frac{\pi}{6} \ln (2+\sqrt{3})+2\left(-\frac{2}{3} G\right)\\& =\frac{2 \pi}{3} \ln (2+\sqrt{3})-\frac{4}{3} G \end{aligned}$$
Example 2
$$
\begin{aligned}
&\int_{0}^{\infty} \frac{\ln \left(1+\sqrt{2} x+x^{2}\right)}{1+x^{2}}\\=&I\left(\frac{\pi}{4}\right) \\
=& \pi \ln \left(2 \cos \frac{\pi}{8}\right)-\frac{\pi}{4} \ln \left(\tan \frac{\pi}{8}\right)-2 \int_{0}^{\frac{\pi}{8}} \ln (\tan x) d x \\
=& \frac{\pi}{2} \ln (2+\sqrt{2})+\frac{\pi}{4} \ln (\sqrt{2}+1) -2\left[\frac{\pi}{8} \ln (\sqrt{2}-1)-\Im\left(\operatorname{Li}_{2}(i(\sqrt{2}-1))\right]\right.\\=& \pi \ln [\sqrt[4]{2}(\sqrt{2}+1)] +2 \Im\left(\operatorname{Li}_{2}(i(\sqrt{2}-1))\right.
\end{aligned}
$$
where the last integral see post
The answer is not satisfactory as the last integral is hard to evaluate!
</s>