<s>[INST] Ask a about hard integral of $\int_{0}^{\infty} \log x \log (\frac{a^2}{x^2}+1) \log(\frac{b^2}{x^2}+1)dx$ I want to evaluate the integral:
$$I(a,b)=\int_{0}^{\infty} \log x \log (\frac{a^2}{x^2}+1) \log(\frac{b^2}{x^2}+1)dx$$
Attempt:$$\frac{\partial ^2I}{\partial a\partial b}=4ab\int_{0}^{\infty}\frac{\log x}{(a^2+x^2)(b^2+x^2)}dx=\frac{4ab}{b^2-a^2}\int_{0}^{\infty}\log x\left(\frac{1}{a^2+x^2}-\frac{1}{b^2+x^2}\right)dx$$
$$=\frac{4ab}{b^2-a^2}\frac{\pi}{2}\left(\frac{\log a}{a}-\frac{\log b}{b}\right)=\frac{2\pi(b\log a-a\log b)}{b^2-a^2}$$
Then $$I(a,b)=2\pi\int_{0}^{b}\int_{0}^{a}\frac{(y\log x-x\log y)}{y^2-x^2}dxdy$$
But this integral very hard to solve,can anyone help me,thank you! 
 [/INST] This derivation is slightly different from Maxim's, because I'm not fluent in Meijer G.  The beginning is the same, and assume $0<b<a.$  Then
$$ I(a,b)=b\,\int_0^\infty (\log{b} + \log{x} ) \log{(1+ \big(\frac{a/b}{x}\big)^2  ) }\log{(1+1/x^2)} dx $$
Let $r=b/a \le 1$ so that some series manipulations are permissible. The integral without the $\log{x}$ is easily performed in Mathematica:
$$  \int_0^\infty  \log{(1+ \big(r\,x\big)^{-2}  ) }\log{(1+1/x^2)} dx = \pi\Big( (1+\frac{1}{r})(\text{arctanh(r)} - \log{(1-r^2)})-2\log{r}\Big)$$
Define 
$$J(s;r)=\int_0^\infty x^s  \log{(1+ \big(r\,x\big)^{-2}  ) }\log{(1+1/x^2)} dx \,.$$
The objective is to find $$\frac{d}{ds}J(s;r)\Big|_{s=0} $$
Within Mathematica J(s;r) can be found in terms of elementary functions and Gauss's hypergeometric $F(a,b;c,x).$
$$\frac{J(s;r)}{\pi}=\sec{(\frac{\pi s}{2})} \Big\{\!\frac{2\,r^2}{
(s\!+\!1)(s\!\!+3)}F(1,\!\frac{s+3}{2};\! \frac{s+5}{2}, r^2)  -
\frac{2\,r^{1-s}}{
(s\!+\!1)(s\!-\!1)}F(1,\!\frac{1-s}{2};\! \frac{3-s}{2}, r^2)$$
$$+ \frac{1}{s\!+\!1}\Big[ r^{-1-s}\log{(1-r^2)} + \log{(-1+1/r^2)}+\frac{2}{s+1} - \pi\,\tan{(\pi\,s/2)}  \Big]
\Big\}$$
Do the derivative and take $s \to 0.$ $F(1,\frac{1}{2}; \frac{3}{2}, r^2)$ and 
 $F(1,\frac{3}{2}; \frac{5}{2}, r^2)$ evaluate to elementary functions.  However the derivatives with respect to $s$ of the hypergeometrics do not.  However, by using the series definition in terms of Pochhammer symbols, an easy calculation shows
$$ \frac{d}{ds} \frac{(3/2+s/2)_k}{(5/2+s/2)_k} \Big|_{s=0} = \frac{2k}{(2k+3)^2} \quad , \quad \frac{d}{ds} \frac{(1/2-s/2)_k}{(3/2-s/2)_k} \Big|_{s=0} = \frac{-2k}{(2k+1)^2}  $$  In detail,
$$\frac{d}{ds} F(1,\frac{1-s}{2}; \frac{3-s}{2}, r^2) \Big|_{s=0} 
= -\sum_{k=0}^\infty\frac{2k}{(2k+1)^2} r^{2k} = -\sum_{k=0}^\infty\frac{2k+1 -1}{(2k+1)^2} r^{2k} = $$
$$=-\frac{\text{arctanh(r)}}{r} + \sum_{k=0}^\infty \frac{r^{2k}}{(2k+1)^2}=
-\frac{\text{arctanh(r)}}{r} + \frac{1}{2r} \Big( \text{Li}_2(r) -  \text{Li}_2(-r) \Big)
$$
The series with the $(2k+3)^2$ in the denominator can be brought to this form with an index shift in the summation.  Collect all the results and you finally get
$$\frac{I(a,b)}{\pi\,b}= \log{b} \Big( (1+\frac{1}{r})(2\,\text{arctanh}(r) + \log{(1-r^2)} ) -2\log{r}  \Big)\, + 
\big(1-\frac{1}{r}\big) \big( \text{Li}_2(r) -  \text{Li}_2(-r) \big) $$
$$-\Big(\frac{\pi^2}{2} + \log{(r^{-2}-1)} +2\,\big(1+\log{r}+\frac{1}{r} \big)\,\text{arctanh}(r) + \frac{1+\log{r}}{r} \, \log{(1-r^2)} \Big) $$
</s>