<s>[INST] Calculating $\int_0^1\frac{\ln^2x\ln(1-x)}{1-x}dx$ without using Beta function and Euler sum. Is it possible to show that
$$\int_0^1\frac{\ln^2x\ln(1-x)}{1-x}dx=-\frac12\zeta(4)$$
without using the Beta function
$$\text{B}(a,b)=\int_0^1 x^{a-1}(1-x)^{b-1}dx=\frac{\Gamma(a)\Gamma(b)}{\Gamma(a+b)}$$
and the generalized Euler sum
$$\sum_{n=1}^\infty \frac{H_n}{n^q}= \left(1+\frac{q}{2} \right)\zeta(q+1)-\frac{1}{2}\sum_{k=1}^{q-2}\zeta(k+1)\zeta(q-k),\quad q\ge 2\ ?$$
By integration by parts I found
$$\int_0^1\frac{\ln^2x\ln(1-x)}{1-x}dx=\color{blue}{\int_0^1\frac{\ln^2(1-x)\ln x}{x}dx}$$
Setting $1-x\to x$ gives the blue integral again.
This integral seems tough under such restrictions. All approaches are welcome.
thanks.
 [/INST] \begin{align}J&=\int_0^1 \frac{\ln^2 x\ln(1-x)}{1-x}dx\\
&\overset{\text{IBP}}=\underbrace{\left[\left(\int_0^x \frac{\ln^2 t}{1-t}dt-\int_0^1 \frac{\ln^2 t}{1-t}dt\right)\ln(1-x)\right]_0^1}_{=0}+\\&\int_0^1 \frac{1}{1-x}\left(\underbrace{\int_0^x \frac{\ln^2 t}{1-t}dt}_{u(t)=\frac{t}{x}}-\int_0^1 \frac{\ln^2 t}{1-t}dt\right)dx\\
&=\int_0^1\int_0^1 \left(\frac{x\ln^2(ux)}{(1-x)(1-ux)}-\frac{\ln^2 u}{(1-x)(1-u)}\right)dudx\\
&=\int_0^1 \int_0^1 \left(\frac{\ln^2 x}{(1-x)(1-u)}+\frac{2\ln x\ln u}{(1-x)(1-u)}-\frac{\ln^2(ux)}{(1-u)(1-ux)}\right)dudx\\
&=2\zeta(2)^2+\int_0^1\frac{1}{1-u}\left(\int_0^1 \frac{\ln^2 x}{1-x}dx-\underbrace{\int_0^1 \frac{\ln^2(ux)}{1-ux}dx}_{z(x)=ux}\right)du\\
&=2\zeta(2)^2+\int_0^1\frac{1}{u(1-u)}\left(u\int_0^1 \frac{\ln^2 z}{1-z}dz-\int_0^u \frac{\ln^2 z}{1-z}dz\right)du\\
&\overset{\text{IPP}}=2\zeta(2)^2-\int_0^1 \ln\left(\frac{u}{1-u}\right)\left(\int_0^1 \frac{\ln^2 z}{1-z}dz-\frac{\ln^2 u}{1-u}\right)du\\
&=2\zeta(2)^2+\int_0^1  \frac{\ln\left(\frac{u}{1-u}\right)\ln^2 u}{1-u}du=\underbrace{2\zeta(2)^2}_{=5\zeta(4)}+\underbrace{\int_0^1 \frac{\ln^3 u}{1-u}du}_{=-6\zeta(4)}-J
\end{align}
Therefore,
\begin{align}\boxed{J=-\frac{1}{2}\zeta(4)}\end{align}
</s>