<s>[INST] How find this integral $I(a,b)=\iiint_{x^2+y^2+z^2\le 1}(ax+by)^2\,dx\,dy\,dz$ Find this integral
$$I(a,b)=\iiint_{x^2+y^2+z^2\le 1}(ax+by)^2\,dx\,dy\,dz$$
since
$$(ax+by)^2=a^2x^2+b^2y^2+2abxy$$
so
$$I=I_{1}+I_{2}+I_{3}$$
where
$$I_{1}=\iiint_{x^2+y^2+z^2\le 1}ax^2\,dV=a\iiint_{x^2+y^2+z^2\le 1}x^2\,dV$$
since
$$\iiint_{x^2+y^2+z^2\le 1}x^2\,dV=\iiint_{x^2+y^2+z^2\le 1}y^2\,dV$$
maybe have other methods? Thank you
 [/INST] Expanding out, your integral is 
$$a^2 \iiint_{x^2+y^2+z^2\le 1}x^2\,dV - 2ab \iiint_{x^2+y^2+z^2\le 1}xy\,dV + b^2 \iiint_{x^2+y^2+z^2\le 1}y^2\,dV$$
The middle term is zero since for fixed $y$ and $z$ the integrand is odd in $x$ and the domain of integration is a symmetric interval in $x$. Furthermore, by symmetry the first and third integrals are the same, so your answer will be 
$$ (a^2 + b^2) \iiint_{x^2+y^2+z^2\le 1}x^2\,dV $$
The integrand is constant in $y$ and $z$, so if we integrate in those variables first, we introduce a factor given by the area of the $yz$-cross section for fixed $x$, namely $\pi(1 - x^2)$ (This is because the cross section is the circle $y^2 + z^2 = 1 - x^2$, of radius $\sqrt{1 - x^2}$).
So the answer is
$$(a^2 + b^2) \int_{-1}^1 \pi(1 - x^2)x^2\,dx$$
$$= \pi(a^2 + b^2)\int_{-1}^1 (x^2 - x^4)\,dx$$
$$= \pi(a^2 + b^2) \bigg({x^3 \over 3} - {x^5 \over 5}\bigg)\bigg|_{x = -1}^{x=1}$$
$$= {4 \over 15}\pi(a^2 + b^2)$$
</s>