<s>[INST] Find maximize value of this expression Let $a;b;c \in [0;2]$
Find maximum value of: $P=ab(a^2-b^2)+bc(b^2-c^2)+ca(c^2-a^2)$
After some factoring, I found:
$P=(a-b)(b-c)(a-c)(a+b+c)$
Then square it: $P^2=(a-b)^2(b-c)^2(c-a)^2(a+b+c)^2$
Now $P^2$ be a symmetric expression. But it' still hard. Can someone help, please? Thank you.
 [/INST] Let $P=f(a,b,c)$ and note that for $k\le \min\{2-a,2-b,2-c\}$
$$f(a+k,b+k,c+k)\ge f(a,b,c)$$
Thus, we can choose $a=2$ (WLOG). Now, we need to maximize
\begin{align}f(2,b,c)&=2b(4-b^2)+bc(b^2-c^2)+2c(c^2-4)
\\
&=2b(4-b^2)+c(b^3-8)+c^3(2-b)
\\
&=2b(4-b^2)+c(b-2)(b^2+2b+4-c^2)
\end{align}
Observe that $$b-2\le 0$$
$$b^2+2b+(4-c^2)\ge 0$$
Thus,
$$f(2,b,0)\ge f(2,b,c)$$
Now, we need to maximize
$$f(2,b,0)=2b(4-b^2)$$
Candidates are boundary values and values that make the first order equal to $0$. Checking all values gives us
$$\frac{32}{3\sqrt{3}}=f(2,\frac{2}{\sqrt{3}},0)\ge f(a,b,c)$$
Remember $f$ is symmetric.
Done!
</s>