<s>[INST] How to prove this inequality $(a^2+bc^4)(b^2+ca^4)(c^2+ab^4) \leq 64$? Question:
If $a,b,c$ are nonnegative real numbers such that $a+b+c=3,$ then

$$(a^2+bc^4)(b^2+ca^4)(c^2+ab^4) \leq 64$$

My try: I found the equality holds only if  $(a,b,c)=(2,0,1)$ or all of its permutations.
But I can't prove this inequality it. I would appreciate very much a proof.
This problem comes from:http://www.artofproblemsolving.com/Forum/viewtopic.php?f=52&t=562119
 [/INST] Without loss of generality, assume $a$ is smallest of $a, b, c$. Also, let $$f(a,b,c)=(a^2+bc^4)(b^2+ca^4)(c^2+ab^4)$$and firstly, if $a\le c\le b$, then$$f(a,b,c)-f(a,c,b)=(b^3-a^3) (c^3 - a^3) (b ^3- c^3) (a b c - 1)<0$$therefore we can assume $a\le b\le c$. Now, we will prove $$f(0,b,a+c)\ge f(a,b,c)$$which is, after full expansion,$$a (-a^6 b^4 c + a^5 b^3 - a^5 c^3 - a^4 b^5 c^5 - a^3 b c^7 - a^2 b^6 + 20 a^2 b^3 c^3 \\+ 15 a b^3 c^4 - a b^2 c^2 - b^7 c^4 + 6 b^3 c^5)+ 6 a^5 b^3 c + 15 a^4 b^3 c^2\ge0$$
and it is obvious that $-a^6 b^4 c + a^5 b^3\ge0$, $- a^5 c^3 - a^4 b^5 c^5 - a^2 b^6 + 20 a^2 b^3 c^3\ge0$ and $6 a^5 b^3 c + 15 a^4 b^3 c^2\ge0$. Therefore it is enough to show$$b c^2 (-a^3 c^5 + 15 a b^2 c^2 - a b - b^6 c^2 + 6 b^2 c^3)\ge0$$and from $-a^3c^5\ge-ab^2c^5$, $-ab\ge-abc^3\ge-b^2c^3$ and $-b^6c^2>-4b^3c^2\ge-4b^2c^3$, it is left to show$$b^2c^2(-a c^3 + 15 a +  c)\ge0$$and we can divide it with three cases.
Case 1) $c^3\le15$: $-a c^3 + 15 a +  c>a(15-c^3)\ge0$.
Case 2) $15^{1/3}< c\le2.6$: Firstly, $c>2.4$. The equation is decreasing with respect to $a$, therefore we need to show only for maximal value of $a$. If $a>0.3$, then $a+b+c\ge2a+c>0.6+2.4=3$, so maximal value is $0.3$. Also, $c^3\le2.6^3<20$. Therefore, $$-a c^3 + 15 a +  c\ge0.3(15-c^3)+c\ge-1.5+2>0$$
Case 3) $2.6<c\le3$: Similarly, it is enough to show for maximal value of $a$ which is $0.2$. Therefore, $$-a c^3 + 15 a +  c\ge0.2(15-27)+2.6>0$$
Therefore, we can assume that $0=a\le b\le c$. Now $f(a,b,c)=b^3c^6\le2^6\left(\frac{3\times b+6\times0.5c}{9}\right)^9=64$ and it is proved.
</s>