<s>[INST] Optimize $xyz$ where $x+y+z=1$ and $x^2+y^2+z^2=1$? Im trying to optimize
$f(x,y,z)=xyz$ restricted to $g(x,y,z)=x+y+z=1$ and $h(x,y,z)=x^2+y^2+z^2=1$.
$∇f=(yz,xz,xy)$,  $∇g=(1,1,1)$ and $∇h=(2x,2y,2z)$.
I tried using the determinant $det(∇f,∇g,∇h)=yz(2z-2y)-xz(2z-2x)+xy(2y-2x)=0$ which I dont know what to do with and I cant simplify the determinant in a good way with row operations.
I also tried solving $z$ from $g=1$. $z=1-x-y$.
$f(x,y,1-x-y)=xy-x^2y-xy^2$ restricted to $h(x,y)=2x^2+2y^2-2x-2y+2xy+1$ with Lagrange multiplier but I made no progress there either as the partial derivatives got too messy.
 [/INST] Notice that $$1=1^2=(x+y+z)^2=x^2+y^2+z^2+2(xy+yz+xy)=1+2(xy+yz+xy)$$
Solving for the products, we learn all elementary symmetric polynomials in $(x,y,z)$: \begin{align*}
x+y+z&=1 \\
xy+xz+yz&=0 \\
xyz&=f
\end{align*} where $f$ is what we want to maximize.  By Vietá's formulae, $x$, $y$, and $z$ are the roots of $$g(t)=t^3-t^2-f=0$$
We must maximize $f$ so that $g$ has $3$ real roots.  To do this, analyze $g$ geometrically.
At local extrema, $$0=g'(t)=3t^2-2t=t(3t-2)$$  There are two extrema: one at $0$ and one at $\frac{2}{3}$.  Since $g(t)$ is increasing at large $|t|$, the former extremum is a maximum; the latter a minimum.
Thus $g$ has three real roots iff $$g(0)\geq0\geq g\left(\frac{2}{3}\right)$$  From the left-hand inequality, $f\leq0$, which is clearly sharp (take $x=y=0$, $z=1$).
</s>