<s>[INST] What are the local extreme values of $f(x,y)=x^3+x^2y-y^2-4y$? $f'_x=3x^2+2xy$, 
$f'_y=x^2-2y-4$
so I have to solve the equation system of 
$3x^2+2xy=0,x^2-2y-4=0$ as solutions I get that $((x=0),(y=0)),((x=-4),(y=6)),(x=1),((y=-\frac{3}{2}))$
After that:
$f''_x=6x+2y$,
$f''_y=-2$,
$f''_{xy}=2x=f''_{yx}$
so I get the determinant
$\begin{vmatrix}
6x+2y & 2x \\ 
2x & -2
\end{vmatrix}$
for $(x=4),(y=6)$
$\begin{vmatrix}
6 \cdot -4+2 \cdot6 & 2 \cdot -4  \\ 
2 \cdot -4 & -2
\end{vmatrix}$=$-40$ there is at this point no local extreme value
for $(x=1),(y=-\frac{3}{2})$
$\begin{vmatrix}
6 \cdot 1+2 \cdot-\frac{3}{2}  & 2   \\ 
2   & -2
\end{vmatrix}$=$-10$ there is at this point no local extreme value
but what about $(x=0)(y=0)$? the value of the determinant will be $0$but it says nothing about the extreme value at this point.
Is my solution correct til this point?
 [/INST] The stationary points are determined as the solutions for
$$
3x^2+2xy = 0\\
x^2-2y-4 = 0
$$
giving the set
$\{(-4,6),(0,-2),(1,-3/2)\}$
Now taking the Hessian
$$
H = \left(\begin{array}{cc}3x+y & x\\ x & -1\end{array}\right)
$$
To qualify the stationary points we should evaluate $H$ in such set, verifying it's eigenvalues. For both eigenvalues negative we have a local maximum. For both eigenvalues positive we have a local minimum and for eigenvalues  with opposite sign we have a saddle point.  In this case the point $(0,2)$ gives negative eigenvalues for $H$ so here we have a local maximum.
Attached a level contour plot showing the local maximum (red) and the two saddle points (blue)

</s>