<s>[INST] Maximizing $x^2y$ given $x^2+y^2=100$, without using the AM-GM inequality and calculus tools Problem says:

Let $x^2+y^2=100$, where $x,y>0$. For which ratio of $x$ to $y$, the value of $x^2y$ will be maximum?

I know these possible tools:

*

*AM-GM inequality


*Calculus tools

Here, I want to escape from all of the tools I mentioned above.
I will try to explain my attempts in the simplest sentences. (my english is not enough, unfortunately). I will not prove any strong theorem and also I'm not sure what I'm doing exactly matches the math, rigorously.

Solution I made:
First, it is not necessary to make these substitutions.  I'm just doing this to work with smaller numbers.
Let, $x=10m, ~y=10n$,
where $0<m<1,~ 0<n<1$, then we have
$$ x^2+y^2=100 \iff m^2+n^2=1$$
$$x^2y=1000m^2n$$
This means,
$$\max\left\{x^2y\right\}=10^3\max\left\{m^2n\right\}$$
$$m^2n=n(1-n^2)=n-n^3$$
Then suppose that,
$$\begin{align}\max\left\{n-n^3 \mid 0<n<1\right\}&=a, a>0&\end{align}$$
This implies
$$n-n^3-a≤0,~ \forall n\in\mathbb (0,1)$$
$$n^3-n+a≥0,~\forall n\in\mathbb (0,1)$$
Then, we observe that
$$\begin{align}n^3-n+a≥0, \forall n\in (0,1) ~ \text{and} ~ \forall n≥1\end{align}$$
This follows
$$ n^3-n+a≥0, ~ \forall n>0.$$

Using the last conclusion, I assume  that there exist $u,v>0$, such that

$$n^3-n+a=(n-u)^2(n+v)≥0.$$
If $n>0$, then the equality occurs, if and only if
$$n=u>0$$


Based on these, we have:
$$\begin{align}n^3-n+a= (n-u)^2(n+v)≥0 \end{align}$$
$$\begin{align}n^3-n+a = & n^3 - n^2(2u-v)+ n(u^2 - 2 u v ) + u^2v & \end{align}$$
$$\begin{align} \begin{cases} 2u-v=0 \\ u^2-2uv=-1 \\u^2v=a \\u,v>0 \end{cases} &\implies \begin{cases} v=2u \\ u^2-4u^2=-1 \\ 2u^3=a \\ u,v>0 \end{cases}\\
&\implies \begin{cases}  u=\frac{\sqrt 3}{3} \\ v=\frac{2\sqrt 3}{3}\\ a=2\left(\frac{\sqrt 3}{3} \right)^3=\frac{2\sqrt 3}{9} \end{cases} \end{align}$$
$$\begin{align}n^3-n+\frac{2\sqrt 3}{9} &=\left(n-\frac{\sqrt 3}{3} \right)^2\left(n+\frac{2\sqrt 3}{3}\right)≥0.&\end{align}$$
As a result, we deduce that
$$\begin{align}n-n^3-\frac{2\sqrt 3}{9} &=-\left(n-\frac{\sqrt 3}{3} \right)^2\left(n+\frac{2\sqrt 3}{3}\right)≤0, &\forall n\in (0,1).&\end{align}$$
$$\begin{align}\max\left\{n-n^3 \mid 0<n<1\right\}&=\frac{2\sqrt 3}{9}, ~ \text{at }~ n=\frac{\sqrt 3}{3}&\end{align}$$
Finally, we obtain
$$m=\sqrt{1-n^2}=\sqrt{1-\frac 13}=\frac{\sqrt 6}{3}$$
$$\frac xy=\frac mn=\sqrt 2.$$

Question:

*

*How much of the things I've done here are correct?

 [/INST] Your approach is good and the easiest way without calculus.

An Approach that Exposes the Core Ideas
The following approach is a slight modification of yours that simplifies the algebra using Vieta's formulas.
If
$$
x^2+y^2=100\tag1
$$
then
$$
\begin{align}
x^2y
&=100\cos^2(\theta)\cdot10\sin(\theta)\tag{2a}\\
&=1000\left(\sin(\theta)-\sin^3(\theta)\right)\tag{2b}
\end{align}
$$
So we wish to maximize $\sin(\theta)-\sin^3(\theta)$ for $0\lt\theta\lt\frac\pi2$ (since $x,y\gt0$). Suppose the maximum is $m$. Then your assumption/lemma says that $s-s^3-m=0$ has a double root. That is,
$$
s^3-s+m=(s-r)^2(s-q)\tag3
$$
Vieta's formulas say that for a monic cubic polynomial,
$$
\begin{align}
r+r+q&=0\tag{4a}\\
r^2+rq+rq&=-1\tag{4b}\\
r^2q&=-m\tag{4c}
\end{align}
$$
Explanation:
$\text{(4a)}$: the sum of the roots is the negative of the coefficient of $s^2$
$\text{(4b)}$: the sum of the pairwise products of the roots is the coefficient of $s$
$\text{(4c)}$: the product of the roots is the negative of the constant coefficient
Now things just fall into place:
$\text{(4a)}$ says that $q=-2r$
$\text{(4b)}$ says that $-3r^2=-1$, that is $r=\frac1{\sqrt3}$
$\text{(4c)}$ says that $-2r^3=-m$, that is $m=\frac2{3\sqrt3}$
Thus, the maximum of $s-s^3$ is $\frac2{3\sqrt3}$ which happens at $s=\frac1{\sqrt3}$, which is a value that $\sin(\theta)$ attains for $0\lt\theta\lt\frac\pi2$.
Thus, the maximum of $x^2y=1000\cdot\frac2{3\sqrt3}=\frac{2000}{3\sqrt3}$.
</s>