<s>[INST] Radius of largest circle in an ellipse Consider an ellipse whose major and minor axis of length $10$ and $8$ unit respectively. Then the radius of the largest circle which can be inscribed in such an ellipse if the circle's center is one focus of the ellipse.
What I tried:
Assuming that major axis and minor axis of an ellipse along coordinate axis.($x$ and $y$ axis respectively)
Then equation of ellipse is $\displaystyle \frac{x^2}{a^2}+\frac{y^2}{b^2}=1.$ where $a=5,b=4$
Then $\displaystyle b^2=a^2(1-e^2)$ . getting $\displaystyle e=3/5.$
coordinate of focus is $(\pm ae,0)=(\pm 3,0)$
Let equation of circle is $(x\pm 3)^2+y^2=r^2$
How do I solve it from here?
 [/INST] You want to minimize $r^2=(x-3)^2+y^2$ subject to $\frac{x^2}{25}+\frac{y^2}{16}=1$:
$$L(x,y,\lambda)=(x-3)^2+y^2+\lambda\left(1-\frac{x^2}{25}-\frac{y^2}{16}\right)\\
\begin{cases}L_x=2x-6-\frac{2x\lambda}{25}=0\\ L_y=2y-\frac{y\lambda}{8}=0\\ L_{\lambda}=1-\frac{x^2}{25}-\frac{y^2}{16}=0\end{cases} \Rightarrow (x,y)=(\pm5,0);\\
r^2(5,0)=2^2 \ \text{(min)}$$
Notes: 
1) WolframAlpha answer. 
2) If you minimize $r^2=(x+3)^2+y^2$ s.t. $\frac{x^2}{25}+\frac{y^2}{16}=1$, you will get $r^2(-5,0)=2^2$.
3) Foci $(\pm c,0)$ can be found from $c^2=a^2-b^2=5^2-4^2=3^2 \Rightarrow c=\pm 3$.
</s>