<s>[INST] Infimum of $\frac{(1+2a)(1+2b)(1+3a-b-ab)}{(1-b)(1+2b)(1+2a-a^2)+3a(1+2a)(1+2b-b^2)}$ Let $0<a,b<1$. What is the infimum of $$f(a,b)=\frac{(1+2a)(1+2b)(1+3a-b-ab)}{(1-b)(1+2b)(1+2a-a^2)+3a(1+2a)(1+2b-b^2)}?$$
It is possible to make the term $1+3a-b-ab$ in the numerator arbitrarily close to $0$ by letting $a\rightarrow 0$ and $b\rightarrow 1$. But this also makes the denominator go to zero, so it doesn't show that $f(a,b)$ can be close to zero.
 [/INST] To find the extrema inside the region $]0,1[ \times ]0,1[$, we should solve
$$\frac{\partial f(a,b)}{\partial a}=0$$
$$\frac{\partial f(a,b)}{\partial b}=0$$
After many algebraic operators, we obtain
$$(1+2b)(b-1)(2a^2b^2-b^2+b+2ab-3a^2b-2a-5a^2)=0$$
$$a(1+2a)(5a-2b+1-10ab+b^2-ab^2-4a^2b+2a^2b^2+8a^2)=0$$
We are only interested in solutions for which $0<a,b<1$, so
$$2a^2b^2-b^2+b+2ab-3a^2b-2a-5a^2=0$$
$$5a-2b+1-10ab+b^2-ab^2-4a^2b+2a^2b^2+8a^2=0$$
Now, I'm not sure how to continue with pen and paper, as the set of equations are quite lengthy and not easy to simplify. Using mathematical software, I received multiple solutions, but only one for which $0<a,b<1$
$$a_{min} = 0.1440863966$$
$$b_{min} = 0.6394981637$$
$$f(a_{min},b_{min})=0.9876434319$$
We have found the minimum inside $]0,1[ \times ]0,1[$, but it is still possible that the global minimum is located at the edges
$$f(a=0,b)=1$$
$$f(a=1,b)=\frac{6(1+2b)(2-b)}{11+20b-13b^2}$$
$$f(a,b=0)=\frac{(1+2a)(1+3a)}{5a^2+5a+1}$$
$$f(a,b=1)=1$$
For $a=1$ and $b=0$, the minimum values are
$$\frac{\partial f(a=1,b)}{\partial b}=0 \Rightarrow b=1, f(a=1,b=1)=1$$
$$\frac{\partial f(a,b=0)}{\partial a}=0 \Rightarrow a=0, f(a=0,b=0)=1$$
The values of $f(a,b)$ at the edges are larger than the minimum value of $f(a,b)$ in $]0,1[ \times ]0,1[$, so as the final answer we have
$$a_{min} = 0.1440863966$$
$$b_{min} = 0.6394981637$$
$$f(a_{min},b_{min})=0.9876434319$$
</s>