<s>[INST] Three squares in a rectangle One of my colleagues gave me the following problem about 15 years ago:
Given three squares inside a 1 by 2 rectangle, with no two squares overlapping, prove that the sum of side lengths is at most 2. (The sides of the squares and the rectangle need not be parallel to each other.)
I couldn't find a solution or even a source for this problem. Does anyone know about it? I was told that this was like an exercise in combinatorial geometry for high school contests such as IMO, but I'm not sure if this problem was listed in such competitions. Has anyone heard of this problem, or does anyone know how to solve it?
Any kind of help will be appreciated.
 [/INST] Since the squares are convex, we can draw lines which separate them. In particular, if two separating lines go from $(b-a,0)$ to $(b,1)$ and from $(c,1)$ to $(c+d,0)$, then we can prove the result in terms of those lines and those variables.
So: let the rectangle go from $(0,0)$ to $(2,1)$. Let $A$ be the leftmost square (or one such square). Let $C$ be the rightmost square (or one such square). Let $B$ be the other square.
Draw a line separating $A$ and $B$, and let $(b,1)$ and $(b-a,0)$ be its intersections with the lines $y=1$ and $y=0$.
Draw a line separating $B$ and $C$, and let $(c,1)$ and $(c+d,0)$ be its intersections with the lines $y=1$ and $y=0$.
Reasoning as in user21820's answer, we assume wlog that:

*

*$0<a$, so $A$ is left and above the line separating $A$ and $B$;

*$0<d$, so $C$ is right and above the line separating $B$ and $C$;

*$0<b$ and $c<2$ and $a-b+c+d>0$, so $B$ is below both lines.

(Since the lines may leave the rectangle, we do not assume $b-a>0$ or $b<2$ or $c>0$ or $c+d<2$.)
Lemma (proved at the end):
\begin{align}
\text{sidelength of }A &\le \min\!\left(\frac{b}{a+1},\,1\right)\\
\text{sidelength of }B &\le \min\!\left((a-b+c+d)u,\,1\right)\\
\text{sidelength of }C &\le \min\!\left(\frac{2-c}{d+1},\,1\right)
\end{align}
where
$$u=\max\left(
\frac{1}{a+d+1},
\frac{\sqrt{a^2+1}}{a^2+a+d+1},
\frac{\sqrt{d^2+1}}{d^2+d+a+1}
\right)$$
The factor $u$ satisfies $1/(a+1)>u$ and $1/(d+1)>u$ so long as $a<3.66$ and $d<3.66$ respectively. I will assume those inequalities for now to show that some functions are increasing or decreasing; I don't have a clean proof for those inequalities or without them yet.
In the corner case of $b=a+1$ and $c=1-d$, the side lengths are $1$, $0$ and $1$, and they sum to exactly $2$. We now use this in analyzing four cases.
Case I, $b\le a+1$ and $c\le 1-d$: The sum of the sidelengths is at most
$$\frac{b}{a+1}+(a-b+c+d)u+1$$ This is increasing in both $b$ and $c$, so its value is at most the corner value of $2$.
Case II, $b\le a+1$ and $c\ge 1-d$: The sum of the sidelengths is at most
$$\frac{b}{a+1}+(a-b+c+d)u+\frac{2-c}{d+1}$$ This is increasing in $b$ and decreasing in $c$, so its value is at most the corner value of $2$.
Case III, $b\ge a+1$ and $c\le 1-d$: The sum of the sidelengths is at most
$$1+(a-b+c+d)u+1$$
This is decreasing in $b$ and increasing in $c$, so its value is at most the corner value of $2$.
Case IV, $b\ge a+1$ and $c\ge 1-d$: The sum of the sidelengths is at most
$$1+(a-b+c+d)u+\frac{2-c}{d+1}$$
This is decreasing in both $b$ and $c$, so its value is at most the corner value of $2$.
So the sum of the sidelengths is at most $2$ in each case.
Proof of Lemma:
The sidelength of $A$ is clearly less than 1, and also clearly less than the maximum sidelength inscribed in the right triangle bounded by $x=0$, $y=1$, and the separator of $A$ and $B$. We use Polya's formula here to calculate the sidelength in the triangle as the maximum of $b/(a+1)$ and $b\sqrt{a^2+1}/(a^2+a+1)$; since $a>0$, the maximum is just $b/(a+1)$.
A similar use of Polya's result bounds the sidelength of $C$. Yet another use of that result, now for a triangle which may be acute or obtuse, bounds the sidelength of $B$ by $|a-b+c+d|u$. Since we assumed $a-b+c+d>0$, we write this bound as $(a-b+c+d)u$.
</s>