<s>[INST] Nested equilateral triangles Let triangle $ABC$ is an equilateral triangle. Triangle $DEF$ is also an equilateral triangle and it is inscribed in triangle $ABC \left(D\in BC,E\in AC,F\in AB\right)$. Find $\cos\measuredangle DEC$ if $AB:DF=8:5$.

Firstly, I would be very grateful if someone can explain to me how I am supposed to draw the diagram. Obviously I have made it by sight.
Let $\measuredangle DEC=\alpha$. We can note that $\triangle AEF \cong \triangle BFD \cong CDE$. This is something we can always use in such configuration. So $$AE=BF=CD, $$ $$AF=BD=CE.$$ Let $AB=BC=AC=8x$ and $DF=DE=EF=5x$. If we denote $CD=y,$ then $CE=AC-AE=AC-CD=8x-y$. Cosine rule on $CED$ gives $$25x^2=(8x-y)^2+y^2-2y\cos60^\circ(8x-y)$$ which is a homogenous equation. I got that $\dfrac{y}{x}=4\pm\sqrt{3}.$ Now using the sine rule on $CED$ $$\dfrac{CD}{DE}=\dfrac{\sin\alpha}{\sin60^\circ}\Rightarrow \sin\alpha=\dfrac{\sqrt{3}}{10}\cdot\dfrac{y}{x}=\dfrac{4\sqrt3\pm3}{10}.$$ Now we can use the trig identity $\sin^2x+\cos^2x=1$ but it doesn't seem very rational. Can you give me a hint? I was able to find $\sin\measuredangle DEC$ in acceptable way, but I can't find $\cos\measuredangle DEC$...
 [/INST] WLOG, $AB = 8, DF = 5$. Say $\angle DEC = \theta$
If $CD = x$ then $CE = 8 - x$
Applying sine law in $\triangle CDE$,
$\displaystyle \frac{\sin 60^0}{5} = \frac{\sin \theta}{x} = \frac{\sin(60^0+\theta)}{8-x}$
From first two, we have $\sin \theta = \frac{x \sqrt3}{10}$
From first and third, $\sin 60^0 \cos \theta + \cos 60^0 \sin\theta = \frac{4\sqrt3}{5} - \frac{x \sqrt3}{10}$
$\implies \cos\theta = \frac{8}{5} - \frac{3x}{10}$
Applying $sin^2\theta + \cos^2\theta = 1$, we get
$x^2 - 8x+\frac{39}{3} = 0 \implies x = 4 \pm \sqrt3$
So, $\cos\theta = \frac{4 \mp 3\sqrt3}{10}$ (please note $\pm$ for $x$ and corresponding $\mp$ for $\cos \theta$)
</s>