<s>[INST] Solve $5a^2 - 4ab - b^2 + 9 = 0$, $ - 21a^2 - 10ab + 40a - b^2 + 8b - 12 = 0$ 
Solve $\left\{\begin{matrix} 5a^2 - 4ab - b^2 + 9 = 0\\ - 21a^2 - 10ab + 40a - b^2 + 8b - 12 = 0. \end{matrix}\right.$

I know that we can use quadratic equation twice, but then we'll get some very complicated steps. Are there any elegant way to solve this? Thank you.
 [/INST] You can notice that many terms of 
$$(b+5a-4)^2=b^2+10ab-8b-40a+25a^2+16$$
appear in the first equation. Similarly, in the first one, you can notice $(b+2a)^2$.
By algebraic manipulation you get that the original equations are equivalent to
$$
\begin{align}
(b+5a-4)^2&=4(a^2+1)\\
(b+2a)^2&=9(a^2+1)
\end{align}
$$
which implies $4(b+2a)^2=9(b+5a-4)^2$ and $2(b+2a)=\pm 3(b+5a-4)$. This should simplify things a little.
(In each of the two possibilities you can express $b$ using $a$ as a linear expression.
Then you will get a quadratic equation in $a$. Or you can start by eliminating $a$.)
</s>