<s>[INST] If $(a-b+5)x^2 + (2b+1)y^2 = b^2-2b-11$ is the equation of a unit circle, then find the sum of the possible values of $ab$. If 
$$(a-b+5)x^2 + (2b+1)y^2 = b^2-2b-11$$ 
is the equation of the unit circle, then what is the sum of the values ​​that $ab$ can take?
 [/INST] Here's an extended hint ...

The standard equation for an origin-centered unit circle is $x^2+y^2=1$. More generally, such a circle has the equation
$$p x^2 + p y^2 = p$$
for any non-zero $p$. Therefore, for $$(a-b+5)x^2+(2b+1)y^2=b^2-2b-11 \tag{1}$$
to be the equation of a unit circle, the coefficients of $x^2$ and $y^2$, and the expression on the right-hand side, must all be equal (and non-zero).
$$a-b+5=2b+1=b^2-2b-11  \quad(\neq 0)\tag{2}$$
The last two parts of $(2)$ allow us to solve for $b$:
$$2b+1=b^2-2b-11 \quad\to\quad b^2-4b-12=0 \tag{3}$$
The first two parts of $(2)$ then tell us what $a$ is:
$$a-b+5=2b+1 \quad\to\quad a=3b-4 \tag{4}$$
The rest of the exercise is left to the reader.
</s>