<s>[INST] Quadratic inequalities 

This is what I tried. I tried finding limits of y and then equating them with the given limits, but I could not simplify it further. The given options for this question are:


*

*a+b=23

*a^2+b^2=277

*a+b=17

*a^2=b^2=149

 [/INST] You want $-5 \le \dfrac{x^2+ax+b}{x^2+2x+3}\le 4$ for all real $x$.
Noting $x^2+2x+3 = (x+1)^2+2 > 0$, we can multiply with this throughout to get
$$-5x^2-10x-15 \le x^2 + ax + b \le 4x^2+8x + 12$$
Now take the left inequality, and we have 
$6x^2+(a+10) x+(15+b) \ge 0$.  As this needs to hold true for all real $x$, we can set a condition that the discriminant should never be positive.  However if you also want equality to hold for some value of x, we want the discriminant to be zero. So $(a+10)^2 = 24(15+b)$ is one condition.
Similarly, looking at the right inequality, we have
$3x^2 + (8-a)x+(12-b) \ge 0$, so the discriminant condition gives us $(8-a)^2 = 12(12-b)$.
Thus we have the two equations to solve for $a, b \in \mathbb{N}$. Solving these, we have $(a, b) = (14, 9)$.
</s>