<s>[INST] Positive definite binary quadratic forms Please help me to solve this question or introduce references that help me: 
Let $f(x,y) = ax^2 + bxy + cy^2$ be a reduced positive definite form. 
Suppose that $\gcd(x, y) = 1$ and that $f(x, y) ≤ a + |b| + c$. Show 
that $f(x,y)$ must be one of the numbers $a, c, a - |b| + c$ or $a + 
|b| + c$.
 [/INST] This answer breaks the problem into three cases: $y = 0$, $y = \pm 1$, and $|y| \ge 2$.  Recall that because $f$ is reduced and positive definite, $|b| \le a \le c$.
Assume first that $y = 0$.  Then because $\gcd(x, y) = \gcd(x, 0) = 1$, $x$ must equal $\pm 1$.  In this case, $f(x, y) = f(\pm 1, 0) = a$, which satisfies the condition $f \le a + |b| + c$.
Next, assume that $y = \pm 1$.  If $x = 0$, then $f(0, \pm 1) = c$.  If $x = \pm 1$, then $f(\pm 1, \pm 1) = a \pm b + c = a \pm |b| + c$.  Each of those three representations meets the conditions $\gcd(x, y) = 1$ and $f \le a + |b| + c$.  If $|x| \ge 2$, we first derive the following inequality starting with the triangle inequality:
\begin{align}
|2ax + by| & \ge |2ax| - |by|\\
           & \ge 4a - b &&\text{because } |x| \ge 2 \text{ and } y = \pm 1\\
           & \ge 4a - a &&\text{because } |b| \le a\\
           & = 3a
\end{align}
We now find a lower bound for $f(x, y)$:
\begin{align}
f(x, y) & = ax^2 + bxy + cy^2\\
& = \frac{1}{4a} \left( 4a^2x^2 + 4abxy + 4acy^2 \right)\\
& = \frac{1}{4a} \left( 4a^2x^2 + 4abxy + b^2y^2 - b^2y^2 + 4acy^2 \right)\\
& = \frac{1}{4a} \left( (2ax + by)^2 + (4ac - b^2)y^2 \right)\\
& \ge \frac{1}{4a} \left( (3a)^2 + 4ac - b^2 \right) &&\text{because } |2ax + by| \ge 3a \text{ and } y = \pm 1\\
& \ge \frac{1}{4a} \left( 9a^2 + 4ac - a^2 \right) &&\text{because } |b| \le a\\
& = 2a + c\\
& = a + a + c\\
& \ge a + |b| + c &&\text{because } |b| \le a
\end{align}
Due to the upper-bound condition $f \le a + |b| + c$, the only possible representation here is $f(x, \pm 1) = a + |b| + c$.
Finally, assume that $|y| \ge 2$.
\begin{align}
f(x, y) & = \frac{1}{4a} \left( (2ax + by)^2 + (4ac - b^2)y^2 \right) &&\text{as above}\\
& \ge \frac{4ac - b^2}{4a}y^2\\
& \ge \frac{4ac - b^2}{4a} \cdot 4 &&\text{because } |y| \ge 2\\
& = \frac{4ac - b^2}{a}\\
& \ge \frac{4ac - ac}{a} &&\text{because } |b| \le a \le c\\
& = 3c
\end{align}
Then
\begin{align}
3c \le f(x, y) & \le a + |b| + c &&\text{as just shown and using a required condition}\\
& \le 3c &&\text{because } |b| \le a \le c
\end{align}
Because both $f(x, y)$ and $a + |b| + c$ are sandwiched between $3c$ and $3c$, the only possible representation here is $f(x, y) = a + |b| + c$.
Thus, for all pairs of integers $(x, y)$ such that $\gcd(x, y) = 1$ and $f(x, y) \le a + |b| + c$, $f(x, y)$ must be one of the numbers $a$, $c$, $a - |b| + c$, or $a + |b| + c$.
Remark: This is Problem 7 of Section 3.7 and is similar to Lemma 3.24 in I. Niven, H. S. Zuckerman, H. L. Montgomery, An Introduction to the Theory of Numbers, 5th ed., Wiley (New York), 1991.  That book is a helpful reference for questions like this.
</s>