<s>[INST] Combinatorics for integer solutions How many integer solutions are there to the following equation?
$x_1 + x_2 + x_3 = 17$
a) if $x_1 > 1, x_2 > 2, x_3 > 3$
b) if $x_1 < 6,  x_3 > 5$ and $x_2$ can be any integer.
c) if $x_1 < 4, x_2 < 3, x_3 < 5$
What is a systematic way to approach and solve these types of problems?
 [/INST] Use generating functions. I.e., the value of $x_1$ is represented by $z^2 + z^3 + \cdots = z^2 (1 - z)^{-1}$, and similarly the others. For all three variables it is in case (a):
$$
[z^{17}] z^2 \cdot z^3 \cdot z^4 \cdot (1 - z)^{-3} = (-1)^{17} \binom{-3}{17}
                     = [z^8] (1 - z)^{-3}
                     = \binom{8 + 3 - 1}{3 - 1}
                     = 45
$$
Case (b) gives $1 + \cdots + z^5 = (1 - z^6) / (1 - z)$ for $x_1$, $z^6 + z^7 + \cdots = z^6 (1 - z)^{-1}$ for $x_2$ and $(1 - z)^{-1}$ for $x_3$ (if we restrict $0 \le x_i$, that is; otherwise there are infinite solutions). Similar to before:
$$
[z^{17}] z^6 \cdot (1 - z^6) \cdot (1 - z)^{-2}
          = [z^{11}] (1 - z^6) \sum_{r \ge 0} (-1)^r \binom{-2}{r} z^r
          = [z^{11}] (1 - z^6) \sum_{r \ge 0} \binom{r + 1}{1} z^r
          = \binom{11 + 1}{1} - \binom{6 + 1}{1}
          = 5
$$
Case (c) is the most interesting one. As before:
\begin{align}
[z^{17}] \frac{1 - z^4}{1 - z} \cdot &\frac{1 - z^3}{1 - z} \cdot \frac{1 - z^5}{1 - z}
   = (1 - z^3 - z^4 - z^5 + z^7 + z^8 + z^9 - z^{12})
      \cdot \sum_{r \ge 0} (-1)^r \binom{-3}{r} z^r \\
   &= (1 - z^3 - z^4 - z^5 + z^7 + z^8 + z^9 - z^{12})
      \cdot \sum_{r \ge 0} \binom{r + 2}{2} z^r \\
   &= \binom{19}{2} - \binom{16}{2} - \binom{15}{2} - \binom{14}{2}
        + \binom{12}{2} + \binom{11}{2} + \binom{10}{2} - \binom{7}{2}
\end{align}
The result is obviously 0, as $3 + 2 + 4 < 17$...
</s>