<s>[INST] coefficient of $x^{17}$ in the expansion of $(1+x^5+x^7)^{20}$ I found this questions from past year maths competition in my country, I've tried any possible way to find it, but it is just way too hard.

find the coefficient of $x^{17}$ in the expansion of $(1+x^5+x^7)^{20}$
(A)3400 (B)3410 (C)3420 (D)3430 (E)3440

so it would be $$x^{140} + ...... + 1$$
This requires binomial theorem and Multinomial theorem, but I'm not sure how to calculate it. Any tips or formula would be appreciate.
 [/INST] So if you think about 
$$ (1 + x^5 + x^7)^{20} $$ 
That intuitively is just 
$$ ((1 + x^5) + x^7) \times ((1 + x^5) + x^7) \times ((1 + x^5) + x^7) ... $$ 
Which can be expanded out term by term. By the Binomial Theorem as 
$$ (1 + x^5)^{20} (x^7)^0 + \begin{pmatrix} 20 \\ 1\end{pmatrix}(1 + x^5)^{19}x^7 + \begin{pmatrix} 20 \\ 2\end{pmatrix}(1 + x^5)^{18}x^{14} ...$$
Now we can take each of the $$(1 + x^5)$$ terms and expand them as well. 
Note that the number 17 can be expressed as sum in terms of multiples of 5 and 7 as
$$ 10 + 7$$
If we consider say $15$ in the sum it's too big, and same for $14$ likewise arguments can be made to show that $5$ by itself won't add to a positive multiple of 7 to make 17. 
So that means every power of 17 is contained in the term
$$\begin{pmatrix} 20 \\ 1\end{pmatrix}(1 + x^5)^{19}x^7 $$ 
Of our binomial expression. We need to find the coefficient of $x^{10}$ in 
$$ (1 + x^5)^{19} $$ 
Call that C. Then 
$$ C \begin{pmatrix} 20 \\ 1\end{pmatrix}$$ 
Is the answer. 
By Binomial Theorem:
$$  (1 + x^5)^{19}  = 1 + \begin{pmatrix} 19 \\ 1\end{pmatrix}x^5 + \begin{pmatrix} 19 \\ 2\end{pmatrix}x^{10} ... $$ 
So then $C = \begin{pmatrix} 19 \\ 2\end{pmatrix} $
So the answer then is 
$$\begin{pmatrix} 19 \\ 2\end{pmatrix} \begin{pmatrix} 20 \\ 1\end{pmatrix} $$
To get more advanced techniques (it is worthwhile to take a look at the multinomial theorems). They generalize these ideas for arbitrarily large sums raised to integer powers like the binomial theorem for 2 elements raised to a power.
Note that
$$\begin{pmatrix} 19 \\ 2\end{pmatrix} \begin{pmatrix} 20 \\ 1\end{pmatrix}  = \frac{19!}{2!17!} 20 = 19 \times 9 \times 20 = 3420$$
</s>