<s>[INST] Calculating coefficient I have a generating function,
$$ \frac{(1-x^7)^6}{(1-x)^6} $$
and I want to calculate the coefficient of $x^{26}$
Solution for this is,
$$ {26+5 \choose 5} - 6{19+5 \choose 5} + 15{12+5 \choose 5} - 20{5+5 \choose 5} $$
Is there formula for this? If there is, what is called?
If there is no formula, how can I calculate it?
Thanks!
 [/INST] It is convenient to use the coefficient of operator $[x^n]$ to denote the coefficient of $x^n$ in a series.

We obtain
  \begin{align*}
\color{blue}{[x^{26}]}&\color{blue}{\frac{\left(1-x^7\right)^6}{\left(1-x\right)^6}}\\
&=[x^{26}]\left(1-x^7\right)^6\sum_{j=0}^{\infty}\binom{-6}{j}(-x)^j\tag{1}\\
&=[x^{26}]\left(1-\binom{6}{1}x^7+\binom{6}{2}x^{14}-\binom{6}{3}x^{21}\right)\sum_{j=0}^\infty\binom{j+5}{5}x^j\tag{2}\\
&=\left([x^{26}]-6[x^{19}]+15[x^{12}]-20[x^5]\right)\sum_{j=0}^\infty\binom{j+5}{5}x^j\tag{3}\\
&\,\,\color{blue}{=\binom{31}{5}-6\binom{24}{5}+15\binom{17}{5}-20\binom{10}{5}}\tag{4}
\end{align*}
  in accordance with the claim.

Comment:


*

*In (1) we expand the denominator using the binomial series expansion.

*In (2) we expand the polynomial up to powers of $x^{21}$ since higher powers do not contribute to $[x^{26}]$ and we apply the binomial identity $\binom{-p}{q}=\binom{p+q-1}{p-1}(-1)^q$.

*In (3) we use the linearity of the coefficient of operator and apply the rule $[x^{p-q}]A(x)=[x^p]x^qA(x)$.

*In (4) we select the coefficients accordingly.
</s>