<s>[INST] about a difficult and weird Probability question Let W be the random variable that counts the number of tails before one gets r heads for a coin whose probability of heads is θ. Without using moment generating function, show that the mean and variance for W are [r(1-θ)]/θ and [r(1-θ)]/θ^2
Please help me with the complete proof. I have some ideas, but I can't write it in a formal way.
Thanks!
 [/INST] Let's apply the binomial theorem to see that
$$
(1-x)^{-n-1} = \sum_{k\geq 0} (-1^k) \binom{-n-1}{k} x^k
$$
We can negate the upper index in the binomial coefficient, incorporating the $(-1)^k$, to get
$$
\frac{1}{(1-x)^{(n+1)}} = \sum_{k\geq 0}   \binom{n+k}{k} x^k = \sum_{k\geq 0}   \binom{n+k}{n} x^k 
$$
and if we multiply that by $z^n$ and note that binomial coefficients with integer top less than the integer bottom are zero, we get our key identity
$$
\frac{x^n}{(1-x)^{(n+1)}} =  \sum_{k\geq 0}   \binom{k}{n} x^k 
$$
Consider $k$ to be the total number of tosses until the $r$-th head turns up on the $k$-th toss. (Obviously $k \geq r$.)  The probability of that happening is
$$f(k;r) = \binom{k-1}{r-1} \theta^{r-1} (1-\theta)^{k-r} \theta $$ which we can read off as " $r-1$ out of the first $k-1$ tosses came up heads; the remaining $k-r$ were tails; and then the last toss was the $r$-th head."
$$f(k;r) = \left( \frac{\theta}{1-\theta} \right)^r \binom{k-1}{r-1} (1-\theta)^k $$
Now let's calculate the expectation of the total number of tosses (not just the tails):
$$E(k;r) = \sum_{k\geq r} k  \left( \frac{\theta}{1-\theta} \right)^r \binom{k-1}{r-1} (1-\theta)^k = \left( \frac{\theta}{1-\theta} \right)^r \sum_{k\geq r} k \binom{k-1}{r-1} (1-\theta)^k
$$
But it is easy to see that $k \binom{k-1}{r-1}  = r \binom{k}{r}$ by incorporating the $k$ factor into the numerator and then seeing how to ket to a binomial coefficient by multiplying the denominator (by $r$).
$$E(k;r) = = \left( \frac{\theta}{1-\theta} \right)^r  r \sum_{k\geq r} \binom{k}{r} (1-\theta)^k
$$
upon which we apply our key identity (with $x = (1-\theta)$)to get
$$E(k;r) =  \left( \frac{\theta}{1-\theta} \right)^r  r \frac{(1-\theta)^r}{\theta^{r+1}} = \frac{r}{\theta}
$$
The mean number of tails is then 
$$\frac{r}{\theta} - r = r \frac{1-\theta}{\theta}$$ as was to be shown.
The variance of the number of tails is the same as the variance of the total number of tosses, as these differ by a fixed constant $r$.  This in turn is 
$E(k^2;r) - \left( E(k;r) \right)^2$.
$$ E(k^2;r) = \left( \frac{\theta}{1-\theta} \right)^r \sum_{k\geq r} k^2 \binom{k-1}{r-1} (1-\theta)^k =  \left( \frac{\theta}{1-\theta} \right)^r \left[ \sum_{k\geq r} (k+1)k \binom{k-1}{r-1} (1-\theta)^k - \sum_{k\geq r} k \binom{k-1}{r-1} (1-\theta)^k \right]
$$
$$= \left( \frac{\theta}{1-\theta} \right)^r \left[  r(r+1) \sum_{k\geq r} \binom{k+1}{r+1}(1-\theta)^k  - r \sum_{k\geq r}\binom{k}{r} (1-\theta)^k \right] =
\left( \frac{\theta}{1-\theta} \right)^r \left[  r(r+1) \sum_{m > r} \binom{m}{r+1}(1-\theta)^{m-1}  - r \frac{(1-\theta)^r}{\theta^{r+1}} \right]=\left( \frac{\theta}{1-\theta} \right)^r \left[  r(r+1) \frac{1}{1-\theta} \sum_{m > r} \binom{m}{r+1}(1-\theta)^m  \right]- \frac{r}{\theta} $$
$$= \left( \frac{\theta}{1-\theta} \right)^r   r(r+1) \frac{1}{1-\theta} \frac{(1-\theta)^{r+1}}{\theta^{r+2}}  - \frac{r}{\theta} = r\frac{r+1 - \theta}{\theta^2}
$$
From which
$$\sigma^2 = E(k^2;r) - \left( E(k;r) \right)^2= r \frac{1-\theta}{\theta^2}$$ 
which was to be shown. 
</s>