<s>[INST] Negative variance. 
I've an urn with 5 red and 3 white balls. I'm randomly drawing a ball
  from the urn each round (without replacement). Let $X$ be the number
  of balls taken from the urn until the first white ball is chosen. I'm
  asked for the pmf, expected value and variance of $X.$

Since we can at most take 5 red balls before there is a white ball with 100 % certainty, we have that $k=1,2,3,4,5.$ We can ignore all the previous balls that we don't look at, the probability will therefore remain the same for all the draws, that is the pdf is $$P(X=k)=3/8, \ \ k=1,2,...,5.$$
For the expected value we then have that 
$$E(X)=\sum_{k=1}^{5}k\cdot\frac{3}{8}=\frac{45}{5}\approx 5.6.$$
For the vairance, we also need $E(X^2),$ so 
$$E(X^2)=\sum_{k=1}^{5}k^2\cdot\frac{3}{8} = \frac{165}{8}\approx 20.6.$$
So $$Var(X)= E(X^2)-E(X)^2=20.6-5.6^2=-10.67.$$
But negative variance, can't be correct! Where am I wrong?
 [/INST] Continuing from comments:
Letting $X$ be the random variable counting the number of balls pulled until pulling the first white ball, we try to find the probability distribution.
To do this, temporarily label each of the balls uniquely.  For the $k$'th ball to be the first white ball, that implies that the first $k-1$ balls are all red.  These $k-1$ red balls can be pulled in $5\frac{k-1}{~}=P(5,k-1)=\frac{5!}{(5-(k-1))!}$ ways.  (For example, pulling two red balls can be done in $5\cdot 4$ ways and pulling three red balls can be done in $5\cdot 4\cdot 3$ ways, etc...).  Then, pulling a white ball can be done in $3$ ways, so multiplying we get a total of $5\frac{k-1}{~}\cdot 3$ ways in which we can pull $k$ balls where only the $k$'th ball is white.  Then, recognize that the number of ways of pulling $k$ balls without regards to color will be $8\frac{k}{~}$.
Taking the ratio then yields the probability we are after: $Pr(X=k)=\dfrac{5\frac{k-1}{~}\cdot 3}{8\frac{k}{~}}$ (for values of $k\geq 1$.  For $k=0$ the probability is zero as pulling a white ball requires pulling at least one ball)
Alternatively, one could see the same result by using conditional probability arguments.  For the $k$'th ball to be the first white ball to occur, that implies that the first ball is red which occurs with probability $\frac{5}{8}$, the second ball is red which occurs with probability $\frac{4}{7}$ given that the first ball was red, the third ball was red... on up until the $k-1$'st ball is red, followed by the final ball being white.
Rewritten and evaluated, this yields the following table:
$\begin{array}{|c|c|c|c|}\hline k&Pr(X=k)&\text{with fractions}&\text{simplified}\\
\hline
1&\dfrac{5\frac{0}{~}\cdot 3}{8\frac{1}{~}}&\dfrac{3}{8}&\dfrac{3}{8}\\\hline
2&\dfrac{5\frac{1}{~}\cdot 3}{8\frac{2}{~}}&\dfrac{5}{8}\cdot\dfrac{3}{7}&\dfrac{15}{56}\\\hline
3&\dfrac{5\frac{2}{~}\cdot 3}{8\frac{3}{~}}&\dfrac{5}{8}\cdot\dfrac{4}{7}\cdot\dfrac{3}{6}&\dfrac{5}{28}\\\hline
4&\dfrac{5\frac{3}{~}\cdot 3}{8\frac{4}{~}}&\dfrac{5}{8}\cdot\dfrac{4}{7}\cdot\dfrac{3}{6}\cdot\dfrac{3}{5}&\dfrac{3}{28}\\\hline
5&\dfrac{5\frac{4}{~}\cdot 3}{8\frac{5}{~}}&\dfrac{5}{8}\cdot\dfrac{4}{7}\cdot\dfrac{3}{6}\cdot\dfrac{2}{5}\cdot\dfrac{3}{4}&\dfrac{3}{56}\\
\hline6&\dfrac{5\frac{5}{~}\cdot 3}{8\frac{6}{~}}&\dfrac{5}{8}\cdot\dfrac{4}{7}\cdot\dfrac{3}{6}\cdot\dfrac{2}{5}\cdot\dfrac{1}{4}\cdot\dfrac{3}{3}&\dfrac{1}{56}\\\hline\end{array}$
As a sanity check, the values in the right column all add up to equal $1$, as expected.
Continue.

As an aside, yet another way you could arrive at the correct probability distribution is through the use of binomial coefficients.  For the $k$'th ball to be the first white one, we need to select $k-1$ red balls and one white ball and further need the white ball selected to occur last.
This occurs with probability $Pr(X=k)=\dfrac{\binom{5}{k-1}\binom{3}{1}}{\binom{8}{k}}\cdot\frac{1}{k}$ which one can check equals the same expression as discussed earlier.

It seems you are insistent on writing this with factorials.  The expression will be
$$Pr(X=k) = \dfrac{ \frac{5!}{(5-k+1)!}\cdot 3}{\frac{8!}{(8-k)!}}$$
Which doesn't simplify much.  If you insisted you could move things around as
$$Pr(X=k)= \dfrac{ (8-k)!}{112\cdot (5-k+1)!}$$
</s>