<s>[INST] find correlation coefficient of $f(x,y)=2$ for $0
Find the correlation coefficient for the random variables $X$ and $Y$ having joint density $f(x,y)=2$ for $0 < x \leq y<1$.
Seem like a simple problem but I'm stuck.
Since $Corr(X,Y) = \frac{Cov(X,Y)}{\sqrt{Var(X)}\sqrt{Var(Y)}}$, I figure I start with $Var(X)$ and $Var(Y)$.

$E(X) = \int_0^1 \int_0^y 2xdxdy = \int_0^1y^2dy = \frac{1}{3}$
$E(X^2) = \int_0^1 \int_0^y 2x^2dxdy = \int_0^1\frac{2}{3}y^3dy = \frac{1}{6}$
$Var(X) = E(X^2) - E(X)^2 = \frac{1}{6} - \frac{1}{9} = \frac{1}{18}$
$E(Y) = \int_0^1 \int_x^1 2ydydx = \int_0^1 1-x^2 dx = \frac{2}{3}$
$E(Y^2) = \int_0^1 \int_x^1 2y^2dydx = \frac{2}{3}\int_0^1 1-x^3dx = \frac{1}{2}$
$Var(Y) = E(Y^2) - E(Y)^2 = \frac{1}{2} - \frac{4}{9} = \frac{1}{18}$
$\frac{1}{\sqrt{Var(X)}\sqrt{Var(Y)}} = \frac{1}{\frac{1}{18}} = 18$

It seems alright up to this point.

$Cov(X,Y) = E(XY) - E(X)E(Y)$
$E(XY) = \int_0^1 \int_0^y 2xy dxdy = \int_0^1 y^3 dy = \frac{1}{4}$
$Cov(X,Y) = \frac{1}{4} - (\frac{1}{18})^2$

And this is where I'm stuck.

$Corr(X,Y) = (\frac{1}{4} - (\frac{1}{18})^2) \times 18 = \frac{18}{4} - \frac{1}{18} > 1$

wolfram says my calculation are correct. so this must be a concept problem.
The $0 < x \leq y<1$ is really throwing me off but Devore doesn't really provide a clear explanation.
Thanks for your hints.
 [/INST] Your integrals look right. Note that $E(X)E(Y)=\frac{2}{9}$, so the covariance calculation is not right. 
The covariance is $\frac{1}{4}-\frac{2}{9}=\frac{1}{36}$. 
</s>