<s>[INST] Given a general region, find the double integral bounded between $y = x$ and $y=3x-x^2 $ $$ J  = \iint_R (x^2-xy)\,dx \,dy, $$
Suppose region R is bounded between $y = x$ and $y=3x-x^2 $
My attempt using vertical integration:
$$ \int^{x=2}_{x=0} \int^{y=3x-x^2}_{y=x} \left({x^2-xy}\right)dy\ dx$$
$$\int^2_0 \left[x^2y-x\frac{y^2}{2}\right]^{3x-x^2}_{x}\, dx$$
$$\int^2_0 \frac{-x^5+4x^4-4x^3}{2} \,dx $$
$$\boxed{J = -\frac{8}{15}}$$

My attempt using horizontal integration :
$$ \int^{y=2}_{y=0} \int^{x=y}_{x=3\,\pm \sqrt{9-y}} \left({x^2-xy}\right)dx\ dy$$
For $ x = 3+\sqrt{9-y}$
$$  \int^2_0 \left[\frac{x^3}{3}-\frac{x^2}{2}y\right]^y_{3+\sqrt{9-y} }\,dy$$
For $ x = 3-\sqrt{9-y}$
$$  \int^2_0 \left[\frac{x^3}{3}-\frac{x^2}{2}y\right]^y_{3-\sqrt{9-y} }\,dy$$

My doubts :
1.) How do I set my limit of integration for horizontal integration, if there is $\pm$ to be considered ?
2.) the answer as negative what does that imply in questions related to double integrals?
Could you guys please help
 [/INST] Your first approach is correct, but your second approach is not. The possible values of $y$ lie in the interval $\left[0,\frac94\right]$, because $\frac94$ is the maximum of $3x-x^2$ when $x\in[0,2]$. When $y\in\left[0,\frac94\right]$, then (see the picture below):

*

*if $y\in[0,2]$, then the possible values of $x$ lie in $\left[\frac{3-\sqrt{9-4y}}2,y\right]$;

*if $y\in\left[2,\frac94\right]$, then the possible values of $x$ lie in $\left[\frac{3-\sqrt{9-4y}}2,\frac{3+\sqrt{9-4y}}2\right]$.

So, the answer is\begin{multline}\int_0^2\int_{\frac{3-\sqrt{9-4y}}2}^yx^2-xy\,\mathrm dy\,\mathrm dx+\int_2^{9/4}\int_{\frac{3-\sqrt{9-4y}}2}^{\frac{3+\sqrt{9-4y}}2}x^2-xy\,\mathrm dy\,\mathrm dx=\\=\int_0^2\frac{1}{12} \left(-2 y^3-6 y^2+\left(45-11 \sqrt{9-4 y}\right)  y+18 \left(\sqrt{9-4 y}-3\right)\right)\,\mathrm dy+\\+\int_2^{9/4}\frac{1}{6}(18-11 y)\sqrt{9-4 y}\,\mathrm dy=-\frac{47}{120}-\frac{17}{120}=-\frac8{15}.\end{multline}

</s>