<s>[INST] Find the area of the portion of the surface of the sphere $x^2+y^2+z^2=4x$ that is cut off by a nappe of the cone $y^2+z^2=x^2$. I'm having trouble with a Calculus question where I am supposed to use the following formula to calculate the area of a surface:

Suppose that $f$ and its first-order partial derivatives are continuous in the closed region $R$ in the $xy$ plane. So, if $S$ is the measure of the area of the surface $z=f(x,y)$ above $R$,
$$ S = \int_R\int \sqrt{f_x^2(x,y)+f_y^2(x,y)+1} dxdy$$
(Here, $f_x(x,y)$ and $f_y(x,y)$ stand for the first-order partial derivatives with respect to $x$ and $y$, respectively.)

The question asks me the following:

Find the area of the portion of the surface of the sphere $x^2+y^2+z^2=4x$ that is cut off by a nappe of the cone $y^2+z^2=x^2$.

I made the following plot of the situation described in the question. To make things simpler, I plotted only the intersection of the sphere and the cone with the $xy$ plane:

The image above shows the part of the sphere (indicated in cyan (light blue) color) that is cut off by a nappe of the cone. Notice that the intersection of the sphere with the $xy$ plane is the circle $y^2=4x-x^2$, and the intersection of the cone with the $xy$ plane are the lines $y=x$ and $y=-x$.
To apply the given formula, I have to find the region $R$ over which the double integral will be computed. From the image, the region of integration $R$ is the region between the line $x=2$ and the portion of the circumference marked in red. The desired portion of the surface of the sphere is above this region $R$.
So, the integration limits are the following:
$$\int_2^4\int_0^\sqrt{4x-x^2} \cdots dydx$$
In other words, the outer integral is computed with $x$ going from $2$ to $4$, and the inner integral is computed with $y$ going from $0$ to $\sqrt{4x-x^2}$ (notice that I have switched $dx$ and $dy$ from the original formula for convenience).
Now, if I compute the first-order partial derivatives indicated in the formula, I obtain the following results: $f_x(x,y) = \dfrac{2-x}{\sqrt{4x-x^2-y^2}}$ and $f_y(x,y) = -\dfrac{y}{\sqrt{4x-x^2-y^2}}$.
The double integral is, then:
$$\int_2^4\int_0^\sqrt{4x-x^2} \sqrt{\frac{(2-x)^2+y^2}{4x-x^2-y^2} + 1} dydx$$
which, after some simplification, becomes:
$$\int_2^4\int_0^\sqrt{4x-x^2} \sqrt{\frac{4}{4x-x^2-y^2}} dydx$$
I get stuck at this point, because I can't seem to find a way of solving this integral. I have tried to solve the inner integral
$$\int_0^\sqrt{4x-x^2} \sqrt{\frac{4}{4x-x^2-y^2}} dy$$
but without any success. Is the set up of the double integral correct? Or is there a way to solve the above integral and get a real result?
 [/INST] I will solve this problem again using the suggestion by Ted Shifrin in the comments to this question above. Doing the integral in polar coordinates, the solution becomes much simpler.
I will take the polar plane to be the $yz$ plane.
In $yz$ coordinates, the desired surface is the function $x=f(y,z)=\sqrt{4-y^2-z^2}+2$ (which corresponds to the right hemisphere of the sphere represented in the picture in the question).
As required by the formula, the surface $x=f(y,z)=\sqrt{4-y^2-z^2}+2$ is above a region $R$ which lies on the $yz$ plane (polar plane). In this case, the region of integration $R$ is the projection of the sphere on the polar plane; in other words, $R$ is a circle of radius 2 centered at $(x,y,z)=(2,0,0)$.
Thus, the limits of integration in polar coordinates will be:
$$\int_0^2\int_0^{2\pi} \cdots r d\theta dr$$
The partial derivatives of $f(x,y,z)$ are $f_y = -\dfrac{y}{\sqrt{4-y^2-z^2}}$ and $f_z = -\dfrac{z}{\sqrt{4-y^2-z^2}}$.
The square root inside the double integral is:
$$\sqrt{f_y^2(y,z)+f_z^2(y,z)+1} = \sqrt{ \dfrac{y^2}{4-y^2-z^2} + \dfrac{z^2}{4-y^2-z^2} + 1 }$$
$$= \sqrt{ \dfrac{4}{4-y^2-z^2} }$$
Converting the above expression to polar coordinates (substituting $y^2+z^2=r^2$):
$$\sqrt{ \dfrac{4}{4-r^2} }$$
Thus, the double integral in polar coordinates is:
$$\int_0^2\int_0^{2\pi} \sqrt{ \dfrac{4}{4-r^2} } r d\theta dr$$
Solving the inner integral, the above expression reduces to $\int_0^2 4 \pi r \sqrt{ \dfrac{1}{4-r^2} } dr$, which can be solved by substitution (for example, by setting $u=4-r^2$ and $du=-2rdr$) and gives $8\pi$ as its result.
</s>