<s>[INST] Visualise $3x^2 - 2xy - 10x +3y^2 -2y + 8=0$ after the $x,y$ term has been eliminated (using rotation) This is a continuation from my previous question. I thought it would be better to start a new one since the old one was answered correctly.
The equation in question is: $3x^2 - 2xy - 10x +3y^2 -2y + 8=0$: 
We introduce the new coordinates for $x,y$: 
$$x = \cos \theta X - \sin \theta Y$$ 
$$y = \cos \theta Y + \sin \theta X $$
We find that $\theta = \frac{\pi}{4}$ eliminates the $xy$ term. This gives us $\cos \theta  = \frac{1}{\sqrt{2}}$, $\sin \theta  = \frac{1}{\sqrt{2}}$.

Now we want to find out what kind of curve $3x^2 - 2xy - 10x +3y^2 -2y + 8=0$ is.
I.E we want to complete the square and find out what kind of properties it has. What is the best method to continue from here?
One attempt is to use the general development of the $Ax^2 + Bxy +Cy^2$ with our new coordinates for $x$ and $y$. Looking at our equation above we have $A=3, B= -2, C=3$.
This gives us:
$$X^2(A\cos^2 \theta + B\cos \theta \sin \theta + Csin^2 \theta)+$$  
$$XY(-2A\cos \theta \sin \theta + B(\cos^2 \theta - \sin^2 \theta) + 2C \cos \theta \sin \theta) +$$
$$Y^2(A\sin^2 \theta - B\sin \theta \cos \theta + Ccos^2 \theta)$$ 
The coefficients for the $XY$ term will be $0$ so we can ignore that one. With the values for $A,B,C$ and $\sin \theta =\cos \theta = \frac{1}{\sqrt{2}}$ we get:
$$X^2(3\frac{1}{2} + (-2)\frac{1}{2} + 3\frac{1}{2})+ Y^2(3\frac{1}{2} - (-2)\frac{1}{2} + 3\frac{1}{2}) = 2X^2 + 4Y^2$$  
The original equation $3x^2 - 2xy - 10x +3y^2 -2y + 8=0$ can now be written as: $$2X^2 + 4Y^2 -10(\cos \theta X - \sin \theta Y) -2(\cos \theta Y + \sin \theta X) + 8 = 0 \rightarrow$$ 
$$2X^2 + 4Y^2 -\frac{10}{\sqrt{2}}X + \frac{10}{\sqrt{2}}Y -\frac{2}{\sqrt{2}}Y - \frac{2}{\sqrt{2}} X +8 = 0 \rightarrow$$ 
$$2X^2 + 4Y^2 -\frac{12}{\sqrt{2}}X + \frac{8}{\sqrt{2}}Y +8 = 0$$
And if I try to complete the square I get:
$$2(x- \frac{3}{\sqrt{2}})^2+(y+2\sqrt{2})^2 = 9$$

The final result in my textbook is: 
$$2(X-1)^2 + (Y-3)^2 = 3$$ where $X = x-y$ and $Y = x+y$
But I am stuck and cannot understand how to get to the result in my textbook. I would love some help (if you have the time to do the actual final calculations, that would be much appreciated!)
 [/INST] These are the steps. Show us your workings for as far as you've gotten:


*

*Plug $x=(X-Y)/\sqrt{2}$ and $y=(X+Y)/\sqrt{2}$ into the equation.

*Rearrange into the form $Ax^2+Bx^2+Cx+Dy+E=1$.

*Complete the square on the LHS to put it in the form of the equation of an ellipse.

</s>