<s>[INST] Let $p, q$ be different primes. Then $p^2+q^2-pq$ is not a perfect square. This question originally comes from the following problem:

Let $ABC$ be a triangle with integer side lengths with $\angle ABC = 60^{\circ}$. Suppose length $\overline{AB}$ and $\overline{BC}$ are prime numbers. Determine and prove what kind of triangle $ABC$ is.

I suspect $ABC$ must be an isosceles triangle where $\overline{AB} = \overline{BC}$. By the law of cosine we have
$$ \overline{AC}^2 = \overline{AB}^2 + \overline{BC}^2 - 2\overline{AB}\overline{BC}\cos 60^{\circ}.$$
Let $p = \overline{AB}$ and $q = \overline{BC}$ the above statement is equivalent to $p^2 + q^2 - pq$ being a perfect square. My job will be proving that this statement holds for $p \neq q$(where $p=q$ corresponds to the case where $\overline{AB} = \overline{BC}$ i.e. $ABC$ being isoceles triangle).
I tried factorizing and discussing case by case and did not work out for me. How can I approach this problem?
 [/INST] We have: $$(p-q)^2<p^2-pq+q^2<(p+q)^2.$$
If $p>q>0$ and $p^2-pq+q^2=c^2,$ for $c>0,$ then  then $q^2\equiv c^2\pmod p,$ so $$c\equiv \pm q\pmod p$$ Now, since $0<p-q< c<p+q,$ this means our options are  $c=q$ or $c=2p-q.$ In both cases, $c$ is in the range only  if $p<2q.$
If $c=q,$ then $p^2-qp=0,$ so $p=0$ or $p=q.$
If $c=2p-q$ then $$p^2-pq+q^2=4p^2-4pq+q^2\\\implies3p^2-3pq=0$$ and again $p=q$ or $p=0.$

So we’ve only really used that the larger of $p,q$ is prime (to conclude $c\equiv\pm q\pmod p.$) Indeed, we could restrict it to the larger of the two being a power of a prime.
So we get:

If $1<m<n$ are relatively prime, and $n$ is the power of an odd prime, or $n=2,4$ then $m^2-mn+n^2$ is not a perfect square.

We can’t use other powers of $2$ because $1$ has more than $2$ square roots modulo $2^n$ for $n>2.$

Not sure if it is possible for $m$ to be an odd prime power if $q$ is not.

We can apply the usual technique to find a formula for all rational solutions to $$x^2-xy+y^2=1.$$
Pick one rational solution $(x,y)=(1,1).$ Every other rational solution is on a line $(1+at,1+bt)$ for some $a,b$ integers and $t\neq 0.$  But given $a,b$ you get:
$$
\begin{align} 
1&=(1+at)^2-(1+at)(1+bt)+(1+bt)^2\\&=1+(2a-(a+b)+2b)t+(a^2-ab+b^2)t^2\\
t&=-\frac{a+b}{a^2-ab+b^2}
\end{align} $$
Now $$\gcd(a+b,a^2-ab+b^2)=1,3$$ with $3$ if $a\equiv $b\pmod 3,$ and $1$ otherwise.
So $$x=1+at=\frac{b^2-2ab}{a^2-ab+b^2},\\
y=1+bt=\frac{a^2-2ab}{a^2-ab+b^2}.$$
So, at least for relatively prime $(m,n)$ if $m^2-nm+n^2$ is a perfect square, then:$$m=b^2-2ab\\n=a^2-2ab$$ for some relatively prime $a,b,$ with $a+b\not\equiv 0\pmod 3,$ or $$m=\frac{b^2-2ab}3\\n=\frac{a^2-2ab}{3}$$
for some relatively prime $a,b$ with $a+b\equiv 0\pmod 3.$
This shows why it is hard for $m,n$ to be prime.
</s>