<s>[INST] Shortest distance between an ellipse and a hyperbola. 
Suppose we have a hyperbola $xy=c$ $(c\neq0)$ and an ellipse $x^2/a^2+y^2/b^2=1$ $(a>b>0)$ that do not intersect. What is a quick way to calculate the shortest distance between these two curves?

I thought of setting a point $(a\cos\theta,b\sin\theta)$ on the ellipse and a point $(t,c/t)$ on the hyperbola, then the distance is given by $$\sqrt{(a\cos\theta-t)^2+(b\sin\theta-\frac{c}{t})^2},$$ whose minimum may be found by calculus methods. But the computations look tedious and difficult to carry out. What are some easier ways to do this?
 [/INST] There doesn't seem to be a nice closed-form solution to this, but the following might be helpful..
For an ellipse $\frac {x^2}{a^2}+\frac {y^2}{a^2}=1$, the equation of the normal at point $P(a \cos\theta, b\sin\theta)$ on the ellipse is
$$\frac {a\sin\theta}{b\cos\theta}x-y=a\left(\frac{a^2-b^2}{ab}\right)\sin\theta\tag{1}$$
A hyperbola which just touches or does not intersect at all with the ellipse above has the equation $xy=\frac {mab}2$ where $m\ge 1$.  The equation of the normal at point $Q (v,\frac {mab}{2v})$ on the hyperbola is
$$\frac {2v^2}{mab}x-y=v\left(\frac {2v^2}{mab}-\frac {mab}{2v^2}\right)\tag{2}$$
The minimum distance between the ellipse and hyperbola is the distance $PQ$ when $(1)=(2)$, i.e. both normals are the same line. 
As the coefficients of $y$ are the same in both $(1),(2)$, equating coefficients of $x$ in $(1),(2)$  gives
$$v=a\sqrt \frac{m\tan\theta}{2}\tag{3}$$
This relationship ensures that the tangents and normals at $P,Q$ are parallel to each other respectively (but the normals are not necessarily the same line). 
Putting $(3)$ in $(2)$ gives
$$\left(\frac ab \tan\theta\right)x-y=a\sqrt{\frac{m\tan\theta} 2}\left(\frac ab\tan\theta-\frac ba\cot\theta\right)\tag{4}$$
To ensure that both normals are the same line, we need to equate RHS of $(1),(4)$. This gives
$$\left(\frac{a^2-b^2}{ab}\right)\sin\theta=\sqrt{\frac{m\tan\theta}2}\left(\frac ab \tan\theta-\frac ba\cot\theta\right)$$
which is equivalent to 
$$(a^2-b^2)\sin\theta=\sqrt{\frac{m\tan\theta}2}\left(\frac{a^2\sin^2\theta-b^2\cos^2\theta}{\sin\theta\cos\theta}\right)\tag{4}$$
Solve numerically $\theta$ in $(4)$, find corresponding value $v$ in $(3)$, then calculate $PQ$. This should give the minimum distance between the ellipse and hyperbola. 

See desmos implementation here.

In the trivial case where $a=b$ (i.e. ellipse is a circle), then $\theta=\frac \pi 4$ and $v=a\sqrt{\frac m2}$ . This gives $P=\left(\frac a{\sqrt{2}}, \frac a{\sqrt{2}}\right)$ and $Q=\left(a\sqrt{\frac m2}, a\sqrt{\frac m2}\right)$ and the distance $PQ=a(\sqrt m-1)$. 
</s>