<s>[INST] image of circle after inversion Given a circle $C$ of radius $r$ and center $(x_0,y_0)$, what is the center and radius of $C$ after inversion in the unit circle?
A simpler question has been asked:
Image of a circle under conformal map $1/z$
 [/INST] You have $(x-x_0)^2+(y-y_0)^2=r^2$ as the original circle. Performing the inversion means replacing $x$ by $\frac x{x^2+y^2}$ and $y$ by $\frac y{x^2+y^2}$. Abbreviating $d:=x^2+y^2$ and $f:=x_0^2+y_0^2-r^2$ one can write the circle after inversion as
\begin{align*}
\left(\frac{x}{d}-x_0\right)^2+\left(\frac{y}{d}-y_0\right)^2 &= r^2 \\
(x-x_0d)^2 + (y-y_0d)^2 &= (rd)^2 \\
x^2-2xx_0d+x_0^2d^2+y^2-2yy_0d+y_0^2d^2 &= r^2d^2 \\
d-2xx_0d+x_0^2d^2-2yy_0d+y_0^2d^2 &= r^2d^2 \\
1-2xx_0+x_0^2d-2yy_0+y_0^2d &= r^2d \\
(x^2+y^2)(x_0^2+y_0^2-r^2)-2xx_0-2yy_0+1 &= 0 \\
(x^2+y^2)f-2xx_0-2yy_0 &= -1 \\
(x^2+y^2)-2x\frac{x_0}f-2y\frac{y_0}f &= -\frac1f \\
\left(x^2-2x\frac{x_0}f\right) +
\left(y^2-2y\frac{y_0}f\right) &= -\frac1f \\
\left(x^2-2x\frac{x_0}f+\left(\frac{x_0}f\right)^2\right) +
\left(y^2-2y\frac{y_0}f+\left(\frac{y_0}f\right)^2\right) &=
\left(\frac{x_0}f\right)^2 + \left(\frac{y_0}f\right)^2 -\frac1f \\
\left(x-\frac{x_0}f\right)^2 + \left(y-\frac{y_0}f\right)^2 &=
\frac{x_0^2+y_0^2-f}{f^2} \\
\left(x-\frac{x_0}{x_0^2+y_0^2-r^2}\right)^2 + \left(y-\frac{y_0}{x_0^2+y_0^2-r^2}\right)^2 &=
\left(\frac{r}{x_0^2+y_0^2-r^2}\right)^2 \\
\end{align*}
So the new center would be at
$$\left(\frac{x_0}{x_0^2+y_0^2-r^2}, \frac{y_0}{x_0^2+y_0^2-r^2}\right)$$
and the new radius would be
$$\frac{r}{\left\lvert x_0^2+y_0^2-r^2\right\rvert}$$
Some sanity checks. If the denominator is zero, that means your circle passes through the origin, so its image will pass through infinity. Makes sense. If the original radius is zero, then so is the new one, as points map to points. Sane as well. And the direction of the new center matches that of the original center, as expected.
</s>