<s>[INST] Find the value of $h$ if $x^2 + y^2 = h$ 
Consider equation $x^2 + y^2 = h$ that touches the line $y=3x+2$ at some point $P$. Find the value of $h$

I know that $x^2 + y^2 = h$ is a circle with radius $\sqrt{h}$.
Also, since $y = 3x + 2 $ is a tangent, we know that the slope of the radius perpendicular to the tangent is $M_{OP}= -\frac{1}{3}$. I'm not sure how I can determine the value of $h$ though?
 [/INST] There are couple of approaches already mentioned in other answers, but I will try to summarize them and add (probably unwarranted) generality. Also, I will describe approach using calculus.
If you are not interested in this generalities, please skip to the last paragraph.


We are given a line $y = ax + b$ and a circle centered $(p,q)$ given
  with $(x-p)^2+(y-q)^2 = h$ and we are looking for $h$ such that given
  line is tangent to the circle.

1. Calculus
We have function $y(x)$ for which relation $(x-p)^2+(y-q)^2 = h$ holds. If we differentiate the relation with respect to $x$ we get $$2(x-p)+2(y-q)y'=0\implies y'=-\frac{x-p}{y-q}$$ Now, since we are looking for a tangent given with $y = ax +b$, we are looking for a point $x$ such that $y'(x) = a$ and $y(x) = ax +b$. Thus we get $$a = -\frac{x-p}{ax + b - q}\implies x =\frac{p-ab+aq}{1+a^2}$$ and also $$y = ax + b  = \frac{b + ap + a^2 q}{1+a^2}$$ Now, we get $$\boxed{h = (x-p)^2 + (y-q)^2 = \frac{(b+ap-q)^2}{1+a^2}}$$
2. Tangent of a circle is perpendicular to its radius
This is your original approach and is completely covered in the answer by Adriano.
Now, the line passing through center $(p,q)$ and is perpendicular to $y = ax + b$ is given by formula $$y - q = -\frac 1 a(x-p)$$ and we can find the intersection with the original line by solving linear system 
\begin{align}
y &= ax + b \\
y - q &= -\frac 1 a(x-p)\end{align}
to get $x = \frac{p - ab + aq}{a^2 + 1},\ y = \frac{b + ap + a^2 q}{a^2 + 1}$ and if we plug this into $(x-p)^2+(y-q)^2 = h$ we get $$\boxed{h = \frac{(b+ap-q)^2}{1+a^2}}$$
3. $h$ is the square of distance of the center to the line
We find this approach in answers by Senex Ægypti Parvi and pajonk.
We start with line $y = ax + b$ and point $(p,q)$. Square of (Euclidean) distance of any point on the line $y = ax + b$ to the point $(p,q)$ is given by
\begin{align}
(x-p)^2 + (y-q)^2 &= (x-p)^2 +(ax+b-q)^2 \\ &= (1 + a^2) x^2 + 2( a b -  p -  a q) x+ (b^2 + p^2 - 2 b q + q^2)\end{align} and $h$ is given by minimal distance.
Now, remember that any quadratic $\alpha x^2 + \beta x + \gamma$ can be written in the form $$\alpha x^2 + \beta x + \gamma = \alpha(x  + \frac\beta{
   2 \alpha})^2 + \gamma - \frac{\beta^2} {4 \alpha}\geq \gamma - \frac{\beta^2}{4\alpha}$$ and thus the minimum of quadratic polynomial ($\alpha > 0$) is given with $\gamma - \frac{\beta^2}{4\alpha}$ and if we set $\alpha = 1+ a^2,\ \beta = 2( a b -  p -  a q),\ \gamma = b^2 + p^2 - 2 b q + q^2$ we get $$\boxed{h = \gamma - \frac{\beta^2}{4\alpha} = \frac{(b+ap-q)^2}{1+a^2}}$$
4. Tangent is a line that intersects circle at exactly one point
I personally like this approach the best because it beautifully intertwines geometry and algebra. This is suggested in answers by cosmin, Emilio Novati and DeepSea.
We want to find intersections of the line $y = ax + b$ and circle $(x-p)^2+(y-q)^2 = h$ so we solve the appropriate system
\begin{align}
(x-p)^2+(y-q)^2 &= h\\
y &= ax + b
\end{align}
and by substituting $y = ax + b$ in the first equation we get $$(x-p)^2+(ax + b - q)^2 - h = 0$$ and analogously to the previous case we get quadratic equation $$ (1 + a^2) x^2 + 2( a b -  p -  a q) x+ (b^2 + p^2 - 2 b q + q^2 - h) = 0 $$
It is well known that there are 3 general cases for the solutions of quadratic equation (with real coefficients) depending on the discriminant $D$:
\begin{array}{ c | c | c }
\textbf{discriminant} & \textbf{algebraic interpretation} & \textbf{geometric interpretation}\\
  \hline   
  D>0 & \text{there are 2 real solution} & \text{line intersects the circle at 2 points} \\
  D=0 & \text{there is a double real solution} & \text{line intersects the circle at 1 point, i.e. it is a tangent} \\
  D<0 & \text{there are 2 complex solutions} & \text{line does not intersect the circle }
\end{array}
We are, thus, interested in case 2), i.e. we want the discriminant of our quadratic equation to be $0$, i.e.:
$$(2 a b - 2 p - 2 a q)^2 = 4 (1 + a^2) (-h + b^2 + p^2 - 2 b q + q^2) \iff \boxed{h = \frac{(b+ap-q)^2}{1+a^2}}$$

TL;DR
What we actually calculated is a well known formula from high school:

For a line $Ax + By + C = 0$, distance of point $(p,q)$ from the line
  is given by formula $$d = \frac{| Ap + Bq + C|}{\sqrt{A^2 + B^2}}$$

How to get the formula from our calculation of $h$? Well:
$$Ax + By + C = 0\implies y = -\frac AB x -\frac CB\implies h = \frac{(-C-Ap-Bq)^2}{B^2(1+\frac{A^2}{B^2})} = \frac{(Ap+Bq+C)^2}{A^2 + B^2}$$ Thus, probably the quickest way to solve your problem is
5. High school mathematics
Transform $y = 3x + 2$ into $3x - y + 2 = 0$, and put it in the above formula with $p = q = 0$ to get $$h = \frac{(3\cdot 0 - 1\cdot 0 + 2)^2}{3^2 + (-1)^2} = \frac 2 5$$
</s>