<s>[INST] Positive recurrent, zero recurrent and transient states How do we prove that a state in a Markov chain process is positive recurrent, zero recurrent or transient? For example, if we have a transition matrix 
$$P=\left(\begin{array}{ccc}
  \frac{1}{3} & \frac{1}{3} & \ 0 & \frac{1}{3} \\
  0 & \frac{1}{2} & \frac{1}{2} & 0\\
  \frac{1}{2} & \ 0 & 0 & \frac{1}{2} \\ 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} 
  \end{array}\right)$$
how do we do it??
Thanks a lot!!
 [/INST] The path $1\to4\to2\to3\to1$ enumerates the state space and has positive probability hence every state is positive recurrent.
</s>