<s>[INST] Simplifying calculations for hitting probabilities using communicating classes When given a rather elaborate transition matrix for 6 state Markov chain, for example 
$$\begin{bmatrix} 0&0&\frac{1}{2}&0&0&\frac{1}{2} \\ \frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5}&0\\ \frac{1}{3}&0&\frac{1}{3}&0&0&\frac{1}{3} \\ \frac{1}{6}&\frac{1}{6}&\frac{1}{6}&\frac{1}{6}&\frac{1}{6}&\frac{1}{6} \\ 0&0&0&0&1&0 \\ \frac{1}{4}&0&\frac{1}{2}&0&0&\frac{1}{4} \end{bmatrix}, $$
if we wanted to find say the hitting probability that we eventually land at state 6 given that we start at state 2, it seems to me that to calculate this manually using the standard method of finding the lowest non-negative vector solution to 
$$ h_{i}^{6} = \begin{cases} 1 & i = 6 \\ \sum_{j} h_{j}^{6}p_{ij} & i \neq 6
\end{cases}$$
where $h_{i}^{6} = \Pr(\text{hitting time to 6 is finite} |X_{0} =i)$, and then setting $i = 2$ seems quite cumbersome and error prone. 
Would it be possible to instead recognise that $\{1, 3, 6 \}$ is a closed communicating class and then merge the states together to form a simplified transition matrix? Reason being since the class is closed we merely need to find the probability that $X_{n} \in \{1, 3, 6 \} $ and we are guaranteed that $(X)_{n} = 6$ eventually? 
For example the simplified transition matrix inscribing states (1,3,6), (2), (4) and (5) would then become 
$$\begin{bmatrix} 1&0&0&0 \\ \frac{2}{5}&\frac{1}{5}&\frac{1}{5}&\frac{1}{5} \\ \frac{1}{2}&\frac{1}{6}&\frac{1}{6}&\frac{1}{6} \\ 0&0&0&1 \end{bmatrix}. $$
Here we can immediately read of that $h_{1,3,6}^{1,3,6} = 1$, $h_{5}^{1,3,6} = 0 $ and solving for the other states becomes a lot easier. Is this a valid strategy to adopt? Is there anything wrong with the reasoning that it is sufficient to find that probability that we enter the closed communicating class to find the probability that we eventually reach a state in that class? Thanks. 
 [/INST] Partition the state space $S=\{1,2,3,4,5,6\}$ into $S=A\cup B\cup C$ with $A=\{1,3,6\}$, $B=\{2,4\}$, $C=\{5\}$. Then $\mathbb P(X_{n+1}\in A\mid X_n\in A) = 1$ and $\mathbb P(X_{n+1}\in C\mid X_n\in C)=1$, so $A$ and $C$ are closed communicating classes, and since $\mathbb P(X_{n+1}\in A\mid X_n\in B)>0$ and $\mathbb P(X_{n+1}\in C\mid X_n\in B)>0$, $B$ is the class of transient states. For $i\in B$, let
\begin{align}
\tau_{i,A} &= \inf\{n>0:X_n\in A\mid X_0 = i \}\\
\tau_{i,C} &= \inf\{n>0:X_n\in C\mid X_0 = i \}.
\end{align}
Denote $\rho_{i,A}:=\mathbb P(\tau_{i,A}<\infty)$ and $\rho_{i,C}:=\mathbb P(\tau_{i,C}<\infty)$, then
\begin{align}
\rho_{i,A}= \sum_{j\in A}p_{ij} + \sum_{j\in B}p_{ij}\rho_{i,A},
\end{align}
and $\rho_{i,C} = 1 - \rho_{i,A}$. It follows that
\begin{align}
\rho_{2,A} &= \frac 25 +\frac15\rho_{2,A} + \frac15\rho_{4,A}\\
\rho_{4,A} &= \frac 12 + \frac16\rho_{2,A} + \frac16\rho_{4,A},
\end{align}
and hence
$$
\rho_{2,A} = \frac{13}{19},\ \rho_{4,A} = \frac{14}{19},\ \rho_{2,C} = \frac6{19},\ \rho_{4,C} = \frac5{19}.
$$
</s>