<s>[INST] Discrete Time Markov Chain question Let $\{X_n : n \ge 0 \}$ be a Markov chain with state space $ \{0, 1, 2, 3\} $ and transition matrix 
$$P=\begin{pmatrix}
\frac{1}{4} & 0 & \frac{1}{2} & \frac{1}{4}\\
0 & \frac{1}{5} & 0 & \frac{4}{5}\\
0 & 1 & 0 & 0\\
\frac{1}{3} & \frac{1}{3} & 0 & \frac{1}{3}\\
\end{pmatrix}
$$
A new process $\{Z_n : n \ge 0\}$ is defined by $Z_n = 0$ if $X_n = 0$ or $1$, and $Z_n = X_n$ if $X_n = 2$ or $3$.
Find $P(Z_{n+1} = 2 \mid Z_n = 0, Z_{n−1} = 2)$.
I'm not too sure how to deal with the fact that $Z_n = 0$ implies that $X_n = 0$ or $1$. I know you're supposed to plug in the corresponding $X_n$ values into the question, but I get $P(Z_{n+1} = 2 \mid Z_n = 0, Z_{n−1} = 2)$ = $P(X_{n+1} = 2 \mid X_n = 0 \ \text{or} \  1, X_{n−1} = 2)$. 
Is this simply equal to $P(X_{n+1} = 2 \mid X_n = 0, X_{n−1} = 2)+P(X_{n+1} = 2\mid X_n = 1, X_{n−1} = 2)$?
 [/INST] First of all, notice that $Z_{n+1} = 2 \Leftrightarrow X_{n+1} = 2$ and $Z_n = 0 \Leftrightarrow X_n \in \{0,1\}$. So
$$P(Z_{n+1} = 2 \mid Z_n = 0, Z_{n-1} = 2) = P(X_{n+1} = 2 \mid X_n \in \{0,1\}, X_{n-1} = 2).$$
Write this quantity as
$$P(X_{n+1} = 2 \mid \{X_n = 0\} \cup \{X_n = 1\}, X_{n-1} = 2).$$
The strong Markov property says that you can drop the $X_{n-1}$ conditioning.
Now try to use laws of conditional probability, along with your transition matrix $P$, to get the answer. I'd use Bayes' theorem, and then you're going to have to do some slightly messy calculations.
</s>