<s>[INST] Diagonalisation of stochastic matrices Suppose that $(X_n)_{n≥0}$ is a Markov chain on a state space $I = {1, 2}$ and stochastic matrix
$$P = \begin{bmatrix} \frac{1}{4} & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3} \end{bmatrix}$$
(a) Find the eigenvalues
$$|P-\lambda I| = \begin{vmatrix} \frac{1}{4}-\lambda & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3}-\lambda \end{vmatrix} = \lambda^2 - \frac{11}{12}\lambda -\frac{1}{12}$$
Then $\lambda_1 = 1, \lambda_2 = -\frac{1}{12}$
(b) You know $P$ and $P^0 = I_2$. Use this to find $a(i,j), b(i,j)$ so that you have an explicit form for $P^n$
We know by diagonalizing $P$, every entry of $P^n$ can be written as $(P^n)_{i,j}=a(i,j)\lambda_1^n+b(i,j)\lambda_2^n$
Edit:
$P^n = \begin{bmatrix} \frac{1}{4} & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3} \end{bmatrix}^n =  \begin{bmatrix} 1 & -9 \\ 1 & 4 \end{bmatrix}  \begin{bmatrix} 1 & 0 \\ 0 & -\frac{1}{12} \end{bmatrix}^n \begin{bmatrix} \frac{4}{13} & \frac{9}{13} \\ -\frac{1}{13} & \frac{1}{13} \end{bmatrix}$
Edit (2):
$\begin{bmatrix} 1 & -9 \\ 1 & 4 \end{bmatrix}  \begin{bmatrix} 1 & 0 \\ 0 & -\frac{1}{12} \end{bmatrix}^n \begin{bmatrix} \frac{4}{13} & \frac{9}{13} \\ -\frac{1}{13} & \frac{1}{13} \end{bmatrix} =  \begin{bmatrix} 1 & -9 \\ 1 & 4 \end{bmatrix}  \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix}^n \begin{bmatrix} 0 & 0 \\ 0 & -\frac{1}{12} \end{bmatrix}^n\begin{bmatrix} \frac{4}{13} & \frac{9}{13} \\ -\frac{1}{13} & \frac{1}{13} \end{bmatrix}$?
 [/INST] You're almost there. We have
\begin{align}
P^n &= 
\begin{bmatrix} 
\frac{1}{4} & \frac{3}{4} \\ \frac{1}{3} & \frac{2}{3} 
\end{bmatrix}^n =  
\begin{bmatrix} 
1 & -9 \\ 1 & 4 
\end{bmatrix}  
\begin{bmatrix} 
1 & 0 \\ 0 & -\frac{1}{12} 
\end{bmatrix}^n 
\begin{bmatrix} 
\frac{4}{13} & \frac{9}{13} \\ -\frac{1}{13} & \frac{1}{13} 
\end{bmatrix}
\\ & =
\frac 1{13}
\begin{bmatrix} 
1 & -9 \\ 1 & 4 
\end{bmatrix}  
\begin{bmatrix} 
1 & 0 \\ 0 & \left(-\frac{1}{12}\right)^n
\end{bmatrix}
\begin{bmatrix} 
4 & 9 \\ -1 & 1
\end{bmatrix}
\\ & = 
\frac 1{13}
\begin{bmatrix} 
1 & -9 \\ 1 & 4 
\end{bmatrix}  
\begin{bmatrix} 
4 & 9 \\ -\left(-\frac{1}{12}\right)^n & \left(-\frac{1}{12}\right)^n
\end{bmatrix}
\\ & = 
\frac 1{13}
\begin{bmatrix}
4 + 9\left(-\frac{1}{12}\right)^n & 9 - 9\left(-\frac{1}{12}\right)^n\\
4 - 4\left(-\frac{1}{12}\right)^n & 9 + 4\left(-\frac{1}{12}\right)^n
\end{bmatrix}.
\end{align}
</s>