<s>[INST] MLE Derivation for AR Model So I am trying to derive the MLE for an AR(1) model. Here are my thoughts thus far:
The AR process is: $z_t = \delta + \psi_1z_{t-1} + \epsilon_t$
The expected value of $z_t = \frac{\delta}{1 - \psi_1}$.
The variance of $z_t = \frac{1}{1 - \psi_1^2}$.
So this is where I am getting caught up.
I have the PDF of $z_t$ as:
\begin{align}
f(z_t;\theta) &= (2 \pi \sigma^2)^{-\frac{1}{2}} 
    \exp \left [-\frac{1}{2} \left (\frac{z_t - \mathbb{E}[z_t]}{\sqrt{\sigma^2}}\right )  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1}{2} \left (\frac{z_t - \frac{\delta}{1 - \psi_1}}
    {\sqrt{\frac{1}{1 - \psi_1^2}}}  \right )^2  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1}{2} \left (\frac{ \left(z_t - \frac{\delta}{1 - \psi_1} \right )^2}{\frac{1}{1 - \psi_1^2}}  \right )  \right] \\
    &= \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1 - \psi_1^2}{2} \left( z_t - \frac{\delta}{1 - \psi_1} \right)^2  \right]
\end{align}
Now, can I assume i.i.d. here? I feel like no because then I would have a time series that is just white noise right? However, if I did assume i.i.d., I would have:
$\mathscr{L} = \prod_{t=1}^T \left (2 \pi \frac{1}{1 - \psi_1^2} \right )^{-\frac{1}{2}}
    \exp \left [-\frac{1 - \psi_1^2}{2} \left( z_t - \frac{\delta}{1 - \psi_1} \right)^2  \right]$
And then from here what exactly would my log likelihood function be? I feel like I am totally screwing this up but this is what I have for it:
$\ln \mathscr{L} = -\frac{T}{2} \ln \left ( 2 \pi \frac{1}{1 - \psi_1^2} \right )
- \frac{(1 - \psi_1^2) \sum_{t=1}^T \left (z_t - \frac{\delta}{1 - \psi_1}  \right )^2}{2}$
Any help is greatly appreciated! Thank you!!
 [/INST] I'm not directly answering your question, but a quick note on the construction of the likelihood function in your model. The likelihood of a $T$-sized sample of $\mathbf{e} = \left[ \epsilon_{1}, \, \epsilon_{2}, \ldots, \, \epsilon_{T} \right]^{\mathsf{T}}$ of i.i.d. normal distributed $\epsilon \sim N(0,\sigma^{2})$ is
$$
L(\mathbf{e}) = (2 \pi \sigma^{2})^{\frac{T}{2}} \exp \left[ \frac{- \mathbf{e}^\mathsf{T} \mathbf{e}}{2\sigma^{2}}   \right]
$$
But obviously you observe $\mathbf{z} = \left[ z_{1}, \, z_{2}, \ldots, \, z_{T} \right]^{\mathsf{T}}$ instead of $\mathbf{e}$. From your AR(1) setup you have $\mathbf{e} = \mathbf{G} \mathbf{z} - \delta \mathbf{I}_{T}$ where
$$
\mathbf{G} = \begin{bmatrix}
\sqrt{1-\psi_{1}^2} & 0 & 0 & \cdots & 0 \\
-\psi_{1} & 1 & 0 & \cdots & 0 \\
0 & -\psi_{1} & 1 & \cdots &  0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
0 & 0 & 0 & \cdots & 1
\end{bmatrix},
$$
(see Prais & Winsten, eqn. 15). Then
$$L(\mathbf{z}) = L(\mathbf{e}) \left| \frac{d \mathbf{e}}{d\mathbf{z}} \right|$$
where $\left| \frac{d \mathbf{e}}{d\mathbf{z}} \right|$ is the Jacobian (determiant) of the transformation, in this case $\operatorname{det} \mathbf{G}$, which works out to be $\left| 1 -\psi_{1}^{2} \right|^{\frac{1}{2}}$ because of all the off-diagonal zeros in $\mathbf{G}$. See Beach & MacKinnon (1978) for more details.
</s>