<s>[INST] $N(\theta,\theta)$: MLE for a Normal where mean=variance $\newcommand{\nd}{\frac{n}{2}}$For an $n$-sample following a Normal$(\mu=\theta,\sigma^2=\theta)$, how do we find the mle?
I can find the root of the score function
$$
\theta=\frac{1\pm\sqrt{1-4\frac{s}{n}}}{2},s=\sum x_i^2,
$$
but I don't see which one is the maximum.
I tried to substitute in the second derivative of the log-likelihood, without success.
For the likelihood, with $x=(x_1,x_2,\ldots,x_n)$,
$$
f(x) = (2\pi)^{-n/2} \theta^{-n/2} \exp\left( -\frac{1}{2\theta}\sum(x_i-\theta)^2\right), 
$$
then, with $s=\sum x_i^2$ and $t=\sum x_i$,
$$
\ln f(x) = -\nd \ln(2\pi) -\nd\ln\theta-\frac{s}{2\theta}-t+\nd\theta,
$$
so that
$$
\partial_\theta \ln f(x) = -\nd\frac{1}{\theta}+\frac{s}{2\theta^2}+\nd, 
$$
and the roots are given by
$$
\theta^2-\theta+\frac{s}{n}=0.
$$
Also,
$$
\partial_{\theta,\theta} \ln f(x) = \nd \frac{1}{\theta^2} - \frac{s}{\theta^3}.
$$
 [/INST] Recall that the normal distribution  $N(\mu, \sigma^2)$ has pdf $f(x\mid \mu ,\sigma ^{2})={\frac {1}{{\sqrt {2\pi \sigma ^{2}}}\ }}\exp {\left(-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}\right)},$ Note here that $\mu = \theta$ and $\sigma^2 = \theta$ and therefore $\sigma = \sqrt{\theta}$
\begin{aligned}
     L(x_1,x_2,...,x_n | \theta) &= \prod_{i=1}^n f(x_i | \theta)
     \\
     &=  \prod_{i=1}^n \frac{1}{\sqrt{2 \pi \theta}}  \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\}
     \\
     & = (2 \pi)^{-n/2}  (\theta)^{-n /2}   \prod_{i=1}^n  \ \exp \Big \{ - \frac{1}{2 \theta} (x_i - \theta)^2 \Big\}
     \\
     & =  (2 \pi)^{-n/2}  (\theta)^{-n /2}  \ \exp \Big \{ - \frac{1}{2 \theta} \sum_{i=1}^n (x_i - \theta)^2 \Big\}
     \\
     \log L& = - \frac{n}{2} \log(2\pi) -  \frac{n}{2} \log(\theta) - \frac{1}{2 \theta}  \sum_{i=1}^n (x_i - \theta)^2
\end{aligned}
Consider the term $\frac{1}{2 \theta}  \sum_{i=1}^n (x_i - \theta)^2$ which can be expanded and simplified
\begin{aligned}
     \frac{1}{2 \theta}  \sum_{i=1}^n (x_i - \theta)^2 & =  \frac{1}{2 \theta}  \sum_{i=1}^n (x_i - \theta)(x_i - \theta)
     \\
     & = \frac{1}{2 \theta} \sum_{i=1}^n \left( x_i^2 - 2 \theta x_i + \theta^2   \right)
     \\
     & =  \frac{1}{2 \theta}  \left( \sum_{i=1}^n (x_i^2) - 2 \theta \sum_{i=1}^n (x_i) + n\theta^2   \right)
     \\
     & = \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) -  \sum_{i=1} (x_i) + \frac{n\theta}{2}
\end{aligned}
We can now compute the derivative with respect to $\theta$, equate to zero and solve for $\theta$
\begin{aligned}
 \log L& = - \frac{n}{2} \log(2\pi) -  \frac{n}{2} \log(\theta) - \left(  \frac{1}{2 \theta} \sum_{i=1}^n (x_i^2) -  \sum_{i=1} (x_i) + \frac{n\theta}{2} \right)
  \\
 \frac{d}{d\theta} \log L & = \frac{-n}{2\theta} - \left( \frac{-1}{2\theta^2}  \sum_{i=1}^n (x_i^2) + \frac{n}{2} \right) = 0
 \\
 & = \frac{-n}{2\theta} + \frac{1}{2\theta^2}  \sum_{i=1}^n (x_i^2) - \frac{n}{2}
 \\
 & = - \theta^2 - \theta + \frac{1}{n} \sum_{i=1}^n (x_i^2)
 \\
 &\text{let $s = \frac{1}{n} \sum_{i=1}^n (x_i^2)$}
 \\
 0 & = - \theta^2 - \theta + s
 \\
 \hat \theta &= \frac{\sqrt{1 + 4s} -1 }{2}
\end{aligned}
</s>