<s>[INST] Fisher information of exponential distribution using the generic formula. I am trying to calculate Fisher information when $X_1...X_n$ are IID and follow the exponential distribution:
$$f(x)=\frac{1}{\theta}e^{-\frac{x}{\theta}}$$
I use the following formula of Fisher information to confirm that the result is indeed the same as with the other formulas:
$$I(\theta) = E[(\frac{\partial}{\partial \theta }logf(X;\theta))^2]$$
I have calculated that for $\frac{\partial}{\partial \theta }logf(X;\theta)$ we have:
$$\frac {-n}{\theta}+ \frac{\sum_1^nX_i}{\theta^2}$$
Squaring that:
$$\frac{n^2}{\theta^2}+\frac{(\sum_1^n X_i)^2}{\theta^4}-\frac{2n\sum_1^nX_i}{\theta^3}$$
And finally taking the expected value and letting it pass through the constants::
$$\frac{n^2}{\theta^2}+\frac{E(\sum_1^n X_i)^2}{\theta^4}-\frac{2nE(\sum_1^nX_i)}{\theta^3}$$
Recalling that $E(X)= \theta$ and $E(X^2)=2\theta^2$ it seems as though the 2nd and 3rd term cancel out leaving $\frac{n^2}{\theta^2}$, but the correct answer is $\frac{n}{\theta^2}$. Where does this go wrong?
 [/INST] Ok using that parameterisation I agree your likelihood is correct! So method one we differentiate again to get
$$
\ell_{\theta \theta} = -\frac{2 \sum x_i }{\theta^3} + \frac{n}{\theta^2}
$$
now since $\mathbb{E} \sum_i x_i = n \theta$ we get 
$$
\begin{align}
\mathcal{I} &= - \mathbb{E}\left[ \ell_{\theta \theta} \right] \\
&= 2 \frac{n\theta}{\theta^3} - \frac{n}{\theta^2} \\\
&= \frac{n}{\theta^2}
\end{align}
$$ 
Alternatively noting that
$$
\begin{align}
\mathbb{E}(\sum_i X_i )^2 &= n\mbox{Var}(X_1) + (n\theta)^2 \\
&= n \theta^2 + n^2 \theta^2
\end{align}
$$
we have
$$
\begin{align}
\mathbb{E} \left[ \ell_{\theta} ^2 \right] &= \mathbb{E} \left[ \left(\frac{1}{\theta^2} \sum_i x_i - \frac{n}{\theta} \right)^2\right] \\
&= \frac{1}{\theta^4}\mathbb{E} \left[\left(\sum_i X_i\right)^2 \right]- \frac{2n}{\theta^3}\mathbb{E}\left[\sum X_i \right]+ \frac{n^2}{\theta^2} \\
&= \frac{n\theta^2}{\theta^4} + \frac{n^2 \theta^2}{\theta^4} - \frac{2 n^2 \theta}{\theta^3} + \frac{n^2}{\theta^2} \\
&=\frac{n}{\theta^2}+\frac{2 n^2}{\theta^2} - \frac{2n^2}{\theta^2} \\
&= \frac{n}{\theta^2}.
\end{align}
$$
</s>