<s>[INST] Cauchy distribution (likelihood and Fisher information) I have a three part question:
1) If I have a $Cauchy(\theta, 1)$ with density:
$p(x-\theta) = \frac{1}{\pi\{1+{(x-\theta)^2}\}}$
and $x_1, ..., x_n$ forms i.i.d sample. I see on Wiki that this will make a likelihood function of:
$l(\theta) = -nlog\pi - \sum_{i=1}^{n} log(1+{(x_i-\theta)^2})$
But how is that derived?
2) What derivative rules would I use to show that:
$l'(\theta) = -2 \sum_{i=1}^{n} \frac{x_i-\theta}{1+(x_i-\theta)^2}$
(I feel this is probably straight-forward, but I am missing it)
3) How would I obtain that the Fisher information is equal to $I(\theta)=\frac{n}{2}$?
I started by:
$\frac{\partial^2f(x;\theta)}{2\theta^2} = \frac{8(x-\theta)^2}{\pi[1+(x-\theta)^2]^3} - \frac{2}{\pi[1+(x-\theta)^2]^2}$
I think that I next need to find the integral of:
$I(\theta) = -E[\frac{\partial^2f(x;\theta)}{2\theta^2}]$
But I cannot get this last step of reducing this information to $\frac{n}{2}$.
 [/INST] The Fisher information for one observation is given by\begin{align*}I(\theta) &= -\mathbb{E}_\theta\left[\frac{\partial^2 \log f(X;\theta)}{\partial\theta^2}\right]\\ &=\mathbb{E}_\theta\left[ \frac{\partial^2 \log \{1+(X-\theta)^2\}}{\partial\theta^2}\right]\\
&=2\mathbb{E}_\theta\left[ -\frac{\partial }{\partial\theta}\frac{(X-\theta)}{1+(X-\theta)^2}\right]\\
&=2\mathbb{E}_\theta\left[\frac{1}{1+(X-\theta)^2}-\frac{2(X-\theta)^2}{[1+(X-\theta)^2]^2}\right]\\
&= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+(x-\theta)^2]^2}-\frac{2(x-\theta)^2}{[1+(x-\theta)^2]^3} \text{d}x\\
&= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2x^2}{[1+x^2]^3} \text{d}x\\
&= \frac{2}{\pi}\int_\mathbb{R} \frac{1}{[1+x^2]^2}-\frac{2}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x\\
&= \frac{2}{\pi}\int_\mathbb{R} \frac{-1}{[1+x^2]^2}+\frac{2}{[1+x^2]^3} \text{d}x
\end{align*}
because the integral (and the information) is translation invariant.
Now it is easy to establish a recurrence relation on$$I_k=\int_\mathbb{R} \frac{1}{[1+x^2]^k}\text{d}x$$Indeed
\begin{align*}
I_k &= \int_\mathbb{R} \frac{1+x^2}{[1+x^2]^{k+1}}\text{d}x\\
&= I_{k+1} + \int_\mathbb{R} \frac{2kx}{[1+x^2]^{k+1}}\frac{x}{2k}\text{d}x\\
&= I_{k+1} + \frac{1}{2k} \int_\mathbb{R} \frac{1}{[1+x^2]^{k}}\text{d}x
= I_{k+1} + \frac{1}{2k} I_k
\end{align*}
by an integration by parts. Hence
$$I_1=\pi\quad\text{and}\quad I_{k+1}=\frac{2k-1}{2k}I_k\quad k>1$$
which implies
$$I_1=\pi\quad I_2=\frac{\pi}{2}\quad I_3=\frac{3\pi}{8}$$
and which leads to the Fisher information:
$$I(\theta)=\frac{2}{\pi}\left\{-I_2+2I_3\right\}=\frac{2}{\pi}\left\{\frac{-\pi}{2}+\frac{3\pi}{4}\right\}=\frac{1}{2}$$
</s>