# Derivation of Expectation and Variance of Power from Thermal Noise

## Preliminaries

For the general likelihood of the 2D power (or even 1D power), one needs to know the contribution to the power (and its uncertainty) from thermal noise. In fact, of course, the thermal noise is added in a non-Gaussian manner (same as everything else, really) because it is strictly positive. However, we assume that it is at least *close* to Gaussian, so that we can describe it purely by its mean and variance.

The other thing that is often assumed is that the thermal noise can be treated independently of all other signals in the chain (eg. EoR and FG). We will show that while this is a reasonable assumption when the expected power (from FG and EoR) is much less than the variance of the noise, it does not hold in general. Ignoring this fact leads to consistent under-estimation of the total variance of the power spectrum. We will first perform the calculation assuming that the FG, EoR and noise can be separated, before doing the full derivation.

Throughout the following, we assume that the variance of the *magnitude* of a visibility from thermal noise is $\sigma^2$. This requires that the variance of the real and imaginary components, $\sigma^2_\mathcal{R}, \sigma^2_\mathcal{I} \equiv \sigma^2 /2$. 

## Independent Calculation


Since the thermal noise is independent of everything else, we can *just* deal with it, and neglect everything else, then add it in at the end. So let's begin.

Let every noise visibility, at frequency $\nu$, be called $V_{i,\nu}$ (we don't label it for being noise, as we won't deal with any other type of visibility in this document). Then the visibility of a $uv\nu$ grid cell is

\begin{equation}
    V_{uv\nu} = \frac{1}{n_{uv\nu}}\sum_i V_{i,\nu} \delta_{u_i,v_i \sim uv},
\end{equation}

where the $\delta$ function really just means "only count baselines in the $uv$ cell", and $n_{uv\nu}$ is just the number of baselines in the $uv$ cell at that frequency. Then of course the fourier-space visibility is 

\begin{equation}
    V_{uv\eta} = \Delta\nu \sum e^{-2\pi i \eta \nu} V_{uv\nu} \phi_\nu,
\end{equation}

where $\phi_\nu$ is some applied taper and $\Delta\nu$ is the frequency channel width.

Thus we can get the expectation and variance of $V_{uv\eta}$:

\begin{align}
    \langle V_{uv\eta} \rangle &= 0, \\
    {\rm Var}(V_{uv\eta}) &= \sigma^2 \sum_\nu \frac{(\Delta\nu)^2 \phi_\nu^2}{n_{uv\nu}} \equiv \sigma^2 / \tilde{n}_{uv}.
\end{align}

With $\tilde{n}_{uv} = \left[\sum_\nu \frac{(\Delta\nu)^2 \phi_\nu^2}{n_{uv\nu}}\right]^{-1}$ an "effective" number of baselines in a $uv\eta$ cell.

Now then, we have $P_{uv\eta} = |V_{uv\eta}|^2$, and we need to evaluate the mean and variance of the power. For this, we simply use the relation that the mean and variance of a Gaussian variable squared are $\sigma^2$ and $2\sigma^4$ respectively, so:

\begin{align}
    \langle P_{uv\eta} \rangle &= \sigma^2 / \tilde{n}_{uv} \equiv {\rm Var}(V_{uv\eta}), \\
    {\rm Var}(P_{uv\eta}) &= \frac{2 \sigma^4_\mathcal{R} + 2\sigma^4_{\mathcal{I}}}{\tilde{n}^2_{uv}} \\
    &= \sigma^4 / \tilde{n}^2_{uv} \equiv \langle P_{uv\eta} \rangle ^2.
\end{align}

Now, the power spectrum is circularly averaged. When doing this, we are careful to use the weights of each grid point, which is properly the inverse of the standard deviation of each bin:

\begin{equation}
    P_{u\eta} = \frac{\sum P_{uv\eta} w_{uv\eta}}{\sum w_{uv\eta}},
\end{equation}

where the sum is over all cells within an annulus, and 

\begin{equation}
    w_{uv\eta}  = \frac{\tilde{n}_{uv}}{\sigma^2}.
\end{equation}

Thus we have

\begin{equation}
    P_{u\eta} = \frac{\sum \tilde{n}_{uv} P_{uv\eta} }{\sum  \tilde{n}_{uv}}.
\end{equation}

Finally, we need to calculate the mean and variance of this quantity:

\begin{align}
    \langle P_{u\eta} \rangle &= \frac{\sum \sigma^2}{\sum \tilde{n}_{uv}} \\
    &=\sigma^2 \frac{n_u}{\sum \tilde{n}_{uv}},
\end{align}

where $n_u$ is the number of $uv$ cells in the $u$ annulus. This averages down roughly as $1/\bar{n}_{uv}$. Then the variance:

\begin{align}
    {\rm Var}(P_{u\eta}) &= \frac{\sigma^4 n_u}{(\sum \tilde{n}_{uv})^2}.
\end{align}

## General Derivation

Here we do a (more) general derivation, where we acknowledge that there is EoR signal (with non-zero expectation) that may contribute to the variance of the power. For simplicity, we attempt to reproduce the conditions under which the ``test_thermal_noise`` script is being run: i.e. zero foregrounds, and deterministic signal. **NOTE: in general neither of these assumptions is true, and this derivwation will have to be done again more generally**.

Let $V_{i,\nu} = V_{S,i,\nu} + V_{N,i,\nu}$, where $S$ stands for *signal*.

We note that every operation up to the squaring of the visibilities is linear, and therefore the equations are valid for the noise term (and other terms can be derived similarly). Thus we require only to do calculations from $\langle P_{uv\eta} \rangle onwards. We have, in particular, 

\begin{align}
    P_{uv\eta} &= |V_{uv\eta}|^2 \\
    &= |V_{S,uv\eta} + V_{N, uv\eta}|^2 \\
    &= ||V_{S, uv\eta}|^2 + |V_{N, uv\eta}|^2 + 2\mathcal{Re}\left\{V_{S, uv\eta}V^*_{N, uv\eta}\right\} 
\end{align}

For the expectation of the power, each term is of course treated separately, and all terms involving a single $V_N$ disappear, as $\langle X Y \rangle$ for $\langle X \rangle = 0$ and $X,Y$ independent is zero. Thus the expectation of the power is

\begin{align}
    \langle P_{uv\eta} \rangle &= {\rm Var}(V_{S, uv\eta}) + \frac{\sigma^2}{\tilde{n}_{uv}} \\
    &= \langle P_{N,uv\eta} \rangle,
\end{align}

since the *variance* of the signal is zero (as it is considered deterministic here).

However, for the variance, we have the independent parts acting in the standard manner, but recall that for independent $X,Y$,

\begin{equation}
    {\rm Var}(XY) = {\rm Var}(X){\rm Var}(Y) + {\rm Var}(X) |\langle Y \rangle|^2 + {\rm Var}(Y) |\langle X \rangle|^2.
\end{equation}

So we have 
\begin{align}
    {\rm Var}(P_{uv\eta}) = &\langle P_{N, uv\eta} \rangle^2 + \langle P_{S, uv\eta} \rangle^2 + 2 {\rm Var}(V_{N,uv\eta})P_{S,uv\eta} + \\
        & 2 {\rm Cov}(|V_{S,uv\eta}|^2, \mathcal{Re}\left\{V_{N, uv\eta}V^*_{S, uv\eta}\right\}) + 2 {\rm Cov}(|V_{N,uv\eta}|^2, \mathcal{Re}\left\{V_{N, uv\eta}V^*_{S, uv\eta}\right\}) 
\end{align}

Notice that the last term on the first line needs reasonably careful handling of the real/imaginary components, and recalling that $S$ is deterministic.

Recall also that 

\begin{align}
{\rm Cov}[XX, XY] &= E[X^3]E[Y] - E[X^2]E[X]E[Y]  \\
\end{align}

for independent $X,Y$. But this is not exactly what we have. Instead, we have  
\begin{align}
{\rm Cov}[|X|^2, \mathcal{Re} (XY)] &= E[X_\mathcal{R}^3]E[Y_\mathcal{R}] - E[X_\mathcal{R}^2]E[X_\mathcal{R}]E[Y_\mathcal{R}] -  E[X_\mathcal{I}^3]E[Y_\mathcal{I}] + E[X_\mathcal{I}^2]E[X_\mathcal{I}]E[Y_\mathcal{I}].
\end{align}

Regardless, this removes all covariance terms, as its noise part  appears to the odd power in each, so we are left with

\begin{align}
    {\rm Var}(P_{uv\eta}) &= \langle P_{N, uv\eta} \rangle^2 + \langle P_{S, uv\eta} \rangle^2 + 2{\rm Var}(V_{N,uv\eta})|\langle V_{S,uv\eta} \rangle|^2 \\
    &= \langle P_{N, uv\eta} \rangle \left[ \langle P_{N, uv\eta} \rangle + 2|\langle V_{S,uv\eta} \rangle|^2 \right] + \langle P_{S, uv\eta} \rangle^2
    \end{align}

Since the signal is deterministic, we can do away with its expectation signs, and note that $|V_S|^2 \equiv P_S$:

\begin{align}
    {\rm Var}(P_{uv\eta}) &= \langle P_{N, uv\eta} \rangle \left[ \langle P_{N, uv\eta} \rangle + 2P_{S,uv\eta} \right] + P_{S, uv\eta}^2
    \end{align}

Thus, when $\langle P_N \rangle \gg 2 P_S$, the solution is the same as the above independent case. But this will not be true where the signal power is large compared to the noise (for which we must assume some bins exist). 

**The question is: is this correct? At the moment, I can only see how this would add *more* power in some bins, but our numerical estimates suggest that the analytical variance is already too high**

31b8be8d2c1d44277a90c046a5eb8c65af69c01a
ipynb
Jupyter Notebook
devel/noise_power_derivation.ipynb
dprelogo/py21cmmc_fg
beee2a2a0a55a22430f37a1eaf4e4d83115421a1
MIT
2018-05-04T06:05:19.000Z
2018-05-04T06:05:19.000Z
devel/noise_power_derivation.ipynb
dprelogo/py21cmmc_fg
beee2a2a0a55a22430f37a1eaf4e4d83115421a1
MIT
2018-05-15T23:47:06.000Z
2019-02-27T07:20:28.000Z
devel/noise_power_derivation.ipynb
dprelogo/py21cmmc_fg
beee2a2a0a55a22430f37a1eaf4e4d83115421a1
MIT
2018-05-04T06:04:51.000Z
2020-01-28T12:50:09.000Z
Qwen/Qwen-72B
1. YES
2. YES