I will ask my question through a toy motivating example.

It is well known that a Poisson process is the continuous time analog to a Bernoulli process (for example: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-262-discrete-stochastic-processes-spring-2011/course-notes/MIT6_262S11_chap02.pdf). There is also a similar well known relationship between the Gamma distribution and the Beta distribution (for example: https://math.stackexchange.com/questions/190670/how-exactly-are-the-beta-and-gamma-distributions-related). I would like to make one related point that seem less obvious in the literature:

If we place a Beta prior upon that Bernoulli process and then compute the marginal likelihood, can we prove that this is the discrete time analog to the corresponding marginal likelihood when we place a Gamma prior upon the Poisson process... only using the underlying correspondences between Beta and Gamma, and between Bernoulli process and Poisson process? Is there a theorem we may invoke and under what conditions may we invoke it, or do we have to show correspondence by working with the marginals directly?

This example in particular is a toy -- it is relatively straightforward to work with the marginal likelihoods in the two cases, starting with the discrete case and shrinking the time increments down to zero. However for a research project I am working on, it is much easier to show correspondence between likelihoods, and between priors, than between marginal likelihoods. In my research, I am considering a Bernoulli($p_1$) process which is `stopped' at some time $\tau$ (i.e., all observations after time point $\tau$ equal 0 with probability 1), where $\tau$ is geometric($p_2$) distributed. Both $p_1$ and $p_2$ have independent Beta priors.

To put this in formulae using generic notation, if the limiting form of the pmf is equal in distribution to another pmf: $$ \lim_{n \rightarrow \infty} p(x | \theta_n) = p(x | \theta) $$

and the limiting form of the prior pdf is equal in distribution to another pdf:

$$ \lim_{n \rightarrow \infty} p(\theta_n | \psi_n) = p(\theta_n | \psi) $$

then under what conditions (any?) can we `take the limit within the integral' and state that marginal likelihoods are equal in distribution as well:

$$ \lim_{n \rightarrow \infty} \int p(x | \theta_n) p(\theta_n | \psi_n) d\theta_n = \int \lim_{n \rightarrow \infty} p(x | \theta_n) p(\theta_n | \psi_n) d\theta_n = \int p(x | \theta) p(\theta | \psi) d\theta \ \ \text{?} $$

Any help would be most appreciated. I've tried to make this as clear as possible.

  • $\begingroup$ +50 seems like a small bounty for a task :) $\endgroup$ – Aksakal Dec 5 '14 at 16:28
  • $\begingroup$ I did't realize how much of a task it was until I got this non-response to my bounty. I thought there might have been an easy answer so I am deducing that perhaps there isn't. $\endgroup$ – Dan Dec 5 '14 at 19:27
  • $\begingroup$ I don't see an obvious answer. This is not the standard problem everybody deals with, so if you're lucky someone will recognize this and give you an answer. Otherwise, one needs to spend some time working out the equations. $\endgroup$ – Aksakal Dec 5 '14 at 19:29
  • $\begingroup$ Agreed Aksakal - although I must say, the answer to this would seem to be quite relevant to many people working with stochastic models for whatever activity they happen to be studying. $\endgroup$ – Dan Dec 6 '14 at 0:18

I think what you want is the dominated convergence theorem. From Wikipedia:

Let $f_n$ be a sequence of real-valued measurable functions on a measure space $(S, Σ, \mu)$. Suppose that the sequence converges pointwise to a function $f$ and is dominated by some integrable function $g$ in the sense that $$|f_n(x)| \le g(x)$$ for all numbers $n$ in the index set of the sequence and all points $x \in S$. Then $f$ is integrable and $$\lim_{n\to\infty} \int_S |f_n-f|\,d\mu = 0$$ which also implies $$\lim_{n\to\infty} \int_S f_n\,d\mu = \int_S f\,d\mu.$$

Here $f_n = p(x |\theta_n) p(\theta_n | \psi_n)$. Since it is a product of probabilities, it seems hopeful that you can bound the $f_n$ by some integrable $g$.

| cite | improve this answer | |
  • $\begingroup$ I was thinking the same thing Ben - many thanks... $\endgroup$ – Dan Dec 8 '14 at 13:53

Your Answer

