Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete.

1) If the base distribution is continuous, drawing from it will get a new value (a new cluster mean, for example) every time.

2) But, you can nest Dirichlet processes. Which implies the base distribution can be discrete. (You can place a prior over your base distribution which is just another Dirichlet process).

Consider the Blackwell-Macqueen urn scheme.

H is a distribution over colors.

Start with an empty urn.

With probability alpha sample a color from H. Note, if H is continuous, you'll get a different color every time. If the H is discrete, you may draw colors already present in the urn.

With probability proportional to n-1, pick a ball from the urn. Put the ball back and another one of the same color.

Reference: http://www.columbia.edu/~jwp2128/Papers/PaisleyWangetal2015.pdf

edit: The paper above demonstrates this is possible. Any distribution drawn for the a DP with a discrete base distribution will have the same atoms as the base distribution, but with different weights.

Still, I wonder how the CRP would work under this scheme since there's a chance you'd select a non-empty table (analogously to the above urn scheme).

  • $\begingroup$ Apparently, after reading a few excerpts from some papers, I think they can be. Not sure then how this fits in with sampling schemes. $\endgroup$ – yalis Sep 30 '17 at 22:46

The base measure can be discrete. An example of the case where it is discrete is the Hierarchichal Dirichlet Process, wherein $F_j \sim \mathcal D(\alpha F_0)$ where $F_0 \sim \mathcal D(\gamma H)$. The base measure of the second level of the hierarchy ($F_0$) is discrete with probability $1$.

| cite | improve this answer | |

Your Answer

