<p>I would expect this to happen in accordance with the group mean distribution on alpha. If you think about it, if the group mean shifts around it will influence all alphas to the same degree. You could confirm this by doing a scatter plot of the group mean trace against some of the alphas. Hierarchical models are in general difficult for most samplers because of these complex interdependencies between group mean and variance and the individual RVs. See <a href="http://arxiv.org/abs/1312.0906" rel="nofollow">http://arxiv.org/abs/1312.0906</a> for more information on this.</p>

<p>In your specific case, the trace doesn't look too worrisome to me, especially after iteration 1000. So you could probably just discard those as burn-in and keep in mind that you have some sampling noise but probably got the right posterior overall. In addition, you might want to perform a posterior predictive check to see if the model can reproduce the patterns in your data you are interested in.</p>

<p>Alternatively, you could try to estimate a better hessian using <code>pm.find_hessian()</code>, e.g. <a href="https://github.com/pymc-devs/pymc/blob/3eb2237a8005286fee32776c304409ed9943cfb3/pymc/examples/hierarchical.py#L51" rel="nofollow">https://github.com/pymc-devs/pymc/blob/3eb2237a8005286fee32776c304409ed9943cfb3/pymc/examples/hierarchical.py#L51</a></p>

<p>I also found this paper which looks interesting (haven't read it yet but might be cool to implement in PyMC3): arxiv-web3.library.cornell.edu/pdf/1406.3843v1.pdf</p>
Odd correlated posterior traceplots in multilevel model
<p>I'm trying out PyMC3 with a simple multilevel model.  When using both fake and real data the traces of the random effect distributions move with each other (see plot below) and appear to be offsets of the same trace. Is this an expected artifact of NUTS or an indication of a problem with my model?</p>

<p>Here is a traceplot on real data:</p>

<p><img src="https://i.stack.imgur.com/w4yvf.png" alt="traceplot"></p>

<p>Here is an <a href="http://nbviewer.ipython.org/gist/bmabey/07e600887276becbaf4f" rel="nofollow noreferrer">IPtyhon notebook</a> of the model and the functions used to create the fake data.  Here is the <a href="https://gist.github.com/bmabey/07e600887276becbaf4f" rel="nofollow noreferrer">corresponding gist</a>.</p>