{"action":"created","issue":{"url":"https://api.github.com/repos/stan-dev/rstanarm/issues/45","labels_url":"https://api.github.com/repos/stan-dev/rstanarm/issues/45/labels{/name}","comments_url":"https://api.github.com/repos/stan-dev/rstanarm/issues/45/comments","events_url":"https://api.github.com/repos/stan-dev/rstanarm/issues/45/events","html_url":"https://github.com/stan-dev/rstanarm/issues/45","id":125246169,"number":45,"title":"Should we really use confint method for Bayesian credible intervals?","user":{"login":"jgabry","id":7796803,"avatar_url":"https://avatars.githubusercontent.com/u/7796803?v=3","gravatar_id":"","url":"https://api.github.com/users/jgabry","html_url":"https://github.com/jgabry","followers_url":"https://api.github.com/users/jgabry/followers","following_url":"https://api.github.com/users/jgabry/following{/other_user}","gists_url":"https://api.github.com/users/jgabry/gists{/gist_id}","starred_url":"https://api.github.com/users/jgabry/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jgabry/subscriptions","organizations_url":"https://api.github.com/users/jgabry/orgs","repos_url":"https://api.github.com/users/jgabry/repos","events_url":"https://api.github.com/users/jgabry/events{/privacy}","received_events_url":"https://api.github.com/users/jgabry/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":21,"created_at":"2016-01-06T19:35:13Z","updated_at":"2016-01-06T22:10:43Z","closed_at":null,"body":"@bgoodri I've been thinking recently that it might avoid a great deal of confusion if we create a separate method for obtaining so-called credible intervals rather than using R's standard `confint`. Perhaps an S3 generic `credint` and a `credint.stanreg` method? (Or just a `credint` function, skipping the S3 method stuff?) If we want to do this then I can make the change quite easily this afternoon.\r\n\r\nHere are some reasons why I think it's worth doing this\r\n\r\n1. The documentation for `stats::confint`, which we can't change, clearly says it's for confidence intervals. \r\n2. It would give us the opportunity to document `credint`, where we can make it clear what the differences are between frequentist/Bayesian confidence/credible intervals. We don't really do this anywhere at the moment but I think the differences in interpretation are important.\r\n3. The cost is small. The only downside I can think of is that some users will try to call `confint` for a model not estimated by optimization, but we can either issue a warning to use `credint` (and allow `confint` to still be usable) or an error that forces the use of `credint`.\r\n4. If we wanted we could provide credible intervals based on posterior quantiles (as we do now) but also give the option to compute HPD intervals (that would be valid assuming there isn't severe multimodality). "},"comment":{"url":"https://api.github.com/repos/stan-dev/rstanarm/issues/comments/169478593","html_url":"https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169478593","issue_url":"https://api.github.com/repos/stan-dev/rstanarm/issues/45","id":169478593,"user":{"login":"andrewgelman","id":3375441,"avatar_url":"https://avatars.githubusercontent.com/u/3375441?v=3","gravatar_id":"","url":"https://api.github.com/users/andrewgelman","html_url":"https://github.com/andrewgelman","followers_url":"https://api.github.com/users/andrewgelman/followers","following_url":"https://api.github.com/users/andrewgelman/following{/other_user}","gists_url":"https://api.github.com/users/andrewgelman/gists{/gist_id}","starred_url":"https://api.github.com/users/andrewgelman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrewgelman/subscriptions","organizations_url":"https://api.github.com/users/andrewgelman/orgs","repos_url":"https://api.github.com/users/andrewgelman/repos","events_url":"https://api.github.com/users/andrewgelman/events{/privacy}","received_events_url":"https://api.github.com/users/andrewgelman/received_events","type":"User","site_admin":false},"created_at":"2016-01-06T22:10:43Z","updated_at":"2016-01-06T22:10:43Z","body":"Here’s the paper:\nhttp://www.stat.columbia.edu/~gelman/research/published/spin.pdf\nShort answer is that brute force intervals are a bit noisy; some smoothing helps but then things get complicated.  The method in our paper works well but it’s a bit too complicated for my taste.  I’ve wanted to go back and do more but haven’t done so.  Noisy intervals _will_ be a problem, I think, especially given that (a) in some settings we don’t recommend taking a huge number of draws, and (b) often people want 95% intervals (which I think is generally a mistake, but that’s another story).  So it could make sense to just implement central intervals for now?\nA\n\n> On Jan 6, 2016, at 5:06 PM, bgoodri <notifications@github.com> wrote:\n> \n> With 1000 draws, it is not going to be that intensive to brute force it\n> relative to estimating the model. If we sort the draws, then there is a\n> finite number of intervals that contain at least 95% of the points and then\n> it is just a which.min on the diff of the admissible intervals.\n> \n> On Wed, Jan 6, 2016 at 5:00 PM, Andrew Gelman <notifications@github.com>\n> wrote:\n> \n> > I don’t know how easy it is to implement spin in practice. In our paper,\n> > Tian and I have an algorithm that has good statistical properties but it’s\n> > a bit complicated, actually I don’t really understand exactly what we did!\n> >\n> > > On Jan 6, 2016, at 4:59 PM, bgoodri <notifications@github.com> wrote:\n> > >\n> > > The spin is fine with me, although for these models multimodality\n> > shouldn't\n> > > be an issue.\n> > >\n> > > On Wed, Jan 6, 2016 at 4:57 PM, Andrew Gelman <notifications@github.com>\n> > > wrote:\n> > >\n> > > > Our messages overlapped . . . I actually prefer shortest probability\n> > > > interval (spin) rather than HPD for the reason that if the density is\n> > > > multimodal we’re still returning just a simple interval.\n> > > > A\n> > > >\n> > > > > On Jan 6, 2016, at 4:56 PM, Jonah Gabry <notifications@github.com>\n> > > > wrote:\n> > > > >\n> > > > > If we restrict ourselves to just using posterior quantiles then that\n> > > > could be ok, but if we ofter HPD intervals (not saying we definitely\n> > > > should, but I have the code to do it if we want) then central interval\n> > > > isn't quite right.\n> > > > >\n> > > > > —\n> > > > > Reply to this email directly or view it on GitHub <\n> > > > https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169475226\n> > >.\n> > > >\n> > > > >\n> > > >\n> > > > —\n> > > > Reply to this email directly or view it on GitHub\n> > > > <https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169475536\n> > >.\n> > > >\n> > > —\n> > > Reply to this email directly or view it on GitHub <\n> > https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169475849>.\n> >\n> > >\n> >\n> > —\n> > Reply to this email directly or view it on GitHub\n> > <https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169476279>.\n> >\n> —\n> Reply to this email directly or view it on GitHub <https://github.com/stan-dev/rstanarm/issues/45#issuecomment-169477645>.\n> \n\n"}}