{"action":"opened","issue":{"url":"https://api.github.com/repos/tensorprob/tensorprob/issues/74","repository_url":"https://api.github.com/repos/tensorprob/tensorprob","labels_url":"https://api.github.com/repos/tensorprob/tensorprob/issues/74/labels{/name}","comments_url":"https://api.github.com/repos/tensorprob/tensorprob/issues/74/comments","events_url":"https://api.github.com/repos/tensorprob/tensorprob/issues/74/events","html_url":"https://github.com/tensorprob/tensorprob/issues/74","id":138794133,"number":74,"title":"Constrained variables","user":{"login":"ibab","id":890531,"avatar_url":"https://avatars.githubusercontent.com/u/890531?v=3","gravatar_id":"","url":"https://api.github.com/users/ibab","html_url":"https://github.com/ibab","followers_url":"https://api.github.com/users/ibab/followers","following_url":"https://api.github.com/users/ibab/following{/other_user}","gists_url":"https://api.github.com/users/ibab/gists{/gist_id}","starred_url":"https://api.github.com/users/ibab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ibab/subscriptions","organizations_url":"https://api.github.com/users/ibab/orgs","repos_url":"https://api.github.com/users/ibab/repos","events_url":"https://api.github.com/users/ibab/events{/privacy}","received_events_url":"https://api.github.com/users/ibab/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-03-06T14:11:57Z","updated_at":"2016-03-06T14:11:57Z","closed_at":null,"body":"There are several cases where it would be useful to have constrained random variables.\r\nOne such example is a `Simplex`, a list of scalars that always sum to `1`.\r\nThis could be super useful to make mixtures easier to use:\r\n```\r\nweights = Simplex(shape=2)\r\nMix(weights, Dist1(), Dist2())\r\n```\r\nNo need to play around with increasingly complicated nesting of `f` and `(1-f)`!\r\nInternally, we only need `N-1` parameters to describe a Simplex with `N` values.\r\n\r\nAnother useful application are covariance matrices (always positive definite):\r\n```\r\nmu = Uniform(shape=3)\r\ncov = CovMatrix(shape=(3,3))\r\nX1, X2, X3 = NormalN(mu, cov)\r\n```\r\nWe only need `N + N` parameters to fully specify a covariance matrix instead of the `N * N` entries in the matrix.\r\n\r\nSo we need to parameterize these constrained variables in a certain way.\r\nWe would also pass the parameterization parameters to our optimization and sampling backends.\r\nBut when the user queries `model.state`, we would return the \"beautified\" parameters (e.g. simplex and covariance matrix).\r\nOne thing that's interesting here is that these parameterizations are pretty much the same thing as the transforms described in #73.\r\nSo we shouldn't forget to add the change of variables factor to the `logp`!\r\nIn fact, maybe we can implement both transformations and constrained variables in one go, using the same code!\r\n\r\nAnother thing that falls under this topic is that we could completely get rid of the need for boundaries by using transformations that map our parameters from their bounded regions into the full `(∞,∞)` range.\r\nThis is also just a regular transformation that needs the `logp` correction factor.\r\n\r\nWhat happens if the user passes in a transformed variable as data?\r\nIn this case, we need to ignore the underlying parameterization and the `logp` correction and simply \"bake\" the observed value into the logp."}}