{"action":"created","comment":{"url":"https://api.github.com/repos/blei-lab/edward/pulls/comments/68962526","id":68962526,"diff_hunk":"@@ -199,12 +199,10 @@ def build_score_loss(self):\n         ELBO = E_{q(z; lambda)} [ log p(x, z) - log q(z; lambda) ]\n         \"\"\"\n         x = self.data.sample(self.n_data)\n-        z, self.samples = self.variational.sample(self.n_minibatch)\n-\n-        q_log_prob = tf.zeros([self.n_minibatch], dtype=tf.float32)\n-        for i in range(self.variational.num_factors):\n-            q_log_prob += self.variational.log_prob_i(i, tf.stop_gradient(z))\n+        self.samples = self.variational.sample(self.n_minibatch)\n+        z = self.samples","path":"edward/inferences.py","position":19,"original_position":19,"commit_id":"9cad3ee0af3377cfd8749ff18b87bc5ba2754b43","original_commit_id":"9cad3ee0af3377cfd8749ff18b87bc5ba2754b43","user":{"login":"akucukelbir","id":329960,"avatar_url":"https://avatars.githubusercontent.com/u/329960?v=3","gravatar_id":"","url":"https://api.github.com/users/akucukelbir","html_url":"https://github.com/akucukelbir","followers_url":"https://api.github.com/users/akucukelbir/followers","following_url":"https://api.github.com/users/akucukelbir/following{/other_user}","gists_url":"https://api.github.com/users/akucukelbir/gists{/gist_id}","starred_url":"https://api.github.com/users/akucukelbir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akucukelbir/subscriptions","organizations_url":"https://api.github.com/users/akucukelbir/orgs","repos_url":"https://api.github.com/users/akucukelbir/repos","events_url":"https://api.github.com/users/akucukelbir/events{/privacy}","received_events_url":"https://api.github.com/users/akucukelbir/received_events","type":"User","site_admin":false},"body":"does `self.samples` always/only contain samples from the variational model?\r\n\r\nif so, consider renaming it `self.z_samples` and avoid statements like `z = self.samples`.","created_at":"2016-06-29T15:02:15Z","updated_at":"2016-06-29T15:02:15Z","html_url":"https://github.com/blei-lab/edward/pull/138#discussion_r68962526","pull_request_url":"https://api.github.com/repos/blei-lab/edward/pulls/138","_links":{"self":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/comments/68962526"},"html":{"href":"https://github.com/blei-lab/edward/pull/138#discussion_r68962526"},"pull_request":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/138"}}},"pull_request":{"url":"https://api.github.com/repos/blei-lab/edward/pulls/138","id":75571466,"html_url":"https://github.com/blei-lab/edward/pull/138","diff_url":"https://github.com/blei-lab/edward/pull/138.diff","patch_url":"https://github.com/blei-lab/edward/pull/138.patch","issue_url":"https://api.github.com/repos/blei-lab/edward/issues/138","number":138,"state":"open","locked":false,"title":"Feature/shapes","user":{"login":"dustinvtran","id":2569867,"avatar_url":"https://avatars.githubusercontent.com/u/2569867?v=3","gravatar_id":"","url":"https://api.github.com/users/dustinvtran","html_url":"https://github.com/dustinvtran","followers_url":"https://api.github.com/users/dustinvtran/followers","following_url":"https://api.github.com/users/dustinvtran/following{/other_user}","gists_url":"https://api.github.com/users/dustinvtran/gists{/gist_id}","starred_url":"https://api.github.com/users/dustinvtran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dustinvtran/subscriptions","organizations_url":"https://api.github.com/users/dustinvtran/orgs","repos_url":"https://api.github.com/users/dustinvtran/repos","events_url":"https://api.github.com/users/dustinvtran/events{/privacy}","received_events_url":"https://api.github.com/users/dustinvtran/received_events","type":"User","site_admin":false},"body":"#### Summary:\r\n\r\n+ issues: makes progress for #7; makes progress for #27; helps refactor for #80 in moving to `tf.contrib.distributions`\r\n\r\nAll distribution objects, i.e., any class inheriting from `Distribution` in `edward.models`, now takes as input a shape and optionally, parameter arguments. The shape denotes the shape of its random variable. For example:\r\n```python\r\nfrom edward.models import Normal\r\n\r\n# 5 x 2 matrix of random variables\r\nx = Normal([5, 2])\r\nx.sample(size=10)\r\n```\r\nThe output is a set of samples with shape `(10, 5, 2)`, where size is the outer dimension.\r\n\r\nThis generalizes the previously cryptic `num_factors` argument and enables tensors of random variables. Note also that as a result, all sample methods—whether in be in the container object `Variational()`, distribution objects `Distribution()`, or `edward.stats` sampling `rvs()`—will draw samples where the size is the outer dimension. For example:\r\n```python\r\nfrom edward.stats import bernoulli\r\n\r\nx = bernoulli.rvs(p=0.5, size=1)\r\nprint(x.shape)\r\n## (1,)\r\nx = bernoulli.rvs(p=np.array([0.5]), size=1)\r\nprint(x.shape)\r\n## (1, 1)\r\nx = bernoulli.rvs(p=np.array([0.5, 0.2]), size=3)\r\nprint(x.shape)\r\n## (3, 2)\r\n```\r\nThis goes against SciPy behavior (which is inconsistent anyways): `size=1` typically results in the same output dimension as the parameter rather than `size x shape(parameter)`. Our standard adheres to the behavior in the `tf.contrib.distributions` library.\r\n\r\nThe container object `Variational()` now works with a list of tensors if there is more than one layer, and a single tensor if only one layer. This arises in the input for `variational.log_prob(xs)` as well as the output for `variational.sample(size)`. Carrying around a list of tensors makes packing and unpacking of latent variables much easier to work with; see the mixture of Gaussians example. Carrying around a tensor if only one layer is convenient for probability models that deal with only one set of latent variables; it's easier to work directly with `zs` rather than do `zs[0]`; see the Beta-Bernoulli or hierarchical linear model examples.\r\n\r\nThere is a nuance worth mentioning why there's a difference in the `log_prob(xs, zs)` methods of `mixture_gaussian.py` compared to `mixture_gaussian_map.py`. The former uses three sets of variational distributions; the latter uses one (a point mass). This means the former takes `zs` as a list of 3 tensors, and the latter takes `zs` as a single tensor. While this isn't satisfactory (the probability model's method should not rely on the variational model downstream), this makes the difference which already currently exists more transparent.\r\n\r\nFinally, I also simplified the `log_prob()` method. All current inference algorithms use a more holistic `variational.log_prob()` rather than explicitly summing over `variational.log_prob_idx()`. We can deal with the latter in future special cases when we do in fact want to take advantage of individual factors.\r\n\r\nExtensions\r\n  + __Matrixvariate random variables, tensorvariate, etc.__ We currently use a flag in the distribution obejcts to check if `is_multivariate`, which is important for indexing in the methods. We can later generalize the boolean to some integer value, e.g., 0 if univariate, 1 if vectorvariate, 2 if matrixvariate, etc.\r\n  + __Smart arguments for the distribution object.__ For example, we may want to do `Normal(3, loc=1, scale=scale)`, where `scale` is `(3, 5)` and the 1 in `loc` automates to `(3, 5)`.\r\n  + __Shape of samples when size=1.__ In a previous series of commits, I used the output `shape` if size = 1 and `size x shape` if size > 1. We can bring this back later on when it may be useful. For example, `tf.contrib.bayesflow.scg` allow this through the `SampleResampleValue` data type.\r\n  + __Higher-order shapes.__ While the construction of the distribution and container objects is general, we are limited to at most matrix random variables because the primitives it calls upon in `edward.stats` are not guaranteed to work for 3-d or higher tensors. This includes: `rvs()`, `logpmf()`/`logpdf()`, `to_simplex()`; and `entropy()` is not guaranteed to work for 2-d or higher tensors if the distribution is univariate.\r\n\r\n#### Intended Effect:\r\n\r\nSee above.\r\n\r\n#### How to Verify:\r\n\r\nRun all examples and tests. They still work.\r\n\r\nWe need additional unit tests for three things:\r\n+ container object\r\n+ distribution objects\r\n+ [x] `rvs()` in `edward.stats`\r\n\r\n#### Side Effects:\r\n\r\nNone.\r\n\r\n#### Documentation:\r\n\r\nI added and clarified documentation to all relevant methods.\r\n\r\n#### Reviewer Suggestions:\r\n\r\nN/A","created_at":"2016-06-29T00:22:51Z","updated_at":"2016-06-29T15:02:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"0a176a4d73df698cae7d5e40498972d3350eac8f","assignee":{"login":"akucukelbir","id":329960,"avatar_url":"https://avatars.githubusercontent.com/u/329960?v=3","gravatar_id":"","url":"https://api.github.com/users/akucukelbir","html_url":"https://github.com/akucukelbir","followers_url":"https://api.github.com/users/akucukelbir/followers","following_url":"https://api.github.com/users/akucukelbir/following{/other_user}","gists_url":"https://api.github.com/users/akucukelbir/gists{/gist_id}","starred_url":"https://api.github.com/users/akucukelbir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akucukelbir/subscriptions","organizations_url":"https://api.github.com/users/akucukelbir/orgs","repos_url":"https://api.github.com/users/akucukelbir/repos","events_url":"https://api.github.com/users/akucukelbir/events{/privacy}","received_events_url":"https://api.github.com/users/akucukelbir/received_events","type":"User","site_admin":false},"assignees":[{"login":"akucukelbir","id":329960,"avatar_url":"https://avatars.githubusercontent.com/u/329960?v=3","gravatar_id":"","url":"https://api.github.com/users/akucukelbir","html_url":"https://github.com/akucukelbir","followers_url":"https://api.github.com/users/akucukelbir/followers","following_url":"https://api.github.com/users/akucukelbir/following{/other_user}","gists_url":"https://api.github.com/users/akucukelbir/gists{/gist_id}","starred_url":"https://api.github.com/users/akucukelbir/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akucukelbir/subscriptions","organizations_url":"https://api.github.com/users/akucukelbir/orgs","repos_url":"https://api.github.com/users/akucukelbir/repos","events_url":"https://api.github.com/users/akucukelbir/events{/privacy}","received_events_url":"https://api.github.com/users/akucukelbir/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/blei-lab/edward/pulls/138/commits","review_comments_url":"https://api.github.com/repos/blei-lab/edward/pulls/138/comments","review_comment_url":"https://api.github.com/repos/blei-lab/edward/pulls/comments{/number}","comments_url":"https://api.github.com/repos/blei-lab/edward/issues/138/comments","statuses_url":"https://api.github.com/repos/blei-lab/edward/statuses/9cad3ee0af3377cfd8749ff18b87bc5ba2754b43","head":{"label":"blei-lab:feature/shapes","ref":"feature/shapes","sha":"9cad3ee0af3377cfd8749ff18b87bc5ba2754b43","user":{"login":"blei-lab","id":9000745,"avatar_url":"https://avatars.githubusercontent.com/u/9000745?v=3","gravatar_id":"","url":"https://api.github.com/users/blei-lab","html_url":"https://github.com/blei-lab","followers_url":"https://api.github.com/users/blei-lab/followers","following_url":"https://api.github.com/users/blei-lab/following{/other_user}","gists_url":"https://api.github.com/users/blei-lab/gists{/gist_id}","starred_url":"https://api.github.com/users/blei-lab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blei-lab/subscriptions","organizations_url":"https://api.github.com/users/blei-lab/orgs","repos_url":"https://api.github.com/users/blei-lab/repos","events_url":"https://api.github.com/users/blei-lab/events{/privacy}","received_events_url":"https://api.github.com/users/blei-lab/received_events","type":"Organization","site_admin":false},"repo":{"id":51468412,"name":"edward","full_name":"blei-lab/edward","owner":{"login":"blei-lab","id":9000745,"avatar_url":"https://avatars.githubusercontent.com/u/9000745?v=3","gravatar_id":"","url":"https://api.github.com/users/blei-lab","html_url":"https://github.com/blei-lab","followers_url":"https://api.github.com/users/blei-lab/followers","following_url":"https://api.github.com/users/blei-lab/following{/other_user}","gists_url":"https://api.github.com/users/blei-lab/gists{/gist_id}","starred_url":"https://api.github.com/users/blei-lab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blei-lab/subscriptions","organizations_url":"https://api.github.com/users/blei-lab/orgs","repos_url":"https://api.github.com/users/blei-lab/repos","events_url":"https://api.github.com/users/blei-lab/events{/privacy}","received_events_url":"https://api.github.com/users/blei-lab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/blei-lab/edward","description":"A library for probabilistic modeling, inference, and criticism. Deep generative models, variational inference. Runs on TensorFlow.","fork":false,"url":"https://api.github.com/repos/blei-lab/edward","forks_url":"https://api.github.com/repos/blei-lab/edward/forks","keys_url":"https://api.github.com/repos/blei-lab/edward/keys{/key_id}","collaborators_url":"https://api.github.com/repos/blei-lab/edward/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/blei-lab/edward/teams","hooks_url":"https://api.github.com/repos/blei-lab/edward/hooks","issue_events_url":"https://api.github.com/repos/blei-lab/edward/issues/events{/number}","events_url":"https://api.github.com/repos/blei-lab/edward/events","assignees_url":"https://api.github.com/repos/blei-lab/edward/assignees{/user}","branches_url":"https://api.github.com/repos/blei-lab/edward/branches{/branch}","tags_url":"https://api.github.com/repos/blei-lab/edward/tags","blobs_url":"https://api.github.com/repos/blei-lab/edward/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/blei-lab/edward/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/blei-lab/edward/git/refs{/sha}","trees_url":"https://api.github.com/repos/blei-lab/edward/git/trees{/sha}","statuses_url":"https://api.github.com/repos/blei-lab/edward/statuses/{sha}","languages_url":"https://api.github.com/repos/blei-lab/edward/languages","stargazers_url":"https://api.github.com/repos/blei-lab/edward/stargazers","contributors_url":"https://api.github.com/repos/blei-lab/edward/contributors","subscribers_url":"https://api.github.com/repos/blei-lab/edward/subscribers","subscription_url":"https://api.github.com/repos/blei-lab/edward/subscription","commits_url":"https://api.github.com/repos/blei-lab/edward/commits{/sha}","git_commits_url":"https://api.github.com/repos/blei-lab/edward/git/commits{/sha}","comments_url":"https://api.github.com/repos/blei-lab/edward/comments{/number}","issue_comment_url":"https://api.github.com/repos/blei-lab/edward/issues/comments{/number}","contents_url":"https://api.github.com/repos/blei-lab/edward/contents/{+path}","compare_url":"https://api.github.com/repos/blei-lab/edward/compare/{base}...{head}","merges_url":"https://api.github.com/repos/blei-lab/edward/merges","archive_url":"https://api.github.com/repos/blei-lab/edward/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/blei-lab/edward/downloads","issues_url":"https://api.github.com/repos/blei-lab/edward/issues{/number}","pulls_url":"https://api.github.com/repos/blei-lab/edward/pulls{/number}","milestones_url":"https://api.github.com/repos/blei-lab/edward/milestones{/number}","notifications_url":"https://api.github.com/repos/blei-lab/edward/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/blei-lab/edward/labels{/name}","releases_url":"https://api.github.com/repos/blei-lab/edward/releases{/id}","deployments_url":"https://api.github.com/repos/blei-lab/edward/deployments","created_at":"2016-02-10T20:06:05Z","updated_at":"2016-06-29T12:45:13Z","pushed_at":"2016-06-29T14:58:26Z","git_url":"git://github.com/blei-lab/edward.git","ssh_url":"git@github.com:blei-lab/edward.git","clone_url":"https://github.com/blei-lab/edward.git","svn_url":"https://github.com/blei-lab/edward","homepage":"http://edwardlib.org","size":3054,"stargazers_count":797,"watchers_count":797,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":83,"mirror_url":null,"open_issues_count":66,"forks":83,"open_issues":66,"watchers":797,"default_branch":"master"}},"base":{"label":"blei-lab:master","ref":"master","sha":"b61dc36ace598f042d5c5610068f0d9a0b9a176f","user":{"login":"blei-lab","id":9000745,"avatar_url":"https://avatars.githubusercontent.com/u/9000745?v=3","gravatar_id":"","url":"https://api.github.com/users/blei-lab","html_url":"https://github.com/blei-lab","followers_url":"https://api.github.com/users/blei-lab/followers","following_url":"https://api.github.com/users/blei-lab/following{/other_user}","gists_url":"https://api.github.com/users/blei-lab/gists{/gist_id}","starred_url":"https://api.github.com/users/blei-lab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blei-lab/subscriptions","organizations_url":"https://api.github.com/users/blei-lab/orgs","repos_url":"https://api.github.com/users/blei-lab/repos","events_url":"https://api.github.com/users/blei-lab/events{/privacy}","received_events_url":"https://api.github.com/users/blei-lab/received_events","type":"Organization","site_admin":false},"repo":{"id":51468412,"name":"edward","full_name":"blei-lab/edward","owner":{"login":"blei-lab","id":9000745,"avatar_url":"https://avatars.githubusercontent.com/u/9000745?v=3","gravatar_id":"","url":"https://api.github.com/users/blei-lab","html_url":"https://github.com/blei-lab","followers_url":"https://api.github.com/users/blei-lab/followers","following_url":"https://api.github.com/users/blei-lab/following{/other_user}","gists_url":"https://api.github.com/users/blei-lab/gists{/gist_id}","starred_url":"https://api.github.com/users/blei-lab/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blei-lab/subscriptions","organizations_url":"https://api.github.com/users/blei-lab/orgs","repos_url":"https://api.github.com/users/blei-lab/repos","events_url":"https://api.github.com/users/blei-lab/events{/privacy}","received_events_url":"https://api.github.com/users/blei-lab/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/blei-lab/edward","description":"A library for probabilistic modeling, inference, and criticism. Deep generative models, variational inference. Runs on TensorFlow.","fork":false,"url":"https://api.github.com/repos/blei-lab/edward","forks_url":"https://api.github.com/repos/blei-lab/edward/forks","keys_url":"https://api.github.com/repos/blei-lab/edward/keys{/key_id}","collaborators_url":"https://api.github.com/repos/blei-lab/edward/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/blei-lab/edward/teams","hooks_url":"https://api.github.com/repos/blei-lab/edward/hooks","issue_events_url":"https://api.github.com/repos/blei-lab/edward/issues/events{/number}","events_url":"https://api.github.com/repos/blei-lab/edward/events","assignees_url":"https://api.github.com/repos/blei-lab/edward/assignees{/user}","branches_url":"https://api.github.com/repos/blei-lab/edward/branches{/branch}","tags_url":"https://api.github.com/repos/blei-lab/edward/tags","blobs_url":"https://api.github.com/repos/blei-lab/edward/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/blei-lab/edward/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/blei-lab/edward/git/refs{/sha}","trees_url":"https://api.github.com/repos/blei-lab/edward/git/trees{/sha}","statuses_url":"https://api.github.com/repos/blei-lab/edward/statuses/{sha}","languages_url":"https://api.github.com/repos/blei-lab/edward/languages","stargazers_url":"https://api.github.com/repos/blei-lab/edward/stargazers","contributors_url":"https://api.github.com/repos/blei-lab/edward/contributors","subscribers_url":"https://api.github.com/repos/blei-lab/edward/subscribers","subscription_url":"https://api.github.com/repos/blei-lab/edward/subscription","commits_url":"https://api.github.com/repos/blei-lab/edward/commits{/sha}","git_commits_url":"https://api.github.com/repos/blei-lab/edward/git/commits{/sha}","comments_url":"https://api.github.com/repos/blei-lab/edward/comments{/number}","issue_comment_url":"https://api.github.com/repos/blei-lab/edward/issues/comments{/number}","contents_url":"https://api.github.com/repos/blei-lab/edward/contents/{+path}","compare_url":"https://api.github.com/repos/blei-lab/edward/compare/{base}...{head}","merges_url":"https://api.github.com/repos/blei-lab/edward/merges","archive_url":"https://api.github.com/repos/blei-lab/edward/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/blei-lab/edward/downloads","issues_url":"https://api.github.com/repos/blei-lab/edward/issues{/number}","pulls_url":"https://api.github.com/repos/blei-lab/edward/pulls{/number}","milestones_url":"https://api.github.com/repos/blei-lab/edward/milestones{/number}","notifications_url":"https://api.github.com/repos/blei-lab/edward/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/blei-lab/edward/labels{/name}","releases_url":"https://api.github.com/repos/blei-lab/edward/releases{/id}","deployments_url":"https://api.github.com/repos/blei-lab/edward/deployments","created_at":"2016-02-10T20:06:05Z","updated_at":"2016-06-29T12:45:13Z","pushed_at":"2016-06-29T14:58:26Z","git_url":"git://github.com/blei-lab/edward.git","ssh_url":"git@github.com:blei-lab/edward.git","clone_url":"https://github.com/blei-lab/edward.git","svn_url":"https://github.com/blei-lab/edward","homepage":"http://edwardlib.org","size":3054,"stargazers_count":797,"watchers_count":797,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":true,"forks_count":83,"mirror_url":null,"open_issues_count":66,"forks":83,"open_issues":66,"watchers":797,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/138"},"html":{"href":"https://github.com/blei-lab/edward/pull/138"},"issue":{"href":"https://api.github.com/repos/blei-lab/edward/issues/138"},"comments":{"href":"https://api.github.com/repos/blei-lab/edward/issues/138/comments"},"review_comments":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/138/comments"},"review_comment":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/blei-lab/edward/pulls/138/commits"},"statuses":{"href":"https://api.github.com/repos/blei-lab/edward/statuses/9cad3ee0af3377cfd8749ff18b87bc5ba2754b43"}}}}
{"actor":{"display_login":"akucukelbir"}}