{"action":"created","comment":{"url":"https://api.github.com/repos/uber/pyro/pulls/comments/177594354","pull_request_review_id":107493765,"id":177594354,"diff_hunk":"@@ -0,0 +1,292 @@\n+{\n+ \"cells\": [\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"# Tensor shapes in Pyro 0.2\\n\",\n+    \"\\n\",\n+    \"This tutorial introduces Pyro's organization of tensor dimensions.\\n\",\n+    \"\\n\",\n+    \"Summary:\\n\",\n+    \"- Tensors broadcast on the left: `torch.ones(3,4,5) + torch.ones(5)`.\\n\",\n+    \"- Distribution `.sample()` has shape `batch_shape + event_shape`.\\n\",\n+    \"- Distribution `.log_prob(x)` has shape `batch_shape` (but not `event_shape`!).\\n\",\n+    \"- Use `my_dist.reshape([2,3,4])` to draw a batch of samples.\\n\",\n+    \"- Use `my_dist.reshape(extra_event_dims=1)` to declare a dimension as dependent.\\n\",\n+    \"- Use `with iarange('name', size):` to declare a dimension as independent.\\n\",\n+    \"- All dimensions must be declared either dependent or independent.\\n\",\n+    \"- Try to support batching on the left. This lets Pyro auto-parallelize.\\n\",\n+    \"  - use negative indices like `x.sum(-1)` rather than `x.sum(2)`\\n\",\n+    \"  - use ellipsis notation like `pixel = image[..., i, j]`\\n\",\n+    \"  \"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 1,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"import torch\\n\",\n+    \"import pyro\\n\",\n+    \"from pyro.distributions import Bernoulli, MultivariateNormal, Normal\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"## Distributions shapes: `batch_shape` and `event_shape`\\n\",\n+    \"\\n\",\n+    \"The simplest distribution shape is a single univariate distribution.\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 2,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"d = Bernoulli(0.5)\\n\",\n+    \"assert d.batch_shape == ()\\n\",\n+    \"assert d.event_shape == ()\\n\",\n+    \"x = d.sample()\\n\",\n+    \"assert x.shape == ()\\n\",\n+    \"assert d.log_prob(x).shape == ()\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"Distributions can be batched by passing in batched parameters.\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 3,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"d = Bernoulli(0.5 * torch.ones(3,4))\\n\",\n+    \"assert d.batch_shape == (3, 4)\\n\",\n+    \"assert d.event_shape == ()\\n\",\n+    \"x = d.sample()\\n\",\n+    \"assert x.shape == (3, 4)\\n\",\n+    \"assert d.log_prob(x).shape == (3, 4)\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"Another way to batch distributions is via the `.reshape()` method. This only works if \\n\",\n+    \"parameters are identical along the leftmost dimensions.\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 4,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"d = Bernoulli(torch.tensor([0.1, 0.2, 0.3, 0.4])).reshape([3])\\n\",\n+    \"assert d.batch_shape == (3, 4)\\n\",\n+    \"assert d.event_shape == ()\\n\",\n+    \"x = d.sample()\\n\",\n+    \"assert x.shape == (3, 4)\\n\",\n+    \"assert d.log_prob(x).shape == (3, 4)\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"Multivariate distributions have nonempty `.event_shape`. For these distributions, the shapes of `.sample()` and `.log_prob(x)` differ:\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 5,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"d = MultivariateNormal(torch.zeros(3), torch.eye(3, 3))\\n\",\n+    \"assert d.batch_shape == ()\\n\",\n+    \"assert d.event_shape == (3,)\\n\",\n+    \"x = d.sample()\\n\",\n+    \"assert x.shape == (3,)            # == batch_shape + event_shape\\n\",\n+    \"assert d.log_prob(x).shape == ()  # == batch_shape\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"In Pyro you can treat a univariate distribution as multivariate by calling the `.reshape(extra_event_dims=_)` property.\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 6,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"d = Bernoulli(0.5 * torch.ones(3,4)).reshape(extra_event_dims=1)\\n\",\n+    \"assert d.batch_shape == (3,)\\n\",\n+    \"assert d.event_shape == (4,)\\n\",\n+    \"x = d.sample()\\n\",\n+    \"assert x.shape == (3, 4)\\n\",\n+    \"assert d.log_prob(x).shape == (3,)\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"While you work with Pyro programs, keep in mind that samples have shape `batch_shape + event_shape`, whereas `.log_prob(x)` values have shape `batch_shape`. You'll need to ensure that `batch_shape` is carefully controlled by either trimming it down with `.reshape(extra_event_dims=_)` or by declaring dimensions as independent via `pyro.iarange`.\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"markdown\",\n+   \"metadata\": {},\n+   \"source\": [\n+    \"## Declare independent dims with `iarange`\"\n+   ]\n+  },\n+  {\n+   \"cell_type\": \"code\",\n+   \"execution_count\": 7,\n+   \"metadata\": {},\n+   \"outputs\": [],\n+   \"source\": [\n+    \"def model():\\n\",\n+    \"    a = sample(\\\"a\\\", Normal(0, 1))\\n\",\n+    \"    b = sample(\\\"b\\\", Normal(zeros(2), 1).reshape(extra_event_dims=1))\\n\",\n+    \"    with iarange(\\\"c\\\", 2):\\n\",\n+    \"        c = sample(\\\"c\\\", Normal(zeros(2), 1))\\n\",","path":"tutorial/source/tensor_shapes.ipynb","position":168,"original_position":168,"commit_id":"ab9ec2649be54077d09843efd6d37ab19c6ef1b1","original_commit_id":"ab9ec2649be54077d09843efd6d37ab19c6ef1b1","user":{"login":"fritzo","id":648532,"avatar_url":"https://avatars0.githubusercontent.com/u/648532?v=4","gravatar_id":"","url":"https://api.github.com/users/fritzo","html_url":"https://github.com/fritzo","followers_url":"https://api.github.com/users/fritzo/followers","following_url":"https://api.github.com/users/fritzo/following{/other_user}","gists_url":"https://api.github.com/users/fritzo/gists{/gist_id}","starred_url":"https://api.github.com/users/fritzo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fritzo/subscriptions","organizations_url":"https://api.github.com/users/fritzo/orgs","repos_url":"https://api.github.com/users/fritzo/repos","events_url":"https://api.github.com/users/fritzo/events{/privacy}","received_events_url":"https://api.github.com/users/fritzo/received_events","type":"User","site_admin":false},"body":"Yes, I'm only abbreviating because of the severe width-constraints in the ascii-art tables","created_at":"2018-03-27T22:43:41Z","updated_at":"2018-03-27T22:43:41Z","html_url":"https://github.com/uber/pyro/pull/936#discussion_r177594354","pull_request_url":"https://api.github.com/repos/uber/pyro/pulls/936","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/uber/pyro/pulls/comments/177594354"},"html":{"href":"https://github.com/uber/pyro/pull/936#discussion_r177594354"},"pull_request":{"href":"https://api.github.com/repos/uber/pyro/pulls/936"}},"in_reply_to_id":177589365},"pull_request":{"url":"https://api.github.com/repos/uber/pyro/pulls/936","id":177629359,"html_url":"https://github.com/uber/pyro/pull/936","diff_url":"https://github.com/uber/pyro/pull/936.diff","patch_url":"https://github.com/uber/pyro/pull/936.patch","issue_url":"https://api.github.com/repos/uber/pyro/issues/936","number":936,"state":"open","locked":false,"title":"Tutorial on tensor shapes","user":{"login":"fritzo","id":648532,"avatar_url":"https://avatars0.githubusercontent.com/u/648532?v=4","gravatar_id":"","url":"https://api.github.com/users/fritzo","html_url":"https://github.com/fritzo","followers_url":"https://api.github.com/users/fritzo/followers","following_url":"https://api.github.com/users/fritzo/following{/other_user}","gists_url":"https://api.github.com/users/fritzo/gists{/gist_id}","starred_url":"https://api.github.com/users/fritzo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fritzo/subscriptions","organizations_url":"https://api.github.com/users/fritzo/orgs","repos_url":"https://api.github.com/users/fritzo/repos","events_url":"https://api.github.com/users/fritzo/events{/privacy}","received_events_url":"https://api.github.com/users/fritzo/received_events","type":"User","site_admin":false},"body":"Resolves #814\r\n \r\nDo you find tensor shapes in Pyro 0.2 confusing?\r\nHave you ever wondered what you could do with *two* nested `iarange`s?\r\nHave you wanted to enumerate discrete latent variables, but been afraid to ask under what conditions Rao-Blackwellization is valid?\r\n\r\nThis PR is your venue to ask!\r\nDon't wait. Comment now.\r\n\r\n(to see a draft, click the `[View]` button just below `[Unified|Split]` on the [Files changed page](https://github.com/uber/pyro/pull/936/files))","created_at":"2018-03-27T04:34:11Z","updated_at":"2018-03-27T22:43:41Z","closed_at":null,"merged_at":null,"merge_commit_sha":"84723fb10d09e80475c82f3c1a3cf87e205b7f94","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":704807577,"url":"https://api.github.com/repos/uber/pyro/labels/WIP","name":"WIP","color":"fbca04","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/uber/pyro/pulls/936/commits","review_comments_url":"https://api.github.com/repos/uber/pyro/pulls/936/comments","review_comment_url":"https://api.github.com/repos/uber/pyro/pulls/comments{/number}","comments_url":"https://api.github.com/repos/uber/pyro/issues/936/comments","statuses_url":"https://api.github.com/repos/uber/pyro/statuses/ab9ec2649be54077d09843efd6d37ab19c6ef1b1","head":{"label":"uber:tutorial-shapes","ref":"tutorial-shapes","sha":"ab9ec2649be54077d09843efd6d37ab19c6ef1b1","user":{"login":"uber","id":538264,"avatar_url":"https://avatars3.githubusercontent.com/u/538264?v=4","gravatar_id":"","url":"https://api.github.com/users/uber","html_url":"https://github.com/uber","followers_url":"https://api.github.com/users/uber/followers","following_url":"https://api.github.com/users/uber/following{/other_user}","gists_url":"https://api.github.com/users/uber/gists{/gist_id}","starred_url":"https://api.github.com/users/uber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uber/subscriptions","organizations_url":"https://api.github.com/users/uber/orgs","repos_url":"https://api.github.com/users/uber/repos","events_url":"https://api.github.com/users/uber/events{/privacy}","received_events_url":"https://api.github.com/users/uber/received_events","type":"Organization","site_admin":false},"repo":{"id":94506832,"name":"pyro","full_name":"uber/pyro","owner":{"login":"uber","id":538264,"avatar_url":"https://avatars3.githubusercontent.com/u/538264?v=4","gravatar_id":"","url":"https://api.github.com/users/uber","html_url":"https://github.com/uber","followers_url":"https://api.github.com/users/uber/followers","following_url":"https://api.github.com/users/uber/following{/other_user}","gists_url":"https://api.github.com/users/uber/gists{/gist_id}","starred_url":"https://api.github.com/users/uber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uber/subscriptions","organizations_url":"https://api.github.com/users/uber/orgs","repos_url":"https://api.github.com/users/uber/repos","events_url":"https://api.github.com/users/uber/events{/privacy}","received_events_url":"https://api.github.com/users/uber/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/uber/pyro","description":"Deep universal probabilistic programming with Python and PyTorch","fork":false,"url":"https://api.github.com/repos/uber/pyro","forks_url":"https://api.github.com/repos/uber/pyro/forks","keys_url":"https://api.github.com/repos/uber/pyro/keys{/key_id}","collaborators_url":"https://api.github.com/repos/uber/pyro/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/uber/pyro/teams","hooks_url":"https://api.github.com/repos/uber/pyro/hooks","issue_events_url":"https://api.github.com/repos/uber/pyro/issues/events{/number}","events_url":"https://api.github.com/repos/uber/pyro/events","assignees_url":"https://api.github.com/repos/uber/pyro/assignees{/user}","branches_url":"https://api.github.com/repos/uber/pyro/branches{/branch}","tags_url":"https://api.github.com/repos/uber/pyro/tags","blobs_url":"https://api.github.com/repos/uber/pyro/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/uber/pyro/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/uber/pyro/git/refs{/sha}","trees_url":"https://api.github.com/repos/uber/pyro/git/trees{/sha}","statuses_url":"https://api.github.com/repos/uber/pyro/statuses/{sha}","languages_url":"https://api.github.com/repos/uber/pyro/languages","stargazers_url":"https://api.github.com/repos/uber/pyro/stargazers","contributors_url":"https://api.github.com/repos/uber/pyro/contributors","subscribers_url":"https://api.github.com/repos/uber/pyro/subscribers","subscription_url":"https://api.github.com/repos/uber/pyro/subscription","commits_url":"https://api.github.com/repos/uber/pyro/commits{/sha}","git_commits_url":"https://api.github.com/repos/uber/pyro/git/commits{/sha}","comments_url":"https://api.github.com/repos/uber/pyro/comments{/number}","issue_comment_url":"https://api.github.com/repos/uber/pyro/issues/comments{/number}","contents_url":"https://api.github.com/repos/uber/pyro/contents/{+path}","compare_url":"https://api.github.com/repos/uber/pyro/compare/{base}...{head}","merges_url":"https://api.github.com/repos/uber/pyro/merges","archive_url":"https://api.github.com/repos/uber/pyro/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/uber/pyro/downloads","issues_url":"https://api.github.com/repos/uber/pyro/issues{/number}","pulls_url":"https://api.github.com/repos/uber/pyro/pulls{/number}","milestones_url":"https://api.github.com/repos/uber/pyro/milestones{/number}","notifications_url":"https://api.github.com/repos/uber/pyro/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/uber/pyro/labels{/name}","releases_url":"https://api.github.com/repos/uber/pyro/releases{/id}","deployments_url":"https://api.github.com/repos/uber/pyro/deployments","created_at":"2017-06-16T05:03:47Z","updated_at":"2018-03-27T22:41:00Z","pushed_at":"2018-03-27T22:41:02Z","git_url":"git://github.com/uber/pyro.git","ssh_url":"git@github.com:uber/pyro.git","clone_url":"https://github.com/uber/pyro.git","svn_url":"https://github.com/uber/pyro","homepage":"http://pyro.ai","size":52195,"stargazers_count":3124,"watchers_count":3124,"language":"Jupyter Notebook","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":314,"mirror_url":null,"archived":false,"open_issues_count":73,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":314,"open_issues":73,"watchers":3124,"default_branch":"dev"}},"base":{"label":"uber:dev","ref":"dev","sha":"9ad52ee41c2f17e0287df0114abef918f61cabf9","user":{"login":"uber","id":538264,"avatar_url":"https://avatars3.githubusercontent.com/u/538264?v=4","gravatar_id":"","url":"https://api.github.com/users/uber","html_url":"https://github.com/uber","followers_url":"https://api.github.com/users/uber/followers","following_url":"https://api.github.com/users/uber/following{/other_user}","gists_url":"https://api.github.com/users/uber/gists{/gist_id}","starred_url":"https://api.github.com/users/uber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uber/subscriptions","organizations_url":"https://api.github.com/users/uber/orgs","repos_url":"https://api.github.com/users/uber/repos","events_url":"https://api.github.com/users/uber/events{/privacy}","received_events_url":"https://api.github.com/users/uber/received_events","type":"Organization","site_admin":false},"repo":{"id":94506832,"name":"pyro","full_name":"uber/pyro","owner":{"login":"uber","id":538264,"avatar_url":"https://avatars3.githubusercontent.com/u/538264?v=4","gravatar_id":"","url":"https://api.github.com/users/uber","html_url":"https://github.com/uber","followers_url":"https://api.github.com/users/uber/followers","following_url":"https://api.github.com/users/uber/following{/other_user}","gists_url":"https://api.github.com/users/uber/gists{/gist_id}","starred_url":"https://api.github.com/users/uber/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/uber/subscriptions","organizations_url":"https://api.github.com/users/uber/orgs","repos_url":"https://api.github.com/users/uber/repos","events_url":"https://api.github.com/users/uber/events{/privacy}","received_events_url":"https://api.github.com/users/uber/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/uber/pyro","description":"Deep universal probabilistic programming with Python and PyTorch","fork":false,"url":"https://api.github.com/repos/uber/pyro","forks_url":"https://api.github.com/repos/uber/pyro/forks","keys_url":"https://api.github.com/repos/uber/pyro/keys{/key_id}","collaborators_url":"https://api.github.com/repos/uber/pyro/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/uber/pyro/teams","hooks_url":"https://api.github.com/repos/uber/pyro/hooks","issue_events_url":"https://api.github.com/repos/uber/pyro/issues/events{/number}","events_url":"https://api.github.com/repos/uber/pyro/events","assignees_url":"https://api.github.com/repos/uber/pyro/assignees{/user}","branches_url":"https://api.github.com/repos/uber/pyro/branches{/branch}","tags_url":"https://api.github.com/repos/uber/pyro/tags","blobs_url":"https://api.github.com/repos/uber/pyro/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/uber/pyro/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/uber/pyro/git/refs{/sha}","trees_url":"https://api.github.com/repos/uber/pyro/git/trees{/sha}","statuses_url":"https://api.github.com/repos/uber/pyro/statuses/{sha}","languages_url":"https://api.github.com/repos/uber/pyro/languages","stargazers_url":"https://api.github.com/repos/uber/pyro/stargazers","contributors_url":"https://api.github.com/repos/uber/pyro/contributors","subscribers_url":"https://api.github.com/repos/uber/pyro/subscribers","subscription_url":"https://api.github.com/repos/uber/pyro/subscription","commits_url":"https://api.github.com/repos/uber/pyro/commits{/sha}","git_commits_url":"https://api.github.com/repos/uber/pyro/git/commits{/sha}","comments_url":"https://api.github.com/repos/uber/pyro/comments{/number}","issue_comment_url":"https://api.github.com/repos/uber/pyro/issues/comments{/number}","contents_url":"https://api.github.com/repos/uber/pyro/contents/{+path}","compare_url":"https://api.github.com/repos/uber/pyro/compare/{base}...{head}","merges_url":"https://api.github.com/repos/uber/pyro/merges","archive_url":"https://api.github.com/repos/uber/pyro/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/uber/pyro/downloads","issues_url":"https://api.github.com/repos/uber/pyro/issues{/number}","pulls_url":"https://api.github.com/repos/uber/pyro/pulls{/number}","milestones_url":"https://api.github.com/repos/uber/pyro/milestones{/number}","notifications_url":"https://api.github.com/repos/uber/pyro/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/uber/pyro/labels{/name}","releases_url":"https://api.github.com/repos/uber/pyro/releases{/id}","deployments_url":"https://api.github.com/repos/uber/pyro/deployments","created_at":"2017-06-16T05:03:47Z","updated_at":"2018-03-27T22:41:00Z","pushed_at":"2018-03-27T22:41:02Z","git_url":"git://github.com/uber/pyro.git","ssh_url":"git@github.com:uber/pyro.git","clone_url":"https://github.com/uber/pyro.git","svn_url":"https://github.com/uber/pyro","homepage":"http://pyro.ai","size":52195,"stargazers_count":3124,"watchers_count":3124,"language":"Jupyter Notebook","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":314,"mirror_url":null,"archived":false,"open_issues_count":73,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":314,"open_issues":73,"watchers":3124,"default_branch":"dev"}},"_links":{"self":{"href":"https://api.github.com/repos/uber/pyro/pulls/936"},"html":{"href":"https://github.com/uber/pyro/pull/936"},"issue":{"href":"https://api.github.com/repos/uber/pyro/issues/936"},"comments":{"href":"https://api.github.com/repos/uber/pyro/issues/936/comments"},"review_comments":{"href":"https://api.github.com/repos/uber/pyro/pulls/936/comments"},"review_comment":{"href":"https://api.github.com/repos/uber/pyro/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/uber/pyro/pulls/936/commits"},"statuses":{"href":"https://api.github.com/repos/uber/pyro/statuses/ab9ec2649be54077d09843efd6d37ab19c6ef1b1"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"fritzo"}}