{"action":"created","issue":{"url":"https://api.github.com/repos/uber/pyro/issues/907","repository_url":"https://api.github.com/repos/uber/pyro","labels_url":"https://api.github.com/repos/uber/pyro/issues/907/labels{/name}","comments_url":"https://api.github.com/repos/uber/pyro/issues/907/comments","events_url":"https://api.github.com/repos/uber/pyro/issues/907/events","html_url":"https://github.com/uber/pyro/pull/907","id":306640406,"number":907,"title":"Demonstrate how to monitor gradients in gmm tutorial","user":{"login":"fritzo","id":648532,"avatar_url":"https://avatars0.githubusercontent.com/u/648532?v=4","gravatar_id":"","url":"https://api.github.com/users/fritzo","html_url":"https://github.com/fritzo","followers_url":"https://api.github.com/users/fritzo/followers","following_url":"https://api.github.com/users/fritzo/following{/other_user}","gists_url":"https://api.github.com/users/fritzo/gists{/gist_id}","starred_url":"https://api.github.com/users/fritzo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fritzo/subscriptions","organizations_url":"https://api.github.com/users/fritzo/orgs","repos_url":"https://api.github.com/users/fritzo/repos","events_url":"https://api.github.com/users/fritzo/events{/privacy}","received_events_url":"https://api.github.com/users/fritzo/received_events","type":"User","site_admin":false},"labels":[{"id":674111554,"url":"https://api.github.com/repos/uber/pyro/labels/Anchor%20examples","name":"Anchor examples","color":"5319e7","default":false},{"id":704871691,"url":"https://api.github.com/repos/uber/pyro/labels/awaiting%20review","name":"awaiting review","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-03-19T21:20:40Z","updated_at":"2018-03-19T21:58:59Z","closed_at":null,"author_association":"MEMBER","pull_request":{"url":"https://api.github.com/repos/uber/pyro/pulls/907","html_url":"https://github.com/uber/pyro/pull/907","diff_url":"https://github.com/uber/pyro/pull/907.diff","patch_url":"https://github.com/uber/pyro/pull/907.patch"},"body":"This adds a tiny bit of code to monitor gradient norms during convergence in the GMM tutorial. The relevant logging snippet is:\r\n```py\r\ngradient_norms = defaultdict(list)\r\ninference.loss(model, guide, data)  # Initializes param store.\r\nfor name, value in pyro.get_param_store().named_parameters():\r\n    value.register_hook(lambda g, name=name: gradient_norms[name].append(g.norm().item()))\r\n```\r\nafter which we can plot `gradient_norms`.\r\n\r\nThis is in response to @yhalk's question https://github.com/uber/pyro/issues/583#issuecomment-374185289"},"comment":{"url":"https://api.github.com/repos/uber/pyro/issues/comments/374395938","html_url":"https://github.com/uber/pyro/pull/907#issuecomment-374395938","issue_url":"https://api.github.com/repos/uber/pyro/issues/907","id":374395938,"user":{"login":"jpchen","id":1869641,"avatar_url":"https://avatars2.githubusercontent.com/u/1869641?v=4","gravatar_id":"","url":"https://api.github.com/users/jpchen","html_url":"https://github.com/jpchen","followers_url":"https://api.github.com/users/jpchen/followers","following_url":"https://api.github.com/users/jpchen/following{/other_user}","gists_url":"https://api.github.com/users/jpchen/gists{/gist_id}","starred_url":"https://api.github.com/users/jpchen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jpchen/subscriptions","organizations_url":"https://api.github.com/users/jpchen/orgs","repos_url":"https://api.github.com/users/jpchen/repos","events_url":"https://api.github.com/users/jpchen/events{/privacy}","received_events_url":"https://api.github.com/users/jpchen/received_events","type":"User","site_admin":false},"created_at":"2018-03-19T21:58:59Z","updated_at":"2018-03-19T21:58:59Z","author_association":"MEMBER","body":"> ```for name, value in pyro.get_param_store().named_parameters():\r\n    value.register_hook(lambda g, name=name: gradient_norms[name].append(g.norm().item()))```\r\n\r\nive been using variations of this snippet a lot in my code to inspect gradients. would this be useful to expose in pyro? "}}
{"actor":{"display_login":"jpchen"}}