{"action":"created","issue":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/7601","repository_url":"https://api.github.com/repos/pytorch/pytorch","labels_url":"https://api.github.com/repos/pytorch/pytorch/issues/7601/labels{/name}","comments_url":"https://api.github.com/repos/pytorch/pytorch/issues/7601/comments","events_url":"https://api.github.com/repos/pytorch/pytorch/issues/7601/events","html_url":"https://github.com/pytorch/pytorch/issues/7601","id":323420582,"node_id":"MDU6SXNzdWUzMjM0MjA1ODI=","number":7601,"title":"Backward kernels are always put on the default stream","user":{"login":"ngimel","id":15841449,"node_id":"MDQ6VXNlcjE1ODQxNDQ5","avatar_url":"https://avatars3.githubusercontent.com/u/15841449?v=4","gravatar_id":"","url":"https://api.github.com/users/ngimel","html_url":"https://github.com/ngimel","followers_url":"https://api.github.com/users/ngimel/followers","following_url":"https://api.github.com/users/ngimel/following{/other_user}","gists_url":"https://api.github.com/users/ngimel/gists{/gist_id}","starred_url":"https://api.github.com/users/ngimel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ngimel/subscriptions","organizations_url":"https://api.github.com/users/ngimel/orgs","repos_url":"https://api.github.com/users/ngimel/repos","events_url":"https://api.github.com/users/ngimel/events{/privacy}","received_events_url":"https://api.github.com/users/ngimel/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-05-15T23:39:46Z","updated_at":"2018-12-18T17:23:56Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"## Issue description\r\nStreams in THC are thread local, so in backward, since autograd creates its own threads, kernels are always put on the default streams, and don't respect stream that was set previously. \r\n## Code example\r\n```\r\nimport torch\r\n\r\na=torch.Tensor(128,512).cuda().uniform_()\r\n\r\nmodel = torch.nn.Linear(512,512).cuda()\r\n\r\nfwd_stream = torch.cuda.Stream()\r\nbwd_stream = torch.cuda.Stream()\r\n\r\n\r\nfor i in range(3):\r\n   with torch.cuda.stream(fwd_stream):\r\n       out = model(a).sum()\r\n   with torch.cuda.stream(bwd_stream):\r\n       out.backward() #kernels are actually on the default stream\r\n   a.detach()\r\n```\r\n\r\nPytorch, current master\r\n"},"comment":{"url":"https://api.github.com/repos/pytorch/pytorch/issues/comments/448300859","html_url":"https://github.com/pytorch/pytorch/issues/7601#issuecomment-448300859","issue_url":"https://api.github.com/repos/pytorch/pytorch/issues/7601","id":448300859,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0ODMwMDg1OQ==","user":{"login":"ezyang","id":13564,"node_id":"MDQ6VXNlcjEzNTY0","avatar_url":"https://avatars0.githubusercontent.com/u/13564?v=4","gravatar_id":"","url":"https://api.github.com/users/ezyang","html_url":"https://github.com/ezyang","followers_url":"https://api.github.com/users/ezyang/followers","following_url":"https://api.github.com/users/ezyang/following{/other_user}","gists_url":"https://api.github.com/users/ezyang/gists{/gist_id}","starred_url":"https://api.github.com/users/ezyang/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ezyang/subscriptions","organizations_url":"https://api.github.com/users/ezyang/orgs","repos_url":"https://api.github.com/users/ezyang/repos","events_url":"https://api.github.com/users/ezyang/events{/privacy}","received_events_url":"https://api.github.com/users/ezyang/received_events","type":"User","site_admin":false},"created_at":"2018-12-18T17:23:56Z","updated_at":"2018-12-18T17:23:56Z","author_association":"CONTRIBUTOR","body":"cc @mruberry "}}
{"actor":{"display_login":"ezyang"}}