{"action":"created","issue":{"url":"https://api.github.com/repos/Kaixhin/Atari/issues/10","repository_url":"https://api.github.com/repos/Kaixhin/Atari","labels_url":"https://api.github.com/repos/Kaixhin/Atari/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/Kaixhin/Atari/issues/10/comments","events_url":"https://api.github.com/repos/Kaixhin/Atari/issues/10/events","html_url":"https://github.com/Kaixhin/Atari/issues/10","id":150268043,"number":10,"title":"Multi gpu support","user":{"login":"lake4790k","id":15705158,"avatar_url":"https://avatars.githubusercontent.com/u/15705158?v=3","gravatar_id":"","url":"https://api.github.com/users/lake4790k","html_url":"https://github.com/lake4790k","followers_url":"https://api.github.com/users/lake4790k/followers","following_url":"https://api.github.com/users/lake4790k/following{/other_user}","gists_url":"https://api.github.com/users/lake4790k/gists{/gist_id}","starred_url":"https://api.github.com/users/lake4790k/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lake4790k/subscriptions","organizations_url":"https://api.github.com/users/lake4790k/orgs","repos_url":"https://api.github.com/users/lake4790k/repos","events_url":"https://api.github.com/users/lake4790k/events{/privacy}","received_events_url":"https://api.github.com/users/lake4790k/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-22T05:05:16Z","updated_at":"2016-04-22T08:09:49Z","closed_at":null,"body":"I got some speedup by using dual gpus dedicating one for the policy, other for the target network, so policy and target network forwards can be done in parallel (in the vanilla dqn), but not the policy backward, so target gpu is not utilized that much, speed up not 2x, rather 30%. At least there's no need to synchronize the networks between gpus as target net is seldom updated. The advanced methods do more forwards, so may benefit more.\r\n\r\nI'll measure the speed gain with the dqn convnet and will add this simple multigpu method if makes sense. Of course one could do a complete scalable parallel solution like the Gorila, but only makes sense if someone can afford a multigpu cluster for running Atari Dqn."},"comment":{"url":"https://api.github.com/repos/Kaixhin/Atari/issues/comments/213315738","html_url":"https://github.com/Kaixhin/Atari/issues/10#issuecomment-213315738","issue_url":"https://api.github.com/repos/Kaixhin/Atari/issues/10","id":213315738,"user":{"login":"Kaixhin","id":991891,"avatar_url":"https://avatars.githubusercontent.com/u/991891?v=3","gravatar_id":"","url":"https://api.github.com/users/Kaixhin","html_url":"https://github.com/Kaixhin","followers_url":"https://api.github.com/users/Kaixhin/followers","following_url":"https://api.github.com/users/Kaixhin/following{/other_user}","gists_url":"https://api.github.com/users/Kaixhin/gists{/gist_id}","starred_url":"https://api.github.com/users/Kaixhin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kaixhin/subscriptions","organizations_url":"https://api.github.com/users/Kaixhin/orgs","repos_url":"https://api.github.com/users/Kaixhin/repos","events_url":"https://api.github.com/users/Kaixhin/events{/privacy}","received_events_url":"https://api.github.com/users/Kaixhin/received_events","type":"User","site_admin":false},"created_at":"2016-04-22T08:09:49Z","updated_at":"2016-04-22T08:09:49Z","body":"Have you tried [DataParallelTable](https://github.com/torch/cunn/blob/master/doc/cunnmodules.md#dataparalleltable) at all (with NCCL)? With the default minibatch size of 32 I doubt it'll work that well, but worth double checking? If not then an optional 2 GPU switch for the target network might be decent.\r\n\r\nAs for a cluster, there's always [DistLearn](https://github.com/twitter/torch-distlearn). Though I would rather focus on integrating single-machine async Q-learning first."}}