{"action":"created","issue":{"url":"https://api.github.com/repos/apache/incubator-singa/issues/216","repository_url":"https://api.github.com/repos/apache/incubator-singa","labels_url":"https://api.github.com/repos/apache/incubator-singa/issues/216/labels{/name}","comments_url":"https://api.github.com/repos/apache/incubator-singa/issues/216/comments","events_url":"https://api.github.com/repos/apache/incubator-singa/issues/216/events","html_url":"https://github.com/apache/incubator-singa/pull/216","id":166255988,"number":216,"title":"SINGA-226 Add parallel training on a single machine for singa v1.0","user":{"login":"ijingo","id":1220005,"avatar_url":"https://avatars.githubusercontent.com/u/1220005?v=3","gravatar_id":"","url":"https://api.github.com/users/ijingo","html_url":"https://github.com/ijingo","followers_url":"https://api.github.com/users/ijingo/followers","following_url":"https://api.github.com/users/ijingo/following{/other_user}","gists_url":"https://api.github.com/users/ijingo/gists{/gist_id}","starred_url":"https://api.github.com/users/ijingo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ijingo/subscriptions","organizations_url":"https://api.github.com/users/ijingo/orgs","repos_url":"https://api.github.com/users/ijingo/repos","events_url":"https://api.github.com/users/ijingo/events{/privacy}","received_events_url":"https://api.github.com/users/ijingo/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-19T06:15:58Z","updated_at":"2016-07-19T15:17:05Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/incubator-singa/pulls/216","html_url":"https://github.com/apache/incubator-singa/pull/216","diff_url":"https://github.com/apache/incubator-singa/pull/216.diff","patch_url":"https://github.com/apache/incubator-singa/pull/216.patch"},"body":"This commit implements a updater class for parallel training\r\nin local-cpu and local-gpu mode. (The mode specification described\r\nin https://issues.apache.org/jira/browse/SINGA-226)\r\n\r\nUpdater class is a wrapper of Optimizer class. It controls the\r\ncommunication pattern among workers. When initializing Updater,\r\nthe user needs to provide the total number of workers and where\r\nthe Updater does aggregation and computation.\r\n\r\nFile changed descibed as follows:\r\n* Put a new folder named cifar-parallel under example, containing\r\n  the single machine multi-gpu parallel training example.\r\n* Replace Optimizer pointer in feed_forward_net class with Updater\r\n  pointer, since Updater is a wrapper of Optimizer. So the compile()\r\n  method is changed accordingly.\r\n* Add a helper function TrainThread() method in feed_forward_net to\r\n  launch a training thread.\r\n* Adapt alexnet.cc original cifar10 example to support the new compile()\r\n* Fixed a bug in memory.cc which happens during initialzing cnmem\r\n  memory pool"},"comment":{"url":"https://api.github.com/repos/apache/incubator-singa/issues/comments/233666118","html_url":"https://github.com/apache/incubator-singa/pull/216#issuecomment-233666118","issue_url":"https://api.github.com/repos/apache/incubator-singa/issues/216","id":233666118,"user":{"login":"ijingo","id":1220005,"avatar_url":"https://avatars.githubusercontent.com/u/1220005?v=3","gravatar_id":"","url":"https://api.github.com/users/ijingo","html_url":"https://github.com/ijingo","followers_url":"https://api.github.com/users/ijingo/followers","following_url":"https://api.github.com/users/ijingo/following{/other_user}","gists_url":"https://api.github.com/users/ijingo/gists{/gist_id}","starred_url":"https://api.github.com/users/ijingo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ijingo/subscriptions","organizations_url":"https://api.github.com/users/ijingo/orgs","repos_url":"https://api.github.com/users/ijingo/repos","events_url":"https://api.github.com/users/ijingo/events{/privacy}","received_events_url":"https://api.github.com/users/ijingo/received_events","type":"User","site_admin":false},"created_at":"2016-07-19T15:17:05Z","updated_at":"2016-07-19T15:17:05Z","body":"It seems copy gradient to cpu and do sum operation on cpu cost times.\r\nFollowing is time cost in each part when train on one batch:\r\n\r\nForward time: 0.000988s  (on gpu)\r\nBackward time: 0.001189s (on gpu)\r\nUpdate time: 0.014970s (on cpu)\r\n\r\nIn update time:\r\nCopy gradient to cpu 0.004150s\r\nadd gradient 0.001950s\r\naverage gradient 0.000852s\r\ncopy parameter to cpu 0.000891s\r\noptimization algorithm on cpu 0.006571s\r\ncopy updated parameter back to gpu 0.000556s"}}
{"actor":{"display_login":"ijingo"}}