{"action":"created","comment":{"url":"https://api.github.com/repos/chainer/chainer/pulls/comments/218302308","pull_request_review_id":156208029,"id":218302308,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxODMwMjMwOA==","diff_hunk":"@@ -24,6 +25,191 @@\n import resnet50\n import resnext50\n \n+try:","path":"examples/imagenet/train_imagenet.py","position":12,"original_position":12,"commit_id":"1548087c67ac7cb25574b42a7c6c9705a9d697d3","original_commit_id":"1548087c67ac7cb25574b42a7c6c9705a9d697d3","user":{"login":"okuta","id":7133388,"node_id":"MDQ6VXNlcjcxMzMzODg=","avatar_url":"https://avatars1.githubusercontent.com/u/7133388?v=4","gravatar_id":"","url":"https://api.github.com/users/okuta","html_url":"https://github.com/okuta","followers_url":"https://api.github.com/users/okuta/followers","following_url":"https://api.github.com/users/okuta/following{/other_user}","gists_url":"https://api.github.com/users/okuta/gists{/gist_id}","starred_url":"https://api.github.com/users/okuta/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/okuta/subscriptions","organizations_url":"https://api.github.com/users/okuta/orgs","repos_url":"https://api.github.com/users/okuta/repos","events_url":"https://api.github.com/users/okuta/events{/privacy}","received_events_url":"https://api.github.com/users/okuta/received_events","type":"User","site_admin":false},"body":"I feel that adding to much.\r\n- Create new train script as `train_imagenet_dali.py` .\r\n- Move these code to separate utility file `dali_util.py` .\r\n\r\n","created_at":"2018-09-18T05:28:32Z","updated_at":"2018-09-18T05:28:38Z","html_url":"https://github.com/chainer/chainer/pull/5067#discussion_r218302308","pull_request_url":"https://api.github.com/repos/chainer/chainer/pulls/5067","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/chainer/chainer/pulls/comments/218302308"},"html":{"href":"https://github.com/chainer/chainer/pull/5067#discussion_r218302308"},"pull_request":{"href":"https://api.github.com/repos/chainer/chainer/pulls/5067"}}},"pull_request":{"url":"https://api.github.com/repos/chainer/chainer/pulls/5067","id":199629352,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk5NjI5MzUy","html_url":"https://github.com/chainer/chainer/pull/5067","diff_url":"https://github.com/chainer/chainer/pull/5067.diff","patch_url":"https://github.com/chainer/chainer/pull/5067.patch","issue_url":"https://api.github.com/repos/chainer/chainer/issues/5067","number":5067,"state":"open","locked":false,"title":"Support DALI","user":{"login":"anaruse","id":20274068,"node_id":"MDQ6VXNlcjIwMjc0MDY4","avatar_url":"https://avatars1.githubusercontent.com/u/20274068?v=4","gravatar_id":"","url":"https://api.github.com/users/anaruse","html_url":"https://github.com/anaruse","followers_url":"https://api.github.com/users/anaruse/followers","following_url":"https://api.github.com/users/anaruse/following{/other_user}","gists_url":"https://api.github.com/users/anaruse/gists{/gist_id}","starred_url":"https://api.github.com/users/anaruse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anaruse/subscriptions","organizations_url":"https://api.github.com/users/anaruse/orgs","repos_url":"https://api.github.com/users/anaruse/repos","events_url":"https://api.github.com/users/anaruse/events{/privacy}","received_events_url":"https://api.github.com/users/anaruse/received_events","type":"User","site_admin":false},"body":"This PR aims to support NVIDIA DALI to accelerate image data loading to GPU.\r\n\r\nIn this PR, a iterator for DALI named `DaliIterator` has been added. Current iterators like `SerialIterator` and `MultiprocessIterator` are created with an instance of `DatasetMixin`, and training data, in other words, examples are loaded through the dataset instance with explicit index. In contract, `DaliIterator` is created with an instance of DALI `Pipeline` and training examples are pre-processed (decode, normalization, augmentation, etc) in the pipeline according to its configurations and the data is passed to the iterator in the form of \"tuple of DALI array\". That means there are big difference between outputs of current iterators and `DaliIterator`. The difference is solved by a converter named `dali_converter` in which an output of DALI `Pipeline` is converted to \"tuple of numpy/cupy array\". I've also edited imagenet example so that you can test DALI with actual training dataset.\r\n\r\nThere are still several things to fix/add (tests for example), but first I would like to hear your thought on whether this basic design is OK or not.","created_at":"2018-07-06T02:21:04Z","updated_at":"2018-09-18T05:28:38Z","closed_at":null,"merged_at":null,"merge_commit_sha":"fcc9990e2326ee0fdfb57f058912bf320d5a9c49","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[{"id":222603817,"node_id":"MDU6TGFiZWwyMjI2MDM4MTc=","url":"https://api.github.com/repos/chainer/chainer/labels/feature","name":"feature","color":"fbca04","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/chainer/chainer/pulls/5067/commits","review_comments_url":"https://api.github.com/repos/chainer/chainer/pulls/5067/comments","review_comment_url":"https://api.github.com/repos/chainer/chainer/pulls/comments{/number}","comments_url":"https://api.github.com/repos/chainer/chainer/issues/5067/comments","statuses_url":"https://api.github.com/repos/chainer/chainer/statuses/1548087c67ac7cb25574b42a7c6c9705a9d697d3","head":{"label":"anaruse:support_dali","ref":"support_dali","sha":"1548087c67ac7cb25574b42a7c6c9705a9d697d3","user":{"login":"anaruse","id":20274068,"node_id":"MDQ6VXNlcjIwMjc0MDY4","avatar_url":"https://avatars1.githubusercontent.com/u/20274068?v=4","gravatar_id":"","url":"https://api.github.com/users/anaruse","html_url":"https://github.com/anaruse","followers_url":"https://api.github.com/users/anaruse/followers","following_url":"https://api.github.com/users/anaruse/following{/other_user}","gists_url":"https://api.github.com/users/anaruse/gists{/gist_id}","starred_url":"https://api.github.com/users/anaruse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anaruse/subscriptions","organizations_url":"https://api.github.com/users/anaruse/orgs","repos_url":"https://api.github.com/users/anaruse/repos","events_url":"https://api.github.com/users/anaruse/events{/privacy}","received_events_url":"https://api.github.com/users/anaruse/received_events","type":"User","site_admin":false},"repo":{"id":63754829,"node_id":"MDEwOlJlcG9zaXRvcnk2Mzc1NDgyOQ==","name":"chainer","full_name":"anaruse/chainer","private":false,"owner":{"login":"anaruse","id":20274068,"node_id":"MDQ6VXNlcjIwMjc0MDY4","avatar_url":"https://avatars1.githubusercontent.com/u/20274068?v=4","gravatar_id":"","url":"https://api.github.com/users/anaruse","html_url":"https://github.com/anaruse","followers_url":"https://api.github.com/users/anaruse/followers","following_url":"https://api.github.com/users/anaruse/following{/other_user}","gists_url":"https://api.github.com/users/anaruse/gists{/gist_id}","starred_url":"https://api.github.com/users/anaruse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/anaruse/subscriptions","organizations_url":"https://api.github.com/users/anaruse/orgs","repos_url":"https://api.github.com/users/anaruse/repos","events_url":"https://api.github.com/users/anaruse/events{/privacy}","received_events_url":"https://api.github.com/users/anaruse/received_events","type":"User","site_admin":false},"html_url":"https://github.com/anaruse/chainer","description":"A flexible framework of neural networks for deep learning","fork":true,"url":"https://api.github.com/repos/anaruse/chainer","forks_url":"https://api.github.com/repos/anaruse/chainer/forks","keys_url":"https://api.github.com/repos/anaruse/chainer/keys{/key_id}","collaborators_url":"https://api.github.com/repos/anaruse/chainer/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/anaruse/chainer/teams","hooks_url":"https://api.github.com/repos/anaruse/chainer/hooks","issue_events_url":"https://api.github.com/repos/anaruse/chainer/issues/events{/number}","events_url":"https://api.github.com/repos/anaruse/chainer/events","assignees_url":"https://api.github.com/repos/anaruse/chainer/assignees{/user}","branches_url":"https://api.github.com/repos/anaruse/chainer/branches{/branch}","tags_url":"https://api.github.com/repos/anaruse/chainer/tags","blobs_url":"https://api.github.com/repos/anaruse/chainer/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/anaruse/chainer/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/anaruse/chainer/git/refs{/sha}","trees_url":"https://api.github.com/repos/anaruse/chainer/git/trees{/sha}","statuses_url":"https://api.github.com/repos/anaruse/chainer/statuses/{sha}","languages_url":"https://api.github.com/repos/anaruse/chainer/languages","stargazers_url":"https://api.github.com/repos/anaruse/chainer/stargazers","contributors_url":"https://api.github.com/repos/anaruse/chainer/contributors","subscribers_url":"https://api.github.com/repos/anaruse/chainer/subscribers","subscription_url":"https://api.github.com/repos/anaruse/chainer/subscription","commits_url":"https://api.github.com/repos/anaruse/chainer/commits{/sha}","git_commits_url":"https://api.github.com/repos/anaruse/chainer/git/commits{/sha}","comments_url":"https://api.github.com/repos/anaruse/chainer/comments{/number}","issue_comment_url":"https://api.github.com/repos/anaruse/chainer/issues/comments{/number}","contents_url":"https://api.github.com/repos/anaruse/chainer/contents/{+path}","compare_url":"https://api.github.com/repos/anaruse/chainer/compare/{base}...{head}","merges_url":"https://api.github.com/repos/anaruse/chainer/merges","archive_url":"https://api.github.com/repos/anaruse/chainer/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/anaruse/chainer/downloads","issues_url":"https://api.github.com/repos/anaruse/chainer/issues{/number}","pulls_url":"https://api.github.com/repos/anaruse/chainer/pulls{/number}","milestones_url":"https://api.github.com/repos/anaruse/chainer/milestones{/number}","notifications_url":"https://api.github.com/repos/anaruse/chainer/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/anaruse/chainer/labels{/name}","releases_url":"https://api.github.com/repos/anaruse/chainer/releases{/id}","deployments_url":"https://api.github.com/repos/anaruse/chainer/deployments","created_at":"2016-07-20T06:06:21Z","updated_at":"2018-06-25T02:55:31Z","pushed_at":"2018-08-28T12:14:21Z","git_url":"git://github.com/anaruse/chainer.git","ssh_url":"git@github.com:anaruse/chainer.git","clone_url":"https://github.com/anaruse/chainer.git","svn_url":"https://github.com/anaruse/chainer","homepage":"http://chainer.org","size":32111,"stargazers_count":2,"watchers_count":2,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":3,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":3,"open_issues":0,"watchers":2,"default_branch":"master"}},"base":{"label":"chainer:master","ref":"master","sha":"5e2b104f302088a13723880a3e2b6ee50b0e4089","user":{"login":"chainer","id":23183003,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIzMTgzMDAz","avatar_url":"https://avatars3.githubusercontent.com/u/23183003?v=4","gravatar_id":"","url":"https://api.github.com/users/chainer","html_url":"https://github.com/chainer","followers_url":"https://api.github.com/users/chainer/followers","following_url":"https://api.github.com/users/chainer/following{/other_user}","gists_url":"https://api.github.com/users/chainer/gists{/gist_id}","starred_url":"https://api.github.com/users/chainer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chainer/subscriptions","organizations_url":"https://api.github.com/users/chainer/orgs","repos_url":"https://api.github.com/users/chainer/repos","events_url":"https://api.github.com/users/chainer/events{/privacy}","received_events_url":"https://api.github.com/users/chainer/received_events","type":"Organization","site_admin":false},"repo":{"id":36914478,"node_id":"MDEwOlJlcG9zaXRvcnkzNjkxNDQ3OA==","name":"chainer","full_name":"chainer/chainer","private":false,"owner":{"login":"chainer","id":23183003,"node_id":"MDEyOk9yZ2FuaXphdGlvbjIzMTgzMDAz","avatar_url":"https://avatars3.githubusercontent.com/u/23183003?v=4","gravatar_id":"","url":"https://api.github.com/users/chainer","html_url":"https://github.com/chainer","followers_url":"https://api.github.com/users/chainer/followers","following_url":"https://api.github.com/users/chainer/following{/other_user}","gists_url":"https://api.github.com/users/chainer/gists{/gist_id}","starred_url":"https://api.github.com/users/chainer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chainer/subscriptions","organizations_url":"https://api.github.com/users/chainer/orgs","repos_url":"https://api.github.com/users/chainer/repos","events_url":"https://api.github.com/users/chainer/events{/privacy}","received_events_url":"https://api.github.com/users/chainer/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/chainer/chainer","description":"A flexible framework of neural networks for deep learning","fork":false,"url":"https://api.github.com/repos/chainer/chainer","forks_url":"https://api.github.com/repos/chainer/chainer/forks","keys_url":"https://api.github.com/repos/chainer/chainer/keys{/key_id}","collaborators_url":"https://api.github.com/repos/chainer/chainer/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/chainer/chainer/teams","hooks_url":"https://api.github.com/repos/chainer/chainer/hooks","issue_events_url":"https://api.github.com/repos/chainer/chainer/issues/events{/number}","events_url":"https://api.github.com/repos/chainer/chainer/events","assignees_url":"https://api.github.com/repos/chainer/chainer/assignees{/user}","branches_url":"https://api.github.com/repos/chainer/chainer/branches{/branch}","tags_url":"https://api.github.com/repos/chainer/chainer/tags","blobs_url":"https://api.github.com/repos/chainer/chainer/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/chainer/chainer/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/chainer/chainer/git/refs{/sha}","trees_url":"https://api.github.com/repos/chainer/chainer/git/trees{/sha}","statuses_url":"https://api.github.com/repos/chainer/chainer/statuses/{sha}","languages_url":"https://api.github.com/repos/chainer/chainer/languages","stargazers_url":"https://api.github.com/repos/chainer/chainer/stargazers","contributors_url":"https://api.github.com/repos/chainer/chainer/contributors","subscribers_url":"https://api.github.com/repos/chainer/chainer/subscribers","subscription_url":"https://api.github.com/repos/chainer/chainer/subscription","commits_url":"https://api.github.com/repos/chainer/chainer/commits{/sha}","git_commits_url":"https://api.github.com/repos/chainer/chainer/git/commits{/sha}","comments_url":"https://api.github.com/repos/chainer/chainer/comments{/number}","issue_comment_url":"https://api.github.com/repos/chainer/chainer/issues/comments{/number}","contents_url":"https://api.github.com/repos/chainer/chainer/contents/{+path}","compare_url":"https://api.github.com/repos/chainer/chainer/compare/{base}...{head}","merges_url":"https://api.github.com/repos/chainer/chainer/merges","archive_url":"https://api.github.com/repos/chainer/chainer/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/chainer/chainer/downloads","issues_url":"https://api.github.com/repos/chainer/chainer/issues{/number}","pulls_url":"https://api.github.com/repos/chainer/chainer/pulls{/number}","milestones_url":"https://api.github.com/repos/chainer/chainer/milestones{/number}","notifications_url":"https://api.github.com/repos/chainer/chainer/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/chainer/chainer/labels{/name}","releases_url":"https://api.github.com/repos/chainer/chainer/releases{/id}","deployments_url":"https://api.github.com/repos/chainer/chainer/deployments","created_at":"2015-06-05T05:50:37Z","updated_at":"2018-09-18T05:18:28Z","pushed_at":"2018-09-18T05:18:11Z","git_url":"git://github.com/chainer/chainer.git","ssh_url":"git@github.com:chainer/chainer.git","clone_url":"https://github.com/chainer/chainer.git","svn_url":"https://github.com/chainer/chainer","homepage":"https://chainer.org","size":28696,"stargazers_count":4123,"watchers_count":4123,"language":"Python","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1090,"mirror_url":null,"archived":false,"open_issues_count":222,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":1090,"open_issues":222,"watchers":4123,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/chainer/chainer/pulls/5067"},"html":{"href":"https://github.com/chainer/chainer/pull/5067"},"issue":{"href":"https://api.github.com/repos/chainer/chainer/issues/5067"},"comments":{"href":"https://api.github.com/repos/chainer/chainer/issues/5067/comments"},"review_comments":{"href":"https://api.github.com/repos/chainer/chainer/pulls/5067/comments"},"review_comment":{"href":"https://api.github.com/repos/chainer/chainer/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/chainer/chainer/pulls/5067/commits"},"statuses":{"href":"https://api.github.com/repos/chainer/chainer/statuses/1548087c67ac7cb25574b42a7c6c9705a9d697d3"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"okuta"}}