{"action":"created","comment":{"url":"https://api.github.com/repos/NVIDIA/DALI/pulls/comments/236736473","pull_request_review_id":178873323,"id":236736473,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzNjczNjQ3Mw==","diff_hunk":"@@ -70,49 +66,37 @@ class DataReader : public Operator<Backend> {\n   }\n \n   // perform the prefetching operation\n-  virtual bool Prefetch() {\n-    // first clear the batch\n-    // TODO(slayton): exchange multiple batches\n+  virtual void Prefetch() {\n+    prefetched_batch_.reserve(Operator<Backend>::batch_size_);\n     prefetched_batch_.clear();\n \n     for (int i = 0; i < Operator<Backend>::batch_size_; ++i) {\n       auto* t = loader_->ReadOne();\n       prefetched_batch_.push_back(t);\n     }\n-\n-    // TODO(slayton): swap prefetched batches around\n-\n-    return true;\n   }\n \n   // Main prefetch work loop\n   void PrefetchWorker() {\n     std::unique_lock<std::mutex> lock(prefetch_access_mutex_);\n \n     // if a result is already ready, wait until it's consumed\n-    while (prefetch_ready_) {\n-      producer_.wait(lock);\n-    }\n+    producer_.wait(lock, [&]() { return !prefetched_batch_ready_.load(); });\n \n     while (!finished_) {\n       try {\n-        prefetched_batch_.reserve(Operator<Backend>::batch_size_);\n-        prefetch_success_ = Prefetch();\n+        Prefetch();\n       } catch (const std::exception& e) {\n-        printf(\"Prefetch Failed\\n\");\n+        std::cout << \"Prefetch Failed\" << std::endl;","path":"dali/pipeline/operators/reader/reader_op.h","position":55,"original_position":55,"commit_id":"740bef6d89d55326e5f1c5ee0b8649026188a6de","original_commit_id":"740bef6d89d55326e5f1c5ee0b8649026188a6de","user":{"login":"Kh4L","id":3193578,"node_id":"MDQ6VXNlcjMxOTM1Nzg=","avatar_url":"https://avatars2.githubusercontent.com/u/3193578?v=4","gravatar_id":"","url":"https://api.github.com/users/Kh4L","html_url":"https://github.com/Kh4L","followers_url":"https://api.github.com/users/Kh4L/followers","following_url":"https://api.github.com/users/Kh4L/following{/other_user}","gists_url":"https://api.github.com/users/Kh4L/gists{/gist_id}","starred_url":"https://api.github.com/users/Kh4L/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kh4L/subscriptions","organizations_url":"https://api.github.com/users/Kh4L/orgs","repos_url":"https://api.github.com/users/Kh4L/repos","events_url":"https://api.github.com/users/Kh4L/events{/privacy}","received_events_url":"https://api.github.com/users/Kh4L/received_events","type":"User","site_admin":false},"body":"Done","created_at":"2018-11-27T16:13:50Z","updated_at":"2018-11-27T16:13:51Z","html_url":"https://github.com/NVIDIA/DALI/pull/316#discussion_r236736473","pull_request_url":"https://api.github.com/repos/NVIDIA/DALI/pulls/316","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/comments/236736473"},"html":{"href":"https://github.com/NVIDIA/DALI/pull/316#discussion_r236736473"},"pull_request":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/316"}},"in_reply_to_id":236734681},"pull_request":{"url":"https://api.github.com/repos/NVIDIA/DALI/pulls/316","id":232776353,"node_id":"MDExOlB1bGxSZXF1ZXN0MjMyNzc2MzUz","html_url":"https://github.com/NVIDIA/DALI/pull/316","diff_url":"https://github.com/NVIDIA/DALI/pull/316.diff","patch_url":"https://github.com/NVIDIA/DALI/pull/316.patch","issue_url":"https://api.github.com/repos/NVIDIA/DALI/issues/316","number":316,"state":"open","locked":false,"title":"[WIP] VideoReader operator","user":{"login":"Kh4L","id":3193578,"node_id":"MDQ6VXNlcjMxOTM1Nzg=","avatar_url":"https://avatars2.githubusercontent.com/u/3193578?v=4","gravatar_id":"","url":"https://api.github.com/users/Kh4L","html_url":"https://github.com/Kh4L","followers_url":"https://api.github.com/users/Kh4L/followers","following_url":"https://api.github.com/users/Kh4L/following{/other_user}","gists_url":"https://api.github.com/users/Kh4L/gists{/gist_id}","starred_url":"https://api.github.com/users/Kh4L/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kh4L/subscriptions","organizations_url":"https://api.github.com/users/Kh4L/orgs","repos_url":"https://api.github.com/users/Kh4L/repos","events_url":"https://api.github.com/users/Kh4L/events{/privacy}","received_events_url":"https://api.github.com/users/Kh4L/received_events","type":"User","site_admin":false},"body":"**VideoReader** operator loads and decodes H.264 and HEVC video codecs from any container format supported by FFmpeg and outputs sequences as a GPU TensorList.\r\n\r\nThe output is a batch, a TensorList, of **SHWC** Tensors, with **S** being defined by the `count` op argument.  \r\n\r\n:warning: This is a work-in-progress. :warning:\r\n\r\n**Initial PR**\r\n- Add new operators `VideoReader` and `VideoLoader`\r\n- Add a plethora of new classes in `dali/pipeline/operators/reader/nvdecoder`, which are parts of the VideoReader, with courtesy of [NVVL](https://github.com/NVIDIA/nvvl)\r\n- Add draft of an example in examples/video (cf. tasklist)\r\n- Now linking with some CUDA driver API for nvcuvid, cuda and nvidia-ml (required by the NVDECODE API)\r\n- Now linking with **FFmpeg** \r\n- Add a `ReaderOp::Run` template specialization to support `GPUBackend` (`TensorList` as an output) - hacky and temporary (cf. the tasklist below)\r\n- Change `CUDA_CALL` to support of `CUresult` (cuda driver API error code enum) in `error_handling.h`\r\n\r\n**TODOs**\r\n- [ ] Define how we want to link against libcuda, libnvidia-ml and libnvcuvid, dynloading?\r\n- [ ] Support multiple container files\r\n- [ ] Redesign the ReaderOp interface\r\n- [ ] Define op parameters\r\n- [ ] Add unit tests\r\n- [ ] Add a full example","created_at":"2018-11-21T19:29:15Z","updated_at":"2018-11-27T16:13:51Z","closed_at":null,"merged_at":null,"merge_commit_sha":"6e3cb9c7c53bdb4760b8acee82d13cc106a37b23","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/NVIDIA/DALI/pulls/316/commits","review_comments_url":"https://api.github.com/repos/NVIDIA/DALI/pulls/316/comments","review_comment_url":"https://api.github.com/repos/NVIDIA/DALI/pulls/comments{/number}","comments_url":"https://api.github.com/repos/NVIDIA/DALI/issues/316/comments","statuses_url":"https://api.github.com/repos/NVIDIA/DALI/statuses/740bef6d89d55326e5f1c5ee0b8649026188a6de","head":{"label":"Kh4L:video_loader_op","ref":"video_loader_op","sha":"740bef6d89d55326e5f1c5ee0b8649026188a6de","user":{"login":"Kh4L","id":3193578,"node_id":"MDQ6VXNlcjMxOTM1Nzg=","avatar_url":"https://avatars2.githubusercontent.com/u/3193578?v=4","gravatar_id":"","url":"https://api.github.com/users/Kh4L","html_url":"https://github.com/Kh4L","followers_url":"https://api.github.com/users/Kh4L/followers","following_url":"https://api.github.com/users/Kh4L/following{/other_user}","gists_url":"https://api.github.com/users/Kh4L/gists{/gist_id}","starred_url":"https://api.github.com/users/Kh4L/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kh4L/subscriptions","organizations_url":"https://api.github.com/users/Kh4L/orgs","repos_url":"https://api.github.com/users/Kh4L/repos","events_url":"https://api.github.com/users/Kh4L/events{/privacy}","received_events_url":"https://api.github.com/users/Kh4L/received_events","type":"User","site_admin":false},"repo":{"id":139641410,"node_id":"MDEwOlJlcG9zaXRvcnkxMzk2NDE0MTA=","name":"DALI","full_name":"Kh4L/DALI","private":false,"owner":{"login":"Kh4L","id":3193578,"node_id":"MDQ6VXNlcjMxOTM1Nzg=","avatar_url":"https://avatars2.githubusercontent.com/u/3193578?v=4","gravatar_id":"","url":"https://api.github.com/users/Kh4L","html_url":"https://github.com/Kh4L","followers_url":"https://api.github.com/users/Kh4L/followers","following_url":"https://api.github.com/users/Kh4L/following{/other_user}","gists_url":"https://api.github.com/users/Kh4L/gists{/gist_id}","starred_url":"https://api.github.com/users/Kh4L/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Kh4L/subscriptions","organizations_url":"https://api.github.com/users/Kh4L/orgs","repos_url":"https://api.github.com/users/Kh4L/repos","events_url":"https://api.github.com/users/Kh4L/events{/privacy}","received_events_url":"https://api.github.com/users/Kh4L/received_events","type":"User","site_admin":false},"html_url":"https://github.com/Kh4L/DALI","description":"A library containing both highly optimized building blocks and an execution engine for data pre-processing in deep learning applications","fork":true,"url":"https://api.github.com/repos/Kh4L/DALI","forks_url":"https://api.github.com/repos/Kh4L/DALI/forks","keys_url":"https://api.github.com/repos/Kh4L/DALI/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Kh4L/DALI/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Kh4L/DALI/teams","hooks_url":"https://api.github.com/repos/Kh4L/DALI/hooks","issue_events_url":"https://api.github.com/repos/Kh4L/DALI/issues/events{/number}","events_url":"https://api.github.com/repos/Kh4L/DALI/events","assignees_url":"https://api.github.com/repos/Kh4L/DALI/assignees{/user}","branches_url":"https://api.github.com/repos/Kh4L/DALI/branches{/branch}","tags_url":"https://api.github.com/repos/Kh4L/DALI/tags","blobs_url":"https://api.github.com/repos/Kh4L/DALI/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Kh4L/DALI/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Kh4L/DALI/git/refs{/sha}","trees_url":"https://api.github.com/repos/Kh4L/DALI/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Kh4L/DALI/statuses/{sha}","languages_url":"https://api.github.com/repos/Kh4L/DALI/languages","stargazers_url":"https://api.github.com/repos/Kh4L/DALI/stargazers","contributors_url":"https://api.github.com/repos/Kh4L/DALI/contributors","subscribers_url":"https://api.github.com/repos/Kh4L/DALI/subscribers","subscription_url":"https://api.github.com/repos/Kh4L/DALI/subscription","commits_url":"https://api.github.com/repos/Kh4L/DALI/commits{/sha}","git_commits_url":"https://api.github.com/repos/Kh4L/DALI/git/commits{/sha}","comments_url":"https://api.github.com/repos/Kh4L/DALI/comments{/number}","issue_comment_url":"https://api.github.com/repos/Kh4L/DALI/issues/comments{/number}","contents_url":"https://api.github.com/repos/Kh4L/DALI/contents/{+path}","compare_url":"https://api.github.com/repos/Kh4L/DALI/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Kh4L/DALI/merges","archive_url":"https://api.github.com/repos/Kh4L/DALI/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Kh4L/DALI/downloads","issues_url":"https://api.github.com/repos/Kh4L/DALI/issues{/number}","pulls_url":"https://api.github.com/repos/Kh4L/DALI/pulls{/number}","milestones_url":"https://api.github.com/repos/Kh4L/DALI/milestones{/number}","notifications_url":"https://api.github.com/repos/Kh4L/DALI/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Kh4L/DALI/labels{/name}","releases_url":"https://api.github.com/repos/Kh4L/DALI/releases{/id}","deployments_url":"https://api.github.com/repos/Kh4L/DALI/deployments","created_at":"2018-07-03T22:05:17Z","updated_at":"2018-11-23T09:33:21Z","pushed_at":"2018-11-27T14:50:13Z","git_url":"git://github.com/Kh4L/DALI.git","ssh_url":"git@github.com:Kh4L/DALI.git","clone_url":"https://github.com/Kh4L/DALI.git","svn_url":"https://github.com/Kh4L/DALI","homepage":"https://docs.nvidia.com/deeplearning/sdk/dali-developer-guide/index.html","size":41603,"stargazers_count":0,"watchers_count":0,"language":"C++","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":1,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":0,"open_issues":1,"watchers":0,"default_branch":"master"}},"base":{"label":"NVIDIA:master","ref":"master","sha":"7e12ffdcff51141c9c730b0767234d5c4e18a607","user":{"login":"NVIDIA","id":1728152,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE3MjgxNTI=","avatar_url":"https://avatars2.githubusercontent.com/u/1728152?v=4","gravatar_id":"","url":"https://api.github.com/users/NVIDIA","html_url":"https://github.com/NVIDIA","followers_url":"https://api.github.com/users/NVIDIA/followers","following_url":"https://api.github.com/users/NVIDIA/following{/other_user}","gists_url":"https://api.github.com/users/NVIDIA/gists{/gist_id}","starred_url":"https://api.github.com/users/NVIDIA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NVIDIA/subscriptions","organizations_url":"https://api.github.com/users/NVIDIA/orgs","repos_url":"https://api.github.com/users/NVIDIA/repos","events_url":"https://api.github.com/users/NVIDIA/events{/privacy}","received_events_url":"https://api.github.com/users/NVIDIA/received_events","type":"Organization","site_admin":false},"repo":{"id":135768037,"node_id":"MDEwOlJlcG9zaXRvcnkxMzU3NjgwMzc=","name":"DALI","full_name":"NVIDIA/DALI","private":false,"owner":{"login":"NVIDIA","id":1728152,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE3MjgxNTI=","avatar_url":"https://avatars2.githubusercontent.com/u/1728152?v=4","gravatar_id":"","url":"https://api.github.com/users/NVIDIA","html_url":"https://github.com/NVIDIA","followers_url":"https://api.github.com/users/NVIDIA/followers","following_url":"https://api.github.com/users/NVIDIA/following{/other_user}","gists_url":"https://api.github.com/users/NVIDIA/gists{/gist_id}","starred_url":"https://api.github.com/users/NVIDIA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NVIDIA/subscriptions","organizations_url":"https://api.github.com/users/NVIDIA/orgs","repos_url":"https://api.github.com/users/NVIDIA/repos","events_url":"https://api.github.com/users/NVIDIA/events{/privacy}","received_events_url":"https://api.github.com/users/NVIDIA/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/NVIDIA/DALI","description":"A library containing both highly optimized building blocks and an execution engine for data pre-processing in deep learning applications","fork":false,"url":"https://api.github.com/repos/NVIDIA/DALI","forks_url":"https://api.github.com/repos/NVIDIA/DALI/forks","keys_url":"https://api.github.com/repos/NVIDIA/DALI/keys{/key_id}","collaborators_url":"https://api.github.com/repos/NVIDIA/DALI/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/NVIDIA/DALI/teams","hooks_url":"https://api.github.com/repos/NVIDIA/DALI/hooks","issue_events_url":"https://api.github.com/repos/NVIDIA/DALI/issues/events{/number}","events_url":"https://api.github.com/repos/NVIDIA/DALI/events","assignees_url":"https://api.github.com/repos/NVIDIA/DALI/assignees{/user}","branches_url":"https://api.github.com/repos/NVIDIA/DALI/branches{/branch}","tags_url":"https://api.github.com/repos/NVIDIA/DALI/tags","blobs_url":"https://api.github.com/repos/NVIDIA/DALI/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/NVIDIA/DALI/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/NVIDIA/DALI/git/refs{/sha}","trees_url":"https://api.github.com/repos/NVIDIA/DALI/git/trees{/sha}","statuses_url":"https://api.github.com/repos/NVIDIA/DALI/statuses/{sha}","languages_url":"https://api.github.com/repos/NVIDIA/DALI/languages","stargazers_url":"https://api.github.com/repos/NVIDIA/DALI/stargazers","contributors_url":"https://api.github.com/repos/NVIDIA/DALI/contributors","subscribers_url":"https://api.github.com/repos/NVIDIA/DALI/subscribers","subscription_url":"https://api.github.com/repos/NVIDIA/DALI/subscription","commits_url":"https://api.github.com/repos/NVIDIA/DALI/commits{/sha}","git_commits_url":"https://api.github.com/repos/NVIDIA/DALI/git/commits{/sha}","comments_url":"https://api.github.com/repos/NVIDIA/DALI/comments{/number}","issue_comment_url":"https://api.github.com/repos/NVIDIA/DALI/issues/comments{/number}","contents_url":"https://api.github.com/repos/NVIDIA/DALI/contents/{+path}","compare_url":"https://api.github.com/repos/NVIDIA/DALI/compare/{base}...{head}","merges_url":"https://api.github.com/repos/NVIDIA/DALI/merges","archive_url":"https://api.github.com/repos/NVIDIA/DALI/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/NVIDIA/DALI/downloads","issues_url":"https://api.github.com/repos/NVIDIA/DALI/issues{/number}","pulls_url":"https://api.github.com/repos/NVIDIA/DALI/pulls{/number}","milestones_url":"https://api.github.com/repos/NVIDIA/DALI/milestones{/number}","notifications_url":"https://api.github.com/repos/NVIDIA/DALI/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/NVIDIA/DALI/labels{/name}","releases_url":"https://api.github.com/repos/NVIDIA/DALI/releases{/id}","deployments_url":"https://api.github.com/repos/NVIDIA/DALI/deployments","created_at":"2018-06-01T22:18:01Z","updated_at":"2018-11-27T12:11:27Z","pushed_at":"2018-11-27T14:50:15Z","git_url":"git://github.com/NVIDIA/DALI.git","ssh_url":"git@github.com:NVIDIA/DALI.git","clone_url":"https://github.com/NVIDIA/DALI.git","svn_url":"https://github.com/NVIDIA/DALI","homepage":"https://docs.nvidia.com/deeplearning/sdk/dali-developer-guide/index.html","size":40753,"stargazers_count":825,"watchers_count":825,"language":"C++","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":89,"mirror_url":null,"archived":false,"open_issues_count":44,"license":{"key":"other","name":"Other","spdx_id":"NOASSERTION","url":null,"node_id":"MDc6TGljZW5zZTA="},"forks":89,"open_issues":44,"watchers":825,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/316"},"html":{"href":"https://github.com/NVIDIA/DALI/pull/316"},"issue":{"href":"https://api.github.com/repos/NVIDIA/DALI/issues/316"},"comments":{"href":"https://api.github.com/repos/NVIDIA/DALI/issues/316/comments"},"review_comments":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/316/comments"},"review_comment":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/NVIDIA/DALI/pulls/316/commits"},"statuses":{"href":"https://api.github.com/repos/NVIDIA/DALI/statuses/740bef6d89d55326e5f1c5ee0b8649026188a6de"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"Kh4L"}}