{"action":"opened","issue":{"url":"https://api.github.com/repos/w3c/webrtc-nv-use-cases/issues/17","repository_url":"https://api.github.com/repos/w3c/webrtc-nv-use-cases","labels_url":"https://api.github.com/repos/w3c/webrtc-nv-use-cases/issues/17/labels{/name}","comments_url":"https://api.github.com/repos/w3c/webrtc-nv-use-cases/issues/17/comments","events_url":"https://api.github.com/repos/w3c/webrtc-nv-use-cases/issues/17/events","html_url":"https://github.com/w3c/webrtc-nv-use-cases/issues/17","id":367778545,"node_id":"MDU6SXNzdWUzNjc3Nzg1NDU=","number":17,"title":"Allocate RAW data capture buffer as shared GPU memory buffer","user":{"login":"astojilj","id":549216,"node_id":"MDQ6VXNlcjU0OTIxNg==","avatar_url":"https://avatars0.githubusercontent.com/u/549216?v=4","gravatar_id":"","url":"https://api.github.com/users/astojilj","html_url":"https://github.com/astojilj","followers_url":"https://api.github.com/users/astojilj/followers","following_url":"https://api.github.com/users/astojilj/following{/other_user}","gists_url":"https://api.github.com/users/astojilj/gists{/gist_id}","starred_url":"https://api.github.com/users/astojilj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astojilj/subscriptions","organizations_url":"https://api.github.com/users/astojilj/orgs","repos_url":"https://api.github.com/users/astojilj/repos","events_url":"https://api.github.com/users/astojilj/events{/privacy}","received_events_url":"https://api.github.com/users/astojilj/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-10-08T12:59:31Z","updated_at":"2018-10-08T12:59:31Z","closed_at":null,"author_association":"NONE","body":"Let's consider current data flow in processing video stream on GPU:\r\ngetUserMedia() => WebGLRenderingContext.texImage2D(HTMLVideoElement)  => processing in WebGL shaders.\r\n\r\nAn example for this is in [TensorFlow.js](https://github.com/tensorflow/tfjs-core) - after video frame is [uploaded to texture](https://github.com/tensorflow/tfjs-core/blob/27766458746bbc5da44cffcee92ef605fdc1ec5c/src/kernels/webgl/gpgpu_util.ts#L234), processing is done on GPU under fenceSync - once the sync is [signaled as ready](https://github.com/tensorflow/tfjs-core/blob/0afc3ef4ac665e344fd50744cb2ae32227e195a2/src/kernels/backend_webgl.ts#L311), data can be [read back to CPU](https://github.com/tensorflow/tfjs-core/blob/27766458746bbc5da44cffcee92ef605fdc1ec5c/src/kernels/webgl/gpgpu_util.ts#L333) without blocking.\r\n\r\nWhile there is a security concern related to mapping buffers that GPU writes to, we can save a copy in WebGLRenderingContext.texImage2D(HTMLVideoElement) - if raw data buffer is [shared GPU memory buffer](https://cs.chromium.org/chromium/src/ui/gfx/gpu_memory_buffer.h).\r\n\r\nI don't think that the engine should do this by default - it could be implemented by hinting to capture pipeline that the raw data will be used on GPU."}}
{"actor":{"display_login":"astojilj"}}