{"action":"created","issue":{"url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/4","repository_url":"https://api.github.com/repos/w3c/media-and-entertainment","labels_url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/4/labels{/name}","comments_url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/4/comments","events_url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/4/events","html_url":"https://github.com/w3c/media-and-entertainment/issues/4","id":331200926,"node_id":"MDU6SXNzdWUzMzEyMDA5MjY=","number":4,"title":"Frame accurate seeking of HTML5 MediaElement","user":{"login":"tidoust","id":1122927,"node_id":"MDQ6VXNlcjExMjI5Mjc=","avatar_url":"https://avatars2.githubusercontent.com/u/1122927?v=4","gravatar_id":"","url":"https://api.github.com/users/tidoust","html_url":"https://github.com/tidoust","followers_url":"https://api.github.com/users/tidoust/followers","following_url":"https://api.github.com/users/tidoust/following{/other_user}","gists_url":"https://api.github.com/users/tidoust/gists{/gist_id}","starred_url":"https://api.github.com/users/tidoust/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tidoust/subscriptions","organizations_url":"https://api.github.com/users/tidoust/orgs","repos_url":"https://api.github.com/users/tidoust/repos","events_url":"https://api.github.com/users/tidoust/events{/privacy}","received_events_url":"https://api.github.com/users/tidoust/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":42,"created_at":"2018-06-11T13:58:32Z","updated_at":"2018-06-15T14:18:21Z","closed_at":null,"author_association":"NONE","body":"I've heard a couple of companies point out that one of the problems that makes it hard (at least harder than it could be) to do **post-production of videos** in Web browsers is that there is no easy way to process media elements on a frame by frame basis, whereas that is the usual default in Non-Linear Editors (NLE).\r\n\r\nThe `currentTime` property takes a time, not a frame number or an SMPTE timecode. Conversion from/to times to/from frame numbers is doable but supposes one knows the framerate of the video, which is not exposed to Web applications (a generic NLE would thus not know about it). Plus that framerate may actually vary over time.\r\n\r\nAlso, internal rounding of time values may mean that one seeks to the end of the previous frame instead of the beginning of a specific video frame.\r\n\r\nDigging around, I've found a number of discussions and issues around the topic, most notably:\r\n1. An long thread from 2011 on Frame accuracy / SMPTE, which led to improvements in the precision of seeks in browser implementations:\r\nhttps://lists.w3.org/Archives/Public/public-whatwg-archive/2011Jan/0120.html\r\n2. A list of use cases from 2012 for seeking to specific frames. Not sure if these use cases remain relevant today:\r\nhttps://www.w3.org/Bugs/Public/show_bug.cgi?id=22678\r\n3. A question from 2013 on whether there was interest to expose \"versions of currentTime, fastSeek(), duration, and the TimeRanges accessors, in frames, for video data\":\r\nhttps://www.w3.org/Bugs/Public/show_bug.cgi?id=8278#c3\r\n4. A proposal from 2016 to add a rational time value for seek() to solve rounding issues (still open as of June 2018):\r\nhttps://github.com/whatwg/html/issues/609\r\n\r\nThere have probably been other discussions around the topic.\r\n\r\nI'm raising this issue to collect practical use cases and requirements for the feature, and gauge interest from media companies to see a solution emerge. It would be good to precisely identify what does not work today, what minimal updates to media elements could solve the issue, and what these updates would imply from an implementation perspective."},"comment":{"url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/comments/397635627","html_url":"https://github.com/w3c/media-and-entertainment/issues/4#issuecomment-397635627","issue_url":"https://api.github.com/repos/w3c/media-and-entertainment/issues/4","id":397635627,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NzYzNTYyNw==","user":{"login":"nigelmegitt","id":4988915,"node_id":"MDQ6VXNlcjQ5ODg5MTU=","avatar_url":"https://avatars0.githubusercontent.com/u/4988915?v=4","gravatar_id":"","url":"https://api.github.com/users/nigelmegitt","html_url":"https://github.com/nigelmegitt","followers_url":"https://api.github.com/users/nigelmegitt/followers","following_url":"https://api.github.com/users/nigelmegitt/following{/other_user}","gists_url":"https://api.github.com/users/nigelmegitt/gists{/gist_id}","starred_url":"https://api.github.com/users/nigelmegitt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nigelmegitt/subscriptions","organizations_url":"https://api.github.com/users/nigelmegitt/orgs","repos_url":"https://api.github.com/users/nigelmegitt/repos","events_url":"https://api.github.com/users/nigelmegitt/events{/privacy}","received_events_url":"https://api.github.com/users/nigelmegitt/received_events","type":"User","site_admin":false},"created_at":"2018-06-15T14:18:21Z","updated_at":"2018-06-15T14:18:21Z","author_association":"NONE","body":"Thinking more about https://github.com/w3c/media-and-entertainment/issues/4#issuecomment-397593999 :\r\n\r\n> transferring the playback state (e.g. currentTime) from player to JS takes more than 0 time, the value received in JS will always be outdated.\r\n\r\nCan this not be modelled or measured adaptively though to minimise the impact?\r\n\r\n> Triggering events with event handlers at 50hz will make increase resource usage by a lot, even if there is no real reason for it. \r\n\r\n@Snarkdoof have you got data to back this up? Running _time marches on_ when the list of events to add to the queue is empty is not going to take many CPU clock cycles. Relative to the resource needed to play back video, I suspect it is tiny. But the advantage of doing it is huge when there is an event to add to the queue. It'd be good to move from speculation (on both our parts!) to measurement."}}
{"actor":{"display_login":"nigelmegitt"}}