{"action":"created","issue":{"url":"https://api.github.com/repos/IIIF/api/issues/1332","repository_url":"https://api.github.com/repos/IIIF/api","labels_url":"https://api.github.com/repos/IIIF/api/issues/1332/labels{/name}","comments_url":"https://api.github.com/repos/IIIF/api/issues/1332/comments","events_url":"https://api.github.com/repos/IIIF/api/issues/1332/events","html_url":"https://github.com/IIIF/api/issues/1332","id":277856002,"number":1332,"title":"how to extract/reference the audio of a video","user":{"login":"jronallo","id":66485,"avatar_url":"https://avatars1.githubusercontent.com/u/66485?v=4","gravatar_id":"","url":"https://api.github.com/users/jronallo","html_url":"https://github.com/jronallo","followers_url":"https://api.github.com/users/jronallo/followers","following_url":"https://api.github.com/users/jronallo/following{/other_user}","gists_url":"https://api.github.com/users/jronallo/gists{/gist_id}","starred_url":"https://api.github.com/users/jronallo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jronallo/subscriptions","organizations_url":"https://api.github.com/users/jronallo/orgs","repos_url":"https://api.github.com/users/jronallo/repos","events_url":"https://api.github.com/users/jronallo/events{/privacy}","received_events_url":"https://api.github.com/users/jronallo/received_events","type":"User","site_admin":false},"labels":[{"id":353588003,"url":"https://api.github.com/repos/IIIF/api/labels/A/V","name":"A/V","color":"990000","default":false},{"id":83115746,"url":"https://api.github.com/repos/IIIF/api/labels/presentation","name":"presentation","color":"5319e7","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/IIIF/api/milestones/8","html_url":"https://github.com/IIIF/api/milestone/8","labels_url":"https://api.github.com/repos/IIIF/api/milestones/8/labels","id":1021541,"number":8,"title":"Presentation 3.0","description":"Breaking changes from the 2.* Presentation branch","creator":{"login":"azaroth42","id":871868,"avatar_url":"https://avatars3.githubusercontent.com/u/871868?v=4","gravatar_id":"","url":"https://api.github.com/users/azaroth42","html_url":"https://github.com/azaroth42","followers_url":"https://api.github.com/users/azaroth42/followers","following_url":"https://api.github.com/users/azaroth42/following{/other_user}","gists_url":"https://api.github.com/users/azaroth42/gists{/gist_id}","starred_url":"https://api.github.com/users/azaroth42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/azaroth42/subscriptions","organizations_url":"https://api.github.com/users/azaroth42/orgs","repos_url":"https://api.github.com/users/azaroth42/repos","events_url":"https://api.github.com/users/azaroth42/events{/privacy}","received_events_url":"https://api.github.com/users/azaroth42/received_events","type":"User","site_admin":false},"open_issues":22,"closed_issues":138,"state":"open","created_at":"2015-03-14T16:32:24Z","updated_at":"2018-02-27T20:58:48Z","due_on":"2018-05-01T07:00:00Z","closed_at":null},"comments":6,"created_at":"2017-11-29T18:16:13Z","updated_at":"2018-02-28T00:30:27Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"I'm moving a discussion from another issue onto a new issue. I'll add what I've written below. Other comments begin here on this other issue: https://github.com/IIIF/api/issues/1190#issuecomment-347654763\r\n\r\n---\r\n\r\nIs there a way to play only the audio from video bitstreams? Or rather what happens if the canvas has only a duration and you paint on video bitstreams/items? \r\n\r\nIf you have a duration-only canvas, what happens if you paint on video items that target the whole canvas? Is this different from what happens when you have a duration-only canvas and paint on video items that explicitly target only the duration of the canvas through a media fragment?\r\n\r\nThe bitstreams/items do not have any indication of their height and width and so could just be treated as audio items. Also in HTML you can make a video the src of an audio element and it plays the audio just fine without displaying the video.\r\n\r\nI agree that if you have a duration-only canvas and you try to target the canvas with a media fragment that includes a `xywh` then it should be an error since the canvas does not have any space for painting on the visual component of the video. What I'm presenting is not that case where you have a clear mismatch between explicit dimensions.\r\n\r\nThis is instead a question of what happens when you intent to reduce a video to just its duration--to just its audio. Use case is that as a manifest creator I want to use the audio from a video without having to extract the audio and publish the audio files myself, especially since I may not have the rights to redistribute the resource myself. Another use case is a video oral history where the video is boring and so the manifest creator only wants to play the audio.\r\n\r\n---\r\n\r\nThe Media Fragments specification (basic) does mention and somewhat define the `track` dimension and give some examples, it just gives no guidance on how to handle this in HTTP or how to process the values. Some of that guidance is in the advanced version but that never got to recommendation status. \r\n\r\n\"The way the user agent retrieves information regarding the available tracks of a particular resource is out of scope for this specification.\" https://www.w3.org/TR/media-frags/#media-fragment-display \r\nIs there any reason why we couldn't use the `track` name and specify which values a IIIF client ought to accept/reject? \r\n\r\nThe basic specification does give guidance on media fragment servers and the case where you're retrieving audio from a video: https://www.w3.org/TR/media-frags/#media-fragment-servers\r\nSince these are in the fragment hash of course you get back a video even if you're just requesting the audio fragment be played. No transformation of the underlying media is suggested.\r\n\r\nIf leaving off the w,h dimensions from a canvas isn't the way to play audio from a video, then it will probably be easy enough to game this by saying the canvas is 1pixel wide and high which ought to result in a player which looks like an audio player even if it is still trying to render the video. That seems silly. There's no reason to force the client to do the work to render the video component of the video if it isn't really intended to be used.\r\n\r\nThis is a common enough case that it was considered in the media fragment work. There's no transformation needed because the browser handles the case of playing just the audio from a video just fine. As far as the bitstream is concerned this is no different than muting the audio of a video and only rendering the video. This is just a case where you're \"muting\" the visual and only \"rendering\" the audio.\r\n\r\nSo one way forward would be to allow the `track` media fragment on a resource and only allow the values of `audio` and `video` (or both). The other way forward that has been mentioned in this issue is to set a time dimension only on the canvas while still having video items annotated on which would result in creation of just an audio element. How else can we achieve this? And are the use cases sufficient to drive us to find a solution?\r\n\r\n/cc @tomcrane @workergnome @cjcolvar @azaroth42 \r\n\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/IIIF/api/issues/comments/369077215","html_url":"https://github.com/IIIF/api/issues/1332#issuecomment-369077215","issue_url":"https://api.github.com/repos/IIIF/api/issues/1332","id":369077215,"user":{"login":"azaroth42","id":871868,"avatar_url":"https://avatars3.githubusercontent.com/u/871868?v=4","gravatar_id":"","url":"https://api.github.com/users/azaroth42","html_url":"https://github.com/azaroth42","followers_url":"https://api.github.com/users/azaroth42/followers","following_url":"https://api.github.com/users/azaroth42/following{/other_user}","gists_url":"https://api.github.com/users/azaroth42/gists{/gist_id}","starred_url":"https://api.github.com/users/azaroth42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/azaroth42/subscriptions","organizations_url":"https://api.github.com/users/azaroth42/orgs","repos_url":"https://api.github.com/users/azaroth42/repos","events_url":"https://api.github.com/users/azaroth42/events{/privacy}","received_events_url":"https://api.github.com/users/azaroth42/received_events","type":"User","site_admin":false},"created_at":"2018-02-28T00:30:27Z","updated_at":"2018-02-28T00:30:27Z","author_association":"OWNER","body":"Editorially ... I think these join the ImageApiSelector defined at http://iiif.io/api/annex/openannotation/ and then have examples in the cookbook?\r\n\r\nPerhaps an example in the main Prezi document for `start` referencing a time point on a canvas with duration? (per #1386)"}}
{"actor":{"display_login":"azaroth42"}}