{"action":"created","comment":{"url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/comments/212781242","pull_request_review_id":149492188,"id":212781242,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIxMjc4MTI0Mg==","diff_hunk":"@@ -0,0 +1,227 @@\n+//\n+//  SampleHandler.swift\n+//  BroadcastExtension\n+//\n+//  Created by Piyush Tank on 7/1/18.\n+//  Copyright © 2018 Twilio. All rights reserved.\n+//\n+\n+import Accelerate\n+import TwilioVideo\n+import ReplayKit\n+\n+class SampleHandler: RPBroadcastSampleHandler, TVIRoomDelegate, TVIVideoCapturer {\n+\n+    public var isScreencast: Bool = true\n+\n+    // Video SDK components\n+    public var room: TVIRoom?\n+    weak var captureConsumer: TVIVideoCaptureConsumer?\n+\n+    static let kDesiredFrameRate = 30\n+    static let kDownScaledFrameWidth = 540\n+    static let kDownScaledFrameHeight = 960","path":"ReplayKitExample/BroadcastExtension/SampleHandler.swift","position":23,"original_position":23,"commit_id":"c7b57eb6036a20e0bb61d1e9a6050684dc6b19b1","original_commit_id":"c7b57eb6036a20e0bb61d1e9a6050684dc6b19b1","user":{"login":"ceaglest","id":1302577,"node_id":"MDQ6VXNlcjEzMDI1Nzc=","avatar_url":"https://avatars2.githubusercontent.com/u/1302577?v=4","gravatar_id":"","url":"https://api.github.com/users/ceaglest","html_url":"https://github.com/ceaglest","followers_url":"https://api.github.com/users/ceaglest/followers","following_url":"https://api.github.com/users/ceaglest/following{/other_user}","gists_url":"https://api.github.com/users/ceaglest/gists{/gist_id}","starred_url":"https://api.github.com/users/ceaglest/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ceaglest/subscriptions","organizations_url":"https://api.github.com/users/ceaglest/orgs","repos_url":"https://api.github.com/users/ceaglest/repos","events_url":"https://api.github.com/users/ceaglest/events{/privacy}","received_events_url":"https://api.github.com/users/ceaglest/received_events","type":"User","site_admin":false},"body":"Make the scaling logic aspect ratio independent so it can work on iPad, iPhone, iPhone X. Consider setting a maximum dimension (either width or height) of 640.","created_at":"2018-08-25T00:31:22Z","updated_at":"2018-08-25T00:39:42Z","html_url":"https://github.com/twilio/video-quickstart-swift/pull/287#discussion_r212781242","pull_request_url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/comments/212781242"},"html":{"href":"https://github.com/twilio/video-quickstart-swift/pull/287#discussion_r212781242"},"pull_request":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287"}}},"pull_request":{"url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287","id":198596125,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk4NTk2MTI1","html_url":"https://github.com/twilio/video-quickstart-swift/pull/287","diff_url":"https://github.com/twilio/video-quickstart-swift/pull/287.diff","patch_url":"https://github.com/twilio/video-quickstart-swift/pull/287.patch","issue_url":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/287","number":287,"state":"open","locked":false,"title":"ISDK-1512 - ReplayKit broadcast service example (WIP)","user":{"login":"piyushtank","id":17037593,"node_id":"MDQ6VXNlcjE3MDM3NTkz","avatar_url":"https://avatars3.githubusercontent.com/u/17037593?v=4","gravatar_id":"","url":"https://api.github.com/users/piyushtank","html_url":"https://github.com/piyushtank","followers_url":"https://api.github.com/users/piyushtank/followers","following_url":"https://api.github.com/users/piyushtank/following{/other_user}","gists_url":"https://api.github.com/users/piyushtank/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushtank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushtank/subscriptions","organizations_url":"https://api.github.com/users/piyushtank/orgs","repos_url":"https://api.github.com/users/piyushtank/repos","events_url":"https://api.github.com/users/piyushtank/events{/privacy}","received_events_url":"https://api.github.com/users/piyushtank/received_events","type":"User","site_admin":false},"body":"This PR demonstrates how to write a broadcast service (extension) using ReplayKit and TwilioVideo. TwilioVideo already supports ScreenSharing APIs however with ReplayKit this is much easier and efficient.\r\n\r\nThe PR is devided into two main areas - \r\n1. The App -  **ReplayKitExample** \r\nThe app accepts user request to start broad cast. Upon receiving the request, app loads `RPBroadcastActivityViewController` with the preferred UI extension. The app holds on to the broadcast controller to stop and start. Once the UI extension finishes request, app starts broadcasting the screens.\r\n\r\n2. The Extension\r\n  2a - The Extension UI -  As name describes this component is responsible for extension UI. Here user can enter the credential. In this PR, the UI asks user to enter the Room name.\r\n  2b - The Extension Sample Handler - \r\n     Once the app starts broadcasting, sample handler receives pixelbuffer and audio samples. PR connects to the Room in the initialization module. Please note the PR implements custom capturer and provides the received pixelbuffer to the Video SDK.\r\n\r\n#### 50 MB size limitation \r\niOS does not allow an extension to grow beyond 50MB of memory limit. Since the pixelbuffer size received from ReplayKit depends on the size of the iOS device, it is easy to consume more than 50MB limit by an extension when end-to-end buffer pipeline gets involved. To overcome this problem, this PR uses H.264 video codec and downscales the pixelbuffer to 960x540. These two parameters can be changed per your requirement. We are discussing on providing better APIs so that developers will not have to do the downscaling in their app extension.\r\n\r\n#### iOS 12\r\niOS 12 allows screen capture outside the app using RPBroadcastPickerView also. The PR has two routes one for iOS 12 and another for the rest. For iOS 12, the app initiates the RPBroadcastPickerView to capture screen outside the app also. Please note for iOS 12 code path, the room name is hardcoded as I was not able to invoke the extension UI from picker view (I am not sure if it is possible).\r\n\r\n#### Audio\r\nSo far my research says we can not start or stop AudioUnit. I tried with RTC and `ExampleAVAudioEngineDevice` but it threw [this](https://developer.apple.com/documentation/avfoundation/avaudiosession/errorcode/cannotstartrecording) error.  Per [this](https://developer.apple.com/library/archive/qa/qa1872/_index.html) post it looks like its definetely not possible for iOS 8 but I need to dig more. \r\n\r\nOne back-up plan can be writing custom audio device which takes remote participants' audio samples however still the playback side is not clear yet. Need more research in this area.\r\n\r\n#### TODOs\r\n- [ ] UI for discussion\r\n- [ ] Audio\r\n  - [ ] Remote participant's Audio Playback\r\n  - [x] Recording does not work consistatly\r\n  - [ ] Audio device changes\r\n- [ ] iOS12 beta bug for pickerview. Get background image.\r\n- [ ] Readme with screenshots\r\n- [ ] Should we show settings page for extension UI ?\r\n- [ ] Should we use RPBroadcastPicker audio property ? Again audio is a big question rightnow.\r\n- [ ] Inconsistency in folder structure (see podfile changes, is it caused by Xcode 10?)\r\n- [ ] Use `RPVideoSampleOrientationKey` for `TVIVideoFrameOrientation`\r\n\r\n#### NOTE\r\nThe PR is work in progress (WIP) and not ready for merge yet. The PR is posted for early discussion.","created_at":"2018-07-02T07:41:12Z","updated_at":"2018-08-25T00:39:42Z","closed_at":null,"merged_at":null,"merge_commit_sha":"90fb9d2750a58524d8ba916507c824f37f810da3","assignee":null,"assignees":[],"requested_reviewers":[{"login":"paynerc","id":25950,"node_id":"MDQ6VXNlcjI1OTUw","avatar_url":"https://avatars0.githubusercontent.com/u/25950?v=4","gravatar_id":"","url":"https://api.github.com/users/paynerc","html_url":"https://github.com/paynerc","followers_url":"https://api.github.com/users/paynerc/followers","following_url":"https://api.github.com/users/paynerc/following{/other_user}","gists_url":"https://api.github.com/users/paynerc/gists{/gist_id}","starred_url":"https://api.github.com/users/paynerc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/paynerc/subscriptions","organizations_url":"https://api.github.com/users/paynerc/orgs","repos_url":"https://api.github.com/users/paynerc/repos","events_url":"https://api.github.com/users/paynerc/events{/privacy}","received_events_url":"https://api.github.com/users/paynerc/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287/commits","review_comments_url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287/comments","review_comment_url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/comments{/number}","comments_url":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/287/comments","statuses_url":"https://api.github.com/repos/twilio/video-quickstart-swift/statuses/c7b57eb6036a20e0bb61d1e9a6050684dc6b19b1","head":{"label":"piyushtank:task/ISDK-1512-ReplayKitExample","ref":"task/ISDK-1512-ReplayKitExample","sha":"c7b57eb6036a20e0bb61d1e9a6050684dc6b19b1","user":{"login":"piyushtank","id":17037593,"node_id":"MDQ6VXNlcjE3MDM3NTkz","avatar_url":"https://avatars3.githubusercontent.com/u/17037593?v=4","gravatar_id":"","url":"https://api.github.com/users/piyushtank","html_url":"https://github.com/piyushtank","followers_url":"https://api.github.com/users/piyushtank/followers","following_url":"https://api.github.com/users/piyushtank/following{/other_user}","gists_url":"https://api.github.com/users/piyushtank/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushtank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushtank/subscriptions","organizations_url":"https://api.github.com/users/piyushtank/orgs","repos_url":"https://api.github.com/users/piyushtank/repos","events_url":"https://api.github.com/users/piyushtank/events{/privacy}","received_events_url":"https://api.github.com/users/piyushtank/received_events","type":"User","site_admin":false},"repo":{"id":50969677,"node_id":"MDEwOlJlcG9zaXRvcnk1MDk2OTY3Nw==","name":"video-quickstart-swift","full_name":"piyushtank/video-quickstart-swift","owner":{"login":"piyushtank","id":17037593,"node_id":"MDQ6VXNlcjE3MDM3NTkz","avatar_url":"https://avatars3.githubusercontent.com/u/17037593?v=4","gravatar_id":"","url":"https://api.github.com/users/piyushtank","html_url":"https://github.com/piyushtank","followers_url":"https://api.github.com/users/piyushtank/followers","following_url":"https://api.github.com/users/piyushtank/following{/other_user}","gists_url":"https://api.github.com/users/piyushtank/gists{/gist_id}","starred_url":"https://api.github.com/users/piyushtank/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/piyushtank/subscriptions","organizations_url":"https://api.github.com/users/piyushtank/orgs","repos_url":"https://api.github.com/users/piyushtank/repos","events_url":"https://api.github.com/users/piyushtank/events{/privacy}","received_events_url":"https://api.github.com/users/piyushtank/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/piyushtank/video-quickstart-swift","description":"Twilio Video (Conversations) Quickstart for iOS with Swift","fork":true,"url":"https://api.github.com/repos/piyushtank/video-quickstart-swift","forks_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/forks","keys_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/keys{/key_id}","collaborators_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/teams","hooks_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/hooks","issue_events_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/issues/events{/number}","events_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/events","assignees_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/assignees{/user}","branches_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/branches{/branch}","tags_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/tags","blobs_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/git/refs{/sha}","trees_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/git/trees{/sha}","statuses_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/statuses/{sha}","languages_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/languages","stargazers_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/stargazers","contributors_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/contributors","subscribers_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/subscribers","subscription_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/subscription","commits_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/commits{/sha}","git_commits_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/git/commits{/sha}","comments_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/comments{/number}","issue_comment_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/issues/comments{/number}","contents_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/contents/{+path}","compare_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/compare/{base}...{head}","merges_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/merges","archive_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/downloads","issues_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/issues{/number}","pulls_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/pulls{/number}","milestones_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/milestones{/number}","notifications_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/labels{/name}","releases_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/releases{/id}","deployments_url":"https://api.github.com/repos/piyushtank/video-quickstart-swift/deployments","created_at":"2016-02-03T02:59:25Z","updated_at":"2016-02-03T02:59:25Z","pushed_at":"2018-08-02T23:18:45Z","git_url":"git://github.com/piyushtank/video-quickstart-swift.git","ssh_url":"git@github.com:piyushtank/video-quickstart-swift.git","clone_url":"https://github.com/piyushtank/video-quickstart-swift.git","svn_url":"https://github.com/piyushtank/video-quickstart-swift","homepage":null,"size":11107,"stargazers_count":0,"watchers_count":0,"language":"Swift","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"twilio:2.3.0-preview","ref":"2.3.0-preview","sha":"55adde07d1b259555b5ef412a41cf02e9f72ef02","user":{"login":"twilio","id":109142,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwOTE0Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/109142?v=4","gravatar_id":"","url":"https://api.github.com/users/twilio","html_url":"https://github.com/twilio","followers_url":"https://api.github.com/users/twilio/followers","following_url":"https://api.github.com/users/twilio/following{/other_user}","gists_url":"https://api.github.com/users/twilio/gists{/gist_id}","starred_url":"https://api.github.com/users/twilio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twilio/subscriptions","organizations_url":"https://api.github.com/users/twilio/orgs","repos_url":"https://api.github.com/users/twilio/repos","events_url":"https://api.github.com/users/twilio/events{/privacy}","received_events_url":"https://api.github.com/users/twilio/received_events","type":"Organization","site_admin":false},"repo":{"id":48068060,"node_id":"MDEwOlJlcG9zaXRvcnk0ODA2ODA2MA==","name":"video-quickstart-swift","full_name":"twilio/video-quickstart-swift","owner":{"login":"twilio","id":109142,"node_id":"MDEyOk9yZ2FuaXphdGlvbjEwOTE0Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/109142?v=4","gravatar_id":"","url":"https://api.github.com/users/twilio","html_url":"https://github.com/twilio","followers_url":"https://api.github.com/users/twilio/followers","following_url":"https://api.github.com/users/twilio/following{/other_user}","gists_url":"https://api.github.com/users/twilio/gists{/gist_id}","starred_url":"https://api.github.com/users/twilio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twilio/subscriptions","organizations_url":"https://api.github.com/users/twilio/orgs","repos_url":"https://api.github.com/users/twilio/repos","events_url":"https://api.github.com/users/twilio/events{/privacy}","received_events_url":"https://api.github.com/users/twilio/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/twilio/video-quickstart-swift","description":"Twilio Video Quickstart for iOS with Swift","fork":false,"url":"https://api.github.com/repos/twilio/video-quickstart-swift","forks_url":"https://api.github.com/repos/twilio/video-quickstart-swift/forks","keys_url":"https://api.github.com/repos/twilio/video-quickstart-swift/keys{/key_id}","collaborators_url":"https://api.github.com/repos/twilio/video-quickstart-swift/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/twilio/video-quickstart-swift/teams","hooks_url":"https://api.github.com/repos/twilio/video-quickstart-swift/hooks","issue_events_url":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/events{/number}","events_url":"https://api.github.com/repos/twilio/video-quickstart-swift/events","assignees_url":"https://api.github.com/repos/twilio/video-quickstart-swift/assignees{/user}","branches_url":"https://api.github.com/repos/twilio/video-quickstart-swift/branches{/branch}","tags_url":"https://api.github.com/repos/twilio/video-quickstart-swift/tags","blobs_url":"https://api.github.com/repos/twilio/video-quickstart-swift/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/twilio/video-quickstart-swift/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/twilio/video-quickstart-swift/git/refs{/sha}","trees_url":"https://api.github.com/repos/twilio/video-quickstart-swift/git/trees{/sha}","statuses_url":"https://api.github.com/repos/twilio/video-quickstart-swift/statuses/{sha}","languages_url":"https://api.github.com/repos/twilio/video-quickstart-swift/languages","stargazers_url":"https://api.github.com/repos/twilio/video-quickstart-swift/stargazers","contributors_url":"https://api.github.com/repos/twilio/video-quickstart-swift/contributors","subscribers_url":"https://api.github.com/repos/twilio/video-quickstart-swift/subscribers","subscription_url":"https://api.github.com/repos/twilio/video-quickstart-swift/subscription","commits_url":"https://api.github.com/repos/twilio/video-quickstart-swift/commits{/sha}","git_commits_url":"https://api.github.com/repos/twilio/video-quickstart-swift/git/commits{/sha}","comments_url":"https://api.github.com/repos/twilio/video-quickstart-swift/comments{/number}","issue_comment_url":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/comments{/number}","contents_url":"https://api.github.com/repos/twilio/video-quickstart-swift/contents/{+path}","compare_url":"https://api.github.com/repos/twilio/video-quickstart-swift/compare/{base}...{head}","merges_url":"https://api.github.com/repos/twilio/video-quickstart-swift/merges","archive_url":"https://api.github.com/repos/twilio/video-quickstart-swift/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/twilio/video-quickstart-swift/downloads","issues_url":"https://api.github.com/repos/twilio/video-quickstart-swift/issues{/number}","pulls_url":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls{/number}","milestones_url":"https://api.github.com/repos/twilio/video-quickstart-swift/milestones{/number}","notifications_url":"https://api.github.com/repos/twilio/video-quickstart-swift/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/twilio/video-quickstart-swift/labels{/name}","releases_url":"https://api.github.com/repos/twilio/video-quickstart-swift/releases{/id}","deployments_url":"https://api.github.com/repos/twilio/video-quickstart-swift/deployments","created_at":"2015-12-15T20:46:04Z","updated_at":"2018-08-07T20:26:16Z","pushed_at":"2018-08-07T20:26:17Z","git_url":"git://github.com/twilio/video-quickstart-swift.git","ssh_url":"git@github.com:twilio/video-quickstart-swift.git","clone_url":"https://github.com/twilio/video-quickstart-swift.git","svn_url":"https://github.com/twilio/video-quickstart-swift","homepage":"https://www.twilio.com/docs/api/video","size":10948,"stargazers_count":137,"watchers_count":137,"language":"Swift","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":57,"mirror_url":null,"archived":false,"open_issues_count":42,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":57,"open_issues":42,"watchers":137,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287"},"html":{"href":"https://github.com/twilio/video-quickstart-swift/pull/287"},"issue":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/287"},"comments":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/issues/287/comments"},"review_comments":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287/comments"},"review_comment":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/pulls/287/commits"},"statuses":{"href":"https://api.github.com/repos/twilio/video-quickstart-swift/statuses/c7b57eb6036a20e0bb61d1e9a6050684dc6b19b1"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"ceaglest"}}