{"action":"created","comment":{"url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/comments/223161960","pull_request_review_id":162233171,"id":223161960,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIyMzE2MTk2MA==","diff_hunk":"@@ -350,10 +447,11 @@ func StartTranscodeServer(orch Orchestrator, bind string, mux *http.ServeMux, wo\n \n \tglog.Info(\"Listening for RPC on \", bind)\n \tsrv := http.Server{\n-\t\tAddr:         bind,\n-\t\tHandler:      &lp,\n-\t\tReadTimeout:  HTTPTimeout,\n-\t\tWriteTimeout: HTTPTimeout,\n+\t\tAddr:    bind,\n+\t\tHandler: &lp,\n+\t\t// XXX doesn't handle streaming RPC well; split remote transcoder RPC?\n+\t\t//ReadTimeout:  HTTPTimeout,\n+\t\t//WriteTimeout: HTTPTimeout,","path":"server/rpc.go","position":156,"original_position":156,"commit_id":"4198a5f6cf7d33daad847198fd5b29ffac0f3694","original_commit_id":"4198a5f6cf7d33daad847198fd5b29ffac0f3694","user":{"login":"j0sh","id":292510,"node_id":"MDQ6VXNlcjI5MjUxMA==","avatar_url":"https://avatars2.githubusercontent.com/u/292510?v=4","gravatar_id":"","url":"https://api.github.com/users/j0sh","html_url":"https://github.com/j0sh","followers_url":"https://api.github.com/users/j0sh/followers","following_url":"https://api.github.com/users/j0sh/following{/other_user}","gists_url":"https://api.github.com/users/j0sh/gists{/gist_id}","starred_url":"https://api.github.com/users/j0sh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/j0sh/subscriptions","organizations_url":"https://api.github.com/users/j0sh/orgs","repos_url":"https://api.github.com/users/j0sh/repos","events_url":"https://api.github.com/users/j0sh/events{/privacy}","received_events_url":"https://api.github.com/users/j0sh/received_events","type":"User","site_admin":false},"body":"Discussion around this issue: https://github.com/livepeer/go-livepeer/issues/576","created_at":"2018-10-05T23:46:52Z","updated_at":"2018-10-05T23:46:52Z","html_url":"https://github.com/livepeer/go-livepeer/pull/575#discussion_r223161960","pull_request_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/comments/223161960"},"html":{"href":"https://github.com/livepeer/go-livepeer/pull/575#discussion_r223161960"},"pull_request":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575"}}},"pull_request":{"url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575","id":220780744,"node_id":"MDExOlB1bGxSZXF1ZXN0MjIwNzgwNzQ0","html_url":"https://github.com/livepeer/go-livepeer/pull/575","diff_url":"https://github.com/livepeer/go-livepeer/pull/575.diff","patch_url":"https://github.com/livepeer/go-livepeer/pull/575.patch","issue_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/575","number":575,"state":"open","locked":false,"title":"WIP: Split orchestrator and transcoder","user":{"login":"j0sh","id":292510,"node_id":"MDQ6VXNlcjI5MjUxMA==","avatar_url":"https://avatars2.githubusercontent.com/u/292510?v=4","gravatar_id":"","url":"https://api.github.com/users/j0sh","html_url":"https://github.com/j0sh","followers_url":"https://api.github.com/users/j0sh/followers","following_url":"https://api.github.com/users/j0sh/following{/other_user}","gists_url":"https://api.github.com/users/j0sh/gists{/gist_id}","starred_url":"https://api.github.com/users/j0sh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/j0sh/subscriptions","organizations_url":"https://api.github.com/users/j0sh/orgs","repos_url":"https://api.github.com/users/j0sh/repos","events_url":"https://api.github.com/users/j0sh/events{/privacy}","received_events_url":"https://api.github.com/users/j0sh/received_events","type":"User","site_admin":false},"body":"The mechanics of the split are as follows:\r\n\r\nAdd a `Transcoder` interface, along with concrete implementations of `LocalTranscoder` and `RemoteTranscoder` . \r\n\r\n```golang\r\ntype Transcoder interface {\r\n\tTranscode(fname string, profiles []ffmpeg.VideoProfile) ([][]byte, error)\r\n}\r\n```\r\nEach orchestrator maintains a list of transcoders. The local transcoder can be included in this list. Whether the local transcoder is included depends on whether the orchestrator uses the `-transcoder` flag (which enables local transcoding) or the `-orchestrator` flag (which disables local transcoding). Discussion of the flags happened [here](https://github.com/livepeer/go-livepeer/issues/567).\r\n\r\nCurrently, only the first transcoder in the list is used. To test remote transcoders, run the orchestrator in `-orchestrator` mode and connect a transcoder by starting another Livepeer node with `-orchAddr` .\r\n\r\nThe networking follows the \"Orchestrator-Transcoder Network Flow\" as described [here]( https://gist.github.com/j0sh/7cfb580cdacfe44700db4e6315a46491#orchestrator-transcoder-network-flow) with only minor changes to the message definitions.\r\n\r\nRemote transcoders are asynchronously assigned \"tasks\" via streaming RPC. Results are POST'd directly via HTTP multipart along with the the task ID. Notification from HTTP to the transcode loop is done via channels, which unblocks the pending segment in the transcode loop. Remote transcoders have a timeout of 8 seconds before cancellation, which partially mitigates https://github.com/livepeer/go-livepeer/issues/570 .\r\n\r\nFurther TODOs (may be split into separate PRs or done later)\r\n\r\n- [ ] Fix existing tests\r\n- [ ] Add new tests\r\n- [ ] Fallbacks in case a transcoder fails\r\n- [ ] Send each profile to a separate transcoder\r\n- [ ] Transcoder rotation, LRU style\r\n- [ ] Reconnect remote transcoder if connection drops\r\n- [ ] Graceful shutdown of transcoder-only nodes (exit listening loop)\r\n- [ ] Disable geth connectivity for transcoder-only nodes\r\n- [ ] Capacity limits (eg, N concurrent transcodes)\r\n- [ ] Write transcoder info to orchestrator DB, incl. remote IP","created_at":"2018-10-05T18:04:23Z","updated_at":"2018-10-05T23:46:52Z","closed_at":null,"merged_at":null,"merge_commit_sha":"85bf8d1e4bf6765a519da5988337920e516ff0ea","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575/commits","review_comments_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575/comments","review_comment_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls/comments{/number}","comments_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/575/comments","statuses_url":"https://api.github.com/repos/livepeer/go-livepeer/statuses/4198a5f6cf7d33daad847198fd5b29ffac0f3694","head":{"label":"livepeer:ja/otsplit","ref":"ja/otsplit","sha":"4198a5f6cf7d33daad847198fd5b29ffac0f3694","user":{"login":"livepeer","id":25355022,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MzU1MDIy","avatar_url":"https://avatars2.githubusercontent.com/u/25355022?v=4","gravatar_id":"","url":"https://api.github.com/users/livepeer","html_url":"https://github.com/livepeer","followers_url":"https://api.github.com/users/livepeer/followers","following_url":"https://api.github.com/users/livepeer/following{/other_user}","gists_url":"https://api.github.com/users/livepeer/gists{/gist_id}","starred_url":"https://api.github.com/users/livepeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/livepeer/subscriptions","organizations_url":"https://api.github.com/users/livepeer/orgs","repos_url":"https://api.github.com/users/livepeer/repos","events_url":"https://api.github.com/users/livepeer/events{/privacy}","received_events_url":"https://api.github.com/users/livepeer/received_events","type":"Organization","site_admin":false},"repo":{"id":94728962,"node_id":"MDEwOlJlcG9zaXRvcnk5NDcyODk2Mg==","name":"go-livepeer","full_name":"livepeer/go-livepeer","private":false,"owner":{"login":"livepeer","id":25355022,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MzU1MDIy","avatar_url":"https://avatars2.githubusercontent.com/u/25355022?v=4","gravatar_id":"","url":"https://api.github.com/users/livepeer","html_url":"https://github.com/livepeer","followers_url":"https://api.github.com/users/livepeer/followers","following_url":"https://api.github.com/users/livepeer/following{/other_user}","gists_url":"https://api.github.com/users/livepeer/gists{/gist_id}","starred_url":"https://api.github.com/users/livepeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/livepeer/subscriptions","organizations_url":"https://api.github.com/users/livepeer/orgs","repos_url":"https://api.github.com/users/livepeer/repos","events_url":"https://api.github.com/users/livepeer/events{/privacy}","received_events_url":"https://api.github.com/users/livepeer/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/livepeer/go-livepeer","description":"Official Go implementation of the Livepeer protocol","fork":false,"url":"https://api.github.com/repos/livepeer/go-livepeer","forks_url":"https://api.github.com/repos/livepeer/go-livepeer/forks","keys_url":"https://api.github.com/repos/livepeer/go-livepeer/keys{/key_id}","collaborators_url":"https://api.github.com/repos/livepeer/go-livepeer/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/livepeer/go-livepeer/teams","hooks_url":"https://api.github.com/repos/livepeer/go-livepeer/hooks","issue_events_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/events{/number}","events_url":"https://api.github.com/repos/livepeer/go-livepeer/events","assignees_url":"https://api.github.com/repos/livepeer/go-livepeer/assignees{/user}","branches_url":"https://api.github.com/repos/livepeer/go-livepeer/branches{/branch}","tags_url":"https://api.github.com/repos/livepeer/go-livepeer/tags","blobs_url":"https://api.github.com/repos/livepeer/go-livepeer/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/livepeer/go-livepeer/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/livepeer/go-livepeer/git/refs{/sha}","trees_url":"https://api.github.com/repos/livepeer/go-livepeer/git/trees{/sha}","statuses_url":"https://api.github.com/repos/livepeer/go-livepeer/statuses/{sha}","languages_url":"https://api.github.com/repos/livepeer/go-livepeer/languages","stargazers_url":"https://api.github.com/repos/livepeer/go-livepeer/stargazers","contributors_url":"https://api.github.com/repos/livepeer/go-livepeer/contributors","subscribers_url":"https://api.github.com/repos/livepeer/go-livepeer/subscribers","subscription_url":"https://api.github.com/repos/livepeer/go-livepeer/subscription","commits_url":"https://api.github.com/repos/livepeer/go-livepeer/commits{/sha}","git_commits_url":"https://api.github.com/repos/livepeer/go-livepeer/git/commits{/sha}","comments_url":"https://api.github.com/repos/livepeer/go-livepeer/comments{/number}","issue_comment_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/comments{/number}","contents_url":"https://api.github.com/repos/livepeer/go-livepeer/contents/{+path}","compare_url":"https://api.github.com/repos/livepeer/go-livepeer/compare/{base}...{head}","merges_url":"https://api.github.com/repos/livepeer/go-livepeer/merges","archive_url":"https://api.github.com/repos/livepeer/go-livepeer/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/livepeer/go-livepeer/downloads","issues_url":"https://api.github.com/repos/livepeer/go-livepeer/issues{/number}","pulls_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls{/number}","milestones_url":"https://api.github.com/repos/livepeer/go-livepeer/milestones{/number}","notifications_url":"https://api.github.com/repos/livepeer/go-livepeer/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/livepeer/go-livepeer/labels{/name}","releases_url":"https://api.github.com/repos/livepeer/go-livepeer/releases{/id}","deployments_url":"https://api.github.com/repos/livepeer/go-livepeer/deployments","created_at":"2017-06-19T02:36:09Z","updated_at":"2018-10-04T18:14:41Z","pushed_at":"2018-10-05T23:40:43Z","git_url":"git://github.com/livepeer/go-livepeer.git","ssh_url":"git@github.com:livepeer/go-livepeer.git","clone_url":"https://github.com/livepeer/go-livepeer.git","svn_url":"https://github.com/livepeer/go-livepeer","homepage":"http://livepeer.org","size":88811,"stargazers_count":161,"watchers_count":161,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":31,"mirror_url":null,"archived":false,"open_issues_count":108,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":31,"open_issues":108,"watchers":161,"default_branch":"master"}},"base":{"label":"livepeer:master","ref":"master","sha":"cd4fe0d650a3e7a36bfdb59a51c8d9465b2bea64","user":{"login":"livepeer","id":25355022,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MzU1MDIy","avatar_url":"https://avatars2.githubusercontent.com/u/25355022?v=4","gravatar_id":"","url":"https://api.github.com/users/livepeer","html_url":"https://github.com/livepeer","followers_url":"https://api.github.com/users/livepeer/followers","following_url":"https://api.github.com/users/livepeer/following{/other_user}","gists_url":"https://api.github.com/users/livepeer/gists{/gist_id}","starred_url":"https://api.github.com/users/livepeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/livepeer/subscriptions","organizations_url":"https://api.github.com/users/livepeer/orgs","repos_url":"https://api.github.com/users/livepeer/repos","events_url":"https://api.github.com/users/livepeer/events{/privacy}","received_events_url":"https://api.github.com/users/livepeer/received_events","type":"Organization","site_admin":false},"repo":{"id":94728962,"node_id":"MDEwOlJlcG9zaXRvcnk5NDcyODk2Mg==","name":"go-livepeer","full_name":"livepeer/go-livepeer","private":false,"owner":{"login":"livepeer","id":25355022,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MzU1MDIy","avatar_url":"https://avatars2.githubusercontent.com/u/25355022?v=4","gravatar_id":"","url":"https://api.github.com/users/livepeer","html_url":"https://github.com/livepeer","followers_url":"https://api.github.com/users/livepeer/followers","following_url":"https://api.github.com/users/livepeer/following{/other_user}","gists_url":"https://api.github.com/users/livepeer/gists{/gist_id}","starred_url":"https://api.github.com/users/livepeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/livepeer/subscriptions","organizations_url":"https://api.github.com/users/livepeer/orgs","repos_url":"https://api.github.com/users/livepeer/repos","events_url":"https://api.github.com/users/livepeer/events{/privacy}","received_events_url":"https://api.github.com/users/livepeer/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/livepeer/go-livepeer","description":"Official Go implementation of the Livepeer protocol","fork":false,"url":"https://api.github.com/repos/livepeer/go-livepeer","forks_url":"https://api.github.com/repos/livepeer/go-livepeer/forks","keys_url":"https://api.github.com/repos/livepeer/go-livepeer/keys{/key_id}","collaborators_url":"https://api.github.com/repos/livepeer/go-livepeer/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/livepeer/go-livepeer/teams","hooks_url":"https://api.github.com/repos/livepeer/go-livepeer/hooks","issue_events_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/events{/number}","events_url":"https://api.github.com/repos/livepeer/go-livepeer/events","assignees_url":"https://api.github.com/repos/livepeer/go-livepeer/assignees{/user}","branches_url":"https://api.github.com/repos/livepeer/go-livepeer/branches{/branch}","tags_url":"https://api.github.com/repos/livepeer/go-livepeer/tags","blobs_url":"https://api.github.com/repos/livepeer/go-livepeer/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/livepeer/go-livepeer/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/livepeer/go-livepeer/git/refs{/sha}","trees_url":"https://api.github.com/repos/livepeer/go-livepeer/git/trees{/sha}","statuses_url":"https://api.github.com/repos/livepeer/go-livepeer/statuses/{sha}","languages_url":"https://api.github.com/repos/livepeer/go-livepeer/languages","stargazers_url":"https://api.github.com/repos/livepeer/go-livepeer/stargazers","contributors_url":"https://api.github.com/repos/livepeer/go-livepeer/contributors","subscribers_url":"https://api.github.com/repos/livepeer/go-livepeer/subscribers","subscription_url":"https://api.github.com/repos/livepeer/go-livepeer/subscription","commits_url":"https://api.github.com/repos/livepeer/go-livepeer/commits{/sha}","git_commits_url":"https://api.github.com/repos/livepeer/go-livepeer/git/commits{/sha}","comments_url":"https://api.github.com/repos/livepeer/go-livepeer/comments{/number}","issue_comment_url":"https://api.github.com/repos/livepeer/go-livepeer/issues/comments{/number}","contents_url":"https://api.github.com/repos/livepeer/go-livepeer/contents/{+path}","compare_url":"https://api.github.com/repos/livepeer/go-livepeer/compare/{base}...{head}","merges_url":"https://api.github.com/repos/livepeer/go-livepeer/merges","archive_url":"https://api.github.com/repos/livepeer/go-livepeer/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/livepeer/go-livepeer/downloads","issues_url":"https://api.github.com/repos/livepeer/go-livepeer/issues{/number}","pulls_url":"https://api.github.com/repos/livepeer/go-livepeer/pulls{/number}","milestones_url":"https://api.github.com/repos/livepeer/go-livepeer/milestones{/number}","notifications_url":"https://api.github.com/repos/livepeer/go-livepeer/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/livepeer/go-livepeer/labels{/name}","releases_url":"https://api.github.com/repos/livepeer/go-livepeer/releases{/id}","deployments_url":"https://api.github.com/repos/livepeer/go-livepeer/deployments","created_at":"2017-06-19T02:36:09Z","updated_at":"2018-10-04T18:14:41Z","pushed_at":"2018-10-05T23:40:43Z","git_url":"git://github.com/livepeer/go-livepeer.git","ssh_url":"git@github.com:livepeer/go-livepeer.git","clone_url":"https://github.com/livepeer/go-livepeer.git","svn_url":"https://github.com/livepeer/go-livepeer","homepage":"http://livepeer.org","size":88811,"stargazers_count":161,"watchers_count":161,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":31,"mirror_url":null,"archived":false,"open_issues_count":108,"license":{"key":"mit","name":"MIT License","spdx_id":"MIT","url":"https://api.github.com/licenses/mit","node_id":"MDc6TGljZW5zZTEz"},"forks":31,"open_issues":108,"watchers":161,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575"},"html":{"href":"https://github.com/livepeer/go-livepeer/pull/575"},"issue":{"href":"https://api.github.com/repos/livepeer/go-livepeer/issues/575"},"comments":{"href":"https://api.github.com/repos/livepeer/go-livepeer/issues/575/comments"},"review_comments":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575/comments"},"review_comment":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/livepeer/go-livepeer/pulls/575/commits"},"statuses":{"href":"https://api.github.com/repos/livepeer/go-livepeer/statuses/4198a5f6cf7d33daad847198fd5b29ffac0f3694"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"j0sh"}}