{"action":"created","comment":{"url":"https://api.github.com/repos/twitter/heron/pulls/comments/82291549","pull_request_review_id":3210558,"id":82291549,"diff_hunk":"@@ -0,0 +1,11 @@\n+# uploader class for transferring the topology jar/tar files to storage\n+heron.class.uploader:         com.twitter.heron.uploader.docker.DockerUploader\n+\n+#base image\n+heron.uploader.docker.base:                      ubuntu:latest","path":"heron/uploaders/src/java/com/twitter/heron/uploader/docker/sample.yaml","position":5,"original_position":5,"commit_id":"cc0fdb7958d1315e66f32842f3bd98c35c02300f","original_commit_id":"cc0fdb7958d1315e66f32842f3bd98c35c02300f","user":{"login":"chris-pardy","id":5663126,"avatar_url":"https://avatars.githubusercontent.com/u/5663126?v=3","gravatar_id":"","url":"https://api.github.com/users/chris-pardy","html_url":"https://github.com/chris-pardy","followers_url":"https://api.github.com/users/chris-pardy/followers","following_url":"https://api.github.com/users/chris-pardy/following{/other_user}","gists_url":"https://api.github.com/users/chris-pardy/gists{/gist_id}","starred_url":"https://api.github.com/users/chris-pardy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chris-pardy/subscriptions","organizations_url":"https://api.github.com/users/chris-pardy/orgs","repos_url":"https://api.github.com/users/chris-pardy/repos","events_url":"https://api.github.com/users/chris-pardy/events{/privacy}","received_events_url":"https://api.github.com/users/chris-pardy/received_events","type":"User","site_admin":false},"body":"I'm personally using a Docker base image with the binaries, but writing them into the sandbox should be fine. I'm going to verify this tonight.","created_at":"2016-10-06T21:40:15Z","updated_at":"2016-10-06T21:40:15Z","html_url":"https://github.com/twitter/heron/pull/1419#discussion_r82291549","pull_request_url":"https://api.github.com/repos/twitter/heron/pulls/1419","_links":{"self":{"href":"https://api.github.com/repos/twitter/heron/pulls/comments/82291549"},"html":{"href":"https://github.com/twitter/heron/pull/1419#discussion_r82291549"},"pull_request":{"href":"https://api.github.com/repos/twitter/heron/pulls/1419"}}},"pull_request":{"url":"https://api.github.com/repos/twitter/heron/pulls/1419","id":86534914,"html_url":"https://github.com/twitter/heron/pull/1419","diff_url":"https://github.com/twitter/heron/pull/1419.diff","patch_url":"https://github.com/twitter/heron/pull/1419.patch","issue_url":"https://api.github.com/repos/twitter/heron/issues/1419","number":1419,"state":"open","locked":false,"title":"Docker Uploader for Heron Topologies","user":{"login":"chris-pardy","id":5663126,"avatar_url":"https://avatars.githubusercontent.com/u/5663126?v=3","gravatar_id":"","url":"https://api.github.com/users/chris-pardy","html_url":"https://github.com/chris-pardy","followers_url":"https://api.github.com/users/chris-pardy/followers","following_url":"https://api.github.com/users/chris-pardy/following{/other_user}","gists_url":"https://api.github.com/users/chris-pardy/gists{/gist_id}","starred_url":"https://api.github.com/users/chris-pardy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chris-pardy/subscriptions","organizations_url":"https://api.github.com/users/chris-pardy/orgs","repos_url":"https://api.github.com/users/chris-pardy/repos","events_url":"https://api.github.com/users/chris-pardy/events{/privacy}","received_events_url":"https://api.github.com/users/chris-pardy/received_events","type":"User","site_admin":false},"body":"This adds support for uploading Topology definitions to a Docker repository by building them into a Docker Image. A base image can be specified in the config file, as well as a repository prefix for the image.\r\n\r\nThe DockerUploader creates a Docker image from a given base image (eg. ubuntu:trusty) and adds the topology definition and binary to the docker image, it then optionally uploads that image to a docker repository.\r\n\r\nThis allows schedulers downstream to deploy the generated docker image, or a new docker image based on the generated docker image. The purpose is to enable running Heron topologies within a Docker container. In practice this allows a correctly configured Aurora job to use the Docker container to setup the job sandbox. It could also serve as the basis for other schedulers that would like to schedule heron instances inside docker containers.\r\n\r\nAll docker functionality is provided by the Docker command line interface, so if Docker is not installed or configured uploading via the DockerUploader will fail.\r\n\r\nIn practice how we're using this:\r\n* We've pre-built a docker image from a minimal trusty + corporate stuff + heron core\r\n* When we \"upload\" topologies we create a new docker image from the heron core base that includes our topology definition, his is pushed into our private docker repository.\r\n* We use a custom Aurora job file that knows how to use leverage these docker containers to start the job.","created_at":"2016-09-23T15:40:39Z","updated_at":"2016-10-06T21:40:15Z","closed_at":null,"merged_at":null,"merge_commit_sha":"934ece0b059df10bc6fbb3fdc6a5928c3377361e","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/twitter/heron/pulls/1419/commits","review_comments_url":"https://api.github.com/repos/twitter/heron/pulls/1419/comments","review_comment_url":"https://api.github.com/repos/twitter/heron/pulls/comments{/number}","comments_url":"https://api.github.com/repos/twitter/heron/issues/1419/comments","statuses_url":"https://api.github.com/repos/twitter/heron/statuses/cc0fdb7958d1315e66f32842f3bd98c35c02300f","head":{"label":"chris-pardy:chris-pardy/docker-uploader","ref":"chris-pardy/docker-uploader","sha":"cc0fdb7958d1315e66f32842f3bd98c35c02300f","user":{"login":"chris-pardy","id":5663126,"avatar_url":"https://avatars.githubusercontent.com/u/5663126?v=3","gravatar_id":"","url":"https://api.github.com/users/chris-pardy","html_url":"https://github.com/chris-pardy","followers_url":"https://api.github.com/users/chris-pardy/followers","following_url":"https://api.github.com/users/chris-pardy/following{/other_user}","gists_url":"https://api.github.com/users/chris-pardy/gists{/gist_id}","starred_url":"https://api.github.com/users/chris-pardy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chris-pardy/subscriptions","organizations_url":"https://api.github.com/users/chris-pardy/orgs","repos_url":"https://api.github.com/users/chris-pardy/repos","events_url":"https://api.github.com/users/chris-pardy/events{/privacy}","received_events_url":"https://api.github.com/users/chris-pardy/received_events","type":"User","site_admin":false},"repo":{"id":68949292,"name":"heron","full_name":"chris-pardy/heron","owner":{"login":"chris-pardy","id":5663126,"avatar_url":"https://avatars.githubusercontent.com/u/5663126?v=3","gravatar_id":"","url":"https://api.github.com/users/chris-pardy","html_url":"https://github.com/chris-pardy","followers_url":"https://api.github.com/users/chris-pardy/followers","following_url":"https://api.github.com/users/chris-pardy/following{/other_user}","gists_url":"https://api.github.com/users/chris-pardy/gists{/gist_id}","starred_url":"https://api.github.com/users/chris-pardy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chris-pardy/subscriptions","organizations_url":"https://api.github.com/users/chris-pardy/orgs","repos_url":"https://api.github.com/users/chris-pardy/repos","events_url":"https://api.github.com/users/chris-pardy/events{/privacy}","received_events_url":"https://api.github.com/users/chris-pardy/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/chris-pardy/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":true,"url":"https://api.github.com/repos/chris-pardy/heron","forks_url":"https://api.github.com/repos/chris-pardy/heron/forks","keys_url":"https://api.github.com/repos/chris-pardy/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/chris-pardy/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/chris-pardy/heron/teams","hooks_url":"https://api.github.com/repos/chris-pardy/heron/hooks","issue_events_url":"https://api.github.com/repos/chris-pardy/heron/issues/events{/number}","events_url":"https://api.github.com/repos/chris-pardy/heron/events","assignees_url":"https://api.github.com/repos/chris-pardy/heron/assignees{/user}","branches_url":"https://api.github.com/repos/chris-pardy/heron/branches{/branch}","tags_url":"https://api.github.com/repos/chris-pardy/heron/tags","blobs_url":"https://api.github.com/repos/chris-pardy/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/chris-pardy/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/chris-pardy/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/chris-pardy/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/chris-pardy/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/chris-pardy/heron/languages","stargazers_url":"https://api.github.com/repos/chris-pardy/heron/stargazers","contributors_url":"https://api.github.com/repos/chris-pardy/heron/contributors","subscribers_url":"https://api.github.com/repos/chris-pardy/heron/subscribers","subscription_url":"https://api.github.com/repos/chris-pardy/heron/subscription","commits_url":"https://api.github.com/repos/chris-pardy/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/chris-pardy/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/chris-pardy/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/chris-pardy/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/chris-pardy/heron/contents/{+path}","compare_url":"https://api.github.com/repos/chris-pardy/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/chris-pardy/heron/merges","archive_url":"https://api.github.com/repos/chris-pardy/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/chris-pardy/heron/downloads","issues_url":"https://api.github.com/repos/chris-pardy/heron/issues{/number}","pulls_url":"https://api.github.com/repos/chris-pardy/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/chris-pardy/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/chris-pardy/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/chris-pardy/heron/labels{/name}","releases_url":"https://api.github.com/repos/chris-pardy/heron/releases{/id}","deployments_url":"https://api.github.com/repos/chris-pardy/heron/deployments","created_at":"2016-09-22T18:17:19Z","updated_at":"2016-09-22T18:17:23Z","pushed_at":"2016-10-06T13:45:47Z","git_url":"git://github.com/chris-pardy/heron.git","ssh_url":"git@github.com:chris-pardy/heron.git","clone_url":"https://github.com/chris-pardy/heron.git","svn_url":"https://github.com/chris-pardy/heron","homepage":"http://heronstreaming.io","size":120618,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"twitter:master","ref":"master","sha":"694c6e6bb02af0686657cbdb1ccbfd40f53d32d5","user":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"repo":{"id":43158694,"name":"heron","full_name":"twitter/heron","owner":{"login":"twitter","id":50278,"avatar_url":"https://avatars.githubusercontent.com/u/50278?v=3","gravatar_id":"","url":"https://api.github.com/users/twitter","html_url":"https://github.com/twitter","followers_url":"https://api.github.com/users/twitter/followers","following_url":"https://api.github.com/users/twitter/following{/other_user}","gists_url":"https://api.github.com/users/twitter/gists{/gist_id}","starred_url":"https://api.github.com/users/twitter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/twitter/subscriptions","organizations_url":"https://api.github.com/users/twitter/orgs","repos_url":"https://api.github.com/users/twitter/repos","events_url":"https://api.github.com/users/twitter/events{/privacy}","received_events_url":"https://api.github.com/users/twitter/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/twitter/heron","description":"Heron is a realtime, distributed, fault-tolerant stream processing engine from Twitter","fork":false,"url":"https://api.github.com/repos/twitter/heron","forks_url":"https://api.github.com/repos/twitter/heron/forks","keys_url":"https://api.github.com/repos/twitter/heron/keys{/key_id}","collaborators_url":"https://api.github.com/repos/twitter/heron/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/twitter/heron/teams","hooks_url":"https://api.github.com/repos/twitter/heron/hooks","issue_events_url":"https://api.github.com/repos/twitter/heron/issues/events{/number}","events_url":"https://api.github.com/repos/twitter/heron/events","assignees_url":"https://api.github.com/repos/twitter/heron/assignees{/user}","branches_url":"https://api.github.com/repos/twitter/heron/branches{/branch}","tags_url":"https://api.github.com/repos/twitter/heron/tags","blobs_url":"https://api.github.com/repos/twitter/heron/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/twitter/heron/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/twitter/heron/git/refs{/sha}","trees_url":"https://api.github.com/repos/twitter/heron/git/trees{/sha}","statuses_url":"https://api.github.com/repos/twitter/heron/statuses/{sha}","languages_url":"https://api.github.com/repos/twitter/heron/languages","stargazers_url":"https://api.github.com/repos/twitter/heron/stargazers","contributors_url":"https://api.github.com/repos/twitter/heron/contributors","subscribers_url":"https://api.github.com/repos/twitter/heron/subscribers","subscription_url":"https://api.github.com/repos/twitter/heron/subscription","commits_url":"https://api.github.com/repos/twitter/heron/commits{/sha}","git_commits_url":"https://api.github.com/repos/twitter/heron/git/commits{/sha}","comments_url":"https://api.github.com/repos/twitter/heron/comments{/number}","issue_comment_url":"https://api.github.com/repos/twitter/heron/issues/comments{/number}","contents_url":"https://api.github.com/repos/twitter/heron/contents/{+path}","compare_url":"https://api.github.com/repos/twitter/heron/compare/{base}...{head}","merges_url":"https://api.github.com/repos/twitter/heron/merges","archive_url":"https://api.github.com/repos/twitter/heron/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/twitter/heron/downloads","issues_url":"https://api.github.com/repos/twitter/heron/issues{/number}","pulls_url":"https://api.github.com/repos/twitter/heron/pulls{/number}","milestones_url":"https://api.github.com/repos/twitter/heron/milestones{/number}","notifications_url":"https://api.github.com/repos/twitter/heron/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/twitter/heron/labels{/name}","releases_url":"https://api.github.com/repos/twitter/heron/releases{/id}","deployments_url":"https://api.github.com/repos/twitter/heron/deployments","created_at":"2015-09-25T16:09:24Z","updated_at":"2016-10-06T16:26:28Z","pushed_at":"2016-10-06T21:12:49Z","git_url":"git://github.com/twitter/heron.git","ssh_url":"git@github.com:twitter/heron.git","clone_url":"https://github.com/twitter/heron.git","svn_url":"https://github.com/twitter/heron","homepage":"http://heronstreaming.io","size":147652,"stargazers_count":2406,"watchers_count":2406,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":401,"mirror_url":null,"open_issues_count":202,"forks":401,"open_issues":202,"watchers":2406,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/twitter/heron/pulls/1419"},"html":{"href":"https://github.com/twitter/heron/pull/1419"},"issue":{"href":"https://api.github.com/repos/twitter/heron/issues/1419"},"comments":{"href":"https://api.github.com/repos/twitter/heron/issues/1419/comments"},"review_comments":{"href":"https://api.github.com/repos/twitter/heron/pulls/1419/comments"},"review_comment":{"href":"https://api.github.com/repos/twitter/heron/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/twitter/heron/pulls/1419/commits"},"statuses":{"href":"https://api.github.com/repos/twitter/heron/statuses/cc0fdb7958d1315e66f32842f3bd98c35c02300f"}}}}
{"actor":{"display_login":"chris-pardy"}}