{"action":"created","comment":{"url":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/51512589","id":51512589,"diff_hunk":"@@ -81,52 +70,142 @@ func NewAttachManager(endpoint, certPath string, attachErrors chan<- error, name\n \t\tclient, err = docker.NewTLSClient(endpoint, cert, key, ca)\n \t}\n \n+\treturn client, err\n+}\n+\n+// Construct an AttachManager and set up the Docker Client\n+func NewAttachManager(endpoint string, certPath string, nameFromEnv string,\n+\tfieldsFromEnv []string, fieldsFromLabels []string) (*AttachManager, error) {\n+\n+\tclient, err := newDockerClient(certPath, endpoint)\n \tif err != nil {\n \t\treturn nil, err\n \t}\n \n \tm := &AttachManager{\n-\t\tattached:      make(map[string]*LogPump),\n-\t\tchannels:      make(map[chan *AttachEvent]struct{}),\n-\t\tclient:        client,\n-\t\terrors:        attachErrors,\n-\t\tevents:        make(chan *docker.APIEvents),\n-\t\tnameFromEnv:   nameFromEnv,\n-\t\tfieldsFromEnv: fieldsFromEnv,\n+\t\tclient:           client,\n+\t\tevents:           make(chan *docker.APIEvents),\n+\t\tnameFromEnv:      nameFromEnv,\n+\t\tfieldsFromEnv:    fieldsFromEnv,\n+\t\tfieldsFromLabels: fieldsFromLabels,\n \t}\n \n-\t// Attach to all currently running containers\n-\tif containers, err := client.ListContainers(docker.ListContainersOptions{}); err == nil {\n-\t\tfor _, listing := range containers {\n-\t\t\tm.attach(listing.ID[:12])\n+\treturn m, nil\n+}\n+\n+// Handler to wrap functions with retry logic\n+func withRetries(doWork func() error) error {","path":"plugins/docker/attacher.go","position":108,"original_position":116,"commit_id":"13faee9c03266286c3e4a9a2313ea368a360ac45","original_commit_id":"0e84d6c6531a219fa9f485686bd2da5e76416cfd","user":{"login":"relistan","id":267761,"avatar_url":"https://avatars.githubusercontent.com/u/267761?v=3","gravatar_id":"","url":"https://api.github.com/users/relistan","html_url":"https://github.com/relistan","followers_url":"https://api.github.com/users/relistan/followers","following_url":"https://api.github.com/users/relistan/following{/other_user}","gists_url":"https://api.github.com/users/relistan/gists{/gist_id}","starred_url":"https://api.github.com/users/relistan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/relistan/subscriptions","organizations_url":"https://api.github.com/users/relistan/orgs","repos_url":"https://api.github.com/users/relistan/repos","events_url":"https://api.github.com/users/relistan/events{/privacy}","received_events_url":"https://api.github.com/users/relistan/received_events","type":"User","site_admin":false},"body":"Done.","created_at":"2016-02-02T02:03:46Z","updated_at":"2016-02-02T02:03:46Z","html_url":"https://github.com/mozilla-services/heka/pull/1843#discussion_r51512589","pull_request_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1843","_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/comments/51512589"},"html":{"href":"https://github.com/mozilla-services/heka/pull/1843#discussion_r51512589"},"pull_request":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1843"}}},"pull_request":{"url":"https://api.github.com/repos/mozilla-services/heka/pulls/1843","id":57277124,"html_url":"https://github.com/mozilla-services/heka/pull/1843","diff_url":"https://github.com/mozilla-services/heka/pull/1843.diff","patch_url":"https://github.com/mozilla-services/heka/pull/1843.patch","issue_url":"https://api.github.com/repos/mozilla-services/heka/issues/1843","number":1843,"state":"open","locked":false,"title":"A major revamp of the DockerLogInput plugin","user":{"login":"relistan","id":267761,"avatar_url":"https://avatars.githubusercontent.com/u/267761?v=3","gravatar_id":"","url":"https://api.github.com/users/relistan","html_url":"https://github.com/relistan","followers_url":"https://api.github.com/users/relistan/followers","following_url":"https://api.github.com/users/relistan/following{/other_user}","gists_url":"https://api.github.com/users/relistan/gists{/gist_id}","starred_url":"https://api.github.com/users/relistan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/relistan/subscriptions","organizations_url":"https://api.github.com/users/relistan/orgs","repos_url":"https://api.github.com/users/relistan/repos","events_url":"https://api.github.com/users/relistan/events{/privacy}","received_events_url":"https://api.github.com/users/relistan/received_events","type":"User","site_admin":false},"body":"This PR is a general cleanup of the Docker input plugin to accomplish three major goals:\r\n\r\n1. Don't lose connections to Docker when the daemon is down for a period of time\r\n2. Support the splitter API in this plugin\r\n3. Log what is happening\r\n\r\nFrom running Heka with Docker for quite some time at my previous employer (New Relic) we saw that Heka would lose connections to the Docker daemon if it was down for a period of time (e.g. not a restart but a full stop for minute). This was because the re-connect strategy was to restart the plugin entirely and Heka would give up doing that after a short time. This pR takes a different approach and tries very hard to stay connected to the Docker daemon while also still exiting the plugin on total failure.\r\n\r\nSecondly a major drawback of the existing plugin is that lack of support for the new splitters means that you are stuck with with-by-line output from this Input. This PR supports splitters in the `DockerLogInput` plugin. At Nitro we're now using this with our `MultilineSplitter` implementation in #1842 to grab stacktraces that are output from apps running under Docker.\r\n\r\nFinally, without logging, it's really hard to tell what the plugin is seeing an not seeing. This adds some minimalist logging that seems to be enough to tell when things are attached properly or not.","created_at":"2016-01-26T21:42:36Z","updated_at":"2016-02-02T02:03:46Z","closed_at":null,"merged_at":null,"merge_commit_sha":"b8056d66a43c7a5555eb1c56abfe29ec964c25e0","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1843/commits","review_comments_url":"https://api.github.com/repos/mozilla-services/heka/pulls/1843/comments","review_comment_url":"https://api.github.com/repos/mozilla-services/heka/pulls/comments{/number}","comments_url":"https://api.github.com/repos/mozilla-services/heka/issues/1843/comments","statuses_url":"https://api.github.com/repos/mozilla-services/heka/statuses/13faee9c03266286c3e4a9a2313ea368a360ac45","head":{"label":"Nitro:pr-dockerinput-plugin","ref":"pr-dockerinput-plugin","sha":"13faee9c03266286c3e4a9a2313ea368a360ac45","user":{"login":"Nitro","id":1775750,"avatar_url":"https://avatars.githubusercontent.com/u/1775750?v=3","gravatar_id":"","url":"https://api.github.com/users/Nitro","html_url":"https://github.com/Nitro","followers_url":"https://api.github.com/users/Nitro/followers","following_url":"https://api.github.com/users/Nitro/following{/other_user}","gists_url":"https://api.github.com/users/Nitro/gists{/gist_id}","starred_url":"https://api.github.com/users/Nitro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nitro/subscriptions","organizations_url":"https://api.github.com/users/Nitro/orgs","repos_url":"https://api.github.com/users/Nitro/repos","events_url":"https://api.github.com/users/Nitro/events{/privacy}","received_events_url":"https://api.github.com/users/Nitro/received_events","type":"Organization","site_admin":false},"repo":{"id":49496209,"name":"heka","full_name":"Nitro/heka","owner":{"login":"Nitro","id":1775750,"avatar_url":"https://avatars.githubusercontent.com/u/1775750?v=3","gravatar_id":"","url":"https://api.github.com/users/Nitro","html_url":"https://github.com/Nitro","followers_url":"https://api.github.com/users/Nitro/followers","following_url":"https://api.github.com/users/Nitro/following{/other_user}","gists_url":"https://api.github.com/users/Nitro/gists{/gist_id}","starred_url":"https://api.github.com/users/Nitro/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Nitro/subscriptions","organizations_url":"https://api.github.com/users/Nitro/orgs","repos_url":"https://api.github.com/users/Nitro/repos","events_url":"https://api.github.com/users/Nitro/events{/privacy}","received_events_url":"https://api.github.com/users/Nitro/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/Nitro/heka","description":"Data collection and processing made easy.","fork":true,"url":"https://api.github.com/repos/Nitro/heka","forks_url":"https://api.github.com/repos/Nitro/heka/forks","keys_url":"https://api.github.com/repos/Nitro/heka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Nitro/heka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Nitro/heka/teams","hooks_url":"https://api.github.com/repos/Nitro/heka/hooks","issue_events_url":"https://api.github.com/repos/Nitro/heka/issues/events{/number}","events_url":"https://api.github.com/repos/Nitro/heka/events","assignees_url":"https://api.github.com/repos/Nitro/heka/assignees{/user}","branches_url":"https://api.github.com/repos/Nitro/heka/branches{/branch}","tags_url":"https://api.github.com/repos/Nitro/heka/tags","blobs_url":"https://api.github.com/repos/Nitro/heka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Nitro/heka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Nitro/heka/git/refs{/sha}","trees_url":"https://api.github.com/repos/Nitro/heka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Nitro/heka/statuses/{sha}","languages_url":"https://api.github.com/repos/Nitro/heka/languages","stargazers_url":"https://api.github.com/repos/Nitro/heka/stargazers","contributors_url":"https://api.github.com/repos/Nitro/heka/contributors","subscribers_url":"https://api.github.com/repos/Nitro/heka/subscribers","subscription_url":"https://api.github.com/repos/Nitro/heka/subscription","commits_url":"https://api.github.com/repos/Nitro/heka/commits{/sha}","git_commits_url":"https://api.github.com/repos/Nitro/heka/git/commits{/sha}","comments_url":"https://api.github.com/repos/Nitro/heka/comments{/number}","issue_comment_url":"https://api.github.com/repos/Nitro/heka/issues/comments{/number}","contents_url":"https://api.github.com/repos/Nitro/heka/contents/{+path}","compare_url":"https://api.github.com/repos/Nitro/heka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Nitro/heka/merges","archive_url":"https://api.github.com/repos/Nitro/heka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Nitro/heka/downloads","issues_url":"https://api.github.com/repos/Nitro/heka/issues{/number}","pulls_url":"https://api.github.com/repos/Nitro/heka/pulls{/number}","milestones_url":"https://api.github.com/repos/Nitro/heka/milestones{/number}","notifications_url":"https://api.github.com/repos/Nitro/heka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Nitro/heka/labels{/name}","releases_url":"https://api.github.com/repos/Nitro/heka/releases{/id}","deployments_url":"https://api.github.com/repos/Nitro/heka/deployments","created_at":"2016-01-12T11:36:00Z","updated_at":"2016-01-12T11:36:04Z","pushed_at":"2016-02-02T02:03:17Z","git_url":"git://github.com/Nitro/heka.git","ssh_url":"git@github.com:Nitro/heka.git","clone_url":"https://github.com/Nitro/heka.git","svn_url":"https://github.com/Nitro/heka","homepage":"http://hekad.readthedocs.org/","size":10987,"stargazers_count":0,"watchers_count":0,"language":"Go","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"dev"}},"base":{"label":"mozilla-services:dev","ref":"dev","sha":"59082a60fea58c54538fd885a3d21d4f1121e8bd","user":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"repo":{"id":6248212,"name":"heka","full_name":"mozilla-services/heka","owner":{"login":"mozilla-services","id":1066228,"avatar_url":"https://avatars.githubusercontent.com/u/1066228?v=3","gravatar_id":"","url":"https://api.github.com/users/mozilla-services","html_url":"https://github.com/mozilla-services","followers_url":"https://api.github.com/users/mozilla-services/followers","following_url":"https://api.github.com/users/mozilla-services/following{/other_user}","gists_url":"https://api.github.com/users/mozilla-services/gists{/gist_id}","starred_url":"https://api.github.com/users/mozilla-services/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mozilla-services/subscriptions","organizations_url":"https://api.github.com/users/mozilla-services/orgs","repos_url":"https://api.github.com/users/mozilla-services/repos","events_url":"https://api.github.com/users/mozilla-services/events{/privacy}","received_events_url":"https://api.github.com/users/mozilla-services/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/mozilla-services/heka","description":"Data collection and processing made easy.","fork":false,"url":"https://api.github.com/repos/mozilla-services/heka","forks_url":"https://api.github.com/repos/mozilla-services/heka/forks","keys_url":"https://api.github.com/repos/mozilla-services/heka/keys{/key_id}","collaborators_url":"https://api.github.com/repos/mozilla-services/heka/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/mozilla-services/heka/teams","hooks_url":"https://api.github.com/repos/mozilla-services/heka/hooks","issue_events_url":"https://api.github.com/repos/mozilla-services/heka/issues/events{/number}","events_url":"https://api.github.com/repos/mozilla-services/heka/events","assignees_url":"https://api.github.com/repos/mozilla-services/heka/assignees{/user}","branches_url":"https://api.github.com/repos/mozilla-services/heka/branches{/branch}","tags_url":"https://api.github.com/repos/mozilla-services/heka/tags","blobs_url":"https://api.github.com/repos/mozilla-services/heka/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/mozilla-services/heka/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/mozilla-services/heka/git/refs{/sha}","trees_url":"https://api.github.com/repos/mozilla-services/heka/git/trees{/sha}","statuses_url":"https://api.github.com/repos/mozilla-services/heka/statuses/{sha}","languages_url":"https://api.github.com/repos/mozilla-services/heka/languages","stargazers_url":"https://api.github.com/repos/mozilla-services/heka/stargazers","contributors_url":"https://api.github.com/repos/mozilla-services/heka/contributors","subscribers_url":"https://api.github.com/repos/mozilla-services/heka/subscribers","subscription_url":"https://api.github.com/repos/mozilla-services/heka/subscription","commits_url":"https://api.github.com/repos/mozilla-services/heka/commits{/sha}","git_commits_url":"https://api.github.com/repos/mozilla-services/heka/git/commits{/sha}","comments_url":"https://api.github.com/repos/mozilla-services/heka/comments{/number}","issue_comment_url":"https://api.github.com/repos/mozilla-services/heka/issues/comments{/number}","contents_url":"https://api.github.com/repos/mozilla-services/heka/contents/{+path}","compare_url":"https://api.github.com/repos/mozilla-services/heka/compare/{base}...{head}","merges_url":"https://api.github.com/repos/mozilla-services/heka/merges","archive_url":"https://api.github.com/repos/mozilla-services/heka/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/mozilla-services/heka/downloads","issues_url":"https://api.github.com/repos/mozilla-services/heka/issues{/number}","pulls_url":"https://api.github.com/repos/mozilla-services/heka/pulls{/number}","milestones_url":"https://api.github.com/repos/mozilla-services/heka/milestones{/number}","notifications_url":"https://api.github.com/repos/mozilla-services/heka/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/mozilla-services/heka/labels{/name}","releases_url":"https://api.github.com/repos/mozilla-services/heka/releases{/id}","deployments_url":"https://api.github.com/repos/mozilla-services/heka/deployments","created_at":"2012-10-16T17:20:51Z","updated_at":"2016-02-02T01:13:23Z","pushed_at":"2016-02-02T02:03:18Z","git_url":"git://github.com/mozilla-services/heka.git","ssh_url":"git@github.com:mozilla-services/heka.git","clone_url":"https://github.com/mozilla-services/heka.git","svn_url":"https://github.com/mozilla-services/heka","homepage":"http://hekad.readthedocs.org/","size":14301,"stargazers_count":2836,"watchers_count":2836,"language":"Go","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":433,"mirror_url":null,"open_issues_count":191,"forks":433,"open_issues":191,"watchers":2836,"default_branch":"dev"}},"_links":{"self":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1843"},"html":{"href":"https://github.com/mozilla-services/heka/pull/1843"},"issue":{"href":"https://api.github.com/repos/mozilla-services/heka/issues/1843"},"comments":{"href":"https://api.github.com/repos/mozilla-services/heka/issues/1843/comments"},"review_comments":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1843/comments"},"review_comment":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/mozilla-services/heka/pulls/1843/commits"},"statuses":{"href":"https://api.github.com/repos/mozilla-services/heka/statuses/13faee9c03266286c3e4a9a2313ea368a360ac45"}}}}