{"action":"created","comment":{"url":"https://api.github.com/repos/cilium/cilium/pulls/comments/187731864","pull_request_review_id":119578579,"id":187731864,"diff_hunk":"@@ -1771,13 +1771,9 @@ func (e *Endpoint) LeaveLocked(owner Owner) []error {\n \terrors := []error{}\n \n \towner.RemoveFromEndpointQueue(uint64(e.ID))\n-\tif c := e.Consumable; c != nil {\n-\t\tc.Mutex.Lock()\n-\t\tif e.RealizedL4Policy != nil {\n-\t\t\t// Passing a new map of nil will purge all redirects\n-\t\t\te.removeOldRedirects(owner, nil)\n-\t\t}\n-\t\tc.Mutex.Unlock()\n+\tif e.SecurityIdentity != nil && e.RealizedL4Policy != nil {","path":"pkg/endpoint/endpoint.go","position":185,"original_position":11,"commit_id":"bd96b28c37437c85b069459f1c6df57db2f5d7d2","original_commit_id":"26c4fd0623ed35dc2de17007a3b4e1752fff7fd6","user":{"login":"ianvernon","id":3858271,"avatar_url":"https://avatars1.githubusercontent.com/u/3858271?v=4","gravatar_id":"","url":"https://api.github.com/users/ianvernon","html_url":"https://github.com/ianvernon","followers_url":"https://api.github.com/users/ianvernon/followers","following_url":"https://api.github.com/users/ianvernon/following{/other_user}","gists_url":"https://api.github.com/users/ianvernon/gists{/gist_id}","starred_url":"https://api.github.com/users/ianvernon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ianvernon/subscriptions","organizations_url":"https://api.github.com/users/ianvernon/orgs","repos_url":"https://api.github.com/users/ianvernon/repos","events_url":"https://api.github.com/users/ianvernon/events{/privacy}","received_events_url":"https://api.github.com/users/ianvernon/received_events","type":"User","site_admin":false},"body":"Created https://github.com/cilium/cilium/issues/4077 ","created_at":"2018-05-11T21:00:03Z","updated_at":"2018-05-11T21:00:03Z","html_url":"https://github.com/cilium/cilium/pull/3965#discussion_r187731864","pull_request_url":"https://api.github.com/repos/cilium/cilium/pulls/3965","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/cilium/cilium/pulls/comments/187731864"},"html":{"href":"https://github.com/cilium/cilium/pull/3965#discussion_r187731864"},"pull_request":{"href":"https://api.github.com/repos/cilium/cilium/pulls/3965"}},"in_reply_to_id":187569917},"pull_request":{"url":"https://api.github.com/repos/cilium/cilium/pulls/3965","id":185086774,"html_url":"https://github.com/cilium/cilium/pull/3965","diff_url":"https://github.com/cilium/cilium/pull/3965.diff","patch_url":"https://github.com/cilium/cilium/pull/3965.patch","issue_url":"https://api.github.com/repos/cilium/cilium/issues/3965","number":3965,"state":"open","locked":false,"title":"policy: factor out endpoint PolicyMap updates into controller and remove Consumable","user":{"login":"ianvernon","id":3858271,"avatar_url":"https://avatars1.githubusercontent.com/u/3858271?v=4","gravatar_id":"","url":"https://api.github.com/users/ianvernon","html_url":"https://github.com/ianvernon","followers_url":"https://api.github.com/users/ianvernon/followers","following_url":"https://api.github.com/users/ianvernon/following{/other_user}","gists_url":"https://api.github.com/users/ianvernon/gists{/gist_id}","starred_url":"https://api.github.com/users/ianvernon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ianvernon/subscriptions","organizations_url":"https://api.github.com/users/ianvernon/orgs","repos_url":"https://api.github.com/users/ianvernon/repos","events_url":"https://api.github.com/users/ianvernon/events{/privacy}","received_events_url":"https://api.github.com/users/ianvernon/received_events","type":"User","site_admin":false},"body":"Before this change, PolicyMap updates were done alongside policy regeneration\r\nfor a given endpoint. When a PolicyMap update failed, an error was returned.\r\nPolicyMap cleanup / updating was never done until the next call to regenerate\r\npolicy for an endpoint. This change now refactors the code to generate the\r\ndesired set of PolicyMap keys for the given endpoint when policy is generated.\r\nThis desired set of PolicyMap keys is now contained within the Endpoint\r\nstructure itself, along with a set of realized keys, or keys that have been\r\nsuccessfully added into the endpoint's PolicyMap. Once policy is recomputed for\r\nan endpoint, a controller is spawned for that endpoint which periodically tries\r\nto ensure that the state of the endpoint's PolicyMap is in sync with the set of\r\ndesired PolicyMap keys. Any keys which are not present in this set of keys are\r\nattempted to be removed from the map. To move to this controller-based PolicyMap\r\nsynchronization, all PolicyMap-related data, apart from L4Policy, is moved out\r\nof the Consumable type, and is now taken care of on a per-endpoint basis within\r\nthe aforementioned controller. This goes in line with the effort to move away\r\nfrom the Consumable, and outlines a future where datapath updates can be\r\nseparate from the policy package all-together. \r\n    \r\nWith regard to the actual function which synchronizes with the datapath, it is\r\nalso called synchronously during regenerateBPF, so that the program for an\r\nendpoint is not regenerated if any map update fails. The controller which invokes\r\nthis function is still instantiated in `regeneratePolicy` for an endpoint so that the\r\ndesired state is attempted to be realized for the endpoint with respect to the\r\npolicy maps. This maintains the same behavior that has previously existed for\r\nregenerating an endpoint's program. This does mean however, that the program\r\nfor an endpoint could  not be installed / regenerated even though the PolicyMap\r\nfor the endpoint itself is updated. A logical next step to fixing this issue is https://github.com/cilium/cilium/issues/3897,\r\nwhich will allow for a more transactional approach to updating the PolicyMap\r\nfor a given endpoint. \r\n\r\nFurthermore, this change does have some impact on the CLI commands \r\n`cilium bpf policy {add, delete}` because if a user adds or deletes an entry\r\nin an endpoint's BPF policy map, and that entry is not part of the desired state\r\nfor said endpoint as determined by the next policy computation, then the entry\r\nwill be evicted from, or re-added to, the PolicyMap.\r\n\r\nGiven that PolicyMap updates are now handled on a per-endpoint basis, this \r\nresulted in gutting the Consumable structure for the endpoint. As a result,\r\nall realized / desired policy for an endpoint is now stored within each endpoint\r\nstructure itself as opposed to a shared structure for endpoints with the same\r\nsecurity identity on the same node. The Consumable type is removed as it is\r\nno longer used in any part of Cilium. This paves the way for an iterative refactoring\r\nof how policy is computed in Cilium for the future to make it more easily testable\r\nand modular.\r\n\r\nFinally, this patch contains some miscellaneous fixes and cleanups found in \r\nthe process of doing this change.\r\n    \r\nSigned-off by: Ian Vernon <ian@cilium.io>\r\n\r\nRelates-to: #3161 ","created_at":"2018-04-30T23:04:54Z","updated_at":"2018-05-11T21:00:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"fca6c7ebfbaf8a0d0d091c4e37e4d46565edc763","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[{"name":"ci","id":2519781,"slug":"ci","description":"CI integration","privacy":"closed","url":"https://api.github.com/teams/2519781","members_url":"https://api.github.com/teams/2519781/members{/member}","repositories_url":"https://api.github.com/teams/2519781/repos","permission":"pull"},{"name":"janitors","id":2519796,"slug":"janitors","description":"Rotating team of janitors for unowned code","privacy":"closed","url":"https://api.github.com/teams/2519796","members_url":"https://api.github.com/teams/2519796/members{/member}","repositories_url":"https://api.github.com/teams/2519796/repos","permission":"pull"}],"labels":[{"id":746643464,"url":"https://api.github.com/repos/cilium/cilium/labels/area/policy","name":"area/policy","color":"c9f97a","default":false},{"id":300384441,"url":"https://api.github.com/repos/cilium/cilium/labels/kind/enhancement","name":"kind/enhancement","color":"84b6eb","default":false},{"id":403149902,"url":"https://api.github.com/repos/cilium/cilium/labels/pending-review","name":"pending-review","color":"fbca04","default":false},{"id":755969022,"url":"https://api.github.com/repos/cilium/cilium/labels/requires-janitor-review","name":"requires-janitor-review","color":"fbca04","default":false}],"milestone":null,"commits_url":"https://api.github.com/repos/cilium/cilium/pulls/3965/commits","review_comments_url":"https://api.github.com/repos/cilium/cilium/pulls/3965/comments","review_comment_url":"https://api.github.com/repos/cilium/cilium/pulls/comments{/number}","comments_url":"https://api.github.com/repos/cilium/cilium/issues/3965/comments","statuses_url":"https://api.github.com/repos/cilium/cilium/statuses/bd96b28c37437c85b069459f1c6df57db2f5d7d2","head":{"label":"cilium:remove-l3-only-identity-maps-to-endpoints","ref":"remove-l3-only-identity-maps-to-endpoints","sha":"bd96b28c37437c85b069459f1c6df57db2f5d7d2","user":{"login":"cilium","id":21054566,"avatar_url":"https://avatars0.githubusercontent.com/u/21054566?v=4","gravatar_id":"","url":"https://api.github.com/users/cilium","html_url":"https://github.com/cilium","followers_url":"https://api.github.com/users/cilium/followers","following_url":"https://api.github.com/users/cilium/following{/other_user}","gists_url":"https://api.github.com/users/cilium/gists{/gist_id}","starred_url":"https://api.github.com/users/cilium/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cilium/subscriptions","organizations_url":"https://api.github.com/users/cilium/orgs","repos_url":"https://api.github.com/users/cilium/repos","events_url":"https://api.github.com/users/cilium/events{/privacy}","received_events_url":"https://api.github.com/users/cilium/received_events","type":"Organization","site_admin":false},"repo":{"id":48109239,"name":"cilium","full_name":"cilium/cilium","owner":{"login":"cilium","id":21054566,"avatar_url":"https://avatars0.githubusercontent.com/u/21054566?v=4","gravatar_id":"","url":"https://api.github.com/users/cilium","html_url":"https://github.com/cilium","followers_url":"https://api.github.com/users/cilium/followers","following_url":"https://api.github.com/users/cilium/following{/other_user}","gists_url":"https://api.github.com/users/cilium/gists{/gist_id}","starred_url":"https://api.github.com/users/cilium/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cilium/subscriptions","organizations_url":"https://api.github.com/users/cilium/orgs","repos_url":"https://api.github.com/users/cilium/repos","events_url":"https://api.github.com/users/cilium/events{/privacy}","received_events_url":"https://api.github.com/users/cilium/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/cilium/cilium","description":"HTTP, gRPC, and Kafka Aware Security and Networking for Containers with BPF and XDP","fork":false,"url":"https://api.github.com/repos/cilium/cilium","forks_url":"https://api.github.com/repos/cilium/cilium/forks","keys_url":"https://api.github.com/repos/cilium/cilium/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cilium/cilium/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cilium/cilium/teams","hooks_url":"https://api.github.com/repos/cilium/cilium/hooks","issue_events_url":"https://api.github.com/repos/cilium/cilium/issues/events{/number}","events_url":"https://api.github.com/repos/cilium/cilium/events","assignees_url":"https://api.github.com/repos/cilium/cilium/assignees{/user}","branches_url":"https://api.github.com/repos/cilium/cilium/branches{/branch}","tags_url":"https://api.github.com/repos/cilium/cilium/tags","blobs_url":"https://api.github.com/repos/cilium/cilium/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cilium/cilium/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cilium/cilium/git/refs{/sha}","trees_url":"https://api.github.com/repos/cilium/cilium/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cilium/cilium/statuses/{sha}","languages_url":"https://api.github.com/repos/cilium/cilium/languages","stargazers_url":"https://api.github.com/repos/cilium/cilium/stargazers","contributors_url":"https://api.github.com/repos/cilium/cilium/contributors","subscribers_url":"https://api.github.com/repos/cilium/cilium/subscribers","subscription_url":"https://api.github.com/repos/cilium/cilium/subscription","commits_url":"https://api.github.com/repos/cilium/cilium/commits{/sha}","git_commits_url":"https://api.github.com/repos/cilium/cilium/git/commits{/sha}","comments_url":"https://api.github.com/repos/cilium/cilium/comments{/number}","issue_comment_url":"https://api.github.com/repos/cilium/cilium/issues/comments{/number}","contents_url":"https://api.github.com/repos/cilium/cilium/contents/{+path}","compare_url":"https://api.github.com/repos/cilium/cilium/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cilium/cilium/merges","archive_url":"https://api.github.com/repos/cilium/cilium/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cilium/cilium/downloads","issues_url":"https://api.github.com/repos/cilium/cilium/issues{/number}","pulls_url":"https://api.github.com/repos/cilium/cilium/pulls{/number}","milestones_url":"https://api.github.com/repos/cilium/cilium/milestones{/number}","notifications_url":"https://api.github.com/repos/cilium/cilium/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cilium/cilium/labels{/name}","releases_url":"https://api.github.com/repos/cilium/cilium/releases{/id}","deployments_url":"https://api.github.com/repos/cilium/cilium/deployments","created_at":"2015-12-16T12:33:31Z","updated_at":"2018-05-11T20:39:28Z","pushed_at":"2018-05-11T20:39:18Z","git_url":"git://github.com/cilium/cilium.git","ssh_url":"git@github.com:cilium/cilium.git","clone_url":"https://github.com/cilium/cilium.git","svn_url":"https://github.com/cilium/cilium","homepage":"","size":48659,"stargazers_count":2225,"watchers_count":2225,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":195,"mirror_url":null,"archived":false,"open_issues_count":516,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":195,"open_issues":516,"watchers":2225,"default_branch":"master"}},"base":{"label":"cilium:master","ref":"master","sha":"d1e423e50f85e05b935d15b29354d1c0578deb47","user":{"login":"cilium","id":21054566,"avatar_url":"https://avatars0.githubusercontent.com/u/21054566?v=4","gravatar_id":"","url":"https://api.github.com/users/cilium","html_url":"https://github.com/cilium","followers_url":"https://api.github.com/users/cilium/followers","following_url":"https://api.github.com/users/cilium/following{/other_user}","gists_url":"https://api.github.com/users/cilium/gists{/gist_id}","starred_url":"https://api.github.com/users/cilium/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cilium/subscriptions","organizations_url":"https://api.github.com/users/cilium/orgs","repos_url":"https://api.github.com/users/cilium/repos","events_url":"https://api.github.com/users/cilium/events{/privacy}","received_events_url":"https://api.github.com/users/cilium/received_events","type":"Organization","site_admin":false},"repo":{"id":48109239,"name":"cilium","full_name":"cilium/cilium","owner":{"login":"cilium","id":21054566,"avatar_url":"https://avatars0.githubusercontent.com/u/21054566?v=4","gravatar_id":"","url":"https://api.github.com/users/cilium","html_url":"https://github.com/cilium","followers_url":"https://api.github.com/users/cilium/followers","following_url":"https://api.github.com/users/cilium/following{/other_user}","gists_url":"https://api.github.com/users/cilium/gists{/gist_id}","starred_url":"https://api.github.com/users/cilium/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cilium/subscriptions","organizations_url":"https://api.github.com/users/cilium/orgs","repos_url":"https://api.github.com/users/cilium/repos","events_url":"https://api.github.com/users/cilium/events{/privacy}","received_events_url":"https://api.github.com/users/cilium/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/cilium/cilium","description":"HTTP, gRPC, and Kafka Aware Security and Networking for Containers with BPF and XDP","fork":false,"url":"https://api.github.com/repos/cilium/cilium","forks_url":"https://api.github.com/repos/cilium/cilium/forks","keys_url":"https://api.github.com/repos/cilium/cilium/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cilium/cilium/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cilium/cilium/teams","hooks_url":"https://api.github.com/repos/cilium/cilium/hooks","issue_events_url":"https://api.github.com/repos/cilium/cilium/issues/events{/number}","events_url":"https://api.github.com/repos/cilium/cilium/events","assignees_url":"https://api.github.com/repos/cilium/cilium/assignees{/user}","branches_url":"https://api.github.com/repos/cilium/cilium/branches{/branch}","tags_url":"https://api.github.com/repos/cilium/cilium/tags","blobs_url":"https://api.github.com/repos/cilium/cilium/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cilium/cilium/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cilium/cilium/git/refs{/sha}","trees_url":"https://api.github.com/repos/cilium/cilium/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cilium/cilium/statuses/{sha}","languages_url":"https://api.github.com/repos/cilium/cilium/languages","stargazers_url":"https://api.github.com/repos/cilium/cilium/stargazers","contributors_url":"https://api.github.com/repos/cilium/cilium/contributors","subscribers_url":"https://api.github.com/repos/cilium/cilium/subscribers","subscription_url":"https://api.github.com/repos/cilium/cilium/subscription","commits_url":"https://api.github.com/repos/cilium/cilium/commits{/sha}","git_commits_url":"https://api.github.com/repos/cilium/cilium/git/commits{/sha}","comments_url":"https://api.github.com/repos/cilium/cilium/comments{/number}","issue_comment_url":"https://api.github.com/repos/cilium/cilium/issues/comments{/number}","contents_url":"https://api.github.com/repos/cilium/cilium/contents/{+path}","compare_url":"https://api.github.com/repos/cilium/cilium/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cilium/cilium/merges","archive_url":"https://api.github.com/repos/cilium/cilium/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cilium/cilium/downloads","issues_url":"https://api.github.com/repos/cilium/cilium/issues{/number}","pulls_url":"https://api.github.com/repos/cilium/cilium/pulls{/number}","milestones_url":"https://api.github.com/repos/cilium/cilium/milestones{/number}","notifications_url":"https://api.github.com/repos/cilium/cilium/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cilium/cilium/labels{/name}","releases_url":"https://api.github.com/repos/cilium/cilium/releases{/id}","deployments_url":"https://api.github.com/repos/cilium/cilium/deployments","created_at":"2015-12-16T12:33:31Z","updated_at":"2018-05-11T20:39:28Z","pushed_at":"2018-05-11T20:39:18Z","git_url":"git://github.com/cilium/cilium.git","ssh_url":"git@github.com:cilium/cilium.git","clone_url":"https://github.com/cilium/cilium.git","svn_url":"https://github.com/cilium/cilium","homepage":"","size":48659,"stargazers_count":2225,"watchers_count":2225,"language":"Go","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":195,"mirror_url":null,"archived":false,"open_issues_count":516,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0"},"forks":195,"open_issues":516,"watchers":2225,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/cilium/cilium/pulls/3965"},"html":{"href":"https://github.com/cilium/cilium/pull/3965"},"issue":{"href":"https://api.github.com/repos/cilium/cilium/issues/3965"},"comments":{"href":"https://api.github.com/repos/cilium/cilium/issues/3965/comments"},"review_comments":{"href":"https://api.github.com/repos/cilium/cilium/pulls/3965/comments"},"review_comment":{"href":"https://api.github.com/repos/cilium/cilium/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/cilium/cilium/pulls/3965/commits"},"statuses":{"href":"https://api.github.com/repos/cilium/cilium/statuses/bd96b28c37437c85b069459f1c6df57db2f5d7d2"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"ianvernon"}}