{"action":"closed","issue":{"url":"https://api.github.com/repos/cloudnativelabs/kube-router/issues/536","repository_url":"https://api.github.com/repos/cloudnativelabs/kube-router","labels_url":"https://api.github.com/repos/cloudnativelabs/kube-router/issues/536/labels{/name}","comments_url":"https://api.github.com/repos/cloudnativelabs/kube-router/issues/536/comments","events_url":"https://api.github.com/repos/cloudnativelabs/kube-router/issues/536/events","html_url":"https://github.com/cloudnativelabs/kube-router/issues/536","id":360036538,"node_id":"MDU6SXNzdWUzNjAwMzY1Mzg=","number":536,"title":"Reduce api calls for disableSourceDestinationCheck","user":{"login":"buddyledungarees","id":3676980,"node_id":"MDQ6VXNlcjM2NzY5ODA=","avatar_url":"https://avatars1.githubusercontent.com/u/3676980?v=4","gravatar_id":"","url":"https://api.github.com/users/buddyledungarees","html_url":"https://github.com/buddyledungarees","followers_url":"https://api.github.com/users/buddyledungarees/followers","following_url":"https://api.github.com/users/buddyledungarees/following{/other_user}","gists_url":"https://api.github.com/users/buddyledungarees/gists{/gist_id}","starred_url":"https://api.github.com/users/buddyledungarees/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/buddyledungarees/subscriptions","organizations_url":"https://api.github.com/users/buddyledungarees/orgs","repos_url":"https://api.github.com/users/buddyledungarees/repos","events_url":"https://api.github.com/users/buddyledungarees/events{/privacy}","received_events_url":"https://api.github.com/users/buddyledungarees/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-09-13T19:36:17Z","updated_at":"2018-09-18T00:18:15Z","closed_at":"2018-09-18T00:18:15Z","author_association":"NONE","body":"I have a 100 node cluster and we're seeing the same problem as #313. We're running v0.2.0 of kube-router (latest release as of this issue).\r\n\r\nI'm wondering if we can reduce the number of calls by having the nodes only try to disable the setting for themselves and only on startup? We run kube-router as a daemonset anyway and the volume of calls over a 30 minute window was averaging 2.5 per sec across all kubernetes nodes in one subnet. Granted that doesn't seem like much but AWS doesn't publish their throttling limits and empirically, we're getting throttled:\r\n\r\n```\r\nE0913 17:09:13.559902       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: 504aa627-d3e8-47e5-b4ae-503aa310fb16\r\nE0913 17:09:35.458108       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: 490e1305-c47f-41d5-9ac2-82e670446ae9\r\nE0913 17:10:03.918433       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: de771669-47e2-458b-8636-0095417347a9\r\nE0913 17:10:28.118314       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: 1da40c06-0c3e-4591-8685-484e9b4a6e21\r\nE0913 17:10:49.015117       1 health_controller.go:125] NetworkService Controller heartbeat missed\r\nE0913 17:10:54.008204       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: ce86e736-de49-4a25-8189-1d5d43946e0a\r\nE0913 17:11:18.179957       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: de0551a6-393b-403c-9e0b-c9c52c71d447\r\nE0913 17:11:43.252905       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: 78b47071-7414-4761-a9b0-86036d07ba07\r\nE0913 17:11:54.033868       1 health_controller.go:125] NetworkService Controller heartbeat missed\r\nE0913 17:12:05.077176       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: a9307324-6dd7-40d3-907d-aae7877c260a\r\nE0913 17:12:29.952005       1 aws.go:61] Failed to disable source destination check due to: RequestLimitExceeded: Request limit exceeded.\r\n        status code: 503, request id: 614eddfb-0688-408b-8ab6-19d616851600\r\n```\r\n\r\nWould it make more sense to have each node disable just themselves?"}}
{"actor":{"display_login":"buddyledungarees"}}