{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/68","repository_url":"https://api.github.com/repos/kubernetes-csi/external-provisioner","labels_url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/68/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/68/comments","events_url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/68/events","html_url":"https://github.com/kubernetes-csi/external-provisioner/issues/68","id":310941473,"node_id":"MDU6SXNzdWUzMTA5NDE0NzM=","number":68,"title":"Scaling issue: kube apiserver throttles external-provisioner when 100 PVCs created at same time","user":{"login":"saad-ali","id":10052848,"node_id":"MDQ6VXNlcjEwMDUyODQ4","avatar_url":"https://avatars1.githubusercontent.com/u/10052848?v=4","gravatar_id":"","url":"https://api.github.com/users/saad-ali","html_url":"https://github.com/saad-ali","followers_url":"https://api.github.com/users/saad-ali/followers","following_url":"https://api.github.com/users/saad-ali/following{/other_user}","gists_url":"https://api.github.com/users/saad-ali/gists{/gist_id}","starred_url":"https://api.github.com/users/saad-ali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saad-ali/subscriptions","organizations_url":"https://api.github.com/users/saad-ali/orgs","repos_url":"https://api.github.com/users/saad-ali/repos","events_url":"https://api.github.com/users/saad-ali/events{/privacy}","received_events_url":"https://api.github.com/users/saad-ali/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-04-03T18:10:54Z","updated_at":"2018-06-06T21:47:23Z","closed_at":null,"author_association":"MEMBER","body":"Reported by @cduchesne \r\n\r\nThe CSI external-provisioner has scaling issues.\r\n\r\nWhen 100 PVCs are created at same time, the CSI external-provisioner hammers the kube apiserver with requests and gets throttled, causing all sorts of issues."},"comment":{"url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/comments/395224658","html_url":"https://github.com/kubernetes-csi/external-provisioner/issues/68#issuecomment-395224658","issue_url":"https://api.github.com/repos/kubernetes-csi/external-provisioner/issues/68","id":395224658,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NTIyNDY1OA==","user":{"login":"saad-ali","id":10052848,"node_id":"MDQ6VXNlcjEwMDUyODQ4","avatar_url":"https://avatars1.githubusercontent.com/u/10052848?v=4","gravatar_id":"","url":"https://api.github.com/users/saad-ali","html_url":"https://github.com/saad-ali","followers_url":"https://api.github.com/users/saad-ali/followers","following_url":"https://api.github.com/users/saad-ali/following{/other_user}","gists_url":"https://api.github.com/users/saad-ali/gists{/gist_id}","starred_url":"https://api.github.com/users/saad-ali/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/saad-ali/subscriptions","organizations_url":"https://api.github.com/users/saad-ali/orgs","repos_url":"https://api.github.com/users/saad-ali/repos","events_url":"https://api.github.com/users/saad-ali/events{/privacy}","received_events_url":"https://api.github.com/users/saad-ali/received_events","type":"User","site_admin":false},"created_at":"2018-06-06T21:47:23Z","updated_at":"2018-06-06T21:47:23Z","author_association":"MEMBER","body":"Thanks for surfacing this @orainxiong. \r\n\r\nProblem sounds like the external-provisioner (and probably external-attacher and driver-registrar), open connections to kube API server without thinking twice. If you provision a lot of volumes at once, the kube API server gets mad and throttles everything.\r\n\r\nWe should do the following:\r\n* Create a worker pool so that no more then N operations can be carried out against the API server at once.\r\n* Allow N to be configured via a flag to the binary.\r\n* Incorporate Kubernetes informer cache in to sidecar containers so that reads/writes are optimized."}}
{"actor":{"display_login":"saad-ali"}}