{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/30871","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/30871/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/30871/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/30871/events","html_url":"https://github.com/kubernetes/kubernetes/issues/30871","id":171889593,"number":30871,"title":"Investigate higher resource consumption in apiserver in 100-node clusters","user":{"login":"wojtek-t","id":10743879,"avatar_url":"https://avatars.githubusercontent.com/u/10743879?v=3","gravatar_id":"","url":"https://api.github.com/users/wojtek-t","html_url":"https://github.com/wojtek-t","followers_url":"https://api.github.com/users/wojtek-t/followers","following_url":"https://api.github.com/users/wojtek-t/following{/other_user}","gists_url":"https://api.github.com/users/wojtek-t/gists{/gist_id}","starred_url":"https://api.github.com/users/wojtek-t/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wojtek-t/subscriptions","organizations_url":"https://api.github.com/users/wojtek-t/orgs","repos_url":"https://api.github.com/users/wojtek-t/repos","events_url":"https://api.github.com/users/wojtek-t/events{/privacy}","received_events_url":"https://api.github.com/users/wojtek-t/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/performance","name":"area/performance","color":"d4c5f9"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/priority/P1","name":"priority/P1","color":"eb6420"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/CSI-API%20Machinery%20SIG","name":"team/CSI-API Machinery SIG","color":"d2b48c"}],"state":"open","locked":false,"assignee":{"login":"lavalamp","id":647318,"avatar_url":"https://avatars.githubusercontent.com/u/647318?v=3","gravatar_id":"","url":"https://api.github.com/users/lavalamp","html_url":"https://github.com/lavalamp","followers_url":"https://api.github.com/users/lavalamp/followers","following_url":"https://api.github.com/users/lavalamp/following{/other_user}","gists_url":"https://api.github.com/users/lavalamp/gists{/gist_id}","starred_url":"https://api.github.com/users/lavalamp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lavalamp/subscriptions","organizations_url":"https://api.github.com/users/lavalamp/orgs","repos_url":"https://api.github.com/users/lavalamp/repos","events_url":"https://api.github.com/users/lavalamp/events{/privacy}","received_events_url":"https://api.github.com/users/lavalamp/received_events","type":"User","site_admin":false},"assignees":[{"login":"lavalamp","id":647318,"avatar_url":"https://avatars.githubusercontent.com/u/647318?v=3","gravatar_id":"","url":"https://api.github.com/users/lavalamp","html_url":"https://github.com/lavalamp","followers_url":"https://api.github.com/users/lavalamp/followers","following_url":"https://api.github.com/users/lavalamp/following{/other_user}","gists_url":"https://api.github.com/users/lavalamp/gists{/gist_id}","starred_url":"https://api.github.com/users/lavalamp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lavalamp/subscriptions","organizations_url":"https://api.github.com/users/lavalamp/orgs","repos_url":"https://api.github.com/users/lavalamp/repos","events_url":"https://api.github.com/users/lavalamp/events{/privacy}","received_events_url":"https://api.github.com/users/lavalamp/received_events","type":"User","site_admin":false},{"login":"caesarxuchao","id":2823529,"avatar_url":"https://avatars.githubusercontent.com/u/2823529?v=3","gravatar_id":"","url":"https://api.github.com/users/caesarxuchao","html_url":"https://github.com/caesarxuchao","followers_url":"https://api.github.com/users/caesarxuchao/followers","following_url":"https://api.github.com/users/caesarxuchao/following{/other_user}","gists_url":"https://api.github.com/users/caesarxuchao/gists{/gist_id}","starred_url":"https://api.github.com/users/caesarxuchao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/caesarxuchao/subscriptions","organizations_url":"https://api.github.com/users/caesarxuchao/orgs","repos_url":"https://api.github.com/users/caesarxuchao/repos","events_url":"https://api.github.com/users/caesarxuchao/events{/privacy}","received_events_url":"https://api.github.com/users/caesarxuchao/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/kubernetes/kubernetes/milestones/22","html_url":"https://github.com/kubernetes/kubernetes/milestones/v1.4","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/milestones/22/labels","id":1777384,"number":22,"title":"v1.4","description":"https://github.com/kubernetes/features/blob/master/release-1.4/Release-1.4.md","creator":{"login":"bgrant0607","id":7725777,"avatar_url":"https://avatars.githubusercontent.com/u/7725777?v=3","gravatar_id":"","url":"https://api.github.com/users/bgrant0607","html_url":"https://github.com/bgrant0607","followers_url":"https://api.github.com/users/bgrant0607/followers","following_url":"https://api.github.com/users/bgrant0607/following{/other_user}","gists_url":"https://api.github.com/users/bgrant0607/gists{/gist_id}","starred_url":"https://api.github.com/users/bgrant0607/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bgrant0607/subscriptions","organizations_url":"https://api.github.com/users/bgrant0607/orgs","repos_url":"https://api.github.com/users/bgrant0607/repos","events_url":"https://api.github.com/users/bgrant0607/events{/privacy}","received_events_url":"https://api.github.com/users/bgrant0607/received_events","type":"User","site_admin":false},"open_issues":200,"closed_issues":323,"state":"open","created_at":"2016-05-19T18:50:19Z","updated_at":"2016-08-22T14:29:34Z","due_on":"2016-09-15T00:00:00Z","closed_at":null},"comments":8,"created_at":"2016-08-18T12:30:40Z","updated_at":"2016-08-22T14:44:18Z","closed_at":null,"body":"We already knew that enabling GarbageCollector increases memory usage of the controller-manager.\r\n\r\nHowever, it appears that it also significantly increases memory usage on apiserver.\r\nIn particular, our scalability tests on 100-node real clusters show that after enabling GarbageCollector in them memory usage of apiserver increased (99th percentile):\r\nfrom ~0.9GB\r\nto ~1.7GB\r\n\r\nThis is huge!\r\n\r\nI didn't have time to dig much deeper into it, but  really suspect that it's all because GarbageCollector is using \"dynamicClient\" underneath, and dynamic client is using JSONs for communication with apiserver. And we already knew that JSONs are waaaaaay more expensive than protobufs.\r\n\r\nI'm not saying this is a hard blocker for launch, but such a huge increase in resource usage needs to be a very conscious decision.\r\n\r\n@caesarxuchao @lavalamp \r\n@kubernetes/sig-api-machinery @kubernetes/sig-scalability @smarterclayton "},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/241435739","html_url":"https://github.com/kubernetes/kubernetes/issues/30871#issuecomment-241435739","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/30871","id":241435739,"user":{"login":"timothysc","id":169553,"avatar_url":"https://avatars.githubusercontent.com/u/169553?v=3","gravatar_id":"","url":"https://api.github.com/users/timothysc","html_url":"https://github.com/timothysc","followers_url":"https://api.github.com/users/timothysc/followers","following_url":"https://api.github.com/users/timothysc/following{/other_user}","gists_url":"https://api.github.com/users/timothysc/gists{/gist_id}","starred_url":"https://api.github.com/users/timothysc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/timothysc/subscriptions","organizations_url":"https://api.github.com/users/timothysc/orgs","repos_url":"https://api.github.com/users/timothysc/repos","events_url":"https://api.github.com/users/timothysc/events{/privacy}","received_events_url":"https://api.github.com/users/timothysc/received_events","type":"User","site_admin":false},"created_at":"2016-08-22T14:44:18Z","updated_at":"2016-08-22T14:44:18Z","body":"We are also seeing this... and will be collecting profiles later today. \r\n\r\n/cc @smarterclayton "}}
{"actor":{"display_login":"timothysc"}}