{"action":"created","issue":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/24246","repository_url":"https://api.github.com/repos/kubernetes/kubernetes","labels_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/24246/labels{/name}","comments_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/24246/comments","events_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/24246/events","html_url":"https://github.com/kubernetes/kubernetes/issues/24246","id":148340752,"number":24246,"title":"Change interface of priority functions for easier \"framework-based\" parallelization","user":{"login":"wojtek-t","id":10743879,"avatar_url":"https://avatars.githubusercontent.com/u/10743879?v=3","gravatar_id":"","url":"https://api.github.com/users/wojtek-t","html_url":"https://github.com/wojtek-t","followers_url":"https://api.github.com/users/wojtek-t/followers","following_url":"https://api.github.com/users/wojtek-t/following{/other_user}","gists_url":"https://api.github.com/users/wojtek-t/gists{/gist_id}","starred_url":"https://api.github.com/users/wojtek-t/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wojtek-t/subscriptions","organizations_url":"https://api.github.com/users/wojtek-t/orgs","repos_url":"https://api.github.com/users/wojtek-t/repos","events_url":"https://api.github.com/users/wojtek-t/events{/privacy}","received_events_url":"https://api.github.com/users/wojtek-t/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/performance","name":"area/performance","color":"d4c5f9"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/area/scheduling","name":"area/scheduling","color":"ab6420"},{"url":"https://api.github.com/repos/kubernetes/kubernetes/labels/team/control-plane","name":"team/control-plane","color":"d2b48c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2016-04-14T12:08:23Z","updated_at":"2016-04-15T09:45:03Z","closed_at":null,"body":"I'm currently trying to improve the overall scheduling throughput. It seems we can achieve significant gains from different kind of parallelization.\r\nHowever, I would like to avoid parallizing individual predicate/priority function and support parallelization at the framework level.\r\n\r\nFor predicate function, it is easy to parallelize computations of them between nodes, since they are independent.\r\nHowever, it's not the case for priorities - the interface of priority function is that it computes priorities for all nodes at the same time.\r\n\r\n\r\nIt would be much easier to parallelize everything, when priority function would also take a node as an argument and would compute priority wrt to current priority function just for this particular node.\r\nHowever, there are two problematic functions for now (maybe more in the future):\r\n- SelectorSpreading\r\n- NodeAffinity\r\n\r\nBoth of them have the same main problem - they are computing some partial scores for nodes independently, but then to compute the final score for the node, they rely on the maximum of those partial scores from all nodes.\r\n\r\n\r\n### Proposal\r\n\r\n1. I'm proposing to change the interface of priority functions to take a node for which they are supposed to compute priority.\r\n2. We also extend the HostPriority type to include \"priorityFunctionName\" string\r\n3. We will split priority functions into two phases:\r\n- per-node computations (which are generally cpu-intensive and would benefit from parallelization)\r\n- aggregation (which is generally much cheaper) and we can afford not parallelizing it\r\n\r\nThoughts?\r\n\r\n@davidopp @kubernetes/sig-scheduling @kubernetes/sig-scalability "},"comment":{"url":"https://api.github.com/repos/kubernetes/kubernetes/issues/comments/210391489","html_url":"https://github.com/kubernetes/kubernetes/issues/24246#issuecomment-210391489","issue_url":"https://api.github.com/repos/kubernetes/kubernetes/issues/24246","id":210391489,"user":{"login":"wojtek-t","id":10743879,"avatar_url":"https://avatars.githubusercontent.com/u/10743879?v=3","gravatar_id":"","url":"https://api.github.com/users/wojtek-t","html_url":"https://github.com/wojtek-t","followers_url":"https://api.github.com/users/wojtek-t/followers","following_url":"https://api.github.com/users/wojtek-t/following{/other_user}","gists_url":"https://api.github.com/users/wojtek-t/gists{/gist_id}","starred_url":"https://api.github.com/users/wojtek-t/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wojtek-t/subscriptions","organizations_url":"https://api.github.com/users/wojtek-t/orgs","repos_url":"https://api.github.com/users/wojtek-t/repos","events_url":"https://api.github.com/users/wojtek-t/events{/privacy}","received_events_url":"https://api.github.com/users/wojtek-t/received_events","type":"User","site_admin":false},"created_at":"2016-04-15T09:45:03Z","updated_at":"2016-04-15T09:45:03Z","body":"@davidopp - thanks\r\n\r\n>\r\n(assuming you think the speedup justifies the extra complexity).\r\n\r\nI wouldn't call it just \"extra complexity\". \r\nIt will definitely complicate the framework, but at the same time, I would say it will simplify the individual priority functions. If so, then I think it's a gain from productivity performance too, because we should very rarely touch framework itself, but we may want to add/modify priority/predicate functions much more often.\r\n\r\nAnyway - I will try to create a POC PR next week and we can discuss further on it.\r\n"}}