{"action":"created","issue":{"url":"https://api.github.com/repos/jupyterhub/binderhub/issues/855","repository_url":"https://api.github.com/repos/jupyterhub/binderhub","labels_url":"https://api.github.com/repos/jupyterhub/binderhub/issues/855/labels{/name}","comments_url":"https://api.github.com/repos/jupyterhub/binderhub/issues/855/comments","events_url":"https://api.github.com/repos/jupyterhub/binderhub/issues/855/events","html_url":"https://github.com/jupyterhub/binderhub/issues/855","id":447783713,"node_id":"MDU6SXNzdWU0NDc3ODM3MTM=","number":855,"title":"Optimize the user-schedulers conf for image locality","user":{"login":"consideRatio","id":3837114,"node_id":"MDQ6VXNlcjM4MzcxMTQ=","avatar_url":"https://avatars3.githubusercontent.com/u/3837114?v=4","gravatar_id":"","url":"https://api.github.com/users/consideRatio","html_url":"https://github.com/consideRatio","followers_url":"https://api.github.com/users/consideRatio/followers","following_url":"https://api.github.com/users/consideRatio/following{/other_user}","gists_url":"https://api.github.com/users/consideRatio/gists{/gist_id}","starred_url":"https://api.github.com/users/consideRatio/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/consideRatio/subscriptions","organizations_url":"https://api.github.com/users/consideRatio/orgs","repos_url":"https://api.github.com/users/consideRatio/repos","events_url":"https://api.github.com/users/consideRatio/events{/privacy}","received_events_url":"https://api.github.com/users/consideRatio/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-05-23T17:23:46Z","updated_at":"2019-05-23T21:21:39Z","closed_at":null,"author_association":"MEMBER","body":"We should try make sure to optimize the user pods to the nodes where their image is available already. This can be done i think, within the z2jh user scheduler configuration by adding weight to image locality.\r\n\r\n## how to value image locality?\r\nBut. How much? Value this to what extent? If you have 10-90% capacity used on one node, and 0% capacity used on another node awaiting scale down but with the image already, where should we schedule?\r\n\r\n/cc: @jhamman "},"comment":{"url":"https://api.github.com/repos/jupyterhub/binderhub/issues/comments/495389785","html_url":"https://github.com/jupyterhub/binderhub/issues/855#issuecomment-495389785","issue_url":"https://api.github.com/repos/jupyterhub/binderhub/issues/855","id":495389785,"node_id":"MDEyOklzc3VlQ29tbWVudDQ5NTM4OTc4NQ==","user":{"login":"betatim","id":1448859,"node_id":"MDQ6VXNlcjE0NDg4NTk=","avatar_url":"https://avatars1.githubusercontent.com/u/1448859?v=4","gravatar_id":"","url":"https://api.github.com/users/betatim","html_url":"https://github.com/betatim","followers_url":"https://api.github.com/users/betatim/followers","following_url":"https://api.github.com/users/betatim/following{/other_user}","gists_url":"https://api.github.com/users/betatim/gists{/gist_id}","starred_url":"https://api.github.com/users/betatim/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/betatim/subscriptions","organizations_url":"https://api.github.com/users/betatim/orgs","repos_url":"https://api.github.com/users/betatim/repos","events_url":"https://api.github.com/users/betatim/events{/privacy}","received_events_url":"https://api.github.com/users/betatim/received_events","type":"User","site_admin":false},"created_at":"2019-05-23T21:21:39Z","updated_at":"2019-05-23T21:21:39Z","author_association":"MEMBER","body":"I think this would be cool and speed up launch times. However one scenario that happens to mybinder.org a lot is courses and workshops. This means 20-70 people all launch the same image at the same time and then continue to do things in unison for the next hour(s). A downside of putting them all on the same node is that they will be affected by the fact that we heavily oversubscribe our nodes -> mybinder.org will feel slow.\r\n\r\nWith the current user-scheduler we already suffer from this because the decision which node is optimal wonn't change (much) while assigning the 20-70 pods that all started within a minute or two of each other.\r\n\r\nAny time we introduce a pattern we will end up with unintended consequences that we will need to understand and then mitigate. If we do things at random things will be random which tends to be a good (not the best) and low complexity strategy (c.f. computing the variance of simple MC integration vs VEGAS)"}}
{"actor":{"display_login":"betatim"}}