{"action":"opened","issue":{"url":"https://api.github.com/repos/uwescience/myria/issues/833","repository_url":"https://api.github.com/repos/uwescience/myria","labels_url":"https://api.github.com/repos/uwescience/myria/issues/833/labels{/name}","comments_url":"https://api.github.com/repos/uwescience/myria/issues/833/comments","events_url":"https://api.github.com/repos/uwescience/myria/issues/833/events","html_url":"https://github.com/uwescience/myria/issues/833","id":164684253,"number":833,"title":"Allow fractional vcores for nodes and workers","user":{"login":"senderista","id":131425,"avatar_url":"https://avatars.githubusercontent.com/u/131425?v=3","gravatar_id":"","url":"https://api.github.com/users/senderista","html_url":"https://github.com/senderista","followers_url":"https://api.github.com/users/senderista/followers","following_url":"https://api.github.com/users/senderista/following{/other_user}","gists_url":"https://api.github.com/users/senderista/gists{/gist_id}","starred_url":"https://api.github.com/users/senderista/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senderista/subscriptions","organizations_url":"https://api.github.com/users/senderista/orgs","repos_url":"https://api.github.com/users/senderista/repos","events_url":"https://api.github.com/users/senderista/events{/privacy}","received_events_url":"https://api.github.com/users/senderista/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/uwescience/myria/labels/Enhancement","name":"Enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":{"login":"senderista","id":131425,"avatar_url":"https://avatars.githubusercontent.com/u/131425?v=3","gravatar_id":"","url":"https://api.github.com/users/senderista","html_url":"https://github.com/senderista","followers_url":"https://api.github.com/users/senderista/followers","following_url":"https://api.github.com/users/senderista/following{/other_user}","gists_url":"https://api.github.com/users/senderista/gists{/gist_id}","starred_url":"https://api.github.com/users/senderista/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senderista/subscriptions","organizations_url":"https://api.github.com/users/senderista/orgs","repos_url":"https://api.github.com/users/senderista/repos","events_url":"https://api.github.com/users/senderista/events{/privacy}","received_events_url":"https://api.github.com/users/senderista/received_events","type":"User","site_admin":false},"assignees":[{"login":"senderista","id":131425,"avatar_url":"https://avatars.githubusercontent.com/u/131425?v=3","gravatar_id":"","url":"https://api.github.com/users/senderista","html_url":"https://github.com/senderista","followers_url":"https://api.github.com/users/senderista/followers","following_url":"https://api.github.com/users/senderista/following{/other_user}","gists_url":"https://api.github.com/users/senderista/gists{/gist_id}","starred_url":"https://api.github.com/users/senderista/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/senderista/subscriptions","organizations_url":"https://api.github.com/users/senderista/orgs","repos_url":"https://api.github.com/users/senderista/repos","events_url":"https://api.github.com/users/senderista/events{/privacy}","received_events_url":"https://api.github.com/users/senderista/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2016-07-09T20:27:33Z","updated_at":"2016-07-09T20:27:33Z","closed_at":null,"body":"Although vcores in YARN represent only shares of (YARN-allocated) CPU resources allocated to containers (and hence only the ratio of container vcores/node vcores is meaningful, not the absolute values of either), we should maintain the illusion that vcores correspond to the number of physical CPUs on a node. Since dividing the physical cores available to workers by the number of workers often yields a fractional number, we should allow decimal values of the `worker_vcores` parameter and perform the rescaling automatically. E.g., if an EC2 instance type has 8 virtual CPUs, and we want to reserve one CPU for the (currently randomly assigned) Driver, then if we need to schedule 4 workers on each instance, we need 7/4 = 1.75 vcores/worker. YARN doesn't allow fractional vcore values, but we can achieve the equivalent by rescaling all vcore values by a factor of 4, so now we have `node_vcores = 32`, `worker_vcores = 7`.\r\n\r\nThis will need supporting changes in the `myria-ec2-ansible` repo (parameter validation, usage messages)."}}
{"actor":{"display_login":"senderista"}}