{"action":"created","issue":{"url":"https://api.github.com/repos/Yelp/paasta/issues/504","repository_url":"https://api.github.com/repos/Yelp/paasta","labels_url":"https://api.github.com/repos/Yelp/paasta/issues/504/labels{/name}","comments_url":"https://api.github.com/repos/Yelp/paasta/issues/504/comments","events_url":"https://api.github.com/repos/Yelp/paasta/issues/504/events","html_url":"https://github.com/Yelp/paasta/issues/504","id":153548998,"number":504,"title":"Support cfs_quota_us / cfs_period_us","user":{"login":"EvanKrall","id":233961,"avatar_url":"https://avatars.githubusercontent.com/u/233961?v=3","gravatar_id":"","url":"https://api.github.com/users/EvanKrall","html_url":"https://github.com/EvanKrall","followers_url":"https://api.github.com/users/EvanKrall/followers","following_url":"https://api.github.com/users/EvanKrall/following{/other_user}","gists_url":"https://api.github.com/users/EvanKrall/gists{/gist_id}","starred_url":"https://api.github.com/users/EvanKrall/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/EvanKrall/subscriptions","organizations_url":"https://api.github.com/users/EvanKrall/orgs","repos_url":"https://api.github.com/users/EvanKrall/repos","events_url":"https://api.github.com/users/EvanKrall/events{/privacy}","received_events_url":"https://api.github.com/users/EvanKrall/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Yelp/paasta/labels/in%20progress","name":"in progress","color":"ededed"}],"state":"open","locked":false,"assignee":{"login":"matthewbentley","id":1930163,"avatar_url":"https://avatars.githubusercontent.com/u/1930163?v=3","gravatar_id":"","url":"https://api.github.com/users/matthewbentley","html_url":"https://github.com/matthewbentley","followers_url":"https://api.github.com/users/matthewbentley/followers","following_url":"https://api.github.com/users/matthewbentley/following{/other_user}","gists_url":"https://api.github.com/users/matthewbentley/gists{/gist_id}","starred_url":"https://api.github.com/users/matthewbentley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthewbentley/subscriptions","organizations_url":"https://api.github.com/users/matthewbentley/orgs","repos_url":"https://api.github.com/users/matthewbentley/repos","events_url":"https://api.github.com/users/matthewbentley/events{/privacy}","received_events_url":"https://api.github.com/users/matthewbentley/received_events","type":"User","site_admin":false},"milestone":null,"comments":2,"created_at":"2016-05-06T22:12:36Z","updated_at":"2016-06-07T20:20:09Z","closed_at":null,"body":"Recently, Mesos added an option:\r\n\r\n> `--[no]-cgroups_enable_cfs`  Cgroups feature flag to enable hard limits on CPU resources via the CFS bandwidth limiting subfeature. (default: false)\r\n\r\nThis is really intriguing, since it provides a way to hard-cap containers. We're not sure whether Mesos would set the appropriate options on Docker, but Marathon allows us to pass arbitrary command line flags to `docker run` [via the `container.docker.parameters` field](https://mesosphere.github.io/marathon/docs/rest-api.html#post-v2-apps), and docker has command line flags for this:\r\n\r\n```\r\n  --cpu-period                    Limit CPU CFS (Completely Fair Scheduler) period\r\n  --cpu-quota                     Limit CPU CFS (Completely Fair Scheduler) quota\r\n```\r\n\r\n@solarkennedy and I played around with this, and it does appear to work:\r\n\r\n```\r\nkrall@dev5-devc:~/ $ docker run -d --cpu-period 100000 --cpu-quota 20000 busybox sh -c 'yes >/dev/null'\r\n481e13359f87800a5979d09f6c0caa3f8b54754bdfe2d6103cd16563c6000366\r\nkrall@dev5-devc:~/ $ docker stats --no-stream 481e13359f87800a5979d09f6c0caa3f8b54754bdfe2d6103cd16563c6000366\r\nCONTAINER                                                          CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O\r\n481e13359f87800a5979d09f6c0caa3f8b54754bdfe2d6103cd16563c6000366   20.12%              409.6 kB / 135.2 GB   0.00%               648 B / 648 B       319.5 kB / 0 B\r\nkrall@dev5-devc:~/ $\r\n```\r\n\r\nWe were thinking it would be useful to add some parameters to marathon.yaml:\r\n\r\n - `cfs_period_us`: would set `--cpu-period` pretty much directly, but would default to 100000 (100ms), which is the default in the kernel.\r\n - `cpu_burst_pct`: Would set --cpu-quota through some math, but would default to 100, allowing services to use up to 2x their CPU allocation before getting throttled. (The CPU shares setting that Mesos/docker is already setting would still apply).\r\n\r\nMath: `cfs_quota_us = cpus * cfs_period_us * (100 + cpu_burst_pct) / 100`.\r\n\r\nI think as described here, this would be a pretty straightforward change but could provide some big wins in preventing services from impacting each other, particularly during startup.\r\n\r\n@Rob-Johnson You're interested in isolation things. Does this seem reasonable to you? Do you want to take this?"},"comment":{"url":"https://api.github.com/repos/Yelp/paasta/issues/comments/224401501","html_url":"https://github.com/Yelp/paasta/issues/504#issuecomment-224401501","issue_url":"https://api.github.com/repos/Yelp/paasta/issues/504","id":224401501,"user":{"login":"matthewbentley","id":1930163,"avatar_url":"https://avatars.githubusercontent.com/u/1930163?v=3","gravatar_id":"","url":"https://api.github.com/users/matthewbentley","html_url":"https://github.com/matthewbentley","followers_url":"https://api.github.com/users/matthewbentley/followers","following_url":"https://api.github.com/users/matthewbentley/following{/other_user}","gists_url":"https://api.github.com/users/matthewbentley/gists{/gist_id}","starred_url":"https://api.github.com/users/matthewbentley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/matthewbentley/subscriptions","organizations_url":"https://api.github.com/users/matthewbentley/orgs","repos_url":"https://api.github.com/users/matthewbentley/repos","events_url":"https://api.github.com/users/matthewbentley/events{/privacy}","received_events_url":"https://api.github.com/users/matthewbentley/received_events","type":"User","site_admin":false},"created_at":"2016-06-07T20:20:08Z","updated_at":"2016-06-07T20:20:08Z","body":"What does `cpus` in `cfs_quota_us = cpus * cfs_period_us * (100 + cpu_burst_pct) / 100` refer to?"}}
{"actor":{"display_login":"matthewbentley"}}