{"action":"created","issue":{"url":"https://api.github.com/repos/ray-project/ray/issues/5226","repository_url":"https://api.github.com/repos/ray-project/ray","labels_url":"https://api.github.com/repos/ray-project/ray/issues/5226/labels{/name}","comments_url":"https://api.github.com/repos/ray-project/ray/issues/5226/comments","events_url":"https://api.github.com/repos/ray-project/ray/issues/5226/events","html_url":"https://github.com/ray-project/ray/pull/5226","id":470016699,"node_id":"MDExOlB1bGxSZXF1ZXN0Mjk5MTEyOTM3","number":5226,"title":"Ray, Tune, and RLlib support for memory, object_store_memory options","user":{"login":"ericl","id":14922,"node_id":"MDQ6VXNlcjE0OTIy","avatar_url":"https://avatars3.githubusercontent.com/u/14922?v=4","gravatar_id":"","url":"https://api.github.com/users/ericl","html_url":"https://github.com/ericl","followers_url":"https://api.github.com/users/ericl/followers","following_url":"https://api.github.com/users/ericl/following{/other_user}","gists_url":"https://api.github.com/users/ericl/gists{/gist_id}","starred_url":"https://api.github.com/users/ericl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericl/subscriptions","organizations_url":"https://api.github.com/users/ericl/orgs","repos_url":"https://api.github.com/users/ericl/repos","events_url":"https://api.github.com/users/ericl/events{/privacy}","received_events_url":"https://api.github.com/users/ericl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"robertnishihara","id":249517,"node_id":"MDQ6VXNlcjI0OTUxNw==","avatar_url":"https://avatars3.githubusercontent.com/u/249517?v=4","gravatar_id":"","url":"https://api.github.com/users/robertnishihara","html_url":"https://github.com/robertnishihara","followers_url":"https://api.github.com/users/robertnishihara/followers","following_url":"https://api.github.com/users/robertnishihara/following{/other_user}","gists_url":"https://api.github.com/users/robertnishihara/gists{/gist_id}","starred_url":"https://api.github.com/users/robertnishihara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robertnishihara/subscriptions","organizations_url":"https://api.github.com/users/robertnishihara/orgs","repos_url":"https://api.github.com/users/robertnishihara/repos","events_url":"https://api.github.com/users/robertnishihara/events{/privacy}","received_events_url":"https://api.github.com/users/robertnishihara/received_events","type":"User","site_admin":false},"assignees":[{"login":"robertnishihara","id":249517,"node_id":"MDQ6VXNlcjI0OTUxNw==","avatar_url":"https://avatars3.githubusercontent.com/u/249517?v=4","gravatar_id":"","url":"https://api.github.com/users/robertnishihara","html_url":"https://github.com/robertnishihara","followers_url":"https://api.github.com/users/robertnishihara/followers","following_url":"https://api.github.com/users/robertnishihara/following{/other_user}","gists_url":"https://api.github.com/users/robertnishihara/gists{/gist_id}","starred_url":"https://api.github.com/users/robertnishihara/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robertnishihara/subscriptions","organizations_url":"https://api.github.com/users/robertnishihara/orgs","repos_url":"https://api.github.com/users/robertnishihara/repos","events_url":"https://api.github.com/users/robertnishihara/events{/privacy}","received_events_url":"https://api.github.com/users/robertnishihara/received_events","type":"User","site_admin":false}],"milestone":null,"comments":18,"created_at":"2019-07-18T22:18:07Z","updated_at":"2019-08-02T02:53:34Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/ray-project/ray/pulls/5226","html_url":"https://github.com/ray-project/ray/pull/5226","diff_url":"https://github.com/ray-project/ray/pull/5226.diff","patch_url":"https://github.com/ray-project/ray/pull/5226.patch"},"body":"<!-- Thank you for your contribution! Please review https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before opening a pull request. -->\r\n\r\n## What do these changes do?\r\n\r\nThis is the Ray-side support for https://github.com/apache/arrow/pull/4885, and in addition heap limits. It adds two new resources for `ray.remote`:\r\n\r\n```\r\nray.remote(memory=<bytes>, object_store_memory=<bytes>)\r\n```\r\n\r\nIf these resources are set, then Ray will schedule the actor or task with these resource requests. In addition, for actors the limits will be enforced: plasma will LRU evict once the actor has used that much memory in the object store, and workers will raise RayMemoryError if they exceed their heap limit. The amount of available heap memory is determined by `system memory - used memory - object store max - redis max (if applicable)`, can be overriden by passing `memory` to ray start.\r\n\r\nCurrently everything is disabled by default except for object store limits on RLlib workers (which should greatly improve stability there). Documentation is to come in another PR.\r\n\r\n*Limitations*: Due to the current way custom resources are tracked in Ray, memory quotas can only be requested in 100MB chunks, and we cannot supported more than 1TB of memory per raylet. This can potentially be addressed in a followup if we refactor the resource id tracking to allow for non-discrete resources.\r\n\r\n## Related issue number\r\n\r\nFirst part of https://github.com/ray-project/ray/issues/5114\r\n\r\n## Linter\r\n\r\n- [x] I've run `scripts/format.sh` to lint the changes in this PR.\r\n"},"comment":{"url":"https://api.github.com/repos/ray-project/ray/issues/comments/517527923","html_url":"https://github.com/ray-project/ray/pull/5226#issuecomment-517527923","issue_url":"https://api.github.com/repos/ray-project/ray/issues/5226","id":517527923,"node_id":"MDEyOklzc3VlQ29tbWVudDUxNzUyNzkyMw==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2019-08-02T02:53:34Z","updated_at":"2019-08-02T02:53:34Z","author_association":"COLLABORATOR","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/15919/\nTest FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}