{"action":"created","issue":{"url":"https://api.github.com/repos/ray-project/ray/issues/1592","repository_url":"https://api.github.com/repos/ray-project/ray","labels_url":"https://api.github.com/repos/ray-project/ray/issues/1592/labels{/name}","comments_url":"https://api.github.com/repos/ray-project/ray/issues/1592/comments","events_url":"https://api.github.com/repos/ray-project/ray/issues/1592/events","html_url":"https://github.com/ray-project/ray/issues/1592","id":299881446,"number":1592,"title":"Huge pages crashes with object sizes >= 100MB","user":{"login":"ericl","id":14922,"avatar_url":"https://avatars3.githubusercontent.com/u/14922?v=4","gravatar_id":"","url":"https://api.github.com/users/ericl","html_url":"https://github.com/ericl","followers_url":"https://api.github.com/users/ericl/followers","following_url":"https://api.github.com/users/ericl/following{/other_user}","gists_url":"https://api.github.com/users/ericl/gists{/gist_id}","starred_url":"https://api.github.com/users/ericl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericl/subscriptions","organizations_url":"https://api.github.com/users/ericl/orgs","repos_url":"https://api.github.com/users/ericl/repos","events_url":"https://api.github.com/users/ericl/events{/privacy}","received_events_url":"https://api.github.com/users/ericl/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-02-23T23:23:23Z","updated_at":"2018-02-23T23:26:58Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"<!--\r\nGeneral questions should be asked on the mailing list ray-dev@googlegroups.com.\r\n\r\nBefore submitting an issue, please fill out the following form.\r\n-->\r\n\r\n### System information\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: 16.04\r\n- **Ray installed from (source or binary)**: source\r\n- **Ray version**: master\r\n- **Python version**: 2.7\r\n- **Exact command to reproduce**: python ray_ps_benchmark.py  --dim=25000000\r\n<!--\r\nYou can obtain the Ray version with\r\n\r\npython -c \"import ray; print(ray.__version__)\"\r\n-->\r\n\r\n### Describe the problem\r\nThe following script will crash if a large dim value is passed in as an arg\r\n```\r\nTraceback (most recent call last):\r\n  File \"plasma.pyx\", line 472, in pyarrow.plasma.PlasmaClient.release\r\n  File \"error.pxi\", line 79, in pyarrow.lib.check_status\r\npyarrow.libOK\r\n.ArrowIOError: Error during munmap\r\nException pyarrow.lib.ArrowIOError: ArrowIOError('Error during munmap',) in 'pyarrow.plasma.PlasmaBuffer.__dealloc__' ignored\r\n```\r\n\r\n```\r\nfrom __future__ import absolute_import\r\nfrom __future__ import division\r\nfrom __future__ import print_function\r\n\r\nimport argparse\r\nimport numpy as np\r\nimport ray\r\n\r\nparser = argparse.ArgumentParser(description=\"Run the synchronous parameter \"\r\n                                             \"server example.\")\r\nparser.add_argument(\"--dim\", default=25*1000*1000, type=int,\r\n                    help=\"The number of parameters.\")\r\nargs = parser.parse_args()\r\n\r\nclass CNN(object):\r\n    def __init__(self, dim):\r\n        self.dim = dim\r\n        self.grads = np.ones(self.dim, dtype=np.float32)\r\n        \r\n    def get_gradients(self, params):\r\n        return self.grads\r\n\r\n@ray.remote\r\nclass Worker(object):\r\n    def __init__(self, dim):\r\n        self.net = CNN(dim)\r\n\r\n    def compute_gradient(self, params):\r\n        return self.net.get_gradients(params)\r\n\r\ndef main():\r\n  # initial parameter value\r\n  params = np.zeros(args.dim, dtype=np.float32)\r\n\r\n  ray.init(huge_pages=True, plasma_directory=\"/mnt/hugepages\")\r\n  worker = Worker.remote(args.dim)\r\n\r\n  net = CNN(args.dim)\r\n\r\n  grads = worker.compute_gradient.remote(params)\r\n  ray.wait([grads])\r\n\r\n\r\nif __name__ == \"__main__\":\r\n  main()\r\n```\r\n\r\n### Source code / logs\r\npython ray_ps_benchmark.py --dim=2500  # OK\r\npython ray_ps_benchmark.py --dim=25000000  # PyArrow error\r\n"},"comment":{"url":"https://api.github.com/repos/ray-project/ray/issues/comments/368168444","html_url":"https://github.com/ray-project/ray/issues/1592#issuecomment-368168444","issue_url":"https://api.github.com/repos/ray-project/ray/issues/1592","id":368168444,"user":{"login":"ericl","id":14922,"avatar_url":"https://avatars3.githubusercontent.com/u/14922?v=4","gravatar_id":"","url":"https://api.github.com/users/ericl","html_url":"https://github.com/ericl","followers_url":"https://api.github.com/users/ericl/followers","following_url":"https://api.github.com/users/ericl/following{/other_user}","gists_url":"https://api.github.com/users/ericl/gists{/gist_id}","starred_url":"https://api.github.com/users/ericl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericl/subscriptions","organizations_url":"https://api.github.com/users/ericl/orgs","repos_url":"https://api.github.com/users/ericl/repos","events_url":"https://api.github.com/users/ericl/events{/privacy}","received_events_url":"https://api.github.com/users/ericl/received_events","type":"User","site_admin":false},"created_at":"2018-02-23T23:26:58Z","updated_at":"2018-02-23T23:26:58Z","author_association":"CONTRIBUTOR","body":"Actually, it also seems to crash with smaller objects, but takes longer (run it in a loop)."}}
{"actor":{"display_login":"ericl"}}