{"action":"created","issue":{"url":"https://api.github.com/repos/ray-project/ray/issues/1653","repository_url":"https://api.github.com/repos/ray-project/ray","labels_url":"https://api.github.com/repos/ray-project/ray/issues/1653/labels{/name}","comments_url":"https://api.github.com/repos/ray-project/ray/issues/1653/comments","events_url":"https://api.github.com/repos/ray-project/ray/issues/1653/events","html_url":"https://github.com/ray-project/ray/issues/1653","id":302146452,"number":1653,"title":"Problems Running on Cluster using the Manager: Slurm","user":{"login":"marianogabitto","id":4081325,"avatar_url":"https://avatars1.githubusercontent.com/u/4081325?v=4","gravatar_id":"","url":"https://api.github.com/users/marianogabitto","html_url":"https://github.com/marianogabitto","followers_url":"https://api.github.com/users/marianogabitto/followers","following_url":"https://api.github.com/users/marianogabitto/following{/other_user}","gists_url":"https://api.github.com/users/marianogabitto/gists{/gist_id}","starred_url":"https://api.github.com/users/marianogabitto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marianogabitto/subscriptions","organizations_url":"https://api.github.com/users/marianogabitto/orgs","repos_url":"https://api.github.com/users/marianogabitto/repos","events_url":"https://api.github.com/users/marianogabitto/events{/privacy}","received_events_url":"https://api.github.com/users/marianogabitto/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-03-05T01:42:05Z","updated_at":"2018-03-05T15:46:21Z","closed_at":null,"author_association":"NONE","body":"<!--\r\nGeneral questions should be asked on the mailing list ray-dev@googlegroups.com.\r\n\r\nBefore submitting an issue, please fill out the following form.\r\n-->\r\n\r\n### System information\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**:\r\n- **Ray installed from (source or binary)**: pip install ray\r\n- **Ray version**: ray (0.3.1)\r\n- **Python version**: 2.7.13\r\n- **Exact command to reproduce**: import ray\r\n\r\n### Describe the problem\r\n<!-- Describe the problem clearly here. -->\r\nI get the following error when I try to import ray:\r\n\r\nimport ray\r\n  File \"/mnt/xfs1/home/mgabitto/python_cpu/lib/python2.7/site-packages/ray/__init__.py\", line 50, in <module>\r\n    from ray.worker import (error_info, init, connect, disconnect,\r\n  File \"/mnt/xfs1/home/mgabitto/python_cpu/lib/python2.7/site-packages/ray/worker.py\", line 946, in <module>\r\n    global_worker = Worker()\r\n  File \"/mnt/xfs1/home/mgabitto/python_cpu/lib/python2.7/site-packages/ray/worker.py\", line 238, in __init__\r\n    self.original_gpu_ids = ray.utils.get_cuda_visible_devices()\r\n  File \"/mnt/xfs1/home/mgabitto/python_cpu/lib/python2.7/site-packages/ray/utils.py\", line 133, in get_cuda_visible_devices\r\n    return [int(i) for i in gpu_ids_str.split(\",\")]\r\nValueError: invalid literal for int() with base 10: 'NoDevFiles'\r\n\r\n\r\n\r\n\r\n### Source code / logs\r\n<!-- Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem. -->\r\n\r\nThis is how I am running the code:\r\nsbatch -N 1 --exclusive runner_cpu.sh \r\n\r\nInside runner_cpu.sh is the following code:\r\n\r\n#!/bin/bash\r\n\r\n# activating the correct environment\r\nmodule load gcc/7.2.0\r\nmodule load cuda\r\nmodule load cudnn/v5.1-cuda-8.0\r\nsource ~/python_cpu/bin/activate\r\n\r\n# Running Command\r\nstdbuf -oL python code.py  >> log.log\r\n\r\nAny help appreciated\r\n\r\nThanks,\r\n\r\nM\r\n"},"comment":{"url":"https://api.github.com/repos/ray-project/ray/issues/comments/370462042","html_url":"https://github.com/ray-project/ray/issues/1653#issuecomment-370462042","issue_url":"https://api.github.com/repos/ray-project/ray/issues/1653","id":370462042,"user":{"login":"marianogabitto","id":4081325,"avatar_url":"https://avatars1.githubusercontent.com/u/4081325?v=4","gravatar_id":"","url":"https://api.github.com/users/marianogabitto","html_url":"https://github.com/marianogabitto","followers_url":"https://api.github.com/users/marianogabitto/followers","following_url":"https://api.github.com/users/marianogabitto/following{/other_user}","gists_url":"https://api.github.com/users/marianogabitto/gists{/gist_id}","starred_url":"https://api.github.com/users/marianogabitto/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marianogabitto/subscriptions","organizations_url":"https://api.github.com/users/marianogabitto/orgs","repos_url":"https://api.github.com/users/marianogabitto/repos","events_url":"https://api.github.com/users/marianogabitto/events{/privacy}","received_events_url":"https://api.github.com/users/marianogabitto/received_events","type":"User","site_admin":false},"created_at":"2018-03-05T15:46:21Z","updated_at":"2018-03-05T15:46:21Z","author_association":"NONE","body":"If I do \r\necho $CUDA_VISIBLE_DEVICES\r\n\r\non the command line it returns an empty line. But what we actually want is to run the command within a bash script. When I do it, I get \"NoDevFiles\"\r\n\r\nso basically I think this is the problem. Why do you have to query how many gpus does the system has? Can we omit this? some option to avoid gpu support?\r\n\r\nM\r\n"}}
{"actor":{"display_login":"marianogabitto"}}