{"action":"opened","issue":{"url":"https://api.github.com/repos/Yelp/mrjob/issues/1826","repository_url":"https://api.github.com/repos/Yelp/mrjob","labels_url":"https://api.github.com/repos/Yelp/mrjob/issues/1826/labels{/name}","comments_url":"https://api.github.com/repos/Yelp/mrjob/issues/1826/comments","events_url":"https://api.github.com/repos/Yelp/mrjob/issues/1826/events","html_url":"https://github.com/Yelp/mrjob/issues/1826","id":355551603,"node_id":"MDU6SXNzdWUzNTU1NTE2MDM=","number":1826,"title":"Cannot input hdfs path to mrjob on local Hadoop","user":{"login":"akshedu","id":13628154,"node_id":"MDQ6VXNlcjEzNjI4MTU0","avatar_url":"https://avatars1.githubusercontent.com/u/13628154?v=4","gravatar_id":"","url":"https://api.github.com/users/akshedu","html_url":"https://github.com/akshedu","followers_url":"https://api.github.com/users/akshedu/followers","following_url":"https://api.github.com/users/akshedu/following{/other_user}","gists_url":"https://api.github.com/users/akshedu/gists{/gist_id}","starred_url":"https://api.github.com/users/akshedu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/akshedu/subscriptions","organizations_url":"https://api.github.com/users/akshedu/orgs","repos_url":"https://api.github.com/users/akshedu/repos","events_url":"https://api.github.com/users/akshedu/events{/privacy}","received_events_url":"https://api.github.com/users/akshedu/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-30T11:54:40Z","updated_at":"2018-08-30T11:54:40Z","closed_at":null,"author_association":"NONE","body":"I think its a straightforward application of the mrjob library. I am not able to use hdfs file input:\r\n\r\n```(mrjob) Workspace $ python cc-mrjob/word_count_classic.py --hadoop-streaming-jar /usr/local/Cellar/hadoop/3.1.1/libexec/share/hadoop/tools/lib/hadoop-streaming-3.1.1.jar -r hadoop hdfs:///workspace/pg2701.txt\r\nUsing configs in /Users/akshanshgupta/.mrjob.conf\r\nCreating temp directory /var/folders/xz/g4k09fps1bd1vm9r4mj8dgnm0000gn/T/word_count_classic.akshanshgupta.20180830.115026.352538\r\nArchiving /Users/akshanshgupta/Workspace/cc-mrjob/packages -> /var/folders/xz/g4k09fps1bd1vm9r4mj8dgnm0000gn/T/word_count_classic.akshanshgupta.20180830.115026.352538/archives/packages.tar.gz\r\nLooking for hadoop binary in $PATH...\r\nFound hadoop binary: /usr/local/bin/hadoop\r\nSTDERR: 2018-08-30 17:20:40,123 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\r\nSTDERR: ls: `hdfs:///workspace/pg2701.txt': No such file or directory\r\nTraceback (most recent call last):\r\n  File \"cc-mrjob/word_count_classic.py\", line 31, in <module>\r\n    MRWordFreqCount.run()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/job.py\", line 436, in run\r\n    mr_job.execute()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/job.py\", line 457, in execute\r\n    super(MRJob, self).execute()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/launch.py\", line 187, in execute\r\n    self.run_job()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/launch.py\", line 235, in run_job\r\n    runner.run()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/runner.py\", line 515, in run\r\n    self._check_input_paths()\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/runner.py\", line 1007, in _check_input_paths\r\n    if not self.fs.exists(path):\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/fs/composite.py\", line 84, in exists\r\n    return self._do_action('exists', path_glob)\r\n  File \"/Users/akshanshgupta/mrjob/lib/python2.7/site-packages/mrjob/fs/composite.py\", line 68, in _do_action\r\n    raise first_exception\r\nIOError: Could not check path hdfs:///workspace/pg2701.txt\r\n```"}}
{"actor":{"display_login":"akshedu"}}