{"action":"opened","issue":{"url":"https://api.github.com/repos/spotify/luigi/issues/1712","repository_url":"https://api.github.com/repos/spotify/luigi","labels_url":"https://api.github.com/repos/spotify/luigi/issues/1712/labels{/name}","comments_url":"https://api.github.com/repos/spotify/luigi/issues/1712/comments","events_url":"https://api.github.com/repos/spotify/luigi/issues/1712/events","html_url":"https://github.com/spotify/luigi/issues/1712","id":158943176,"number":1712,"title":"Hive Examples Needed","user":{"login":"ericjmiller","id":5026753,"avatar_url":"https://avatars.githubusercontent.com/u/5026753?v=3","gravatar_id":"","url":"https://api.github.com/users/ericjmiller","html_url":"https://github.com/ericjmiller","followers_url":"https://api.github.com/users/ericjmiller/followers","following_url":"https://api.github.com/users/ericjmiller/following{/other_user}","gists_url":"https://api.github.com/users/ericjmiller/gists{/gist_id}","starred_url":"https://api.github.com/users/ericjmiller/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ericjmiller/subscriptions","organizations_url":"https://api.github.com/users/ericjmiller/orgs","repos_url":"https://api.github.com/users/ericjmiller/repos","events_url":"https://api.github.com/users/ericjmiller/events{/privacy}","received_events_url":"https://api.github.com/users/ericjmiller/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-06-07T14:50:53Z","updated_at":"2016-06-07T14:50:53Z","closed_at":null,"body":"I'm having difficulty creating a trivial workflow script.  I simply want to create a table (if it's not already there) viz CreateTable and put query results into that table viz HiveQuery.  Here is what I've writting so far:\r\n\r\n\r\n\r\n```\r\nimport luigi\r\nimport luigi.contrib.hive\r\n\r\n\r\nclass CreateTable(luigi.contrib.hive.HiveQueryTask):\r\n    def query(self):\r\n        return 'create table temp_batting_agg (col_value STRING);'\r\n\r\n\r\nclass HiveQuery(luigi.contrib.hive.HiveQueryTask):\r\n    def requires(self):\r\n        return CreateTable()\r\n\r\n    def query(self):\r\n        return 'select * from temp_batting limit 5;'\r\n\r\n    def output(self):\r\n        return luigi.contrib.hive.HiveTableTarget('temp_batting_agg')\r\n\r\nif __name__ == \"__main__\":\r\n    luigi.run()\r\n\r\n```\r\n\r\nThe current output:\r\n\r\n```\r\n[root@sandbox luigi]# PYTHONPATH=. luigi --module batting_flow HiveQuery --local-scheduler\r\nDEBUG: Checking if HiveQuery() is complete\r\nDEBUG: Checking Hive table 'default.temp_batting_agg' exists\r\nWARNING: Will not schedule HiveQuery() or any dependencies due to error in complete() method:\r\nTraceback (most recent call last):\r\n  File \"/usr/local/lib/python3.5/site-packages/luigi/worker.py\", line 285, in check_complete\r\n    is_complete = task.complete()\r\n  File \"/usr/local/lib/python3.5/site-packages/luigi/task.py\", line 389, in complete\r\n    return all(map(lambda output: output.exists(), outputs))\r\n  File \"/usr/local/lib/python3.5/site-packages/luigi/task.py\", line 389, in <lambda>\r\n    return all(map(lambda output: output.exists(), outputs))\r\n  File \"/usr/local/lib/python3.5/site-packages/luigi/contrib/hive.py\", line 382, in exists\r\n    return self.client.table_exists(self.table, self.database)\r\n  File \"/usr/local/lib/python3.5/site-packages/luigi/contrib/hive.py\", line 141, in table_exists\r\n    return stdout and table.lower() in stdout\r\nTypeError: a bytes-like object is required, not 'str'\r\n\r\nINFO: Skipping error email. Set `error-email` in the `core` section of the luigi config file or override `owner_email`in the task to receive error emails.\r\nINFO: Done scheduling tasks\r\nINFO: Running Worker with 1 processes\r\nDEBUG: Asking scheduler for work...\r\nDEBUG: Done\r\nDEBUG: There are no more tasks to run at this time\r\nINFO: Worker Worker(salt=620827500, workers=1, host=sandbox.hortonworks.com, username=root, pid=30746) was stopped. Shutting down Keep-Alive thread\r\nINFO:\r\n===== Luigi Execution Summary =====\r\n\r\nDid not schedule any tasks\r\n\r\n===== Luigi Execution Summary =====\r\n```\r\n\r\nI'll submit a PR with a solid Hive example - I think it's needed because I'm having significant trouble doing anything with the results of a query.  The query will run, but I can't manipulate them or write them to a table, etc.\r\n\r\nThanks!"}}
{"actor":{"display_login":"ericjmiller"}}