{"action":"created","issue":{"url":"https://api.github.com/repos/jghoman/haivvreo/issues/31","repository_url":"https://api.github.com/repos/jghoman/haivvreo","labels_url":"https://api.github.com/repos/jghoman/haivvreo/issues/31/labels{/name}","comments_url":"https://api.github.com/repos/jghoman/haivvreo/issues/31/comments","events_url":"https://api.github.com/repos/jghoman/haivvreo/issues/31/events","html_url":"https://github.com/jghoman/haivvreo/issues/31","id":130751751,"number":31,"title":"How to read binary encoded data","user":{"login":"vkuznet","id":125138,"avatar_url":"https://avatars.githubusercontent.com/u/125138?v=3","gravatar_id":"","url":"https://api.github.com/users/vkuznet","html_url":"https://github.com/vkuznet","followers_url":"https://api.github.com/users/vkuznet/followers","following_url":"https://api.github.com/users/vkuznet/following{/other_user}","gists_url":"https://api.github.com/users/vkuznet/gists{/gist_id}","starred_url":"https://api.github.com/users/vkuznet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vkuznet/subscriptions","organizations_url":"https://api.github.com/users/vkuznet/orgs","repos_url":"https://api.github.com/users/vkuznet/repos","events_url":"https://api.github.com/users/vkuznet/events{/privacy}","received_events_url":"https://api.github.com/users/vkuznet/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-02-02T17:19:37Z","updated_at":"2016-02-03T00:50:16Z","closed_at":"2016-02-03T00:37:26Z","body":"Hi, we're trying to use hive over external data we store on HDFS. The data are in avro format but their content was binary encoded. We used python software to that and it happy provides a way to encode our data in binary format. Now, when I setup Hive table and pointed it to use our scheme it complains with the following exception:\r\n```\r\nFailed with exception java.io.IOException:java.io.IOException: Not a data file.\r\n16/02/02 17:53:39 [main]: ERROR CliDriver: Failed with exception java.io.IOException:java.io.IOException: Not a data file.\r\njava.io.IOException: java.io.IOException: Not a data file.\r\n        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)\r\n        at org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)\r\n        at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:138)\r\n        at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1655)\r\n        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:227)\r\n        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159)\r\n        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370)\r\n        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:756)\r\n        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)\r\n        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)\r\n        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\r\n        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\r\n        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\r\n        at java.lang.reflect.Method.invoke(Method.java:606)\r\n        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)\r\n        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)\r\nCaused by: java.io.IOException: Not a data file.\r\n        at org.apache.avro.file.DataFileStream.initialize(DataFileStream.java:105)\r\n        at org.apache.avro.file.DataFileReader.<init>(DataFileReader.java:97)\r\n        at org.apache.hadoop.hive.ql.io.avro.AvroGenericRecordReader.<init>(AvroGenericRecordReader.java:81)\r\n        at org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat.getRecordReader(AvroContainerInputFormat.java:51)\r\n        at org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:667)\r\n        at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323)\r\n        at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)\r\n        ... 15 more\r\n```\r\nIf I read exception right, it tries to use AvroGenericRecordReader class to read the data. Therefore I want to know if it is possible to instruct hive to use binary Avro encoder to read data stored in binary avro format. I'm not java expert and would appreciate if you'll provide as much details as possible that we can think how to address this problem. Thanks, Valentin."},"comment":{"url":"https://api.github.com/repos/jghoman/haivvreo/issues/comments/178923702","html_url":"https://github.com/jghoman/haivvreo/issues/31#issuecomment-178923702","issue_url":"https://api.github.com/repos/jghoman/haivvreo/issues/31","id":178923702,"user":{"login":"jghoman","id":95350,"avatar_url":"https://avatars.githubusercontent.com/u/95350?v=3","gravatar_id":"","url":"https://api.github.com/users/jghoman","html_url":"https://github.com/jghoman","followers_url":"https://api.github.com/users/jghoman/followers","following_url":"https://api.github.com/users/jghoman/following{/other_user}","gists_url":"https://api.github.com/users/jghoman/gists{/gist_id}","starred_url":"https://api.github.com/users/jghoman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jghoman/subscriptions","organizations_url":"https://api.github.com/users/jghoman/orgs","repos_url":"https://api.github.com/users/jghoman/repos","events_url":"https://api.github.com/users/jghoman/events{/privacy}","received_events_url":"https://api.github.com/users/jghoman/received_events","type":"User","site_admin":false},"created_at":"2016-02-03T00:50:16Z","updated_at":"2016-02-03T00:50:16Z","body":"Awesome! Glad to help."}}