{"action":"created","issue":{"url":"https://api.github.com/repos/apache/carbondata/issues/3019","repository_url":"https://api.github.com/repos/apache/carbondata","labels_url":"https://api.github.com/repos/apache/carbondata/issues/3019/labels{/name}","comments_url":"https://api.github.com/repos/apache/carbondata/issues/3019/comments","events_url":"https://api.github.com/repos/apache/carbondata/issues/3019/events","html_url":"https://github.com/apache/carbondata/pull/3019","id":393775653,"node_id":"MDExOlB1bGxSZXF1ZXN0MjQwNjk3ODQx","number":3019,"title":"[CARBONDATA-3194] Integrating Carbon with Presto using hive connector","user":{"login":"ravipesala","id":8134086,"node_id":"MDQ6VXNlcjgxMzQwODY=","avatar_url":"https://avatars2.githubusercontent.com/u/8134086?v=4","gravatar_id":"","url":"https://api.github.com/users/ravipesala","html_url":"https://github.com/ravipesala","followers_url":"https://api.github.com/users/ravipesala/followers","following_url":"https://api.github.com/users/ravipesala/following{/other_user}","gists_url":"https://api.github.com/users/ravipesala/gists{/gist_id}","starred_url":"https://api.github.com/users/ravipesala/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ravipesala/subscriptions","organizations_url":"https://api.github.com/users/ravipesala/orgs","repos_url":"https://api.github.com/users/ravipesala/repos","events_url":"https://api.github.com/users/ravipesala/events{/privacy}","received_events_url":"https://api.github.com/users/ravipesala/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":27,"created_at":"2018-12-23T19:38:43Z","updated_at":"2018-12-28T13:58:52Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/carbondata/pulls/3019","html_url":"https://github.com/apache/carbondata/pull/3019","diff_url":"https://github.com/apache/carbondata/pull/3019.diff","patch_url":"https://github.com/apache/carbondata/pull/3019.patch"},"body":"This PR depends on PR https://github.com/apache/carbondata/pull/3004\r\n\r\nProblem:\r\nCurrent Carbon Presto integration added a new presto connector that takes \r\nthe carbon store folder and lists the databases and tables from the folders. \r\nIn this implementation, we have many issues like. \r\n1. DB and table always need to be in specific order and name of the folders \r\nshould always match the DB name and table name. \r\n2. The table which is created in presto cannot be reflected directly in \r\nother execution engines like Spark. \r\n3. DB with location and table with location cannot work. \r\n4. There will not be any access control on tables. \r\n5. There is no interoperability between hive tables like ORC or Parquet with \r\ncarbon. Like if we want to join some hive table with Carbon Table then it \r\nwon't be possible. \r\n\r\nSolution:\r\nExtend the `CarbondataConnectorFactory` with `HiveConnectorFactory`, so that all features of HIve presto connector will be inherited to carbon as well.\r\nIt simplifies the integration so removed lot of old code.\r\n\r\nNow with carbon connector user can create all the tables supported by presto Hive along with carbon.\r\n```\r\ncreate table sdk_output.files(name varchar, age int, height double) with(format='CARBON')\r\n```\r\nPresto Hive supports three types of metastores \r\n1. ThriftMetastoreModule, used to connect to Hive metastore and run the queries on it.\r\n2. FileMetastoreModule, used to store schema on the folder itself when creating the tables. It is mostly used for testing, not for production.\r\n3. GlueMetastoreModule, used to connect to Amazon's glue metastore.\r\n\r\nSo now on carbon also can support all the above three metastores.\r\n\r\nNote: Old file based store itegration cannot work now.\r\n\r\nBe sure to do all of the following checklist to help us incorporate \r\nyour contribution quickly and easily:\r\n\r\n - [ ] Any interfaces changed?\r\n \r\n - [ ] Any backward compatibility impacted?\r\n \r\n - [ ] Document update required?\r\n\r\n - [ ] Testing done\r\n        Please provide details on \r\n        - Whether new unit test cases have been added or why no new tests are required?\r\n        - How it is tested? Please attach test report.\r\n        - Is it a performance related change? Please attach the performance test report.\r\n        - Any additional information to help reviewers in testing this change.\r\n       \r\n - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. \r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/carbondata/issues/comments/450363173","html_url":"https://github.com/apache/carbondata/pull/3019#issuecomment-450363173","issue_url":"https://api.github.com/repos/apache/carbondata/issues/3019","id":450363173,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MDM2MzE3Mw==","user":{"login":"CarbonDataQA","id":24402133,"node_id":"MDQ6VXNlcjI0NDAyMTMz","avatar_url":"https://avatars0.githubusercontent.com/u/24402133?v=4","gravatar_id":"","url":"https://api.github.com/users/CarbonDataQA","html_url":"https://github.com/CarbonDataQA","followers_url":"https://api.github.com/users/CarbonDataQA/followers","following_url":"https://api.github.com/users/CarbonDataQA/following{/other_user}","gists_url":"https://api.github.com/users/CarbonDataQA/gists{/gist_id}","starred_url":"https://api.github.com/users/CarbonDataQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/CarbonDataQA/subscriptions","organizations_url":"https://api.github.com/users/CarbonDataQA/orgs","repos_url":"https://api.github.com/users/CarbonDataQA/repos","events_url":"https://api.github.com/users/CarbonDataQA/events{/privacy}","received_events_url":"https://api.github.com/users/CarbonDataQA/received_events","type":"User","site_admin":false},"created_at":"2018-12-28T13:58:52Z","updated_at":"2018-12-28T13:58:52Z","author_association":"NONE","body":"Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2070/\n"}}
{"actor":{"display_login":"CarbonDataQA"}}