{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/21306","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/21306/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/21306/comments","events_url":"https://api.github.com/repos/apache/spark/issues/21306/events","html_url":"https://github.com/apache/spark/pull/21306","id":322438565,"node_id":"MDExOlB1bGxSZXF1ZXN0MTg3NTcyOTEz","number":21306,"title":"[SPARK-24252][SQL] Add catalog registration and table catalog APIs.","user":{"login":"rdblue","id":87915,"node_id":"MDQ6VXNlcjg3OTE1","avatar_url":"https://avatars1.githubusercontent.com/u/87915?v=4","gravatar_id":"","url":"https://api.github.com/users/rdblue","html_url":"https://github.com/rdblue","followers_url":"https://api.github.com/users/rdblue/followers","following_url":"https://api.github.com/users/rdblue/following{/other_user}","gists_url":"https://api.github.com/users/rdblue/gists{/gist_id}","starred_url":"https://api.github.com/users/rdblue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rdblue/subscriptions","organizations_url":"https://api.github.com/users/rdblue/orgs","repos_url":"https://api.github.com/users/rdblue/repos","events_url":"https://api.github.com/users/rdblue/events{/privacy}","received_events_url":"https://api.github.com/users/rdblue/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":24,"created_at":"2018-05-11T21:21:56Z","updated_at":"2018-08-18T00:32:21Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/21306","html_url":"https://github.com/apache/spark/pull/21306","diff_url":"https://github.com/apache/spark/pull/21306.diff","patch_url":"https://github.com/apache/spark/pull/21306.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis adds a v2 API for storage implementations to provide catalog instances to Spark. There are two parts:\r\n\r\n* `CatalogProvider` is used to instantiate and initialize catalogs via reflection, similar to data sources\r\n* `TableCatalog` provides table operations proposed in the [Table Metadata API SPIP](https://docs.google.com/document/d/1zLFiA1VuaWeVxeTDXNg8bL6GP3BVoOZBkewFtEnjEoo/edit#)\r\n\r\nThis also adds helper classes:\r\n\r\n* `Catalogs` loads and initializes catalogs using configuration from a `SQLConf`\r\n* `CaseInsensitiveStringMap` is used to pass configuration to `CatalogProvider` via `initialize`\r\n* `PartitionTransform` is used to pass table partitioning without using `Expression`\r\n\r\nCatalogs are configured by adding config properties starting with `spark.sql.catalog.(name)`. The name property must specify a class that implements `CatalogProvider`. Other properties under the namespace (`spark.sql.catalog.(name).(prop)`) are passed to the provider during initialization.\r\n\r\n## How was this patch tested?\r\n\r\nThis includes a suite for `CaseInsensitiveStringMap` and one that tests loading catalog providers."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/414018667","html_url":"https://github.com/apache/spark/pull/21306#issuecomment-414018667","issue_url":"https://api.github.com/repos/apache/spark/issues/21306","id":414018667,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNDAxODY2Nw==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-08-18T00:32:21Z","updated_at":"2018-08-18T00:32:21Z","author_association":"NONE","body":"Merged build finished. Test PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}