{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12618","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12618/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12618/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12618/events","html_url":"https://github.com/apache/spark/pull/12618","id":150461782,"number":12618,"title":"[SPARK-14857] [SQL] Table/Database Name Validation in SessionCatalog","user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":23,"created_at":"2016-04-22T20:16:24Z","updated_at":"2016-05-23T23:13:12Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12618","html_url":"https://github.com/apache/spark/pull/12618","diff_url":"https://github.com/apache/spark/pull/12618.diff","patch_url":"https://github.com/apache/spark/pull/12618.patch"},"body":"#### What changes were proposed in this pull request?\r\nTheir PR is to validate the database/table names before storing these information in `ExternalCatalog`. \r\n\r\nThis PR is part of the efforts to reduce the dependencies on Hive meta-store for catching any illegal input. \r\n\r\nFor example, if users use `backstick` to quote the table/database names containing illegal characters, these names are allowed by Spark Parser, but Hive metastore does not allow them. We need to catch them in SessionCatalog and issue an appropriate error message. We are facing the same issues for creating data source tables.\r\n```\r\nCREATE TABLE `tab:1`  ...\r\n```\r\n\r\nThis PR enforces the name rules of Spark SQL for `table`/`database`/`view`: `only can contain alphanumeric and underscore characters.` Different from Hive, we allow the names with starting underscore characters. \r\n\r\nWhen the `ExternalCatalog` is Hive Metastore, we add an extra check by using `MetaStoreUtils.validateName` to verify its validity. \r\n\r\nIn the future PRs, we will continue the work on validation of function/column names. \r\n\r\n**Updated**: Running SQL on files directly could temporarily break the naming rules. For example,\r\n```SQL\r\n  select id from `org.apache.spark.sql.parquet`.`path/to/parquet/files` as p\r\n```\r\nThis was resolved by an `Analyzer` rule: `ResolveDataSource`. Thus, it should not exist after finishing `Analyzer`. Thus, it is ok to keep enforcing the existing naming rule, if we do not validate the names in `databaseExists` and `tableExists`. \r\n\r\n#### How was this patch tested?\r\nAdded test cases for ensuring the coverage."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/221123110","html_url":"https://github.com/apache/spark/pull/12618#issuecomment-221123110","issue_url":"https://api.github.com/repos/apache/spark/issues/12618","id":221123110,"user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"created_at":"2016-05-23T23:13:11Z","updated_at":"2016-05-23T23:13:11Z","body":"retest this please"}}
{"actor":{"display_login":"gatorsmile"}}