{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/14962","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/14962/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/14962/comments","events_url":"https://api.github.com/repos/apache/spark/issues/14962/events","html_url":"https://github.com/apache/spark/pull/14962","id":175078383,"number":14962,"title":"[SPARK-17402][SQL] separate the management of temp views and metastore tables/views in SessionCatalog","user":{"login":"cloud-fan","id":3182036,"avatar_url":"https://avatars.githubusercontent.com/u/3182036?v=3","gravatar_id":"","url":"https://api.github.com/users/cloud-fan","html_url":"https://github.com/cloud-fan","followers_url":"https://api.github.com/users/cloud-fan/followers","following_url":"https://api.github.com/users/cloud-fan/following{/other_user}","gists_url":"https://api.github.com/users/cloud-fan/gists{/gist_id}","starred_url":"https://api.github.com/users/cloud-fan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cloud-fan/subscriptions","organizations_url":"https://api.github.com/users/cloud-fan/orgs","repos_url":"https://api.github.com/users/cloud-fan/repos","events_url":"https://api.github.com/users/cloud-fan/events{/privacy}","received_events_url":"https://api.github.com/users/cloud-fan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-09-05T14:04:07Z","updated_at":"2016-09-06T08:37:44Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/14962","html_url":"https://github.com/apache/spark/pull/14962","diff_url":"https://github.com/apache/spark/pull/14962.diff","patch_url":"https://github.com/apache/spark/pull/14962.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nIn `SessionCatalog`, we have several operations(`getTableMetadata`, `tableExists`, `renameTable`, `dropTable`) that handle both temp views and metastore tables/views. They can save some lines of code for some commands that need to deal with both temp views and metastore tables/views, but also introduce bugs for other commands, because the operation names say nothing about temp views and are very likely to be misused:\r\n\r\n* `DataFrameWriter.saveAsTable`/`CREATE TABLE USING` will fail if a same-name temp view exits\r\n* `Catalog.dropTempView` may drop metastore table mistakenly\r\n* `ALTER TABLE RECOVER PARTITIONS`/`LOAD DATA`/`TRUNCATE TABLE`/`SHOW CREATE TABLE` should report \"table not found\" instead of \"temp view is not supported\", if a same-name temp view exists, because these commands don't need to deal with temp views.\r\n\r\nIn some commands we support temp views mistakenly without mentioning it in document: `ShowTablePropertiesCommand`, `ShowColumnsCommand`, `Catalog.listColumns`.\r\n\r\nMixing the handling of temp views and metastore tables/views also makes it harder to implement thread-safe operations. e.g. `AlterViewAsCommand` checks `isTemporaryTable` first then `createTempView`, which is not atomic. Most temp view related operations in `SessionCatalog` hold a lock on the `SessionCatalog` object, which is unnecessary.\r\n\r\n\r\nThis PR separates the management of temp views and metastore tables/views in `SessionCatalog`, any commands that need to deal with temp views should explicitly call temp view related operations in `SessionCatalog`, to fix existing bugs and prevent future bug like this.\r\n\r\n## How was this patch tested?\r\n\r\nexisting tests and 3 new tests."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/244886092","html_url":"https://github.com/apache/spark/pull/14962#issuecomment-244886092","issue_url":"https://api.github.com/repos/apache/spark/issues/14962","id":244886092,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-09-06T08:37:44Z","updated_at":"2016-09-06T08:37:44Z","body":"**[Test build #64984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/64984/consoleFull)** for PR 14962 at commit [`7314a33`](https://github.com/apache/spark/commit/7314a33e1d5455bf551594474ff787d0ce91dcec)."}}
{"actor":{"display_login":"SparkQA"}}