{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12117","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12117/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12117/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12117/events","html_url":"https://github.com/apache/spark/pull/12117","id":145295081,"number":12117,"title":"[SPARK-14123] [SQL] Handle CreateFunction/DropFunction","user":{"login":"yhuai","id":2072857,"avatar_url":"https://avatars.githubusercontent.com/u/2072857?v=3","gravatar_id":"","url":"https://api.github.com/users/yhuai","html_url":"https://github.com/yhuai","followers_url":"https://api.github.com/users/yhuai/followers","following_url":"https://api.github.com/users/yhuai/following{/other_user}","gists_url":"https://api.github.com/users/yhuai/gists{/gist_id}","starred_url":"https://api.github.com/users/yhuai/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/yhuai/subscriptions","organizations_url":"https://api.github.com/users/yhuai/orgs","repos_url":"https://api.github.com/users/yhuai/repos","events_url":"https://api.github.com/users/yhuai/events{/privacy}","received_events_url":"https://api.github.com/users/yhuai/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":12,"created_at":"2016-04-01T21:56:31Z","updated_at":"2016-04-04T05:27:34Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12117","html_url":"https://github.com/apache/spark/pull/12117","diff_url":"https://github.com/apache/spark/pull/12117.diff","patch_url":"https://github.com/apache/spark/pull/12117.patch"},"body":"## What changes were proposed in this pull request?\r\nThis PR implements CreateFunction and DropFunction commands. Besides implementing these two commands, we also change how to manage functions. Here are the main changes.\r\n* `FunctionRegistry` will be a container to store all functions builders and it will not actively load any functions. Because of this change, we do not need to maintain a separate registry for HiveContext. So, `HiveFunctionRegistry` is deleted.\r\n* SessionCatalog takes care the job of loading a function if this function is not in the `FunctionRegistry` but its metadata is stored in the external catalog. For this case, SessionCatalog will (1) load the metadata from the external catalog, (2) load all needed resources (i.e. jars and files), (3) create a function builder based on the function definition, (4) register the function builder in the `FunctionRegistry`.\r\n* A `UnresolvedGenerator` is created. So, the parser will not need to call `FunctionRegistry` directly during parsing, which is not a good time to create a Hive UDTF. In the analysis phase, we will resolve `UnresolvedGenerator`.\r\n\r\nThis PR is based on @viirya's https://github.com/apache/spark/pull/12036/ \r\n\r\n## How was this patch tested?\r\nExisting tests and new tests.\r\n\r\n## TODOs\r\n[x] Self-review\r\n[x] Cleanup\r\n[x] More tests for create/drop functions (we need to more tests for permanent functions).\r\n[ ] File JIRAs for all TODOs\r\n[x] Standardize the error message when a function does not exist."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/205143016","html_url":"https://github.com/apache/spark/pull/12117#issuecomment-205143016","issue_url":"https://api.github.com/repos/apache/spark/issues/12117","id":205143016,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-04-04T05:27:34Z","updated_at":"2016-04-04T05:27:34Z","body":"**[Test build #54824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/54824/consoleFull)** for PR 12117 at commit [`9d39a83`](https://github.com/apache/spark/commit/9d39a835a25ea2a509fa490099a7923b70e8bcbd)."}}