{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/22307","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/22307/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/22307/comments","events_url":"https://api.github.com/repos/apache/spark/issues/22307/events","html_url":"https://github.com/apache/spark/pull/22307","id":356174240,"node_id":"MDExOlB1bGxSZXF1ZXN0MjEyNTMwNzcz","number":22307,"title":"[SPARK-25301][SQL] When a view uses an UDF from a non default database, Spark analyser throws AnalysisException","user":{"login":"vinodkc","id":9795818,"node_id":"MDQ6VXNlcjk3OTU4MTg=","avatar_url":"https://avatars3.githubusercontent.com/u/9795818?v=4","gravatar_id":"","url":"https://api.github.com/users/vinodkc","html_url":"https://github.com/vinodkc","followers_url":"https://api.github.com/users/vinodkc/followers","following_url":"https://api.github.com/users/vinodkc/following{/other_user}","gists_url":"https://api.github.com/users/vinodkc/gists{/gist_id}","starred_url":"https://api.github.com/users/vinodkc/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/vinodkc/subscriptions","organizations_url":"https://api.github.com/users/vinodkc/orgs","repos_url":"https://api.github.com/users/vinodkc/repos","events_url":"https://api.github.com/users/vinodkc/events{/privacy}","received_events_url":"https://api.github.com/users/vinodkc/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-09-01T05:32:42Z","updated_at":"2018-09-01T07:39:51Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/22307","html_url":"https://github.com/apache/spark/pull/22307","diff_url":"https://github.com/apache/spark/pull/22307.diff","patch_url":"https://github.com/apache/spark/pull/22307.patch"},"body":"## What changes were proposed in this pull request?\r\nWhen a hive view uses an UDF from a non default database, Spark analyser throws AnalysisException\r\n\r\nSteps to simulate this issue\r\n-----------------------------\r\nIn Hive\r\n--------\r\n1) CREATE DATABASE d100;\r\n2) ADD JAR /usr/udf/masking.jar // masking.jar has a custom udf class 'com.uzx.udf.Masking'\r\n3) create function d100.udf100 as \"com.uzx.udf.Masking\"; // Note: udf100 is created in d100\r\n4) create view d100.v100 as select d100.udf100(name)  from default.emp; // Note : table default.emp has two columns 'nanme', 'address', \r\n5) select * from d100.v100; // query on view d100.v100 gives correct result\r\n\r\nIn Spark\r\n-------------\r\n1) spark.sql(\"select * from d100.v100\").show\r\nthrows \r\n```\r\norg.apache.spark.sql.AnalysisException: Undefined function: 'd100.udf100'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'\r\n```\r\n\r\nThis is because, while parsing the SQL statement of the View \r\n``` 'select `d100.udf100`(`emp`.`name`) from `default`.`emp`' ``` , spark parser fails to split database name and udf name and hence Spark function registry tries to load the UDF 'd100.udf100' from 'default' database.\r\n\r\nTo solve this issue, before creating 'FunctionIdentifier' ,  try to  get  actual database name and then create FunctionIdentifier using  that database name and function name \r\n\r\n## How was this patch tested?\r\nAdded 1 unit test "},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/417840276","html_url":"https://github.com/apache/spark/pull/22307#issuecomment-417840276","issue_url":"https://api.github.com/repos/apache/spark/issues/22307","id":417840276,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNzg0MDI3Ng==","user":{"login":"AmplabJenkins","id":4031291,"node_id":"MDQ6VXNlcjQwMzEyOTE=","avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-09-01T07:39:51Z","updated_at":"2018-09-01T07:39:51Z","author_association":"NONE","body":"Test FAILed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95573/\nTest FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}