{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/20611","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/20611/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/20611/comments","events_url":"https://api.github.com/repos/apache/spark/issues/20611/events","html_url":"https://github.com/apache/spark/pull/20611","id":297206298,"number":20611,"title":"[SPARK-23425][SQL]When wild card is been used  in load command system is throwing  analysis exception","user":{"login":"sujith71955","id":12999161,"avatar_url":"https://avatars1.githubusercontent.com/u/12999161?v=4","gravatar_id":"","url":"https://api.github.com/users/sujith71955","html_url":"https://github.com/sujith71955","followers_url":"https://api.github.com/users/sujith71955/followers","following_url":"https://api.github.com/users/sujith71955/following{/other_user}","gists_url":"https://api.github.com/users/sujith71955/gists{/gist_id}","starred_url":"https://api.github.com/users/sujith71955/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sujith71955/subscriptions","organizations_url":"https://api.github.com/users/sujith71955/orgs","repos_url":"https://api.github.com/users/sujith71955/repos","events_url":"https://api.github.com/users/sujith71955/events{/privacy}","received_events_url":"https://api.github.com/users/sujith71955/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-02-14T18:54:16Z","updated_at":"2018-02-18T16:00:38Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/20611","html_url":"https://github.com/apache/spark/pull/20611","diff_url":"https://github.com/apache/spark/pull/20611.diff","patch_url":"https://github.com/apache/spark/pull/20611.patch"},"body":"## What changes were proposed in this pull request?\r\nA validaton logic is been added for non local files, Error will be thrown,If hdfs path doest not exist or if no files matches the wild card defined in load path.\r\nfs.exists(srcPath) API cannot resolve the path with wild card pattern, so globStatus() api is been used which can resolve the paths with hdfs supported wildcard\r\nstring like *,? etc.\r\n## How was this patch tested?\r\nManually tested in hdfs-Yarn cluster, please find the attached verification report below.\r\nBefore fix\r\n![hdfs_path_snapshot](https://user-images.githubusercontent.com/12999161/36221919-aa322450-11e5-11e8-976a-d01db5d2cebd.PNG)\r\n![wildcard_issue](https://user-images.githubusercontent.com/12999161/36221932-b50bb0da-11e5-11e8-8c64-b7c0f2ce1ec7.PNG)\r\n\r\nAfter Fix\r\n![afterfix_wildcard](https://user-images.githubusercontent.com/12999161/36221953-ca18cc74-11e5-11e8-940e-b25cff212066.PNG)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/366526013","html_url":"https://github.com/apache/spark/pull/20611#issuecomment-366526013","issue_url":"https://api.github.com/repos/apache/spark/issues/20611","id":366526013,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-02-18T16:00:37Z","updated_at":"2018-02-18T16:00:37Z","author_association":"NONE","body":"Merged build finished. Test FAILed."}}
{"actor":{"display_login":"AmplabJenkins"}}