{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/13759","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/13759/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/13759/comments","events_url":"https://api.github.com/repos/apache/spark/issues/13759/events","html_url":"https://github.com/apache/spark/pull/13759","id":161011179,"number":13759,"title":"[SPARK-16044][SQL] input_file_name() returns empty strings in data sources based on NewHadoopRDD","user":{"login":"HyukjinKwon","id":6477701,"avatar_url":"https://avatars.githubusercontent.com/u/6477701?v=3","gravatar_id":"","url":"https://api.github.com/users/HyukjinKwon","html_url":"https://github.com/HyukjinKwon","followers_url":"https://api.github.com/users/HyukjinKwon/followers","following_url":"https://api.github.com/users/HyukjinKwon/following{/other_user}","gists_url":"https://api.github.com/users/HyukjinKwon/gists{/gist_id}","starred_url":"https://api.github.com/users/HyukjinKwon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HyukjinKwon/subscriptions","organizations_url":"https://api.github.com/users/HyukjinKwon/orgs","repos_url":"https://api.github.com/users/HyukjinKwon/repos","events_url":"https://api.github.com/users/HyukjinKwon/events{/privacy}","received_events_url":"https://api.github.com/users/HyukjinKwon/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-06-18T07:07:22Z","updated_at":"2016-06-21T04:58:53Z","closed_at":"2016-06-21T04:56:58Z","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/13759","html_url":"https://github.com/apache/spark/pull/13759","diff_url":"https://github.com/apache/spark/pull/13759.diff","patch_url":"https://github.com/apache/spark/pull/13759.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR makes `input_file_name()` function return the file paths not empty strings for external data sources based on `NewHadoopRDD`, such as [spark-redshift](https://github.com/databricks/spark-redshift/blob/cba5eee1ab79ae8f0fa9e668373a54d2b5babf6b/src/main/scala/com/databricks/spark/redshift/RedshiftRelation.scala#L149) and [spark-xml](https://github.com/databricks/spark-xml/blob/master/src/main/scala/com/databricks/spark/xml/util/XmlFile.scala#L39-L47).\r\n\r\nThe codes with the external data sources below:\r\n\r\n```scala\r\ndf.select(input_file_name).show()\r\n```\r\n\r\nwill produce\r\n\r\n- **Before**\r\n  ```\r\n+-----------------+\r\n|input_file_name()|\r\n+-----------------+\r\n|                 |\r\n+-----------------+\r\n```\r\n\r\n- **After**\r\n  ```\r\n+--------------------+\r\n|   input_file_name()|\r\n+--------------------+\r\n|file:/private/var...|\r\n+--------------------+\r\n```\r\n\r\n## How was this patch tested?\r\n\r\nUnit tests in `ColumnExpressionSuite`."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/227340828","html_url":"https://github.com/apache/spark/pull/13759#issuecomment-227340828","issue_url":"https://api.github.com/repos/apache/spark/issues/13759","id":227340828,"user":{"login":"rxin","id":323388,"avatar_url":"https://avatars.githubusercontent.com/u/323388?v=3","gravatar_id":"","url":"https://api.github.com/users/rxin","html_url":"https://github.com/rxin","followers_url":"https://api.github.com/users/rxin/followers","following_url":"https://api.github.com/users/rxin/following{/other_user}","gists_url":"https://api.github.com/users/rxin/gists{/gist_id}","starred_url":"https://api.github.com/users/rxin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rxin/subscriptions","organizations_url":"https://api.github.com/users/rxin/orgs","repos_url":"https://api.github.com/users/rxin/repos","events_url":"https://api.github.com/users/rxin/events{/privacy}","received_events_url":"https://api.github.com/users/rxin/received_events","type":"User","site_admin":false},"created_at":"2016-06-21T04:58:53Z","updated_at":"2016-06-21T04:58:53Z","body":"Can you submit a pr for 1.6? Thanks.\r\n"}}
{"actor":{"display_login":"rxin"}}