{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/14371","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/14371/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/14371/comments","events_url":"https://api.github.com/repos/apache/spark/issues/14371/events","html_url":"https://github.com/apache/spark/pull/14371","id":167669898,"number":14371,"title":"[SPARK-16736] WiP Core+ SQL superfluous fs calls","user":{"login":"steveloughran","id":162090,"avatar_url":"https://avatars.githubusercontent.com/u/162090?v=3","gravatar_id":"","url":"https://api.github.com/users/steveloughran","html_url":"https://github.com/steveloughran","followers_url":"https://api.github.com/users/steveloughran/followers","following_url":"https://api.github.com/users/steveloughran/following{/other_user}","gists_url":"https://api.github.com/users/steveloughran/gists{/gist_id}","starred_url":"https://api.github.com/users/steveloughran/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/steveloughran/subscriptions","organizations_url":"https://api.github.com/users/steveloughran/orgs","repos_url":"https://api.github.com/users/steveloughran/repos","events_url":"https://api.github.com/users/steveloughran/events{/privacy}","received_events_url":"https://api.github.com/users/steveloughran/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2016-07-26T17:51:34Z","updated_at":"2016-07-26T19:59:38Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/14371","html_url":"https://github.com/apache/spark/pull/14371","diff_url":"https://github.com/apache/spark/pull/14371.diff","patch_url":"https://github.com/apache/spark/pull/14371.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nA review of the code, working back from Hadoop's `FileSystem.exists()` and `FileSystem.isDirectory()` code, then removing uses of the calls when superfluous.\r\n\r\n1. delete is harmless if called on a nonexistent path, so don't do any checks before deletes\r\n1. any `FileSystem.exists()`  check before `getFileStatus()` or `open()` is superfluous as the operation itself does the check. Instead the `FileNotFoundException` is caught and triggers the downgraded path. When a `FileNotFoundException` was thrown before, the code still creates a new FNFE with the error messages. Though now the inner exceptions are nested, for easier diagnostics.\r\n\r\n## How was this patch tested?\r\n\r\nInitially, relying on Jenkins test runs.\r\n\r\nOne troublespot here is that some of the codepaths are clearly error situations; it's not clear that they have coverage anyway. Trying to create the failure conditions in tests would be ideal, but it will also be hard."},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/235386476","html_url":"https://github.com/apache/spark/pull/14371#issuecomment-235386476","issue_url":"https://api.github.com/repos/apache/spark/issues/14371","id":235386476,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars.githubusercontent.com/u/4031291?v=3","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2016-07-26T19:59:38Z","updated_at":"2016-07-26T19:59:38Z","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/62898/\nTest PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}