{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/20717","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/20717/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/20717/comments","events_url":"https://api.github.com/repos/apache/spark/issues/20717/events","html_url":"https://github.com/apache/spark/pull/20717","id":301827499,"number":20717,"title":"[SPARK-23564][SQL] Add isNotNull check for left anti and outer joins","user":{"login":"mgaido91","id":8821783,"avatar_url":"https://avatars1.githubusercontent.com/u/8821783?v=4","gravatar_id":"","url":"https://api.github.com/users/mgaido91","html_url":"https://github.com/mgaido91","followers_url":"https://api.github.com/users/mgaido91/followers","following_url":"https://api.github.com/users/mgaido91/following{/other_user}","gists_url":"https://api.github.com/users/mgaido91/gists{/gist_id}","starred_url":"https://api.github.com/users/mgaido91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mgaido91/subscriptions","organizations_url":"https://api.github.com/users/mgaido91/orgs","repos_url":"https://api.github.com/users/mgaido91/repos","events_url":"https://api.github.com/users/mgaido91/events{/privacy}","received_events_url":"https://api.github.com/users/mgaido91/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-03-02T16:36:02Z","updated_at":"2018-03-10T14:03:47Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/20717","html_url":"https://github.com/apache/spark/pull/20717","diff_url":"https://github.com/apache/spark/pull/20717.diff","patch_url":"https://github.com/apache/spark/pull/20717.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nIn order to optimize queries, some conditions can be added to the join condition for LEFT ANTI and OUTER joins. Unfortunately, so far this was not done since we are using only constraints which can be enforced on the output of the operator (in this case of the JOIN).\r\n\r\nWe can enforce some `isNotNull` conditions on one side, which are not valid conditions on the output of the Join, though. The PR adds these conditions in the `Optimizer` phase, in order to improve performance in some cases.\r\n\r\n## How was this patch tested?\r\n\r\nAdded UTs\r\n\r\nPlease review http://spark.apache.org/contributing.html before opening a pull request.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/372032344","html_url":"https://github.com/apache/spark/pull/20717#issuecomment-372032344","issue_url":"https://api.github.com/repos/apache/spark/issues/20717","id":372032344,"user":{"login":"AmplabJenkins","id":4031291,"avatar_url":"https://avatars3.githubusercontent.com/u/4031291?v=4","gravatar_id":"","url":"https://api.github.com/users/AmplabJenkins","html_url":"https://github.com/AmplabJenkins","followers_url":"https://api.github.com/users/AmplabJenkins/followers","following_url":"https://api.github.com/users/AmplabJenkins/following{/other_user}","gists_url":"https://api.github.com/users/AmplabJenkins/gists{/gist_id}","starred_url":"https://api.github.com/users/AmplabJenkins/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AmplabJenkins/subscriptions","organizations_url":"https://api.github.com/users/AmplabJenkins/orgs","repos_url":"https://api.github.com/users/AmplabJenkins/repos","events_url":"https://api.github.com/users/AmplabJenkins/events{/privacy}","received_events_url":"https://api.github.com/users/AmplabJenkins/received_events","type":"User","site_admin":false},"created_at":"2018-03-10T14:03:47Z","updated_at":"2018-03-10T14:03:47Z","author_association":"NONE","body":"Test PASSed.\nRefer to this link for build results (access rights to CI server needed): \nhttps://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88149/\nTest PASSed."}}
{"actor":{"display_login":"AmplabJenkins"}}