{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/10630","labels_url":"https://api.github.com/repos/apache/spark/issues/10630/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/10630/comments","events_url":"https://api.github.com/repos/apache/spark/issues/10630/events","html_url":"https://github.com/apache/spark/pull/10630","id":125327667,"number":10630,"title":"[SPARK-12656] [SQL] Implement Intersect with Left-semi Join","user":{"login":"gatorsmile","id":11567269,"avatar_url":"https://avatars.githubusercontent.com/u/11567269?v=3","gravatar_id":"","url":"https://api.github.com/users/gatorsmile","html_url":"https://github.com/gatorsmile","followers_url":"https://api.github.com/users/gatorsmile/followers","following_url":"https://api.github.com/users/gatorsmile/following{/other_user}","gists_url":"https://api.github.com/users/gatorsmile/gists{/gist_id}","starred_url":"https://api.github.com/users/gatorsmile/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gatorsmile/subscriptions","organizations_url":"https://api.github.com/users/gatorsmile/orgs","repos_url":"https://api.github.com/users/gatorsmile/repos","events_url":"https://api.github.com/users/gatorsmile/events{/privacy}","received_events_url":"https://api.github.com/users/gatorsmile/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2016-01-07T05:48:33Z","updated_at":"2016-01-07T11:26:45Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/10630","html_url":"https://github.com/apache/spark/pull/10630","diff_url":"https://github.com/apache/spark/pull/10630.diff","patch_url":"https://github.com/apache/spark/pull/10630.patch"},"body":"Our current Intersect physical operator simply delegates to RDD.intersect. We should remove the Intersect physical operator and simply transform a logical intersect into a semi-join. This way, we can take advantage of all the benefits of join implementations (e.g. managed memory, code generation, broadcast joins).\r\n\r\nAfter a search, I found one of the mainstream RDBMS did the same. In their query explain, Intersect is replaced by Left-semi Join. Left-semi Join could help outer-join elimination in Optimizer, as shown in the PR: https://github.com/apache/spark/pull/10566"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/169635154","html_url":"https://github.com/apache/spark/pull/10630#issuecomment-169635154","issue_url":"https://api.github.com/repos/apache/spark/issues/10630","id":169635154,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-01-07T11:26:45Z","updated_at":"2016-01-07T11:26:45Z","body":"**[Test build #48912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/48912/consoleFull)** for PR 10630 at commit [`6742984`](https://github.com/apache/spark/commit/6742984f5ec0d480435988712e161838c3e8d6d5).\n * This patch **fails from timeout after a configured wait of \\`250m\\`**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}