{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/12246","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/12246/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/12246/comments","events_url":"https://api.github.com/repos/apache/spark/issues/12246/events","html_url":"https://github.com/apache/spark/pull/12246","id":146770234,"number":12246,"title":"[WIP, DO-NOT-MERGE][SPARK-14473][SQL] Define analysis rules to catch operations not supported in streaming","user":{"login":"tdas","id":663212,"avatar_url":"https://avatars.githubusercontent.com/u/663212?v=3","gravatar_id":"","url":"https://api.github.com/users/tdas","html_url":"https://github.com/tdas","followers_url":"https://api.github.com/users/tdas/followers","following_url":"https://api.github.com/users/tdas/following{/other_user}","gists_url":"https://api.github.com/users/tdas/gists{/gist_id}","starred_url":"https://api.github.com/users/tdas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tdas/subscriptions","organizations_url":"https://api.github.com/users/tdas/orgs","repos_url":"https://api.github.com/users/tdas/repos","events_url":"https://api.github.com/users/tdas/events{/privacy}","received_events_url":"https://api.github.com/users/tdas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":8,"created_at":"2016-04-07T23:25:25Z","updated_at":"2016-04-12T01:57:36Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/12246","html_url":"https://github.com/apache/spark/pull/12246","diff_url":"https://github.com/apache/spark/pull/12246.diff","patch_url":"https://github.com/apache/spark/pull/12246.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThere are many operations that are currently not supported in the streaming execution. For example:\r\n - joining two streams\r\n - unioning a stream and a batch source\r\n - sorting\r\n - window functions (not time windows)\r\n - distinct aggregates\r\n\r\nFurthermore, executing a query with a stream source as a batch query should also fail.\r\n\r\nThis patch add an additional step after analysis in the QueryExecution which will check that all the operations in the analyzed logical plan is supported or not.\r\n\r\nTodo\r\n- [ ] Logical Plans\r\n  - [ ] Decide whether the following are supported \r\n    - [ ] Pivot\r\n  - [ ] Tests for each unsupported operation\r\n- [ ] Dataset API \r\n  - [ ] actions like count, collect, etc.\r\n  - [ ] dedup?\r\n\r\n\r\n## How was this patch tested?\r\nunit tests.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/208662269","html_url":"https://github.com/apache/spark/pull/12246#issuecomment-208662269","issue_url":"https://api.github.com/repos/apache/spark/issues/12246","id":208662269,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-04-12T01:57:36Z","updated_at":"2016-04-12T01:57:36Z","body":"**[Test build #55572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/55572/consoleFull)** for PR 12246 at commit [`64ed243`](https://github.com/apache/spark/commit/64ed24347de349e8e734ad50c11392eb17237d4e)."}}