{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/22482","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/22482/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/22482/comments","events_url":"https://api.github.com/repos/apache/spark/issues/22482/events","html_url":"https://github.com/apache/spark/pull/22482","id":362018103,"node_id":"MDExOlB1bGxSZXF1ZXN0MjE2ODQ0NDc1","number":22482,"title":"WIP - [SPARK-10816][SS] Support session window natively","user":{"login":"HeartSaVioR","id":1317309,"node_id":"MDQ6VXNlcjEzMTczMDk=","avatar_url":"https://avatars2.githubusercontent.com/u/1317309?v=4","gravatar_id":"","url":"https://api.github.com/users/HeartSaVioR","html_url":"https://github.com/HeartSaVioR","followers_url":"https://api.github.com/users/HeartSaVioR/followers","following_url":"https://api.github.com/users/HeartSaVioR/following{/other_user}","gists_url":"https://api.github.com/users/HeartSaVioR/gists{/gist_id}","starred_url":"https://api.github.com/users/HeartSaVioR/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/HeartSaVioR/subscriptions","organizations_url":"https://api.github.com/users/HeartSaVioR/orgs","repos_url":"https://api.github.com/users/HeartSaVioR/repos","events_url":"https://api.github.com/users/HeartSaVioR/events{/privacy}","received_events_url":"https://api.github.com/users/HeartSaVioR/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":15,"created_at":"2018-09-20T04:53:09Z","updated_at":"2018-10-02T07:05:01Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/22482","html_url":"https://github.com/apache/spark/pull/22482","diff_url":"https://github.com/apache/spark/pull/22482.diff","patch_url":"https://github.com/apache/spark/pull/22482.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis patch proposes native support of session window, like Spark has been supporting for time window.\r\n\r\nPlease refer the attached doc in [SPARK-10816](https://issues.apache.org/jira/browse/SPARK-10816) for more details on rationalization, concepts, and limitation, etc.\r\n\r\nIn point of end users' view, only the change is addition of \"session\" SQL function. End users could define query with session window as replacing \"window\" function to \"session\" function, and \"window\" column to \"session\" column. After then the patch will provide same experience with time window.\r\n\r\nInternally, this patch will change the physical plan of aggregation a bit: if there's session function being used in query, it will sort the input rows as \"grouping keys\" + \"session\", and merge overlapped sessions into one with applying aggregations, so it's like a sort based aggregation but the unit of group is grouping keys + session.\r\n\r\nDue to handle late event, there's a case multiple session windows co-exist per key which are not yet to evict. This patch handles the case via borrowing state implementation from streaming join which can handle multiple values for given key.\r\n\r\n## How was this patch tested?\r\n\r\nMany UTs are added to verify session window queries for both batch and streaming.\r\n\r\nPlease review http://spark.apache.org/contributing.html before opening a pull request.\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/426170539","html_url":"https://github.com/apache/spark/pull/22482#issuecomment-426170539","issue_url":"https://api.github.com/repos/apache/spark/issues/22482","id":426170539,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNjE3MDUzOQ==","user":{"login":"SparkQA","id":8108735,"node_id":"MDQ6VXNlcjgxMDg3MzU=","avatar_url":"https://avatars1.githubusercontent.com/u/8108735?v=4","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2018-10-02T07:05:01Z","updated_at":"2018-10-02T07:05:01Z","author_association":"NONE","body":"**[Test build #96838 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96838/testReport)** for PR 22482 at commit [`9a60cf3`](https://github.com/apache/spark/commit/9a60cf36461fb86dc716f0a9ec6e25c4ef6031c4).\n * This patch **fails due to an unknown error code, -9**.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}