{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/16272","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/16272/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/16272/comments","events_url":"https://api.github.com/repos/apache/spark/issues/16272/events","html_url":"https://github.com/apache/spark/pull/16272","id":195383838,"number":16272,"title":"[SPARK-18850][SS]Make StreamExecution serializable","user":{"login":"zsxwing","id":1000778,"avatar_url":"https://avatars.githubusercontent.com/u/1000778?v=3","gravatar_id":"","url":"https://api.github.com/users/zsxwing","html_url":"https://github.com/zsxwing","followers_url":"https://api.github.com/users/zsxwing/followers","following_url":"https://api.github.com/users/zsxwing/following{/other_user}","gists_url":"https://api.github.com/users/zsxwing/gists{/gist_id}","starred_url":"https://api.github.com/users/zsxwing/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zsxwing/subscriptions","organizations_url":"https://api.github.com/users/zsxwing/orgs","repos_url":"https://api.github.com/users/zsxwing/repos","events_url":"https://api.github.com/users/zsxwing/events{/privacy}","received_events_url":"https://api.github.com/users/zsxwing/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-12-13T22:12:57Z","updated_at":"2016-12-15T18:49:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/16272","html_url":"https://github.com/apache/spark/pull/16272","diff_url":"https://github.com/apache/spark/pull/16272.diff","patch_url":"https://github.com/apache/spark/pull/16272.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nThis PR adds StreamingQueryWrapper to make StreamExecution serializable because it is too easy for it to get captured with normal usage. If StreamingQueryWrapper gets captured in a closure but no place calls its methods, it should not fail the Spark tasks. However if its methods are called, then this PR will throw a better message.\r\n\r\n## How was this patch tested?\r\n\r\n`test(\"StreamingQuery should be Serializable but cannot be used in executors\")`"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/267408548","html_url":"https://github.com/apache/spark/pull/16272#issuecomment-267408548","issue_url":"https://api.github.com/repos/apache/spark/issues/16272","id":267408548,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-12-15T18:49:32Z","updated_at":"2016-12-15T18:49:32Z","body":"**[Test build #70198 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70198/testReport)** for PR 16272 at commit [`4469b0e`](https://github.com/apache/spark/commit/4469b0e335ed913cf66e490cfcf5f23932a07af1).\n * This patch **fails to build**.\n * This patch merges cleanly.\n * This patch adds the following public classes _(experimental)_:\n  * `case class AlterTableChangeColumnCommand(`\n  * `case class EventTimeStats(var max: Long, var min: Long, var sum: Long, var count: Long) `\n  * `class EventTimeStatsAccum(protected var currentStats: EventTimeStats = EventTimeStats.zero)`"}}
{"actor":{"display_login":"SparkQA"}}