{"action":"created","issue":{"url":"https://api.github.com/repos/apache/spark/issues/15859","repository_url":"https://api.github.com/repos/apache/spark","labels_url":"https://api.github.com/repos/apache/spark/issues/15859/labels{/name}","comments_url":"https://api.github.com/repos/apache/spark/issues/15859/comments","events_url":"https://api.github.com/repos/apache/spark/issues/15859/events","html_url":"https://github.com/apache/spark/pull/15859","id":188882029,"number":15859,"title":"[SPARK-18416][Structured Streaming] Fixed temp file leak in state store","user":{"login":"tdas","id":663212,"avatar_url":"https://avatars.githubusercontent.com/u/663212?v=3","gravatar_id":"","url":"https://api.github.com/users/tdas","html_url":"https://github.com/tdas","followers_url":"https://api.github.com/users/tdas/followers","following_url":"https://api.github.com/users/tdas/following{/other_user}","gists_url":"https://api.github.com/users/tdas/gists{/gist_id}","starred_url":"https://api.github.com/users/tdas/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tdas/subscriptions","organizations_url":"https://api.github.com/users/tdas/orgs","repos_url":"https://api.github.com/users/tdas/repos","events_url":"https://api.github.com/users/tdas/events{/privacy}","received_events_url":"https://api.github.com/users/tdas/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-11-12T01:04:21Z","updated_at":"2016-11-12T03:49:09Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/apache/spark/pulls/15859","html_url":"https://github.com/apache/spark/pull/15859","diff_url":"https://github.com/apache/spark/pull/15859.diff","patch_url":"https://github.com/apache/spark/pull/15859.patch"},"body":"## What changes were proposed in this pull request?\r\n\r\nStateStore.get() causes temporary files to be created immediately, even if the store is not used to make updates for new version. The temp file is not closed as store.commit() is not called in those cases, thus keeping the output stream to temp file open forever.\r\n\r\nThis PR fixes it by opening the temp file only when there are updates being made.\r\n\r\n## How was this patch tested?\r\n\r\nNew unit test\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/apache/spark/issues/comments/260099546","html_url":"https://github.com/apache/spark/pull/15859#issuecomment-260099546","issue_url":"https://api.github.com/repos/apache/spark/issues/15859","id":260099546,"user":{"login":"SparkQA","id":8108735,"avatar_url":"https://avatars.githubusercontent.com/u/8108735?v=3","gravatar_id":"","url":"https://api.github.com/users/SparkQA","html_url":"https://github.com/SparkQA","followers_url":"https://api.github.com/users/SparkQA/followers","following_url":"https://api.github.com/users/SparkQA/following{/other_user}","gists_url":"https://api.github.com/users/SparkQA/gists{/gist_id}","starred_url":"https://api.github.com/users/SparkQA/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/SparkQA/subscriptions","organizations_url":"https://api.github.com/users/SparkQA/orgs","repos_url":"https://api.github.com/users/SparkQA/repos","events_url":"https://api.github.com/users/SparkQA/events{/privacy}","received_events_url":"https://api.github.com/users/SparkQA/received_events","type":"User","site_admin":false},"created_at":"2016-11-12T03:49:08Z","updated_at":"2016-11-12T03:49:08Z","body":"**[Test build #68549 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/68549/consoleFull)** for PR 15859 at commit [`b60485e`](https://github.com/apache/spark/commit/b60485e7b3b76e33f46c3c017733f2eaed57f922).\n * This patch passes all tests.\n * This patch merges cleanly.\n * This patch adds no public classes."}}
{"actor":{"display_login":"SparkQA"}}