{"action":"closed","issue":{"url":"https://api.github.com/repos/chermenin/spark-states/issues/1","repository_url":"https://api.github.com/repos/chermenin/spark-states","labels_url":"https://api.github.com/repos/chermenin/spark-states/issues/1/labels{/name}","comments_url":"https://api.github.com/repos/chermenin/spark-states/issues/1/comments","events_url":"https://api.github.com/repos/chermenin/spark-states/issues/1/events","html_url":"https://github.com/chermenin/spark-states/issues/1","id":363290893,"node_id":"MDU6SXNzdWUzNjMyOTA4OTM=","number":1,"title":"Implement ProcessingTimeTimeout for RocksDBStateStore","user":{"login":"chitralverma","id":11135032,"node_id":"MDQ6VXNlcjExMTM1MDMy","avatar_url":"https://avatars2.githubusercontent.com/u/11135032?v=4","gravatar_id":"","url":"https://api.github.com/users/chitralverma","html_url":"https://github.com/chitralverma","followers_url":"https://api.github.com/users/chitralverma/followers","following_url":"https://api.github.com/users/chitralverma/following{/other_user}","gists_url":"https://api.github.com/users/chitralverma/gists{/gist_id}","starred_url":"https://api.github.com/users/chitralverma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chitralverma/subscriptions","organizations_url":"https://api.github.com/users/chitralverma/orgs","repos_url":"https://api.github.com/users/chitralverma/repos","events_url":"https://api.github.com/users/chitralverma/events{/privacy}","received_events_url":"https://api.github.com/users/chitralverma/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"chitralverma","id":11135032,"node_id":"MDQ6VXNlcjExMTM1MDMy","avatar_url":"https://avatars2.githubusercontent.com/u/11135032?v=4","gravatar_id":"","url":"https://api.github.com/users/chitralverma","html_url":"https://github.com/chitralverma","followers_url":"https://api.github.com/users/chitralverma/followers","following_url":"https://api.github.com/users/chitralverma/following{/other_user}","gists_url":"https://api.github.com/users/chitralverma/gists{/gist_id}","starred_url":"https://api.github.com/users/chitralverma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chitralverma/subscriptions","organizations_url":"https://api.github.com/users/chitralverma/orgs","repos_url":"https://api.github.com/users/chitralverma/repos","events_url":"https://api.github.com/users/chitralverma/events{/privacy}","received_events_url":"https://api.github.com/users/chitralverma/received_events","type":"User","site_admin":false},"assignees":[{"login":"chitralverma","id":11135032,"node_id":"MDQ6VXNlcjExMTM1MDMy","avatar_url":"https://avatars2.githubusercontent.com/u/11135032?v=4","gravatar_id":"","url":"https://api.github.com/users/chitralverma","html_url":"https://github.com/chitralverma","followers_url":"https://api.github.com/users/chitralverma/followers","following_url":"https://api.github.com/users/chitralverma/following{/other_user}","gists_url":"https://api.github.com/users/chitralverma/gists{/gist_id}","starred_url":"https://api.github.com/users/chitralverma/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chitralverma/subscriptions","organizations_url":"https://api.github.com/users/chitralverma/orgs","repos_url":"https://api.github.com/users/chitralverma/repos","events_url":"https://api.github.com/users/chitralverma/events{/privacy}","received_events_url":"https://api.github.com/users/chitralverma/received_events","type":"User","site_admin":false}],"milestone":null,"comments":5,"created_at":"2018-09-24T19:38:25Z","updated_at":"2018-12-10T10:42:17Z","closed_at":"2018-12-10T10:42:17Z","author_association":"CONTRIBUTOR","body":"Extending the capabilities of this state store, I propose the feature of state expiration/ timeout based on processing time. This feature is available when you do stateful aggregations using Spark's FlatMapGroupWithState (FMGWS) API. \r\n\r\nFMGWS is not very flexible in terms of options and usage and the timeouts require query to progress in order timeout keys. With addition of this feature to RocksDB State Store, key expiration is truly decoupled from the query engine in structured streaming, with the DB itself taking care of TTLs.\r\n\r\nWill raise a pull request for this shortly, along with certain test scenarios."}}
{"actor":{"display_login":"chitralverma"}}