{"action":"created","comment":{"url":"https://api.github.com/repos/apache/flink/pulls/comments/86379858","pull_request_review_id":7038782,"id":86379858,"diff_hunk":"@@ -273,32 +284,93 @@ public void onProcessingTime(InternalTimer<K, W> timer) throws Exception {\n \t\t\treturn;\n \t\t}\n \n-\t\tTriggerResult triggerResult = context.onProcessingTime(timer.getTimestamp());\n-\t\tif (triggerResult.isFire()) {\n-\t\t\tfire(context.window, contents);\n-\t\t}\n+\t\t\t\tTriggerResult triggerResult = context.onProcessingTime(timer.getTimestamp());\n+\t\t\t\tif (triggerResult.isFire()) {\n+\t\t\t\t\tfire(context.window, contents, windowState);\n+\t\t\t\t}\n \n \t\tif (triggerResult.isPurge() || (!windowAssigner.isEventTime() && isCleanupTime(context.window, timer.getTimestamp()))) {\n \t\t\tcleanup(context.window, windowState, mergingWindows);\n \t\t}\n \t}\n \n-\tprivate void fire(W window, Iterable<StreamRecord<IN>> contents) throws Exception {\n+\tprivate void fire(W window, Iterable<StreamRecord<IN>> contents, ListState<StreamRecord<IN>> windowState) throws Exception {\n \t\ttimestampedCollector.setAbsoluteTimestamp(window.maxTimestamp());\n \n \t\t// Work around type system restrictions...\n-\t\tint toEvict = evictor.evict((Iterable) contents, Iterables.size(contents), context.window);\n-\n-\t\tFluentIterable<IN> projectedContents = FluentIterable\n+\t\tFluentIterable<TimestampedValue<IN>> recordsWithTimestamp = FluentIterable\n \t\t\t.from(contents)\n-\t\t\t.skip(toEvict)\n-\t\t\t.transform(new Function<StreamRecord<IN>, IN>() {\n+\t\t\t.transform(new Function<StreamRecord<IN>, TimestampedValue<IN>>() {\n+\t\t\t\t@Override\n+\t\t\t\tpublic TimestampedValue<IN> apply(StreamRecord<IN> input) {\n+\t\t\t\t\treturn new TimestampedValue<>(input.getValue(), input.getTimestamp());\n+\t\t\t\t}\n+\t\t\t});\n+\t\tevictorContext.evictBefore(recordsWithTimestamp, Iterables.size(recordsWithTimestamp));\n+\n+\t\tFluentIterable<IN> projectedContents = recordsWithTimestamp\n+\t\t\t.transform(new Function<TimestampedValue<IN>, IN>() {\n \t\t\t\t@Override\n-\t\t\t\tpublic IN apply(StreamRecord<IN> input) {\n+\t\t\t\tpublic IN apply(TimestampedValue<IN> input) {\n \t\t\t\t\treturn input.getValue();\n \t\t\t\t}\n \t\t\t});\n+\n \t\tuserFunction.apply(context.key, context.window, projectedContents, timestampedCollector);\n+\t\tevictorContext.evictAfter(recordsWithTimestamp, Iterables.size(recordsWithTimestamp));\n+\n+\n+\t\t//work around to fix FLINK-4369, remove the evicted elements from the windowState.\n+\t\t//this is inefficient, but there is no other way to remove elements from ListState, which is an AppendingState.\n+\t\twindowState.clear();\n+\t\tfor(TimestampedValue<IN> record : recordsWithTimestamp) {\n+\t\t\tif (record.getTimestamp() < 0) {","path":"flink-streaming-java/src/main/java/org/apache/flink/streaming/runtime/operators/windowing/EvictingWindowOperator.java","position":152,"original_position":152,"commit_id":"bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2","original_commit_id":"bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2","user":{"login":"aljoscha","id":68551,"avatar_url":"https://avatars.githubusercontent.com/u/68551?v=3","gravatar_id":"","url":"https://api.github.com/users/aljoscha","html_url":"https://github.com/aljoscha","followers_url":"https://api.github.com/users/aljoscha/followers","following_url":"https://api.github.com/users/aljoscha/following{/other_user}","gists_url":"https://api.github.com/users/aljoscha/gists{/gist_id}","starred_url":"https://api.github.com/users/aljoscha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aljoscha/subscriptions","organizations_url":"https://api.github.com/users/aljoscha/orgs","repos_url":"https://api.github.com/users/aljoscha/repos","events_url":"https://api.github.com/users/aljoscha/events{/privacy}","received_events_url":"https://api.github.com/users/aljoscha/received_events","type":"User","site_admin":false},"body":"I think this does not work properly in all cases because negative timestamps are in fact also valid timestamps. You can look at `StreamRecord` for how this should be solved in `TimestampedValue` and also for how the copying between the two should work.","created_at":"2016-11-03T16:05:11Z","updated_at":"2016-11-03T16:46:32Z","html_url":"https://github.com/apache/flink/pull/2736#discussion_r86379858","pull_request_url":"https://api.github.com/repos/apache/flink/pulls/2736","_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/comments/86379858"},"html":{"href":"https://github.com/apache/flink/pull/2736#discussion_r86379858"},"pull_request":{"href":"https://api.github.com/repos/apache/flink/pulls/2736"}}},"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/2736","id":91727097,"html_url":"https://github.com/apache/flink/pull/2736","diff_url":"https://github.com/apache/flink/pull/2736.diff","patch_url":"https://github.com/apache/flink/pull/2736.patch","issue_url":"https://api.github.com/repos/apache/flink/issues/2736","number":2736,"state":"open","locked":false,"title":"[FLINK-4174] Enhance evictor functionality","user":{"login":"soniclavier","id":707701,"avatar_url":"https://avatars.githubusercontent.com/u/707701?v=3","gravatar_id":"","url":"https://api.github.com/users/soniclavier","html_url":"https://github.com/soniclavier","followers_url":"https://api.github.com/users/soniclavier/followers","following_url":"https://api.github.com/users/soniclavier/following{/other_user}","gists_url":"https://api.github.com/users/soniclavier/gists{/gist_id}","starred_url":"https://api.github.com/users/soniclavier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soniclavier/subscriptions","organizations_url":"https://api.github.com/users/soniclavier/orgs","repos_url":"https://api.github.com/users/soniclavier/repos","events_url":"https://api.github.com/users/soniclavier/events{/privacy}","received_events_url":"https://api.github.com/users/soniclavier/received_events","type":"User","site_admin":false},"body":"The PR implements [FLINK-4174](https://issues.apache.org/jira/browse/FLINK-4174) Enhance window evictor as proposed in [FLIP-4](https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor)\r\n\r\nChanges made:\r\n\r\n- Modified the Evictor API, added two new methods evictBefore and evictAfter. Removed existing evict method\r\n- Modified the corresponding implementations of CountEvictor, DeltaEvictor and TimeEvictor\r\n- Created EvictorContext in the class EvictingWindowOperator, which is passed to the evictor methods\r\n- Created TimestampedValue class which holds the value with corresponding timestamp. This class is exposed to the user via the evictBefore and evictAfter methods.\r\n- Modified EvictingWindowOperator class \r\n\t- to call evictBefore before calling window function and evictAfter after calling window function.\r\n\t- create FluentIterable<TimestampedValue<IN>> from Iterable<StreamRecord<IN>> contents, which is passed to the evictor methods.\r\n\t- to clear the windowstate and add the remaining element(after the eviction) back to the state. (this fixes [FLINK-4369](https://issues.apache.org/jira/browse/FLINK-4369))\r\n- Added test cases in EvictingWindowOperatorTest.","created_at":"2016-11-01T03:43:39Z","updated_at":"2016-11-03T16:46:32Z","closed_at":null,"merged_at":null,"merge_commit_sha":"531dbf0aed486bfa4eeb76695fb2d537466d2214","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/flink/pulls/2736/commits","review_comments_url":"https://api.github.com/repos/apache/flink/pulls/2736/comments","review_comment_url":"https://api.github.com/repos/apache/flink/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/flink/issues/2736/comments","statuses_url":"https://api.github.com/repos/apache/flink/statuses/bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2","head":{"label":"soniclavier:FLINK-4174","ref":"FLINK-4174","sha":"bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2","user":{"login":"soniclavier","id":707701,"avatar_url":"https://avatars.githubusercontent.com/u/707701?v=3","gravatar_id":"","url":"https://api.github.com/users/soniclavier","html_url":"https://github.com/soniclavier","followers_url":"https://api.github.com/users/soniclavier/followers","following_url":"https://api.github.com/users/soniclavier/following{/other_user}","gists_url":"https://api.github.com/users/soniclavier/gists{/gist_id}","starred_url":"https://api.github.com/users/soniclavier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soniclavier/subscriptions","organizations_url":"https://api.github.com/users/soniclavier/orgs","repos_url":"https://api.github.com/users/soniclavier/repos","events_url":"https://api.github.com/users/soniclavier/events{/privacy}","received_events_url":"https://api.github.com/users/soniclavier/received_events","type":"User","site_admin":false},"repo":{"id":62566931,"name":"flink","full_name":"soniclavier/flink","owner":{"login":"soniclavier","id":707701,"avatar_url":"https://avatars.githubusercontent.com/u/707701?v=3","gravatar_id":"","url":"https://api.github.com/users/soniclavier","html_url":"https://github.com/soniclavier","followers_url":"https://api.github.com/users/soniclavier/followers","following_url":"https://api.github.com/users/soniclavier/following{/other_user}","gists_url":"https://api.github.com/users/soniclavier/gists{/gist_id}","starred_url":"https://api.github.com/users/soniclavier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/soniclavier/subscriptions","organizations_url":"https://api.github.com/users/soniclavier/orgs","repos_url":"https://api.github.com/users/soniclavier/repos","events_url":"https://api.github.com/users/soniclavier/events{/privacy}","received_events_url":"https://api.github.com/users/soniclavier/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/soniclavier/flink","description":"Mirror of Apache Flink","fork":true,"url":"https://api.github.com/repos/soniclavier/flink","forks_url":"https://api.github.com/repos/soniclavier/flink/forks","keys_url":"https://api.github.com/repos/soniclavier/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/soniclavier/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/soniclavier/flink/teams","hooks_url":"https://api.github.com/repos/soniclavier/flink/hooks","issue_events_url":"https://api.github.com/repos/soniclavier/flink/issues/events{/number}","events_url":"https://api.github.com/repos/soniclavier/flink/events","assignees_url":"https://api.github.com/repos/soniclavier/flink/assignees{/user}","branches_url":"https://api.github.com/repos/soniclavier/flink/branches{/branch}","tags_url":"https://api.github.com/repos/soniclavier/flink/tags","blobs_url":"https://api.github.com/repos/soniclavier/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/soniclavier/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/soniclavier/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/soniclavier/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/soniclavier/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/soniclavier/flink/languages","stargazers_url":"https://api.github.com/repos/soniclavier/flink/stargazers","contributors_url":"https://api.github.com/repos/soniclavier/flink/contributors","subscribers_url":"https://api.github.com/repos/soniclavier/flink/subscribers","subscription_url":"https://api.github.com/repos/soniclavier/flink/subscription","commits_url":"https://api.github.com/repos/soniclavier/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/soniclavier/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/soniclavier/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/soniclavier/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/soniclavier/flink/contents/{+path}","compare_url":"https://api.github.com/repos/soniclavier/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/soniclavier/flink/merges","archive_url":"https://api.github.com/repos/soniclavier/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/soniclavier/flink/downloads","issues_url":"https://api.github.com/repos/soniclavier/flink/issues{/number}","pulls_url":"https://api.github.com/repos/soniclavier/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/soniclavier/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/soniclavier/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/soniclavier/flink/labels{/name}","releases_url":"https://api.github.com/repos/soniclavier/flink/releases{/id}","deployments_url":"https://api.github.com/repos/soniclavier/flink/deployments","created_at":"2016-07-04T14:07:53Z","updated_at":"2016-07-04T14:08:13Z","pushed_at":"2016-10-31T23:21:14Z","git_url":"git://github.com/soniclavier/flink.git","ssh_url":"git@github.com:soniclavier/flink.git","clone_url":"https://github.com/soniclavier/flink.git","svn_url":"https://github.com/soniclavier/flink","homepage":null,"size":98434,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"1574249181304e124f8ac32b7d2a8fafdb0f9da9","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":20587599,"name":"flink","full_name":"apache/flink","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/flink","description":"Mirror of Apache Flink","fork":false,"url":"https://api.github.com/repos/apache/flink","forks_url":"https://api.github.com/repos/apache/flink/forks","keys_url":"https://api.github.com/repos/apache/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/flink/teams","hooks_url":"https://api.github.com/repos/apache/flink/hooks","issue_events_url":"https://api.github.com/repos/apache/flink/issues/events{/number}","events_url":"https://api.github.com/repos/apache/flink/events","assignees_url":"https://api.github.com/repos/apache/flink/assignees{/user}","branches_url":"https://api.github.com/repos/apache/flink/branches{/branch}","tags_url":"https://api.github.com/repos/apache/flink/tags","blobs_url":"https://api.github.com/repos/apache/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/flink/languages","stargazers_url":"https://api.github.com/repos/apache/flink/stargazers","contributors_url":"https://api.github.com/repos/apache/flink/contributors","subscribers_url":"https://api.github.com/repos/apache/flink/subscribers","subscription_url":"https://api.github.com/repos/apache/flink/subscription","commits_url":"https://api.github.com/repos/apache/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/flink/contents/{+path}","compare_url":"https://api.github.com/repos/apache/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/flink/merges","archive_url":"https://api.github.com/repos/apache/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/flink/downloads","issues_url":"https://api.github.com/repos/apache/flink/issues{/number}","pulls_url":"https://api.github.com/repos/apache/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/flink/labels{/name}","releases_url":"https://api.github.com/repos/apache/flink/releases{/id}","deployments_url":"https://api.github.com/repos/apache/flink/deployments","created_at":"2014-06-07T07:00:10Z","updated_at":"2016-11-03T11:00:06Z","pushed_at":"2016-11-03T16:03:48Z","git_url":"git://github.com/apache/flink.git","ssh_url":"git@github.com:apache/flink.git","clone_url":"https://github.com/apache/flink.git","svn_url":"https://github.com/apache/flink","homepage":null,"size":137602,"stargazers_count":1706,"watchers_count":1706,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1161,"mirror_url":"git://git.apache.org/flink.git","open_issues_count":152,"forks":1161,"open_issues":152,"watchers":1706,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/2736"},"html":{"href":"https://github.com/apache/flink/pull/2736"},"issue":{"href":"https://api.github.com/repos/apache/flink/issues/2736"},"comments":{"href":"https://api.github.com/repos/apache/flink/issues/2736/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/flink/pulls/2736/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/flink/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/flink/pulls/2736/commits"},"statuses":{"href":"https://api.github.com/repos/apache/flink/statuses/bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2"}}}}
{"actor":{"display_login":"aljoscha"}}