{"action":"created","comment":{"url":"https://api.github.com/repos/apache/flink/pulls/comments/79598708","id":79598708,"diff_hunk":"@@ -128,17 +130,22 @@ public void prepare() throws Exception{\n \t\t\t\tConfigConstants.DEFAULT_RUNTIME_HASH_JOIN_BLOOM_FILTERS);\n \n \t\t// create and return joining iterator according to provided local strategy.\n-\t\tif (objectReuseEnabled) {\n-\t\t\tswitch (ls) {\n-\t\t\t\tcase INNER_MERGE:\n-\t\t\t\t\tthis.joinIterator = new ReusingMergeInnerJoinIterator<>(in1, in2, \n+\t\tif (reset) {\n+\t\t\tresetForIterativeTasks(in1, in2, serializer1, serializer2, comparator1, comparator2, pairComparatorFactory);\n+\t\t\treset = false;\n+\t\t}\n+\t\tif (joinIterator == null) {","path":"flink-runtime/src/main/java/org/apache/flink/runtime/operators/JoinDriver.java","position":30,"original_position":30,"commit_id":"e17462881cc2f0137cc8412eb578fb4c42baeb15","original_commit_id":"e17462881cc2f0137cc8412eb578fb4c42baeb15","user":{"login":"ramkrish86","id":3690658,"avatar_url":"https://avatars.githubusercontent.com/u/3690658?v=3","gravatar_id":"","url":"https://api.github.com/users/ramkrish86","html_url":"https://github.com/ramkrish86","followers_url":"https://api.github.com/users/ramkrish86/followers","following_url":"https://api.github.com/users/ramkrish86/following{/other_user}","gists_url":"https://api.github.com/users/ramkrish86/gists{/gist_id}","starred_url":"https://api.github.com/users/ramkrish86/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ramkrish86/subscriptions","organizations_url":"https://api.github.com/users/ramkrish86/orgs","repos_url":"https://api.github.com/users/ramkrish86/repos","events_url":"https://api.github.com/users/ramkrish86/events{/privacy}","received_events_url":"https://api.github.com/users/ramkrish86/received_events","type":"User","site_admin":false},"body":"I am not very sure how to make use of this Resettable framework without having to use a boolean reset or checking for null for the joinIterator. Ideally the reset() API is not needed.\r\nIn a normal way this is the flow\r\n\r\n```\r\ninitialize() (for 1st iteration)\r\n  or\r\nreset() (for all other iterations)\r\nfollowed by\r\nsetup()\r\nprepare()\r\nrun()\r\ncleanup().\r\n\r\n```\r\nNow if i try to use `reset()` to reset my iterator using the current taskContext in1 and in2. Then when the `run() `is called for the next iteration we don't have the latest in1 and in2. As said above see test case `PageRankITCase#testPageRankSmallNumberOfIterations`. This test fails. It expects the in1 and in2 to be updated after all the `resetInput()` happens from the batch task.\r\n\r\nIn that case the best way is to set a boolean in `reset()` and in prepare you do the actual iterator reset and reset the boolean flag after that. (here allow all the `prepare()` to do the actual initializataion).\r\n\r\nIn case I move all the current code in `prepare() `to `initialize()` then am not sure how to use the `prepare() and reset()` method to ensure that we actually do the iterator reset. Here again I need a reset boolean flag to do that.  \r\n","created_at":"2016-09-20T12:32:03Z","updated_at":"2016-09-20T12:32:03Z","html_url":"https://github.com/apache/flink/pull/2510#discussion_r79598708","pull_request_url":"https://api.github.com/repos/apache/flink/pulls/2510","_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/comments/79598708"},"html":{"href":"https://github.com/apache/flink/pull/2510#discussion_r79598708"},"pull_request":{"href":"https://api.github.com/repos/apache/flink/pulls/2510"}}},"pull_request":{"url":"https://api.github.com/repos/apache/flink/pulls/2510","id":85769615,"html_url":"https://github.com/apache/flink/pull/2510","diff_url":"https://github.com/apache/flink/pull/2510.diff","patch_url":"https://github.com/apache/flink/pull/2510.patch","issue_url":"https://api.github.com/repos/apache/flink/issues/2510","number":2510,"state":"open","locked":false,"title":"FLINK-3322 Allow drivers and iterators to reuse the memory segments","user":{"login":"ramkrish86","id":3690658,"avatar_url":"https://avatars.githubusercontent.com/u/3690658?v=3","gravatar_id":"","url":"https://api.github.com/users/ramkrish86","html_url":"https://github.com/ramkrish86","followers_url":"https://api.github.com/users/ramkrish86/followers","following_url":"https://api.github.com/users/ramkrish86/following{/other_user}","gists_url":"https://api.github.com/users/ramkrish86/gists{/gist_id}","starred_url":"https://api.github.com/users/ramkrish86/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ramkrish86/subscriptions","organizations_url":"https://api.github.com/users/ramkrish86/orgs","repos_url":"https://api.github.com/users/ramkrish86/repos","events_url":"https://api.github.com/users/ramkrish86/events{/privacy}","received_events_url":"https://api.github.com/users/ramkrish86/received_events","type":"User","site_admin":false},"body":"Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.\r\nIf your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).\r\nIn addition to going through the list, please provide a meaningful description of your changes.\r\n\r\n- [ ] General\r\n  - The pull request references the related JIRA issue (\"[FLINK-XXX] Jira title text\")\r\n  - The pull request addresses only one issue\r\n  - Each commit in the PR has a meaningful commit message (including the JIRA id)\r\n\r\n- [ ] Documentation\r\n  - Documentation has been added for new functionality\r\n  - Old documentation affected by the pull request has been updated\r\n  - JavaDoc for public methods has been added\r\n\r\n- [ ] Tests & Build\r\n  - Functionality added by the pull request is covered by tests\r\n  - `mvn clean verify` has been executed successfully locally or a Travis build has passed\r\n\r\nUpdated PR based on @ggevay 's feedback. Now the drivers are all made of type ResettableDrivers and we use the existing APIs to reuse the driver and the iterators. The JoinTaskITerator has now a new method called reset() that allows to reset the iterator's state.\r\nIf we go with the approach then it makes sense to remove the ResettableDriver and add all the API in it to the Driver interface itself. That is for a later PR. \r\n@ggevay - Pls review and give your valuable feedback.\r\nI tried mvn clean verify and got the tests to pass in flink-runtime and flink-gelly-examples. One failed 'org.apache.flink.runtime.io.disk.ChannelViewsTest.testWriteAndReadLongRecords' and when I ran locally it passed again. \r\n","created_at":"2016-09-19T10:31:50Z","updated_at":"2016-09-20T12:32:03Z","closed_at":null,"merged_at":null,"merge_commit_sha":"9fc4fa7ff8f8516c9d9a594cc5a5cc5b2252a524","assignee":null,"assignees":[],"milestone":null,"commits_url":"https://api.github.com/repos/apache/flink/pulls/2510/commits","review_comments_url":"https://api.github.com/repos/apache/flink/pulls/2510/comments","review_comment_url":"https://api.github.com/repos/apache/flink/pulls/comments{/number}","comments_url":"https://api.github.com/repos/apache/flink/issues/2510/comments","statuses_url":"https://api.github.com/repos/apache/flink/statuses/e17462881cc2f0137cc8412eb578fb4c42baeb15","head":{"label":"ramkrish86:FLINK-4615_new","ref":"FLINK-4615_new","sha":"e17462881cc2f0137cc8412eb578fb4c42baeb15","user":{"login":"ramkrish86","id":3690658,"avatar_url":"https://avatars.githubusercontent.com/u/3690658?v=3","gravatar_id":"","url":"https://api.github.com/users/ramkrish86","html_url":"https://github.com/ramkrish86","followers_url":"https://api.github.com/users/ramkrish86/followers","following_url":"https://api.github.com/users/ramkrish86/following{/other_user}","gists_url":"https://api.github.com/users/ramkrish86/gists{/gist_id}","starred_url":"https://api.github.com/users/ramkrish86/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ramkrish86/subscriptions","organizations_url":"https://api.github.com/users/ramkrish86/orgs","repos_url":"https://api.github.com/users/ramkrish86/repos","events_url":"https://api.github.com/users/ramkrish86/events{/privacy}","received_events_url":"https://api.github.com/users/ramkrish86/received_events","type":"User","site_admin":false},"repo":{"id":50035316,"name":"flink","full_name":"ramkrish86/flink","owner":{"login":"ramkrish86","id":3690658,"avatar_url":"https://avatars.githubusercontent.com/u/3690658?v=3","gravatar_id":"","url":"https://api.github.com/users/ramkrish86","html_url":"https://github.com/ramkrish86","followers_url":"https://api.github.com/users/ramkrish86/followers","following_url":"https://api.github.com/users/ramkrish86/following{/other_user}","gists_url":"https://api.github.com/users/ramkrish86/gists{/gist_id}","starred_url":"https://api.github.com/users/ramkrish86/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ramkrish86/subscriptions","organizations_url":"https://api.github.com/users/ramkrish86/orgs","repos_url":"https://api.github.com/users/ramkrish86/repos","events_url":"https://api.github.com/users/ramkrish86/events{/privacy}","received_events_url":"https://api.github.com/users/ramkrish86/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/ramkrish86/flink","description":"Mirror of Apache Flink","fork":true,"url":"https://api.github.com/repos/ramkrish86/flink","forks_url":"https://api.github.com/repos/ramkrish86/flink/forks","keys_url":"https://api.github.com/repos/ramkrish86/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/ramkrish86/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/ramkrish86/flink/teams","hooks_url":"https://api.github.com/repos/ramkrish86/flink/hooks","issue_events_url":"https://api.github.com/repos/ramkrish86/flink/issues/events{/number}","events_url":"https://api.github.com/repos/ramkrish86/flink/events","assignees_url":"https://api.github.com/repos/ramkrish86/flink/assignees{/user}","branches_url":"https://api.github.com/repos/ramkrish86/flink/branches{/branch}","tags_url":"https://api.github.com/repos/ramkrish86/flink/tags","blobs_url":"https://api.github.com/repos/ramkrish86/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/ramkrish86/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/ramkrish86/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/ramkrish86/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/ramkrish86/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/ramkrish86/flink/languages","stargazers_url":"https://api.github.com/repos/ramkrish86/flink/stargazers","contributors_url":"https://api.github.com/repos/ramkrish86/flink/contributors","subscribers_url":"https://api.github.com/repos/ramkrish86/flink/subscribers","subscription_url":"https://api.github.com/repos/ramkrish86/flink/subscription","commits_url":"https://api.github.com/repos/ramkrish86/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/ramkrish86/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/ramkrish86/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/ramkrish86/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/ramkrish86/flink/contents/{+path}","compare_url":"https://api.github.com/repos/ramkrish86/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/ramkrish86/flink/merges","archive_url":"https://api.github.com/repos/ramkrish86/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/ramkrish86/flink/downloads","issues_url":"https://api.github.com/repos/ramkrish86/flink/issues{/number}","pulls_url":"https://api.github.com/repos/ramkrish86/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/ramkrish86/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/ramkrish86/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/ramkrish86/flink/labels{/name}","releases_url":"https://api.github.com/repos/ramkrish86/flink/releases{/id}","deployments_url":"https://api.github.com/repos/ramkrish86/flink/deployments","created_at":"2016-01-20T14:27:43Z","updated_at":"2016-01-20T14:27:55Z","pushed_at":"2016-09-19T10:28:14Z","git_url":"git://github.com/ramkrish86/flink.git","ssh_url":"git@github.com:ramkrish86/flink.git","clone_url":"https://github.com/ramkrish86/flink.git","svn_url":"https://github.com/ramkrish86/flink","homepage":null,"size":92130,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"apache:master","ref":"master","sha":"d7b59d761601baba6765bb4fc407bcd9fd6a9387","user":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"repo":{"id":20587599,"name":"flink","full_name":"apache/flink","owner":{"login":"apache","id":47359,"avatar_url":"https://avatars.githubusercontent.com/u/47359?v=3","gravatar_id":"","url":"https://api.github.com/users/apache","html_url":"https://github.com/apache","followers_url":"https://api.github.com/users/apache/followers","following_url":"https://api.github.com/users/apache/following{/other_user}","gists_url":"https://api.github.com/users/apache/gists{/gist_id}","starred_url":"https://api.github.com/users/apache/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/apache/subscriptions","organizations_url":"https://api.github.com/users/apache/orgs","repos_url":"https://api.github.com/users/apache/repos","events_url":"https://api.github.com/users/apache/events{/privacy}","received_events_url":"https://api.github.com/users/apache/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/apache/flink","description":"Mirror of Apache Flink","fork":false,"url":"https://api.github.com/repos/apache/flink","forks_url":"https://api.github.com/repos/apache/flink/forks","keys_url":"https://api.github.com/repos/apache/flink/keys{/key_id}","collaborators_url":"https://api.github.com/repos/apache/flink/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/apache/flink/teams","hooks_url":"https://api.github.com/repos/apache/flink/hooks","issue_events_url":"https://api.github.com/repos/apache/flink/issues/events{/number}","events_url":"https://api.github.com/repos/apache/flink/events","assignees_url":"https://api.github.com/repos/apache/flink/assignees{/user}","branches_url":"https://api.github.com/repos/apache/flink/branches{/branch}","tags_url":"https://api.github.com/repos/apache/flink/tags","blobs_url":"https://api.github.com/repos/apache/flink/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/apache/flink/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/apache/flink/git/refs{/sha}","trees_url":"https://api.github.com/repos/apache/flink/git/trees{/sha}","statuses_url":"https://api.github.com/repos/apache/flink/statuses/{sha}","languages_url":"https://api.github.com/repos/apache/flink/languages","stargazers_url":"https://api.github.com/repos/apache/flink/stargazers","contributors_url":"https://api.github.com/repos/apache/flink/contributors","subscribers_url":"https://api.github.com/repos/apache/flink/subscribers","subscription_url":"https://api.github.com/repos/apache/flink/subscription","commits_url":"https://api.github.com/repos/apache/flink/commits{/sha}","git_commits_url":"https://api.github.com/repos/apache/flink/git/commits{/sha}","comments_url":"https://api.github.com/repos/apache/flink/comments{/number}","issue_comment_url":"https://api.github.com/repos/apache/flink/issues/comments{/number}","contents_url":"https://api.github.com/repos/apache/flink/contents/{+path}","compare_url":"https://api.github.com/repos/apache/flink/compare/{base}...{head}","merges_url":"https://api.github.com/repos/apache/flink/merges","archive_url":"https://api.github.com/repos/apache/flink/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/apache/flink/downloads","issues_url":"https://api.github.com/repos/apache/flink/issues{/number}","pulls_url":"https://api.github.com/repos/apache/flink/pulls{/number}","milestones_url":"https://api.github.com/repos/apache/flink/milestones{/number}","notifications_url":"https://api.github.com/repos/apache/flink/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/apache/flink/labels{/name}","releases_url":"https://api.github.com/repos/apache/flink/releases{/id}","deployments_url":"https://api.github.com/repos/apache/flink/deployments","created_at":"2014-06-07T07:00:10Z","updated_at":"2016-09-19T08:01:54Z","pushed_at":"2016-09-20T12:05:56Z","git_url":"git://github.com/apache/flink.git","ssh_url":"git@github.com:apache/flink.git","clone_url":"https://github.com/apache/flink.git","svn_url":"https://github.com/apache/flink","homepage":null,"size":126137,"stargazers_count":1588,"watchers_count":1588,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":1063,"mirror_url":"git://git.apache.org/flink.git","open_issues_count":154,"forks":1063,"open_issues":154,"watchers":1588,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/apache/flink/pulls/2510"},"html":{"href":"https://github.com/apache/flink/pull/2510"},"issue":{"href":"https://api.github.com/repos/apache/flink/issues/2510"},"comments":{"href":"https://api.github.com/repos/apache/flink/issues/2510/comments"},"review_comments":{"href":"https://api.github.com/repos/apache/flink/pulls/2510/comments"},"review_comment":{"href":"https://api.github.com/repos/apache/flink/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/apache/flink/pulls/2510/commits"},"statuses":{"href":"https://api.github.com/repos/apache/flink/statuses/e17462881cc2f0137cc8412eb578fb4c42baeb15"}}}}
{"actor":{"display_login":"ramkrish86"}}