{"action":"created","comment":{"url":"https://api.github.com/repos/pravega/pravega/pulls/comments/228317897","pull_request_review_id":168565507,"id":228317897,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIyODMxNzg5Nw==","diff_hunk":"@@ -162,7 +162,14 @@ private String updateGroupStateIfNeeded() throws ReinitializationRequiredExcepti\n             atCheckpoint = groupState.getCheckpoint();\n             if (atCheckpoint != null) {\n                 log.info(\"{} at checkpoint {}\", this, atCheckpoint);\n-                return atCheckpoint;\n+                if (groupState.isCheckpointSilent(atCheckpoint)) {\n+                    // Checkpoint the reader immediately with the current position. Checkpoint Event is not generated.\n+                    groupState.checkpoint(atCheckpoint, getPosition());","path":"client/src/main/java/io/pravega/client/stream/impl/EventStreamReaderImpl.java","position":7,"original_position":7,"commit_id":"cc5c26a2a1fa61638832877f3339b413ba06d55f","original_commit_id":"9bcc08484c8f367d57d6ef0734400abaa2b25480","user":{"login":"fpj","id":572920,"node_id":"MDQ6VXNlcjU3MjkyMA==","avatar_url":"https://avatars3.githubusercontent.com/u/572920?v=4","gravatar_id":"","url":"https://api.github.com/users/fpj","html_url":"https://github.com/fpj","followers_url":"https://api.github.com/users/fpj/followers","following_url":"https://api.github.com/users/fpj/following{/other_user}","gists_url":"https://api.github.com/users/fpj/gists{/gist_id}","starred_url":"https://api.github.com/users/fpj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fpj/subscriptions","organizations_url":"https://api.github.com/users/fpj/orgs","repos_url":"https://api.github.com/users/fpj/repos","events_url":"https://api.github.com/users/fpj/events{/privacy}","received_events_url":"https://api.github.com/users/fpj/received_events","type":"User","site_admin":false},"body":"A couple of questions here:\r\n\r\n1- I'd like to clarify the contract that we need to satisfy. The contract that @tkaitchuck is referring to is with respect to visible only (triggered by the application or automatic) or all checkpoints? This PR is adding silent checkpoints, so I wonder whether the contract should take such checkpoints into account. \r\n2- In the case that we update offsets as @shrids proposes, is the concern that we don't know whether the reader has processed any event since the last checkpoint, and as such, we conservatively reset to the position of the last checkpoint? I also assume that if we provide a `Position` object to the reader offline call, then it doesn't need to match the offsets of a checkpoint.","created_at":"2018-10-25T20:11:32Z","updated_at":"2018-10-25T20:11:32Z","html_url":"https://github.com/pravega/pravega/pull/2980#discussion_r228317897","pull_request_url":"https://api.github.com/repos/pravega/pravega/pulls/2980","author_association":"COLLABORATOR","_links":{"self":{"href":"https://api.github.com/repos/pravega/pravega/pulls/comments/228317897"},"html":{"href":"https://github.com/pravega/pravega/pull/2980#discussion_r228317897"},"pull_request":{"href":"https://api.github.com/repos/pravega/pravega/pulls/2980"}},"in_reply_to_id":227555881},"pull_request":{"url":"https://api.github.com/repos/pravega/pravega/pulls/2980","id":222821958,"node_id":"MDExOlB1bGxSZXF1ZXN0MjIyODIxOTU4","html_url":"https://github.com/pravega/pravega/pull/2980","diff_url":"https://github.com/pravega/pravega/pull/2980.diff","patch_url":"https://github.com/pravega/pravega/pull/2980.patch","issue_url":"https://api.github.com/repos/pravega/pravega/issues/2980","number":2980,"state":"open","locked":false,"title":"Issue 2979: Improve precision of StreamCuts obtained via ReaderGroup","user":{"login":"shrids","id":11114165,"node_id":"MDQ6VXNlcjExMTE0MTY1","avatar_url":"https://avatars1.githubusercontent.com/u/11114165?v=4","gravatar_id":"","url":"https://api.github.com/users/shrids","html_url":"https://github.com/shrids","followers_url":"https://api.github.com/users/shrids/followers","following_url":"https://api.github.com/users/shrids/following{/other_user}","gists_url":"https://api.github.com/users/shrids/gists{/gist_id}","starred_url":"https://api.github.com/users/shrids/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shrids/subscriptions","organizations_url":"https://api.github.com/users/shrids/orgs","repos_url":"https://api.github.com/users/shrids/repos","events_url":"https://api.github.com/users/shrids/events{/privacy}","received_events_url":"https://api.github.com/users/shrids/received_events","type":"User","site_admin":false},"body":"**Change log description**  \r\n- Enable `ReaderGroup.generateStreamCuts()` api to obtain a StreamCut based on the latest position of all the readers.\r\n\r\n**Purpose of the change**  \r\nFixes#2979\r\n\r\n**What the code does**  \r\nAn API has been added to the `ReaderGroup` to generate a `StreamCut` after coordinating with all the readers to fetch their respective latest segment read offset .\r\n  ```\r\n   /**\r\n     * Generate a StreamCut after co-ordinating with all the readers using\r\n     * {@link io.pravega.client.state.StateSynchronizer}. All the readers return\r\n     * the latest read offsets of their assigned segments and this is used to generate\r\n     * a {@link StreamCut}.\r\n     * The configuration {@link ReaderGroupConfig#getGroupRefreshTimeMillis()} decides\r\n     * the maximum delay by which the readers return the latest read offsets of their\r\n     * assigned segments.\r\n     * <p>\r\n     * The {@link StreamCut} generated by this API can be used by the application as\r\n     * reference to position in the Stream, which is guaranteed to be greater than or equal to the\r\n     * position of the readers at the point of invocation of the API.\r\n     *\r\n     * @param backgroundExecutor A thread pool that will be used to poll if the\r\n     *                           positions from all the readers have been fetched.\r\n     * @return A future to a Map of Streams (that this group is reading from) to\r\n     * its corresponding cuts.\r\n     */\r\n    @Beta\r\n    CompletableFuture<Map<Stream, StreamCut>> generateStreamCuts(ScheduledExecutorService backgroundExecutor);\r\n```\r\nThis uses a similar mechanism as Checkpointing to fetch the position updates from all the readers to generate the `StreamCut`.\r\n\r\nThe StreamCut obtained by this method is guaranteed to be equal to (or greater than) the current position of the all the readers for the stream(s) managed by the ReaderGroup.\r\n\r\n**How to verify it**  \r\nAll the existing and newly added tests should pass.\r\n\r\n","created_at":"2018-10-15T08:31:18Z","updated_at":"2018-10-25T20:11:32Z","closed_at":null,"merged_at":null,"merge_commit_sha":"a70f2f0be4e41753c8e3d184cc70c8ba19c1e46d","assignee":{"login":"fpj","id":572920,"node_id":"MDQ6VXNlcjU3MjkyMA==","avatar_url":"https://avatars3.githubusercontent.com/u/572920?v=4","gravatar_id":"","url":"https://api.github.com/users/fpj","html_url":"https://github.com/fpj","followers_url":"https://api.github.com/users/fpj/followers","following_url":"https://api.github.com/users/fpj/following{/other_user}","gists_url":"https://api.github.com/users/fpj/gists{/gist_id}","starred_url":"https://api.github.com/users/fpj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fpj/subscriptions","organizations_url":"https://api.github.com/users/fpj/orgs","repos_url":"https://api.github.com/users/fpj/repos","events_url":"https://api.github.com/users/fpj/events{/privacy}","received_events_url":"https://api.github.com/users/fpj/received_events","type":"User","site_admin":false},"assignees":[{"login":"fpj","id":572920,"node_id":"MDQ6VXNlcjU3MjkyMA==","avatar_url":"https://avatars3.githubusercontent.com/u/572920?v=4","gravatar_id":"","url":"https://api.github.com/users/fpj","html_url":"https://github.com/fpj","followers_url":"https://api.github.com/users/fpj/followers","following_url":"https://api.github.com/users/fpj/following{/other_user}","gists_url":"https://api.github.com/users/fpj/gists{/gist_id}","starred_url":"https://api.github.com/users/fpj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fpj/subscriptions","organizations_url":"https://api.github.com/users/fpj/orgs","repos_url":"https://api.github.com/users/fpj/repos","events_url":"https://api.github.com/users/fpj/events{/privacy}","received_events_url":"https://api.github.com/users/fpj/received_events","type":"User","site_admin":false},{"login":"tkaitchuck","id":4756752,"node_id":"MDQ6VXNlcjQ3NTY3NTI=","avatar_url":"https://avatars1.githubusercontent.com/u/4756752?v=4","gravatar_id":"","url":"https://api.github.com/users/tkaitchuck","html_url":"https://github.com/tkaitchuck","followers_url":"https://api.github.com/users/tkaitchuck/followers","following_url":"https://api.github.com/users/tkaitchuck/following{/other_user}","gists_url":"https://api.github.com/users/tkaitchuck/gists{/gist_id}","starred_url":"https://api.github.com/users/tkaitchuck/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tkaitchuck/subscriptions","organizations_url":"https://api.github.com/users/tkaitchuck/orgs","repos_url":"https://api.github.com/users/tkaitchuck/repos","events_url":"https://api.github.com/users/tkaitchuck/events{/privacy}","received_events_url":"https://api.github.com/users/tkaitchuck/received_events","type":"User","site_admin":false}],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/pravega/pravega/pulls/2980/commits","review_comments_url":"https://api.github.com/repos/pravega/pravega/pulls/2980/comments","review_comment_url":"https://api.github.com/repos/pravega/pravega/pulls/comments{/number}","comments_url":"https://api.github.com/repos/pravega/pravega/issues/2980/comments","statuses_url":"https://api.github.com/repos/pravega/pravega/statuses/cc5c26a2a1fa61638832877f3339b413ba06d55f","head":{"label":"shrids:issue-precise-streamCut-api","ref":"issue-precise-streamCut-api","sha":"cc5c26a2a1fa61638832877f3339b413ba06d55f","user":{"login":"shrids","id":11114165,"node_id":"MDQ6VXNlcjExMTE0MTY1","avatar_url":"https://avatars1.githubusercontent.com/u/11114165?v=4","gravatar_id":"","url":"https://api.github.com/users/shrids","html_url":"https://github.com/shrids","followers_url":"https://api.github.com/users/shrids/followers","following_url":"https://api.github.com/users/shrids/following{/other_user}","gists_url":"https://api.github.com/users/shrids/gists{/gist_id}","starred_url":"https://api.github.com/users/shrids/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shrids/subscriptions","organizations_url":"https://api.github.com/users/shrids/orgs","repos_url":"https://api.github.com/users/shrids/repos","events_url":"https://api.github.com/users/shrids/events{/privacy}","received_events_url":"https://api.github.com/users/shrids/received_events","type":"User","site_admin":false},"repo":{"id":90976985,"node_id":"MDEwOlJlcG9zaXRvcnk5MDk3Njk4NQ==","name":"pravega","full_name":"shrids/pravega","private":false,"owner":{"login":"shrids","id":11114165,"node_id":"MDQ6VXNlcjExMTE0MTY1","avatar_url":"https://avatars1.githubusercontent.com/u/11114165?v=4","gravatar_id":"","url":"https://api.github.com/users/shrids","html_url":"https://github.com/shrids","followers_url":"https://api.github.com/users/shrids/followers","following_url":"https://api.github.com/users/shrids/following{/other_user}","gists_url":"https://api.github.com/users/shrids/gists{/gist_id}","starred_url":"https://api.github.com/users/shrids/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shrids/subscriptions","organizations_url":"https://api.github.com/users/shrids/orgs","repos_url":"https://api.github.com/users/shrids/repos","events_url":"https://api.github.com/users/shrids/events{/privacy}","received_events_url":"https://api.github.com/users/shrids/received_events","type":"User","site_admin":false},"html_url":"https://github.com/shrids/pravega","description":"Pravega - Streaming as a new software defined storage primitive","fork":true,"url":"https://api.github.com/repos/shrids/pravega","forks_url":"https://api.github.com/repos/shrids/pravega/forks","keys_url":"https://api.github.com/repos/shrids/pravega/keys{/key_id}","collaborators_url":"https://api.github.com/repos/shrids/pravega/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/shrids/pravega/teams","hooks_url":"https://api.github.com/repos/shrids/pravega/hooks","issue_events_url":"https://api.github.com/repos/shrids/pravega/issues/events{/number}","events_url":"https://api.github.com/repos/shrids/pravega/events","assignees_url":"https://api.github.com/repos/shrids/pravega/assignees{/user}","branches_url":"https://api.github.com/repos/shrids/pravega/branches{/branch}","tags_url":"https://api.github.com/repos/shrids/pravega/tags","blobs_url":"https://api.github.com/repos/shrids/pravega/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/shrids/pravega/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/shrids/pravega/git/refs{/sha}","trees_url":"https://api.github.com/repos/shrids/pravega/git/trees{/sha}","statuses_url":"https://api.github.com/repos/shrids/pravega/statuses/{sha}","languages_url":"https://api.github.com/repos/shrids/pravega/languages","stargazers_url":"https://api.github.com/repos/shrids/pravega/stargazers","contributors_url":"https://api.github.com/repos/shrids/pravega/contributors","subscribers_url":"https://api.github.com/repos/shrids/pravega/subscribers","subscription_url":"https://api.github.com/repos/shrids/pravega/subscription","commits_url":"https://api.github.com/repos/shrids/pravega/commits{/sha}","git_commits_url":"https://api.github.com/repos/shrids/pravega/git/commits{/sha}","comments_url":"https://api.github.com/repos/shrids/pravega/comments{/number}","issue_comment_url":"https://api.github.com/repos/shrids/pravega/issues/comments{/number}","contents_url":"https://api.github.com/repos/shrids/pravega/contents/{+path}","compare_url":"https://api.github.com/repos/shrids/pravega/compare/{base}...{head}","merges_url":"https://api.github.com/repos/shrids/pravega/merges","archive_url":"https://api.github.com/repos/shrids/pravega/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/shrids/pravega/downloads","issues_url":"https://api.github.com/repos/shrids/pravega/issues{/number}","pulls_url":"https://api.github.com/repos/shrids/pravega/pulls{/number}","milestones_url":"https://api.github.com/repos/shrids/pravega/milestones{/number}","notifications_url":"https://api.github.com/repos/shrids/pravega/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/shrids/pravega/labels{/name}","releases_url":"https://api.github.com/repos/shrids/pravega/releases{/id}","deployments_url":"https://api.github.com/repos/shrids/pravega/deployments","created_at":"2017-05-11T12:31:46Z","updated_at":"2018-10-25T04:19:32Z","pushed_at":"2018-10-25T09:24:37Z","git_url":"git://github.com/shrids/pravega.git","ssh_url":"git@github.com:shrids/pravega.git","clone_url":"https://github.com/shrids/pravega.git","svn_url":"https://github.com/shrids/pravega","homepage":"http://pravega.io","size":26643,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":7,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":7,"watchers":0,"default_branch":"master"}},"base":{"label":"pravega:master","ref":"master","sha":"811c9c29d77d091911ca7cd5bb0c428d8799c063","user":{"login":"pravega","id":25698199,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1Njk4MTk5","avatar_url":"https://avatars1.githubusercontent.com/u/25698199?v=4","gravatar_id":"","url":"https://api.github.com/users/pravega","html_url":"https://github.com/pravega","followers_url":"https://api.github.com/users/pravega/followers","following_url":"https://api.github.com/users/pravega/following{/other_user}","gists_url":"https://api.github.com/users/pravega/gists{/gist_id}","starred_url":"https://api.github.com/users/pravega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pravega/subscriptions","organizations_url":"https://api.github.com/users/pravega/orgs","repos_url":"https://api.github.com/users/pravega/repos","events_url":"https://api.github.com/users/pravega/events{/privacy}","received_events_url":"https://api.github.com/users/pravega/received_events","type":"Organization","site_admin":false},"repo":{"id":63096822,"node_id":"MDEwOlJlcG9zaXRvcnk2MzA5NjgyMg==","name":"pravega","full_name":"pravega/pravega","private":false,"owner":{"login":"pravega","id":25698199,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1Njk4MTk5","avatar_url":"https://avatars1.githubusercontent.com/u/25698199?v=4","gravatar_id":"","url":"https://api.github.com/users/pravega","html_url":"https://github.com/pravega","followers_url":"https://api.github.com/users/pravega/followers","following_url":"https://api.github.com/users/pravega/following{/other_user}","gists_url":"https://api.github.com/users/pravega/gists{/gist_id}","starred_url":"https://api.github.com/users/pravega/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pravega/subscriptions","organizations_url":"https://api.github.com/users/pravega/orgs","repos_url":"https://api.github.com/users/pravega/repos","events_url":"https://api.github.com/users/pravega/events{/privacy}","received_events_url":"https://api.github.com/users/pravega/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/pravega/pravega","description":"Pravega - Streaming as a new software defined storage primitive","fork":false,"url":"https://api.github.com/repos/pravega/pravega","forks_url":"https://api.github.com/repos/pravega/pravega/forks","keys_url":"https://api.github.com/repos/pravega/pravega/keys{/key_id}","collaborators_url":"https://api.github.com/repos/pravega/pravega/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/pravega/pravega/teams","hooks_url":"https://api.github.com/repos/pravega/pravega/hooks","issue_events_url":"https://api.github.com/repos/pravega/pravega/issues/events{/number}","events_url":"https://api.github.com/repos/pravega/pravega/events","assignees_url":"https://api.github.com/repos/pravega/pravega/assignees{/user}","branches_url":"https://api.github.com/repos/pravega/pravega/branches{/branch}","tags_url":"https://api.github.com/repos/pravega/pravega/tags","blobs_url":"https://api.github.com/repos/pravega/pravega/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/pravega/pravega/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/pravega/pravega/git/refs{/sha}","trees_url":"https://api.github.com/repos/pravega/pravega/git/trees{/sha}","statuses_url":"https://api.github.com/repos/pravega/pravega/statuses/{sha}","languages_url":"https://api.github.com/repos/pravega/pravega/languages","stargazers_url":"https://api.github.com/repos/pravega/pravega/stargazers","contributors_url":"https://api.github.com/repos/pravega/pravega/contributors","subscribers_url":"https://api.github.com/repos/pravega/pravega/subscribers","subscription_url":"https://api.github.com/repos/pravega/pravega/subscription","commits_url":"https://api.github.com/repos/pravega/pravega/commits{/sha}","git_commits_url":"https://api.github.com/repos/pravega/pravega/git/commits{/sha}","comments_url":"https://api.github.com/repos/pravega/pravega/comments{/number}","issue_comment_url":"https://api.github.com/repos/pravega/pravega/issues/comments{/number}","contents_url":"https://api.github.com/repos/pravega/pravega/contents/{+path}","compare_url":"https://api.github.com/repos/pravega/pravega/compare/{base}...{head}","merges_url":"https://api.github.com/repos/pravega/pravega/merges","archive_url":"https://api.github.com/repos/pravega/pravega/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/pravega/pravega/downloads","issues_url":"https://api.github.com/repos/pravega/pravega/issues{/number}","pulls_url":"https://api.github.com/repos/pravega/pravega/pulls{/number}","milestones_url":"https://api.github.com/repos/pravega/pravega/milestones{/number}","notifications_url":"https://api.github.com/repos/pravega/pravega/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/pravega/pravega/labels{/name}","releases_url":"https://api.github.com/repos/pravega/pravega/releases{/id}","deployments_url":"https://api.github.com/repos/pravega/pravega/deployments","created_at":"2016-07-11T19:41:03Z","updated_at":"2018-10-25T19:43:37Z","pushed_at":"2018-10-25T19:43:31Z","git_url":"git://github.com/pravega/pravega.git","ssh_url":"git@github.com:pravega/pravega.git","clone_url":"https://github.com/pravega/pravega.git","svn_url":"https://github.com/pravega/pravega","homepage":"http://pravega.io","size":26747,"stargazers_count":277,"watchers_count":277,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":91,"mirror_url":null,"archived":false,"open_issues_count":384,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":91,"open_issues":384,"watchers":277,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/pravega/pravega/pulls/2980"},"html":{"href":"https://github.com/pravega/pravega/pull/2980"},"issue":{"href":"https://api.github.com/repos/pravega/pravega/issues/2980"},"comments":{"href":"https://api.github.com/repos/pravega/pravega/issues/2980/comments"},"review_comments":{"href":"https://api.github.com/repos/pravega/pravega/pulls/2980/comments"},"review_comment":{"href":"https://api.github.com/repos/pravega/pravega/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/pravega/pravega/pulls/2980/commits"},"statuses":{"href":"https://api.github.com/repos/pravega/pravega/statuses/cc5c26a2a1fa61638832877f3339b413ba06d55f"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"fpj"}}