{"action":"created","comment":{"url":"https://api.github.com/repos/kmjung/beam/pulls/comments/192891200","pull_request_review_id":125471123,"id":192891200,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5Mjg5MTIwMA==","diff_hunk":"@@ -234,24 +147,53 @@ public synchronized long getEstimatedSizeBytes(PipelineOptions options)\n   }\n \n   /**\n-   * You can potentially create multiple readers on a source.\n+   * Gets the estimated number of bytes returned by the current source. For sources created by a\n+   * split() operation, this is a fraction of the total table size.\n    */\n+  @Override\n+  public synchronized long getEstimatedSizeBytes(PipelineOptions options)\n+      throws IOException, InterruptedException {\n+    if (cachedReadSizeBytes == null) {\n+      BigQueryOptions bigQueryOptions = options.as(BigQueryOptions.class);\n+      TableReference tableReference = getEffectiveTableReference(bigQueryOptions);\n+      Table table = bqServices.getDatasetService(bigQueryOptions).getTable(tableReference);\n+      if (table != null) {\n+        cachedReadSizeBytes = table.getNumBytes();\n+      }\n+      if (cachedReadSizeBytes == null) {\n+        cachedReadSizeBytes = 0L;\n+      }\n+    }\n+\n+    return cachedReadSizeBytes;\n+  }\n+\n   @Override\n   public BoundedReader<T> createReader(PipelineOptions options) {\n-    return new BigQueryParallelReader<>(\n-        readSession,\n-        readLocation,\n-        (readSessionOptions != null && readSessionOptions.getRowBatchSize() != null)\n-            ? readSessionOptions.getRowBatchSize()\n-            : 1000,\n-        parseFn,\n-        this,\n-        bqServices,\n-        options.as(BigQueryOptions.class));\n+    throw new UnsupportedOperationException(\"BigQuery source must be split before being read\");","path":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryParallelReadTableSource.java","position":251,"original_position":251,"commit_id":"2be5ae20bcc77807869354104d7509bcb9b397a7","original_commit_id":"2be5ae20bcc77807869354104d7509bcb9b397a7","user":{"login":"chamikaramj","id":10819857,"node_id":"MDQ6VXNlcjEwODE5ODU3","avatar_url":"https://avatars3.githubusercontent.com/u/10819857?v=4","gravatar_id":"","url":"https://api.github.com/users/chamikaramj","html_url":"https://github.com/chamikaramj","followers_url":"https://api.github.com/users/chamikaramj/followers","following_url":"https://api.github.com/users/chamikaramj/following{/other_user}","gists_url":"https://api.github.com/users/chamikaramj/gists{/gist_id}","starred_url":"https://api.github.com/users/chamikaramj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chamikaramj/subscriptions","organizations_url":"https://api.github.com/users/chamikaramj/orgs","repos_url":"https://api.github.com/users/chamikaramj/repos","events_url":"https://api.github.com/users/chamikaramj/events{/privacy}","received_events_url":"https://api.github.com/users/chamikaramj/received_events","type":"User","site_admin":false},"body":"This will fail for DirectRunner that does not perform splitting I believe. Also, this will fail if the source is used as a side input.","created_at":"2018-06-04T21:49:54Z","updated_at":"2018-06-04T21:55:16Z","html_url":"https://github.com/kmjung/beam/pull/4#discussion_r192891200","pull_request_url":"https://api.github.com/repos/kmjung/beam/pulls/4","author_association":"NONE","_links":{"self":{"href":"https://api.github.com/repos/kmjung/beam/pulls/comments/192891200"},"html":{"href":"https://github.com/kmjung/beam/pull/4#discussion_r192891200"},"pull_request":{"href":"https://api.github.com/repos/kmjung/beam/pulls/4"}}},"pull_request":{"url":"https://api.github.com/repos/kmjung/beam/pulls/4","id":186745841,"node_id":"MDExOlB1bGxSZXF1ZXN0MTg2NzQ1ODQx","html_url":"https://github.com/kmjung/beam/pull/4","diff_url":"https://github.com/kmjung/beam/pull/4.diff","patch_url":"https://github.com/kmjung/beam/pull/4.patch","issue_url":"https://api.github.com/repos/kmjung/beam/issues/4","number":4,"state":"open","locked":false,"title":"Allow BigQuery parallel read connector to read from queries.","user":{"login":"kmjung","id":10439787,"node_id":"MDQ6VXNlcjEwNDM5Nzg3","avatar_url":"https://avatars0.githubusercontent.com/u/10439787?v=4","gravatar_id":"","url":"https://api.github.com/users/kmjung","html_url":"https://github.com/kmjung","followers_url":"https://api.github.com/users/kmjung/followers","following_url":"https://api.github.com/users/kmjung/following{/other_user}","gists_url":"https://api.github.com/users/kmjung/gists{/gist_id}","starred_url":"https://api.github.com/users/kmjung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmjung/subscriptions","organizations_url":"https://api.github.com/users/kmjung/orgs","repos_url":"https://api.github.com/users/kmjung/repos","events_url":"https://api.github.com/users/kmjung/events{/privacy}","received_events_url":"https://api.github.com/users/kmjung/received_events","type":"User","site_admin":false},"body":"This change implements a query-based source for the parallel read\r\nsource and refactors the common portions out from the table source\r\ncode and into a new stream-based source and reader.\r\n\r\nDESCRIPTION HERE\r\n\r\n------------------------\r\n\r\nFollow this checklist to help us incorporate your contribution quickly and easily:\r\n\r\n - [ ] Make sure there is a [JIRA issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the change (usually before you start working on it).  Trivial changes like typos do not require a JIRA issue.  Your pull request should address just this issue, without pulling in other changes.\r\n - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue.\r\n - [ ] Write a pull request description that is detailed enough to understand:\r\n   - [ ] What the pull request does\r\n   - [ ] Why it does it\r\n   - [ ] How it does it\r\n   - [ ] Why this approach\r\n - [ ] Each commit in the pull request should have a meaningful subject line and body.\r\n - [ ] Run `mvn clean verify` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.\r\n - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).\r\n\r\n","created_at":"2018-05-08T20:24:08Z","updated_at":"2018-06-04T21:55:16Z","closed_at":null,"merged_at":null,"merge_commit_sha":"bbbd88dd200af7867376a4d9c61413ca53d2edff","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/kmjung/beam/pulls/4/commits","review_comments_url":"https://api.github.com/repos/kmjung/beam/pulls/4/comments","review_comment_url":"https://api.github.com/repos/kmjung/beam/pulls/comments{/number}","comments_url":"https://api.github.com/repos/kmjung/beam/issues/4/comments","statuses_url":"https://api.github.com/repos/kmjung/beam/statuses/2be5ae20bcc77807869354104d7509bcb9b397a7","head":{"label":"kmjung:bq_connector_query","ref":"bq_connector_query","sha":"2be5ae20bcc77807869354104d7509bcb9b397a7","user":{"login":"kmjung","id":10439787,"node_id":"MDQ6VXNlcjEwNDM5Nzg3","avatar_url":"https://avatars0.githubusercontent.com/u/10439787?v=4","gravatar_id":"","url":"https://api.github.com/users/kmjung","html_url":"https://github.com/kmjung","followers_url":"https://api.github.com/users/kmjung/followers","following_url":"https://api.github.com/users/kmjung/following{/other_user}","gists_url":"https://api.github.com/users/kmjung/gists{/gist_id}","starred_url":"https://api.github.com/users/kmjung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmjung/subscriptions","organizations_url":"https://api.github.com/users/kmjung/orgs","repos_url":"https://api.github.com/users/kmjung/repos","events_url":"https://api.github.com/users/kmjung/events{/privacy}","received_events_url":"https://api.github.com/users/kmjung/received_events","type":"User","site_admin":false},"repo":{"id":118979692,"node_id":"MDEwOlJlcG9zaXRvcnkxMTg5Nzk2OTI=","name":"beam","full_name":"kmjung/beam","owner":{"login":"kmjung","id":10439787,"node_id":"MDQ6VXNlcjEwNDM5Nzg3","avatar_url":"https://avatars0.githubusercontent.com/u/10439787?v=4","gravatar_id":"","url":"https://api.github.com/users/kmjung","html_url":"https://github.com/kmjung","followers_url":"https://api.github.com/users/kmjung/followers","following_url":"https://api.github.com/users/kmjung/following{/other_user}","gists_url":"https://api.github.com/users/kmjung/gists{/gist_id}","starred_url":"https://api.github.com/users/kmjung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmjung/subscriptions","organizations_url":"https://api.github.com/users/kmjung/orgs","repos_url":"https://api.github.com/users/kmjung/repos","events_url":"https://api.github.com/users/kmjung/events{/privacy}","received_events_url":"https://api.github.com/users/kmjung/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/kmjung/beam","description":"Apache Beam","fork":true,"url":"https://api.github.com/repos/kmjung/beam","forks_url":"https://api.github.com/repos/kmjung/beam/forks","keys_url":"https://api.github.com/repos/kmjung/beam/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kmjung/beam/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kmjung/beam/teams","hooks_url":"https://api.github.com/repos/kmjung/beam/hooks","issue_events_url":"https://api.github.com/repos/kmjung/beam/issues/events{/number}","events_url":"https://api.github.com/repos/kmjung/beam/events","assignees_url":"https://api.github.com/repos/kmjung/beam/assignees{/user}","branches_url":"https://api.github.com/repos/kmjung/beam/branches{/branch}","tags_url":"https://api.github.com/repos/kmjung/beam/tags","blobs_url":"https://api.github.com/repos/kmjung/beam/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kmjung/beam/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kmjung/beam/git/refs{/sha}","trees_url":"https://api.github.com/repos/kmjung/beam/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kmjung/beam/statuses/{sha}","languages_url":"https://api.github.com/repos/kmjung/beam/languages","stargazers_url":"https://api.github.com/repos/kmjung/beam/stargazers","contributors_url":"https://api.github.com/repos/kmjung/beam/contributors","subscribers_url":"https://api.github.com/repos/kmjung/beam/subscribers","subscription_url":"https://api.github.com/repos/kmjung/beam/subscription","commits_url":"https://api.github.com/repos/kmjung/beam/commits{/sha}","git_commits_url":"https://api.github.com/repos/kmjung/beam/git/commits{/sha}","comments_url":"https://api.github.com/repos/kmjung/beam/comments{/number}","issue_comment_url":"https://api.github.com/repos/kmjung/beam/issues/comments{/number}","contents_url":"https://api.github.com/repos/kmjung/beam/contents/{+path}","compare_url":"https://api.github.com/repos/kmjung/beam/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kmjung/beam/merges","archive_url":"https://api.github.com/repos/kmjung/beam/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kmjung/beam/downloads","issues_url":"https://api.github.com/repos/kmjung/beam/issues{/number}","pulls_url":"https://api.github.com/repos/kmjung/beam/pulls{/number}","milestones_url":"https://api.github.com/repos/kmjung/beam/milestones{/number}","notifications_url":"https://api.github.com/repos/kmjung/beam/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kmjung/beam/labels{/name}","releases_url":"https://api.github.com/repos/kmjung/beam/releases{/id}","deployments_url":"https://api.github.com/repos/kmjung/beam/deployments","created_at":"2018-01-25T23:26:58Z","updated_at":"2018-05-17T16:05:52Z","pushed_at":"2018-05-17T16:05:41Z","git_url":"git://github.com/kmjung/beam.git","ssh_url":"git@github.com:kmjung/beam.git","clone_url":"https://github.com/kmjung/beam.git","svn_url":"https://github.com/kmjung/beam","homepage":"","size":60686,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":2,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":2,"watchers":0,"default_branch":"master"}},"base":{"label":"kmjung:bq_connector_release_master","ref":"bq_connector_release_master","sha":"e708cae71883e34560fae797160e62599688e1de","user":{"login":"kmjung","id":10439787,"node_id":"MDQ6VXNlcjEwNDM5Nzg3","avatar_url":"https://avatars0.githubusercontent.com/u/10439787?v=4","gravatar_id":"","url":"https://api.github.com/users/kmjung","html_url":"https://github.com/kmjung","followers_url":"https://api.github.com/users/kmjung/followers","following_url":"https://api.github.com/users/kmjung/following{/other_user}","gists_url":"https://api.github.com/users/kmjung/gists{/gist_id}","starred_url":"https://api.github.com/users/kmjung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmjung/subscriptions","organizations_url":"https://api.github.com/users/kmjung/orgs","repos_url":"https://api.github.com/users/kmjung/repos","events_url":"https://api.github.com/users/kmjung/events{/privacy}","received_events_url":"https://api.github.com/users/kmjung/received_events","type":"User","site_admin":false},"repo":{"id":118979692,"node_id":"MDEwOlJlcG9zaXRvcnkxMTg5Nzk2OTI=","name":"beam","full_name":"kmjung/beam","owner":{"login":"kmjung","id":10439787,"node_id":"MDQ6VXNlcjEwNDM5Nzg3","avatar_url":"https://avatars0.githubusercontent.com/u/10439787?v=4","gravatar_id":"","url":"https://api.github.com/users/kmjung","html_url":"https://github.com/kmjung","followers_url":"https://api.github.com/users/kmjung/followers","following_url":"https://api.github.com/users/kmjung/following{/other_user}","gists_url":"https://api.github.com/users/kmjung/gists{/gist_id}","starred_url":"https://api.github.com/users/kmjung/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kmjung/subscriptions","organizations_url":"https://api.github.com/users/kmjung/orgs","repos_url":"https://api.github.com/users/kmjung/repos","events_url":"https://api.github.com/users/kmjung/events{/privacy}","received_events_url":"https://api.github.com/users/kmjung/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/kmjung/beam","description":"Apache Beam","fork":true,"url":"https://api.github.com/repos/kmjung/beam","forks_url":"https://api.github.com/repos/kmjung/beam/forks","keys_url":"https://api.github.com/repos/kmjung/beam/keys{/key_id}","collaborators_url":"https://api.github.com/repos/kmjung/beam/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/kmjung/beam/teams","hooks_url":"https://api.github.com/repos/kmjung/beam/hooks","issue_events_url":"https://api.github.com/repos/kmjung/beam/issues/events{/number}","events_url":"https://api.github.com/repos/kmjung/beam/events","assignees_url":"https://api.github.com/repos/kmjung/beam/assignees{/user}","branches_url":"https://api.github.com/repos/kmjung/beam/branches{/branch}","tags_url":"https://api.github.com/repos/kmjung/beam/tags","blobs_url":"https://api.github.com/repos/kmjung/beam/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/kmjung/beam/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/kmjung/beam/git/refs{/sha}","trees_url":"https://api.github.com/repos/kmjung/beam/git/trees{/sha}","statuses_url":"https://api.github.com/repos/kmjung/beam/statuses/{sha}","languages_url":"https://api.github.com/repos/kmjung/beam/languages","stargazers_url":"https://api.github.com/repos/kmjung/beam/stargazers","contributors_url":"https://api.github.com/repos/kmjung/beam/contributors","subscribers_url":"https://api.github.com/repos/kmjung/beam/subscribers","subscription_url":"https://api.github.com/repos/kmjung/beam/subscription","commits_url":"https://api.github.com/repos/kmjung/beam/commits{/sha}","git_commits_url":"https://api.github.com/repos/kmjung/beam/git/commits{/sha}","comments_url":"https://api.github.com/repos/kmjung/beam/comments{/number}","issue_comment_url":"https://api.github.com/repos/kmjung/beam/issues/comments{/number}","contents_url":"https://api.github.com/repos/kmjung/beam/contents/{+path}","compare_url":"https://api.github.com/repos/kmjung/beam/compare/{base}...{head}","merges_url":"https://api.github.com/repos/kmjung/beam/merges","archive_url":"https://api.github.com/repos/kmjung/beam/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/kmjung/beam/downloads","issues_url":"https://api.github.com/repos/kmjung/beam/issues{/number}","pulls_url":"https://api.github.com/repos/kmjung/beam/pulls{/number}","milestones_url":"https://api.github.com/repos/kmjung/beam/milestones{/number}","notifications_url":"https://api.github.com/repos/kmjung/beam/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/kmjung/beam/labels{/name}","releases_url":"https://api.github.com/repos/kmjung/beam/releases{/id}","deployments_url":"https://api.github.com/repos/kmjung/beam/deployments","created_at":"2018-01-25T23:26:58Z","updated_at":"2018-05-17T16:05:52Z","pushed_at":"2018-05-17T16:05:41Z","git_url":"git://github.com/kmjung/beam.git","ssh_url":"git@github.com:kmjung/beam.git","clone_url":"https://github.com/kmjung/beam.git","svn_url":"https://github.com/kmjung/beam","homepage":"","size":60686,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":2,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":2,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/kmjung/beam/pulls/4"},"html":{"href":"https://github.com/kmjung/beam/pull/4"},"issue":{"href":"https://api.github.com/repos/kmjung/beam/issues/4"},"comments":{"href":"https://api.github.com/repos/kmjung/beam/issues/4/comments"},"review_comments":{"href":"https://api.github.com/repos/kmjung/beam/pulls/4/comments"},"review_comment":{"href":"https://api.github.com/repos/kmjung/beam/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/kmjung/beam/pulls/4/commits"},"statuses":{"href":"https://api.github.com/repos/kmjung/beam/statuses/2be5ae20bcc77807869354104d7509bcb9b397a7"}},"author_association":"OWNER"}}
{"actor":{"display_login":"chamikaramj"}}