{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/123","repository_url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/123/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/123/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/123/events","html_url":"https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/123","id":136952521,"number":123,"title":"BigQueryIO.Write creates empty table with DirectPipelineRunner","user":{"login":"nevillelyh","id":468514,"avatar_url":"https://avatars.githubusercontent.com/u/468514?v=3","gravatar_id":"","url":"https://api.github.com/users/nevillelyh","html_url":"https://github.com/nevillelyh","followers_url":"https://api.github.com/users/nevillelyh/followers","following_url":"https://api.github.com/users/nevillelyh/following{/other_user}","gists_url":"https://api.github.com/users/nevillelyh/gists{/gist_id}","starred_url":"https://api.github.com/users/nevillelyh/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nevillelyh/subscriptions","organizations_url":"https://api.github.com/users/nevillelyh/orgs","repos_url":"https://api.github.com/users/nevillelyh/repos","events_url":"https://api.github.com/users/nevillelyh/events{/privacy}","received_events_url":"https://api.github.com/users/nevillelyh/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-02-27T18:34:23Z","updated_at":"2016-02-29T05:56:40Z","closed_at":null,"body":"When running with DirectPipelineRunner, the following job creates the output table but writes nothing to it.\r\n\r\n```\r\npackage com.google.cloud.dataflow.examples;\r\n\r\nimport com.google.api.services.bigquery.model.TableFieldSchema;\r\nimport com.google.api.services.bigquery.model.TableRow;\r\nimport com.google.api.services.bigquery.model.TableSchema;\r\nimport com.google.cloud.dataflow.sdk.Pipeline;\r\nimport com.google.cloud.dataflow.sdk.io.BigQueryIO;\r\nimport com.google.cloud.dataflow.sdk.options.DataflowPipelineOptions;\r\nimport com.google.cloud.dataflow.sdk.options.PipelineOptionsFactory;\r\nimport com.google.cloud.dataflow.sdk.repackaged.com.google.common.collect.Lists;\r\nimport com.google.cloud.dataflow.sdk.transforms.Create;\r\nimport com.google.cloud.dataflow.sdk.transforms.DoFn;\r\nimport com.google.cloud.dataflow.sdk.transforms.ParDo;\r\n\r\nimport java.util.Random;\r\n\r\npublic class BigQueryOut {\r\n  public static void main(String[] args) {\r\n    TableSchema schema = new TableSchema().setFields(Lists.newArrayList(\r\n        new TableFieldSchema().setName(\"key\").setType(\"INTEGER\"),\r\n        new TableFieldSchema().setName(\"value\").setType(\"INTEGER\")\r\n    ));\r\n\r\n    DataflowPipelineOptions opts = PipelineOptionsFactory.fromArgs(args).as(DataflowPipelineOptions.class);\r\n    Pipeline pipeline = Pipeline.create(opts);\r\n    pipeline\r\n        .apply(Create.of(1, 2, 3, 4, 5))\r\n        .apply(ParDo.of(new DoFn<Integer, TableRow>() {\r\n          @Override\r\n          public void processElement(ProcessContext c) throws Exception {\r\n            TableRow row = new TableRow().set(\"key\", c.element()).set(\"value\", new Random().nextInt());\r\n            c.output(row);\r\n          }\r\n        }))\r\n        .apply(BigQueryIO.Write.to(\"neville_test.bqo\").withSchema(schema));\r\n    pipeline.run();\r\n  }\r\n}\r\n```\r\n\r\n```\r\n> runMain com.google.cloud.dataflow.examples.BigQueryOut --project=steel-ridge-91615\r\n[info] Compiling 1 Scala source and 1 Java source to /Users/neville/src/gcp/scio/scio-examples/target/scala-2.11/classes...\r\n[warn] bootstrap class path not set in conjunction with -source 1.7\r\n[info] Running com.google.cloud.dataflow.examples.BigQueryOut --project=steel-ridge-91615\r\n[run-main-7] WARN com.google.cloud.dataflow.sdk.io.BigQueryIO - No project specified for BigQuery table \"neville_test.bqo\". Assuming it is in \"steel-ridge-91615\". If the table is in a different project please specify it as a part of the B\r\nigQuery table definition.\r\n[run-main-7] INFO com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner - Executing pipeline using the DirectPipelineRunner.\r\n[run-main-7] INFO com.google.cloud.dataflow.sdk.io.BigQueryIO - Writing to BigQuery table steel-ridge-91615:neville_test.bqo\r\n[run-main-7] INFO com.google.cloud.dataflow.sdk.util.BigQueryTableInserter - Trying to create BigQuery table: steel-ridge-91615:neville_test.bqo\r\n[run-main-7] INFO com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner - Pipeline execution complete.\r\n```"},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/comments/190057091","html_url":"https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/123#issuecomment-190057091","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/DataflowJavaSDK/issues/123","id":190057091,"user":{"login":"davorbonaci","id":8442684,"avatar_url":"https://avatars.githubusercontent.com/u/8442684?v=3","gravatar_id":"","url":"https://api.github.com/users/davorbonaci","html_url":"https://github.com/davorbonaci","followers_url":"https://api.github.com/users/davorbonaci/followers","following_url":"https://api.github.com/users/davorbonaci/following{/other_user}","gists_url":"https://api.github.com/users/davorbonaci/gists{/gist_id}","starred_url":"https://api.github.com/users/davorbonaci/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/davorbonaci/subscriptions","organizations_url":"https://api.github.com/users/davorbonaci/orgs","repos_url":"https://api.github.com/users/davorbonaci/repos","events_url":"https://api.github.com/users/davorbonaci/events{/privacy}","received_events_url":"https://api.github.com/users/davorbonaci/received_events","type":"User","site_admin":false},"created_at":"2016-02-29T05:56:40Z","updated_at":"2016-02-29T05:56:40Z","body":"A quick question: how are you confirming that the table is empty? Do you query a table from the BigQuery UI? (A screenshot might be useful.)"}}