{"action":"created","comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/comments/193454615","pull_request_review_id":126430320,"id":193454615,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDE5MzQ1NDYxNQ==","diff_hunk":"@@ -0,0 +1,52 @@\n+# Copyright 2017 Google Inc.\n+#\n+# Licensed under the Apache License, Version 2.0 (the \"License\");\n+# you may not use this file except in compliance with the License.\n+# You may obtain a copy of the License at\n+#\n+#            http://www.apache.org/licenses/LICENSE-2.0\n+#\n+# Unless required by applicable law or agreed to in writing, software\n+# distributed under the License is distributed on an \"AS IS\" BASIS,\n+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n+# See the License for the specific language governing permissions and\n+# limitations under the License.\n+\n+\n+import logging\n+import unittest\n+\n+from dataflow_python_examples.image_labels import detect_labels_uri\n+\n+\n+class TestDataGenerator(unittest.TestCase):\n+    \"\"\"The test cases are focused on the business logic.  In this case this is how we parse the\n+    schemas, generate data and label images.\n+\n+    Execution Note:\n+        This script is stored in professional-services/data-analytics/dataflow-python-examples/tests\n+        but should be copied to professional-services/data-analytics/dataflow-python-examples/ and\n+        run from there.\n+    \"\"\"\n+    def setUp(self):\n+        logging.basicConfig(level=logging.DEBUG)\n+\n+    def test_detect_labels_uri_return(self):","path":"data-analytics/dataflow-python-examples/tests/test_image_labels.py","position":34,"original_position":34,"commit_id":"39dcbff607cb9b089dfbee6672991c77943c0d96","original_commit_id":"39dcbff607cb9b089dfbee6672991c77943c0d96","user":{"login":"joshmcginley","id":8662458,"node_id":"MDQ6VXNlcjg2NjI0NTg=","avatar_url":"https://avatars3.githubusercontent.com/u/8662458?v=4","gravatar_id":"","url":"https://api.github.com/users/joshmcginley","html_url":"https://github.com/joshmcginley","followers_url":"https://api.github.com/users/joshmcginley/followers","following_url":"https://api.github.com/users/joshmcginley/following{/other_user}","gists_url":"https://api.github.com/users/joshmcginley/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmcginley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmcginley/subscriptions","organizations_url":"https://api.github.com/users/joshmcginley/orgs","repos_url":"https://api.github.com/users/joshmcginley/repos","events_url":"https://api.github.com/users/joshmcginley/events{/privacy}","received_events_url":"https://api.github.com/users/joshmcginley/received_events","type":"User","site_admin":false},"body":"Thanks for the comment.  We struggled with what to put in this particular unit test, as the pipeline is very simple.  \r\n\r\nWe're trying to demonstrate the best practice in each pipeline of unit testing.   By design we didn't include much \"business logic\" in these pipelines, to keep them simple and reusable.  \r\n\r\nI'd be reluctant to have a pipeline without unit tests, as we expect many people to copy/paste these pipelines and modify them to kickstart their development.  They're a lot more likely to follow the unit test best practice if that's what the example does.\r\n\r\nPerhaps we add a comment here explaining that the unit tests are here as an example.  It's primarily there to demonstrate how to structure your code to separate the \"business logic\" from the actual pipeline building, which is better tested through integration tests.  ","created_at":"2018-06-06T15:29:47Z","updated_at":"2018-06-06T15:29:47Z","html_url":"https://github.com/GoogleCloudPlatform/professional-services/pull/36#discussion_r193454615","pull_request_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/comments/193454615"},"html":{"href":"https://github.com/GoogleCloudPlatform/professional-services/pull/36#discussion_r193454615"},"pull_request":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36"}},"in_reply_to_id":193262704},"pull_request":{"url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36","id":187930052,"node_id":"MDExOlB1bGxSZXF1ZXN0MTg3OTMwMDUy","html_url":"https://github.com/GoogleCloudPlatform/professional-services/pull/36","diff_url":"https://github.com/GoogleCloudPlatform/professional-services/pull/36.diff","patch_url":"https://github.com/GoogleCloudPlatform/professional-services/pull/36.patch","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/36","number":36,"state":"open","locked":false,"title":"Data Generator and Image Labeler Pipelines","user":{"login":"joshmcginley","id":8662458,"node_id":"MDQ6VXNlcjg2NjI0NTg=","avatar_url":"https://avatars3.githubusercontent.com/u/8662458?v=4","gravatar_id":"","url":"https://api.github.com/users/joshmcginley","html_url":"https://github.com/joshmcginley","followers_url":"https://api.github.com/users/joshmcginley/followers","following_url":"https://api.github.com/users/joshmcginley/following{/other_user}","gists_url":"https://api.github.com/users/joshmcginley/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmcginley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmcginley/subscriptions","organizations_url":"https://api.github.com/users/joshmcginley/orgs","repos_url":"https://api.github.com/users/joshmcginley/repos","events_url":"https://api.github.com/users/joshmcginley/events{/privacy}","received_events_url":"https://api.github.com/users/joshmcginley/received_events","type":"User","site_admin":false},"body":"Add two new pipelines to our Python Dataflow examples:\r\n\r\n- A data generator used to generate fake data in BigQuery.  This is useful for quickly estimating costs and testing performance before uploading real data.  \r\n- An image labeler showing an example of using the ML Vision API.","created_at":"2018-05-14T19:42:01Z","updated_at":"2018-06-06T15:29:47Z","closed_at":null,"merged_at":null,"merge_commit_sha":"3ec39d5f865d8e3253848769b320d30bb94391b5","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36/commits","review_comments_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36/comments","review_comment_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/comments{/number}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/36/comments","statuses_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/statuses/39dcbff607cb9b089dfbee6672991c77943c0d96","head":{"label":"joshmcginley:master","ref":"master","sha":"39dcbff607cb9b089dfbee6672991c77943c0d96","user":{"login":"joshmcginley","id":8662458,"node_id":"MDQ6VXNlcjg2NjI0NTg=","avatar_url":"https://avatars3.githubusercontent.com/u/8662458?v=4","gravatar_id":"","url":"https://api.github.com/users/joshmcginley","html_url":"https://github.com/joshmcginley","followers_url":"https://api.github.com/users/joshmcginley/followers","following_url":"https://api.github.com/users/joshmcginley/following{/other_user}","gists_url":"https://api.github.com/users/joshmcginley/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmcginley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmcginley/subscriptions","organizations_url":"https://api.github.com/users/joshmcginley/orgs","repos_url":"https://api.github.com/users/joshmcginley/repos","events_url":"https://api.github.com/users/joshmcginley/events{/privacy}","received_events_url":"https://api.github.com/users/joshmcginley/received_events","type":"User","site_admin":false},"repo":{"id":113065220,"node_id":"MDEwOlJlcG9zaXRvcnkxMTMwNjUyMjA=","name":"professional-services","full_name":"joshmcginley/professional-services","owner":{"login":"joshmcginley","id":8662458,"node_id":"MDQ6VXNlcjg2NjI0NTg=","avatar_url":"https://avatars3.githubusercontent.com/u/8662458?v=4","gravatar_id":"","url":"https://api.github.com/users/joshmcginley","html_url":"https://github.com/joshmcginley","followers_url":"https://api.github.com/users/joshmcginley/followers","following_url":"https://api.github.com/users/joshmcginley/following{/other_user}","gists_url":"https://api.github.com/users/joshmcginley/gists{/gist_id}","starred_url":"https://api.github.com/users/joshmcginley/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joshmcginley/subscriptions","organizations_url":"https://api.github.com/users/joshmcginley/orgs","repos_url":"https://api.github.com/users/joshmcginley/repos","events_url":"https://api.github.com/users/joshmcginley/events{/privacy}","received_events_url":"https://api.github.com/users/joshmcginley/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/joshmcginley/professional-services","description":null,"fork":true,"url":"https://api.github.com/repos/joshmcginley/professional-services","forks_url":"https://api.github.com/repos/joshmcginley/professional-services/forks","keys_url":"https://api.github.com/repos/joshmcginley/professional-services/keys{/key_id}","collaborators_url":"https://api.github.com/repos/joshmcginley/professional-services/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/joshmcginley/professional-services/teams","hooks_url":"https://api.github.com/repos/joshmcginley/professional-services/hooks","issue_events_url":"https://api.github.com/repos/joshmcginley/professional-services/issues/events{/number}","events_url":"https://api.github.com/repos/joshmcginley/professional-services/events","assignees_url":"https://api.github.com/repos/joshmcginley/professional-services/assignees{/user}","branches_url":"https://api.github.com/repos/joshmcginley/professional-services/branches{/branch}","tags_url":"https://api.github.com/repos/joshmcginley/professional-services/tags","blobs_url":"https://api.github.com/repos/joshmcginley/professional-services/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/joshmcginley/professional-services/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/joshmcginley/professional-services/git/refs{/sha}","trees_url":"https://api.github.com/repos/joshmcginley/professional-services/git/trees{/sha}","statuses_url":"https://api.github.com/repos/joshmcginley/professional-services/statuses/{sha}","languages_url":"https://api.github.com/repos/joshmcginley/professional-services/languages","stargazers_url":"https://api.github.com/repos/joshmcginley/professional-services/stargazers","contributors_url":"https://api.github.com/repos/joshmcginley/professional-services/contributors","subscribers_url":"https://api.github.com/repos/joshmcginley/professional-services/subscribers","subscription_url":"https://api.github.com/repos/joshmcginley/professional-services/subscription","commits_url":"https://api.github.com/repos/joshmcginley/professional-services/commits{/sha}","git_commits_url":"https://api.github.com/repos/joshmcginley/professional-services/git/commits{/sha}","comments_url":"https://api.github.com/repos/joshmcginley/professional-services/comments{/number}","issue_comment_url":"https://api.github.com/repos/joshmcginley/professional-services/issues/comments{/number}","contents_url":"https://api.github.com/repos/joshmcginley/professional-services/contents/{+path}","compare_url":"https://api.github.com/repos/joshmcginley/professional-services/compare/{base}...{head}","merges_url":"https://api.github.com/repos/joshmcginley/professional-services/merges","archive_url":"https://api.github.com/repos/joshmcginley/professional-services/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/joshmcginley/professional-services/downloads","issues_url":"https://api.github.com/repos/joshmcginley/professional-services/issues{/number}","pulls_url":"https://api.github.com/repos/joshmcginley/professional-services/pulls{/number}","milestones_url":"https://api.github.com/repos/joshmcginley/professional-services/milestones{/number}","notifications_url":"https://api.github.com/repos/joshmcginley/professional-services/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/joshmcginley/professional-services/labels{/name}","releases_url":"https://api.github.com/repos/joshmcginley/professional-services/releases{/id}","deployments_url":"https://api.github.com/repos/joshmcginley/professional-services/deployments","created_at":"2017-12-04T16:14:42Z","updated_at":"2018-05-15T14:23:35Z","pushed_at":"2018-05-15T14:23:33Z","git_url":"git://github.com/joshmcginley/professional-services.git","ssh_url":"git@github.com:joshmcginley/professional-services.git","clone_url":"https://github.com/joshmcginley/professional-services.git","svn_url":"https://github.com/joshmcginley/professional-services","homepage":null,"size":2270,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":false,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"GoogleCloudPlatform:master","ref":"master","sha":"6d07f76e9233fa0d2ce5e2cbcd8b3c6cd497110f","user":{"login":"GoogleCloudPlatform","id":2810941,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI4MTA5NDE=","avatar_url":"https://avatars0.githubusercontent.com/u/2810941?v=4","gravatar_id":"","url":"https://api.github.com/users/GoogleCloudPlatform","html_url":"https://github.com/GoogleCloudPlatform","followers_url":"https://api.github.com/users/GoogleCloudPlatform/followers","following_url":"https://api.github.com/users/GoogleCloudPlatform/following{/other_user}","gists_url":"https://api.github.com/users/GoogleCloudPlatform/gists{/gist_id}","starred_url":"https://api.github.com/users/GoogleCloudPlatform/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GoogleCloudPlatform/subscriptions","organizations_url":"https://api.github.com/users/GoogleCloudPlatform/orgs","repos_url":"https://api.github.com/users/GoogleCloudPlatform/repos","events_url":"https://api.github.com/users/GoogleCloudPlatform/events{/privacy}","received_events_url":"https://api.github.com/users/GoogleCloudPlatform/received_events","type":"Organization","site_admin":false},"repo":{"id":91730359,"node_id":"MDEwOlJlcG9zaXRvcnk5MTczMDM1OQ==","name":"professional-services","full_name":"GoogleCloudPlatform/professional-services","owner":{"login":"GoogleCloudPlatform","id":2810941,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI4MTA5NDE=","avatar_url":"https://avatars0.githubusercontent.com/u/2810941?v=4","gravatar_id":"","url":"https://api.github.com/users/GoogleCloudPlatform","html_url":"https://github.com/GoogleCloudPlatform","followers_url":"https://api.github.com/users/GoogleCloudPlatform/followers","following_url":"https://api.github.com/users/GoogleCloudPlatform/following{/other_user}","gists_url":"https://api.github.com/users/GoogleCloudPlatform/gists{/gist_id}","starred_url":"https://api.github.com/users/GoogleCloudPlatform/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GoogleCloudPlatform/subscriptions","organizations_url":"https://api.github.com/users/GoogleCloudPlatform/orgs","repos_url":"https://api.github.com/users/GoogleCloudPlatform/repos","events_url":"https://api.github.com/users/GoogleCloudPlatform/events{/privacy}","received_events_url":"https://api.github.com/users/GoogleCloudPlatform/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/GoogleCloudPlatform/professional-services","description":null,"fork":false,"url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services","forks_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/forks","keys_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/keys{/key_id}","collaborators_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/teams","hooks_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/hooks","issue_events_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/events{/number}","events_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/events","assignees_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/assignees{/user}","branches_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/branches{/branch}","tags_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/tags","blobs_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/git/refs{/sha}","trees_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/git/trees{/sha}","statuses_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/statuses/{sha}","languages_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/languages","stargazers_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/stargazers","contributors_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/contributors","subscribers_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/subscribers","subscription_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/subscription","commits_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/commits{/sha}","git_commits_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/git/commits{/sha}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/comments{/number}","issue_comment_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/comments{/number}","contents_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/contents/{+path}","compare_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/compare/{base}...{head}","merges_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/merges","archive_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/downloads","issues_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues{/number}","pulls_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls{/number}","milestones_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/milestones{/number}","notifications_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/labels{/name}","releases_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/releases{/id}","deployments_url":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/deployments","created_at":"2017-05-18T19:29:27Z","updated_at":"2018-05-30T17:37:53Z","pushed_at":"2018-06-03T09:39:00Z","git_url":"git://github.com/GoogleCloudPlatform/professional-services.git","ssh_url":"git@github.com:GoogleCloudPlatform/professional-services.git","clone_url":"https://github.com/GoogleCloudPlatform/professional-services.git","svn_url":"https://github.com/GoogleCloudPlatform/professional-services","homepage":null,"size":18722,"stargazers_count":34,"watchers_count":34,"language":"Python","has_issues":false,"has_projects":true,"has_downloads":false,"has_wiki":false,"has_pages":false,"forks_count":18,"mirror_url":null,"archived":false,"open_issues_count":13,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":18,"open_issues":13,"watchers":34,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36"},"html":{"href":"https://github.com/GoogleCloudPlatform/professional-services/pull/36"},"issue":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/36"},"comments":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/issues/36/comments"},"review_comments":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36/comments"},"review_comment":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/pulls/36/commits"},"statuses":{"href":"https://api.github.com/repos/GoogleCloudPlatform/professional-services/statuses/39dcbff607cb9b089dfbee6672991c77943c0d96"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"joshmcginley"}}