{"action":"created","comment":{"url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/comments/241717852","pull_request_review_id":185060866,"id":241717852,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDI0MTcxNzg1Mg==","diff_hunk":"@@ -739,7 +742,40 @@ protected function _runTransformations(Collection $transformations, $output = tr\n                     throw new Exception($message, $e->getCode(), $e, null, array(\"message\" => $e->getMessage()));\n                 }\n             }\n-\n+            if ($deferredJobIds) {\n+                $event = new StorageApi\\Event();\n+                $logMessage = \"Waiting for \" . count($deferredJobIds) . \" Storage jobs\";\n+                $logData = array(\n+                    \"jobs\" => $deferredJobIds,\n+                    \"runId\" => $this->getRunId()\n+                );\n+                $event->setComponent(Util::getLog()->getAppName())\n+                    ->setMessage($logMessage)\n+                    ->setParams($logData);\n+                $event->setType(StorageApi\\Event::TYPE_INFO);\n+                $this->getConfiguration()->getStorageApiClient()->createEvent($event);","path":"library/Keboola/Transformation/Manager.php","position":43,"original_position":43,"commit_id":"bb2c36b989d82bc3dfee3f165f2d65c5ac157328","original_commit_id":"bb2c36b989d82bc3dfee3f165f2d65c5ac157328","user":{"login":"ondrejhlavacek","id":497675,"node_id":"MDQ6VXNlcjQ5NzY3NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/497675?v=4","gravatar_id":"","url":"https://api.github.com/users/ondrejhlavacek","html_url":"https://github.com/ondrejhlavacek","followers_url":"https://api.github.com/users/ondrejhlavacek/followers","following_url":"https://api.github.com/users/ondrejhlavacek/following{/other_user}","gists_url":"https://api.github.com/users/ondrejhlavacek/gists{/gist_id}","starred_url":"https://api.github.com/users/ondrejhlavacek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ondrejhlavacek/subscriptions","organizations_url":"https://api.github.com/users/ondrejhlavacek/orgs","repos_url":"https://api.github.com/users/ondrejhlavacek/repos","events_url":"https://api.github.com/users/ondrejhlavacek/events{/privacy}","received_events_url":"https://api.github.com/users/ondrejhlavacek/received_events","type":"User","site_admin":false},"body":"To souhlas, pěkný to není, ale nechci začít refactorovat nesouvisející kód, tak bych to odložil. ","created_at":"2018-12-14T11:01:41Z","updated_at":"2018-12-14T11:01:41Z","html_url":"https://github.com/keboola/transformation-bundle/pull/129#discussion_r241717852","pull_request_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129","author_association":"MEMBER","_links":{"self":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/comments/241717852"},"html":{"href":"https://github.com/keboola/transformation-bundle/pull/129#discussion_r241717852"},"pull_request":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129"}},"in_reply_to_id":241688331},"pull_request":{"url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129","id":238010059,"node_id":"MDExOlB1bGxSZXF1ZXN0MjM4MDEwMDU5","html_url":"https://github.com/keboola/transformation-bundle/pull/129","diff_url":"https://github.com/keboola/transformation-bundle/pull/129.diff","patch_url":"https://github.com/keboola/transformation-bundle/pull/129.patch","issue_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/129","number":129,"state":"open","locked":false,"title":"feat: parallel unloads","user":{"login":"ondrejhlavacek","id":497675,"node_id":"MDQ6VXNlcjQ5NzY3NQ==","avatar_url":"https://avatars1.githubusercontent.com/u/497675?v=4","gravatar_id":"","url":"https://api.github.com/users/ondrejhlavacek","html_url":"https://github.com/ondrejhlavacek","followers_url":"https://api.github.com/users/ondrejhlavacek/followers","following_url":"https://api.github.com/users/ondrejhlavacek/following{/other_user}","gists_url":"https://api.github.com/users/ondrejhlavacek/gists{/gist_id}","starred_url":"https://api.github.com/users/ondrejhlavacek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ondrejhlavacek/subscriptions","organizations_url":"https://api.github.com/users/ondrejhlavacek/orgs","repos_url":"https://api.github.com/users/ondrejhlavacek/repos","events_url":"https://api.github.com/users/ondrejhlavacek/events{/privacy}","received_events_url":"https://api.github.com/users/ondrejhlavacek/received_events","type":"User","site_admin":false},"body":"- U Snowflake transformací se místo `writeTableAsyncDirect` použije `queueTableImport`, a to jen pro loady do existujících tabulek. \r\n- Z transformací probublají nahoru idčka jobů, kde se na ně pak počká. \r\n- Zduplikoval jsem odchytávání chyb, ale chybové zprávy budou obecnější. Při předávání idček jobů se ztrácí vazba na konkrétní output mapping i na konkrétní transformaci. Možná by to šlo zachovat, ale vyžádalo by si to trochu víc kódu nebo refactoring.","created_at":"2018-12-12T11:53:06Z","updated_at":"2018-12-14T11:01:41Z","closed_at":null,"merged_at":null,"merge_commit_sha":"1a184c55edcd85163d08b5d99a91bdec2d4e319e","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129/commits","review_comments_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129/comments","review_comment_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls/comments{/number}","comments_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/129/comments","statuses_url":"https://api.github.com/repos/keboola/transformation-bundle/statuses/bb2c36b989d82bc3dfee3f165f2d65c5ac157328","head":{"label":"keboola:najlos-parallel-unloads","ref":"najlos-parallel-unloads","sha":"bb2c36b989d82bc3dfee3f165f2d65c5ac157328","user":{"login":"keboola","id":1424387,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE0MjQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/1424387?v=4","gravatar_id":"","url":"https://api.github.com/users/keboola","html_url":"https://github.com/keboola","followers_url":"https://api.github.com/users/keboola/followers","following_url":"https://api.github.com/users/keboola/following{/other_user}","gists_url":"https://api.github.com/users/keboola/gists{/gist_id}","starred_url":"https://api.github.com/users/keboola/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keboola/subscriptions","organizations_url":"https://api.github.com/users/keboola/orgs","repos_url":"https://api.github.com/users/keboola/repos","events_url":"https://api.github.com/users/keboola/events{/privacy}","received_events_url":"https://api.github.com/users/keboola/received_events","type":"Organization","site_admin":false},"repo":{"id":27892267,"node_id":"MDEwOlJlcG9zaXRvcnkyNzg5MjI2Nw==","name":"transformation-bundle","full_name":"keboola/transformation-bundle","private":false,"owner":{"login":"keboola","id":1424387,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE0MjQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/1424387?v=4","gravatar_id":"","url":"https://api.github.com/users/keboola","html_url":"https://github.com/keboola","followers_url":"https://api.github.com/users/keboola/followers","following_url":"https://api.github.com/users/keboola/following{/other_user}","gists_url":"https://api.github.com/users/keboola/gists{/gist_id}","starred_url":"https://api.github.com/users/keboola/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keboola/subscriptions","organizations_url":"https://api.github.com/users/keboola/orgs","repos_url":"https://api.github.com/users/keboola/repos","events_url":"https://api.github.com/users/keboola/events{/privacy}","received_events_url":"https://api.github.com/users/keboola/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/keboola/transformation-bundle","description":null,"fork":false,"url":"https://api.github.com/repos/keboola/transformation-bundle","forks_url":"https://api.github.com/repos/keboola/transformation-bundle/forks","keys_url":"https://api.github.com/repos/keboola/transformation-bundle/keys{/key_id}","collaborators_url":"https://api.github.com/repos/keboola/transformation-bundle/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/keboola/transformation-bundle/teams","hooks_url":"https://api.github.com/repos/keboola/transformation-bundle/hooks","issue_events_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/events{/number}","events_url":"https://api.github.com/repos/keboola/transformation-bundle/events","assignees_url":"https://api.github.com/repos/keboola/transformation-bundle/assignees{/user}","branches_url":"https://api.github.com/repos/keboola/transformation-bundle/branches{/branch}","tags_url":"https://api.github.com/repos/keboola/transformation-bundle/tags","blobs_url":"https://api.github.com/repos/keboola/transformation-bundle/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/keboola/transformation-bundle/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/keboola/transformation-bundle/git/refs{/sha}","trees_url":"https://api.github.com/repos/keboola/transformation-bundle/git/trees{/sha}","statuses_url":"https://api.github.com/repos/keboola/transformation-bundle/statuses/{sha}","languages_url":"https://api.github.com/repos/keboola/transformation-bundle/languages","stargazers_url":"https://api.github.com/repos/keboola/transformation-bundle/stargazers","contributors_url":"https://api.github.com/repos/keboola/transformation-bundle/contributors","subscribers_url":"https://api.github.com/repos/keboola/transformation-bundle/subscribers","subscription_url":"https://api.github.com/repos/keboola/transformation-bundle/subscription","commits_url":"https://api.github.com/repos/keboola/transformation-bundle/commits{/sha}","git_commits_url":"https://api.github.com/repos/keboola/transformation-bundle/git/commits{/sha}","comments_url":"https://api.github.com/repos/keboola/transformation-bundle/comments{/number}","issue_comment_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/comments{/number}","contents_url":"https://api.github.com/repos/keboola/transformation-bundle/contents/{+path}","compare_url":"https://api.github.com/repos/keboola/transformation-bundle/compare/{base}...{head}","merges_url":"https://api.github.com/repos/keboola/transformation-bundle/merges","archive_url":"https://api.github.com/repos/keboola/transformation-bundle/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/keboola/transformation-bundle/downloads","issues_url":"https://api.github.com/repos/keboola/transformation-bundle/issues{/number}","pulls_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls{/number}","milestones_url":"https://api.github.com/repos/keboola/transformation-bundle/milestones{/number}","notifications_url":"https://api.github.com/repos/keboola/transformation-bundle/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/keboola/transformation-bundle/labels{/name}","releases_url":"https://api.github.com/repos/keboola/transformation-bundle/releases{/id}","deployments_url":"https://api.github.com/repos/keboola/transformation-bundle/deployments","created_at":"2014-12-11T21:23:09Z","updated_at":"2018-12-12T11:23:57Z","pushed_at":"2018-12-13T12:01:44Z","git_url":"git://github.com/keboola/transformation-bundle.git","ssh_url":"git@github.com:keboola/transformation-bundle.git","clone_url":"https://github.com/keboola/transformation-bundle.git","svn_url":"https://github.com/keboola/transformation-bundle","homepage":null,"size":6190,"stargazers_count":0,"watchers_count":0,"language":"PHP","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":30,"license":null,"forks":0,"open_issues":30,"watchers":0,"default_branch":"master"}},"base":{"label":"keboola:master","ref":"master","sha":"1616c405a6b4cb03d7dd0588f9a493897c9cb763","user":{"login":"keboola","id":1424387,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE0MjQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/1424387?v=4","gravatar_id":"","url":"https://api.github.com/users/keboola","html_url":"https://github.com/keboola","followers_url":"https://api.github.com/users/keboola/followers","following_url":"https://api.github.com/users/keboola/following{/other_user}","gists_url":"https://api.github.com/users/keboola/gists{/gist_id}","starred_url":"https://api.github.com/users/keboola/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keboola/subscriptions","organizations_url":"https://api.github.com/users/keboola/orgs","repos_url":"https://api.github.com/users/keboola/repos","events_url":"https://api.github.com/users/keboola/events{/privacy}","received_events_url":"https://api.github.com/users/keboola/received_events","type":"Organization","site_admin":false},"repo":{"id":27892267,"node_id":"MDEwOlJlcG9zaXRvcnkyNzg5MjI2Nw==","name":"transformation-bundle","full_name":"keboola/transformation-bundle","private":false,"owner":{"login":"keboola","id":1424387,"node_id":"MDEyOk9yZ2FuaXphdGlvbjE0MjQzODc=","avatar_url":"https://avatars1.githubusercontent.com/u/1424387?v=4","gravatar_id":"","url":"https://api.github.com/users/keboola","html_url":"https://github.com/keboola","followers_url":"https://api.github.com/users/keboola/followers","following_url":"https://api.github.com/users/keboola/following{/other_user}","gists_url":"https://api.github.com/users/keboola/gists{/gist_id}","starred_url":"https://api.github.com/users/keboola/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keboola/subscriptions","organizations_url":"https://api.github.com/users/keboola/orgs","repos_url":"https://api.github.com/users/keboola/repos","events_url":"https://api.github.com/users/keboola/events{/privacy}","received_events_url":"https://api.github.com/users/keboola/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/keboola/transformation-bundle","description":null,"fork":false,"url":"https://api.github.com/repos/keboola/transformation-bundle","forks_url":"https://api.github.com/repos/keboola/transformation-bundle/forks","keys_url":"https://api.github.com/repos/keboola/transformation-bundle/keys{/key_id}","collaborators_url":"https://api.github.com/repos/keboola/transformation-bundle/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/keboola/transformation-bundle/teams","hooks_url":"https://api.github.com/repos/keboola/transformation-bundle/hooks","issue_events_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/events{/number}","events_url":"https://api.github.com/repos/keboola/transformation-bundle/events","assignees_url":"https://api.github.com/repos/keboola/transformation-bundle/assignees{/user}","branches_url":"https://api.github.com/repos/keboola/transformation-bundle/branches{/branch}","tags_url":"https://api.github.com/repos/keboola/transformation-bundle/tags","blobs_url":"https://api.github.com/repos/keboola/transformation-bundle/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/keboola/transformation-bundle/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/keboola/transformation-bundle/git/refs{/sha}","trees_url":"https://api.github.com/repos/keboola/transformation-bundle/git/trees{/sha}","statuses_url":"https://api.github.com/repos/keboola/transformation-bundle/statuses/{sha}","languages_url":"https://api.github.com/repos/keboola/transformation-bundle/languages","stargazers_url":"https://api.github.com/repos/keboola/transformation-bundle/stargazers","contributors_url":"https://api.github.com/repos/keboola/transformation-bundle/contributors","subscribers_url":"https://api.github.com/repos/keboola/transformation-bundle/subscribers","subscription_url":"https://api.github.com/repos/keboola/transformation-bundle/subscription","commits_url":"https://api.github.com/repos/keboola/transformation-bundle/commits{/sha}","git_commits_url":"https://api.github.com/repos/keboola/transformation-bundle/git/commits{/sha}","comments_url":"https://api.github.com/repos/keboola/transformation-bundle/comments{/number}","issue_comment_url":"https://api.github.com/repos/keboola/transformation-bundle/issues/comments{/number}","contents_url":"https://api.github.com/repos/keboola/transformation-bundle/contents/{+path}","compare_url":"https://api.github.com/repos/keboola/transformation-bundle/compare/{base}...{head}","merges_url":"https://api.github.com/repos/keboola/transformation-bundle/merges","archive_url":"https://api.github.com/repos/keboola/transformation-bundle/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/keboola/transformation-bundle/downloads","issues_url":"https://api.github.com/repos/keboola/transformation-bundle/issues{/number}","pulls_url":"https://api.github.com/repos/keboola/transformation-bundle/pulls{/number}","milestones_url":"https://api.github.com/repos/keboola/transformation-bundle/milestones{/number}","notifications_url":"https://api.github.com/repos/keboola/transformation-bundle/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/keboola/transformation-bundle/labels{/name}","releases_url":"https://api.github.com/repos/keboola/transformation-bundle/releases{/id}","deployments_url":"https://api.github.com/repos/keboola/transformation-bundle/deployments","created_at":"2014-12-11T21:23:09Z","updated_at":"2018-12-12T11:23:57Z","pushed_at":"2018-12-13T12:01:44Z","git_url":"git://github.com/keboola/transformation-bundle.git","ssh_url":"git@github.com:keboola/transformation-bundle.git","clone_url":"https://github.com/keboola/transformation-bundle.git","svn_url":"https://github.com/keboola/transformation-bundle","homepage":null,"size":6190,"stargazers_count":0,"watchers_count":0,"language":"PHP","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":30,"license":null,"forks":0,"open_issues":30,"watchers":0,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129"},"html":{"href":"https://github.com/keboola/transformation-bundle/pull/129"},"issue":{"href":"https://api.github.com/repos/keboola/transformation-bundle/issues/129"},"comments":{"href":"https://api.github.com/repos/keboola/transformation-bundle/issues/129/comments"},"review_comments":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129/comments"},"review_comment":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/keboola/transformation-bundle/pulls/129/commits"},"statuses":{"href":"https://api.github.com/repos/keboola/transformation-bundle/statuses/bb2c36b989d82bc3dfee3f165f2d65c5ac157328"}},"author_association":"MEMBER"}}
{"actor":{"display_login":"ondrejhlavacek"}}