{"action":"created","comment":{"url":"https://api.github.com/repos/spotify/styx/pulls/comments/232517581","pull_request_review_id":173730174,"id":232517581,"node_id":"MDI0OlB1bGxSZXF1ZXN0UmV2aWV3Q29tbWVudDIzMjUxNzU4MQ==","diff_hunk":"@@ -736,25 +736,48 @@ void postResource(Resource resource) throws IOException {\n     storeWithRetries(() -> runInTransaction(transaction -> {\n       transaction.store(resource);\n       return null;\n-      // TODO store just in one place, eliminate one of the two calls ^?\n     }));\n   }\n \n   List<Resource> getResources() throws IOException {\n-    final EntityQuery query = Query.newEntityQueryBuilder().setKind(KIND_RESOURCE).build();\n+    final EntityQuery query = Query.newEntityQueryBuilder().setKind(KIND_COUNTER_LIMIT).build();\n     final List<Resource> resources = Lists.newArrayList();\n     datastore.query(query, entity ->\n         resources.add(entityToResource(entity)));\n     return resources;\n   }\n \n-  private Resource entityToResource(Entity entity) {\n-    return Resource.create(entity.getKey().getName(), entity.getLong(PROPERTY_CONCURRENCY));\n+  /**\n+   * Delete resource by id. Deletes both counter shards and counter limit if it exists.\n+   *\n+   * <p>Due to Datastore limitations (modify max 25 entity groups per transaction),\n+   * deletion of shards is done in batches of 25 shards.\n+   */\n+  void deleteResource(String id) throws IOException {\n+    storeWithRetries(() -> runInTransaction(tx -> {\n+      tx.deleteCounterLimit(id);","path":"styx-common/src/main/java/com/spotify/styx/storage/DatastoreStorage.java","position":52,"original_position":52,"commit_id":"665dd91fdead7a468619299ad9b163636ac627be","original_commit_id":"665dd91fdead7a468619299ad9b163636ac627be","user":{"login":"danielnorberg","id":393289,"node_id":"MDQ6VXNlcjM5MzI4OQ==","avatar_url":"https://avatars1.githubusercontent.com/u/393289?v=4","gravatar_id":"","url":"https://api.github.com/users/danielnorberg","html_url":"https://github.com/danielnorberg","followers_url":"https://api.github.com/users/danielnorberg/followers","following_url":"https://api.github.com/users/danielnorberg/following{/other_user}","gists_url":"https://api.github.com/users/danielnorberg/gists{/gist_id}","starred_url":"https://api.github.com/users/danielnorberg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danielnorberg/subscriptions","organizations_url":"https://api.github.com/users/danielnorberg/orgs","repos_url":"https://api.github.com/users/danielnorberg/repos","events_url":"https://api.github.com/users/danielnorberg/events{/privacy}","received_events_url":"https://api.github.com/users/danielnorberg/received_events","type":"User","site_admin":false},"body":"No need for transaction here, a single delete is atomic anyway.","created_at":"2018-11-12T01:44:34Z","updated_at":"2018-11-12T01:44:34Z","html_url":"https://github.com/spotify/styx/pull/621#discussion_r232517581","pull_request_url":"https://api.github.com/repos/spotify/styx/pulls/621","author_association":"CONTRIBUTOR","_links":{"self":{"href":"https://api.github.com/repos/spotify/styx/pulls/comments/232517581"},"html":{"href":"https://github.com/spotify/styx/pull/621#discussion_r232517581"},"pull_request":{"href":"https://api.github.com/repos/spotify/styx/pulls/621"}}},"pull_request":{"url":"https://api.github.com/repos/spotify/styx/pulls/621","id":229093917,"node_id":"MDExOlB1bGxSZXF1ZXN0MjI5MDkzOTE3","html_url":"https://github.com/spotify/styx/pull/621","diff_url":"https://github.com/spotify/styx/pull/621.diff","patch_url":"https://github.com/spotify/styx/pull/621.patch","issue_url":"https://api.github.com/repos/spotify/styx/issues/621","number":621,"state":"open","locked":false,"title":"remove resource kind","user":{"login":"honnix","id":158892,"node_id":"MDQ6VXNlcjE1ODg5Mg==","avatar_url":"https://avatars0.githubusercontent.com/u/158892?v=4","gravatar_id":"","url":"https://api.github.com/users/honnix","html_url":"https://github.com/honnix","followers_url":"https://api.github.com/users/honnix/followers","following_url":"https://api.github.com/users/honnix/following{/other_user}","gists_url":"https://api.github.com/users/honnix/gists{/gist_id}","starred_url":"https://api.github.com/users/honnix/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/honnix/subscriptions","organizations_url":"https://api.github.com/users/honnix/orgs","repos_url":"https://api.github.com/users/honnix/repos","events_url":"https://api.github.com/users/honnix/events{/privacy}","received_events_url":"https://api.github.com/users/honnix/received_events","type":"User","site_admin":false},"body":"# Hey, I just made a Pull Request!\r\n\r\n## Description\r\n<!--- Describe your changes -->\r\nRework of #489\r\n\r\nRemove `Resource` kind because it is not used by scheduler.\r\n\r\nSo the idea (or the way I choose to convince myself regarding resource vs. counter) is, resource is the concept we expose to users while counter is only referred internally.\r\n\r\n## Motivation and Context\r\n<!--- Why is this change required? What problem does it solve? -->\r\n<!--- If it fixes an open issue, please link to the issue here. -->\r\n\r\n`Resource` kind is not used for controlling resource usage anymore.\r\n\r\n## Have you tested this? If so, how?\r\n<!--- Valid responses are \"I have included unit tests.\" or --> \r\n<!--- \"I ran my jobs with this code and it works for me.\" -->\r\n\r\nUnit test\r\n\r\n## Checklist for PR author(s)\r\n<!--- Put an `x` in all the boxes that apply: -->\r\n- [x] Changes are covered by unit test\r\n- [x] All tests pass\r\n- [x] Code coverage check passes\r\n- [x] Error handling is tested\r\n- [x] Errors are handled at the appropriate layer\r\n- [x] Errors that cannot be handled where they occur are propagated\r\n- [x] (optional) Changes are covered by system test\r\n- [ ] Relevant documentation updated\r\n- [x] This PR has NO breaking change to public API\r\n- [ ] This PR has breaking change to public API and it is documented\r\n\r\n## Checklist for PR reviewer(s)\r\n<!--- Put an `x` in all the boxes that apply: -->\r\n- [ ] This PR has been incorporated in release note for the coming version\r\n- [ ] Risky changes introduced by this PR have been all considered\r\n\r\n<!---\r\nfor more information on how to submit valuable contributions,\r\nsee https://opensource.guide/how-to-contribute/#how-to-submit-a-contribution\r\n-->\r\n","created_at":"2018-11-07T16:43:22Z","updated_at":"2018-11-12T01:44:34Z","closed_at":null,"merged_at":null,"merge_commit_sha":"1b0329764814d6de6370773c328d2ac4d51c35fc","assignee":null,"assignees":[],"requested_reviewers":[{"login":"NatashaL","id":4315174,"node_id":"MDQ6VXNlcjQzMTUxNzQ=","avatar_url":"https://avatars1.githubusercontent.com/u/4315174?v=4","gravatar_id":"","url":"https://api.github.com/users/NatashaL","html_url":"https://github.com/NatashaL","followers_url":"https://api.github.com/users/NatashaL/followers","following_url":"https://api.github.com/users/NatashaL/following{/other_user}","gists_url":"https://api.github.com/users/NatashaL/gists{/gist_id}","starred_url":"https://api.github.com/users/NatashaL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/NatashaL/subscriptions","organizations_url":"https://api.github.com/users/NatashaL/orgs","repos_url":"https://api.github.com/users/NatashaL/repos","events_url":"https://api.github.com/users/NatashaL/events{/privacy}","received_events_url":"https://api.github.com/users/NatashaL/received_events","type":"User","site_admin":false}],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/spotify/styx/pulls/621/commits","review_comments_url":"https://api.github.com/repos/spotify/styx/pulls/621/comments","review_comment_url":"https://api.github.com/repos/spotify/styx/pulls/comments{/number}","comments_url":"https://api.github.com/repos/spotify/styx/issues/621/comments","statuses_url":"https://api.github.com/repos/spotify/styx/statuses/665dd91fdead7a468619299ad9b163636ac627be","head":{"label":"spotify:no-resource-kind","ref":"no-resource-kind","sha":"665dd91fdead7a468619299ad9b163636ac627be","user":{"login":"spotify","id":251374,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTM3NA==","avatar_url":"https://avatars2.githubusercontent.com/u/251374?v=4","gravatar_id":"","url":"https://api.github.com/users/spotify","html_url":"https://github.com/spotify","followers_url":"https://api.github.com/users/spotify/followers","following_url":"https://api.github.com/users/spotify/following{/other_user}","gists_url":"https://api.github.com/users/spotify/gists{/gist_id}","starred_url":"https://api.github.com/users/spotify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spotify/subscriptions","organizations_url":"https://api.github.com/users/spotify/orgs","repos_url":"https://api.github.com/users/spotify/repos","events_url":"https://api.github.com/users/spotify/events{/privacy}","received_events_url":"https://api.github.com/users/spotify/received_events","type":"Organization","site_admin":false},"repo":{"id":74431644,"node_id":"MDEwOlJlcG9zaXRvcnk3NDQzMTY0NA==","name":"styx","full_name":"spotify/styx","private":false,"owner":{"login":"spotify","id":251374,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTM3NA==","avatar_url":"https://avatars2.githubusercontent.com/u/251374?v=4","gravatar_id":"","url":"https://api.github.com/users/spotify","html_url":"https://github.com/spotify","followers_url":"https://api.github.com/users/spotify/followers","following_url":"https://api.github.com/users/spotify/following{/other_user}","gists_url":"https://api.github.com/users/spotify/gists{/gist_id}","starred_url":"https://api.github.com/users/spotify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spotify/subscriptions","organizations_url":"https://api.github.com/users/spotify/orgs","repos_url":"https://api.github.com/users/spotify/repos","events_url":"https://api.github.com/users/spotify/events{/privacy}","received_events_url":"https://api.github.com/users/spotify/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/spotify/styx","description":"\"The path to execution\", Styx is a service that schedules batch data processing jobs in Docker containers on Kubernetes.","fork":false,"url":"https://api.github.com/repos/spotify/styx","forks_url":"https://api.github.com/repos/spotify/styx/forks","keys_url":"https://api.github.com/repos/spotify/styx/keys{/key_id}","collaborators_url":"https://api.github.com/repos/spotify/styx/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/spotify/styx/teams","hooks_url":"https://api.github.com/repos/spotify/styx/hooks","issue_events_url":"https://api.github.com/repos/spotify/styx/issues/events{/number}","events_url":"https://api.github.com/repos/spotify/styx/events","assignees_url":"https://api.github.com/repos/spotify/styx/assignees{/user}","branches_url":"https://api.github.com/repos/spotify/styx/branches{/branch}","tags_url":"https://api.github.com/repos/spotify/styx/tags","blobs_url":"https://api.github.com/repos/spotify/styx/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/spotify/styx/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/spotify/styx/git/refs{/sha}","trees_url":"https://api.github.com/repos/spotify/styx/git/trees{/sha}","statuses_url":"https://api.github.com/repos/spotify/styx/statuses/{sha}","languages_url":"https://api.github.com/repos/spotify/styx/languages","stargazers_url":"https://api.github.com/repos/spotify/styx/stargazers","contributors_url":"https://api.github.com/repos/spotify/styx/contributors","subscribers_url":"https://api.github.com/repos/spotify/styx/subscribers","subscription_url":"https://api.github.com/repos/spotify/styx/subscription","commits_url":"https://api.github.com/repos/spotify/styx/commits{/sha}","git_commits_url":"https://api.github.com/repos/spotify/styx/git/commits{/sha}","comments_url":"https://api.github.com/repos/spotify/styx/comments{/number}","issue_comment_url":"https://api.github.com/repos/spotify/styx/issues/comments{/number}","contents_url":"https://api.github.com/repos/spotify/styx/contents/{+path}","compare_url":"https://api.github.com/repos/spotify/styx/compare/{base}...{head}","merges_url":"https://api.github.com/repos/spotify/styx/merges","archive_url":"https://api.github.com/repos/spotify/styx/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/spotify/styx/downloads","issues_url":"https://api.github.com/repos/spotify/styx/issues{/number}","pulls_url":"https://api.github.com/repos/spotify/styx/pulls{/number}","milestones_url":"https://api.github.com/repos/spotify/styx/milestones{/number}","notifications_url":"https://api.github.com/repos/spotify/styx/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/spotify/styx/labels{/name}","releases_url":"https://api.github.com/repos/spotify/styx/releases{/id}","deployments_url":"https://api.github.com/repos/spotify/styx/deployments","created_at":"2016-11-22T03:44:55Z","updated_at":"2018-11-06T08:09:59Z","pushed_at":"2018-11-08T09:34:44Z","git_url":"git://github.com/spotify/styx.git","ssh_url":"git@github.com:spotify/styx.git","clone_url":"https://github.com/spotify/styx.git","svn_url":"https://github.com/spotify/styx","homepage":"","size":6247,"stargazers_count":118,"watchers_count":118,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":21,"mirror_url":null,"archived":false,"open_issues_count":20,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":21,"open_issues":20,"watchers":118,"default_branch":"master"}},"base":{"label":"spotify:master","ref":"master","sha":"34e1bfb05590b215d0a6d4a72e734538f138747d","user":{"login":"spotify","id":251374,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTM3NA==","avatar_url":"https://avatars2.githubusercontent.com/u/251374?v=4","gravatar_id":"","url":"https://api.github.com/users/spotify","html_url":"https://github.com/spotify","followers_url":"https://api.github.com/users/spotify/followers","following_url":"https://api.github.com/users/spotify/following{/other_user}","gists_url":"https://api.github.com/users/spotify/gists{/gist_id}","starred_url":"https://api.github.com/users/spotify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spotify/subscriptions","organizations_url":"https://api.github.com/users/spotify/orgs","repos_url":"https://api.github.com/users/spotify/repos","events_url":"https://api.github.com/users/spotify/events{/privacy}","received_events_url":"https://api.github.com/users/spotify/received_events","type":"Organization","site_admin":false},"repo":{"id":74431644,"node_id":"MDEwOlJlcG9zaXRvcnk3NDQzMTY0NA==","name":"styx","full_name":"spotify/styx","private":false,"owner":{"login":"spotify","id":251374,"node_id":"MDEyOk9yZ2FuaXphdGlvbjI1MTM3NA==","avatar_url":"https://avatars2.githubusercontent.com/u/251374?v=4","gravatar_id":"","url":"https://api.github.com/users/spotify","html_url":"https://github.com/spotify","followers_url":"https://api.github.com/users/spotify/followers","following_url":"https://api.github.com/users/spotify/following{/other_user}","gists_url":"https://api.github.com/users/spotify/gists{/gist_id}","starred_url":"https://api.github.com/users/spotify/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/spotify/subscriptions","organizations_url":"https://api.github.com/users/spotify/orgs","repos_url":"https://api.github.com/users/spotify/repos","events_url":"https://api.github.com/users/spotify/events{/privacy}","received_events_url":"https://api.github.com/users/spotify/received_events","type":"Organization","site_admin":false},"html_url":"https://github.com/spotify/styx","description":"\"The path to execution\", Styx is a service that schedules batch data processing jobs in Docker containers on Kubernetes.","fork":false,"url":"https://api.github.com/repos/spotify/styx","forks_url":"https://api.github.com/repos/spotify/styx/forks","keys_url":"https://api.github.com/repos/spotify/styx/keys{/key_id}","collaborators_url":"https://api.github.com/repos/spotify/styx/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/spotify/styx/teams","hooks_url":"https://api.github.com/repos/spotify/styx/hooks","issue_events_url":"https://api.github.com/repos/spotify/styx/issues/events{/number}","events_url":"https://api.github.com/repos/spotify/styx/events","assignees_url":"https://api.github.com/repos/spotify/styx/assignees{/user}","branches_url":"https://api.github.com/repos/spotify/styx/branches{/branch}","tags_url":"https://api.github.com/repos/spotify/styx/tags","blobs_url":"https://api.github.com/repos/spotify/styx/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/spotify/styx/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/spotify/styx/git/refs{/sha}","trees_url":"https://api.github.com/repos/spotify/styx/git/trees{/sha}","statuses_url":"https://api.github.com/repos/spotify/styx/statuses/{sha}","languages_url":"https://api.github.com/repos/spotify/styx/languages","stargazers_url":"https://api.github.com/repos/spotify/styx/stargazers","contributors_url":"https://api.github.com/repos/spotify/styx/contributors","subscribers_url":"https://api.github.com/repos/spotify/styx/subscribers","subscription_url":"https://api.github.com/repos/spotify/styx/subscription","commits_url":"https://api.github.com/repos/spotify/styx/commits{/sha}","git_commits_url":"https://api.github.com/repos/spotify/styx/git/commits{/sha}","comments_url":"https://api.github.com/repos/spotify/styx/comments{/number}","issue_comment_url":"https://api.github.com/repos/spotify/styx/issues/comments{/number}","contents_url":"https://api.github.com/repos/spotify/styx/contents/{+path}","compare_url":"https://api.github.com/repos/spotify/styx/compare/{base}...{head}","merges_url":"https://api.github.com/repos/spotify/styx/merges","archive_url":"https://api.github.com/repos/spotify/styx/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/spotify/styx/downloads","issues_url":"https://api.github.com/repos/spotify/styx/issues{/number}","pulls_url":"https://api.github.com/repos/spotify/styx/pulls{/number}","milestones_url":"https://api.github.com/repos/spotify/styx/milestones{/number}","notifications_url":"https://api.github.com/repos/spotify/styx/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/spotify/styx/labels{/name}","releases_url":"https://api.github.com/repos/spotify/styx/releases{/id}","deployments_url":"https://api.github.com/repos/spotify/styx/deployments","created_at":"2016-11-22T03:44:55Z","updated_at":"2018-11-06T08:09:59Z","pushed_at":"2018-11-08T09:34:44Z","git_url":"git://github.com/spotify/styx.git","ssh_url":"git@github.com:spotify/styx.git","clone_url":"https://github.com/spotify/styx.git","svn_url":"https://github.com/spotify/styx","homepage":"","size":6247,"stargazers_count":118,"watchers_count":118,"language":"Java","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":true,"forks_count":21,"mirror_url":null,"archived":false,"open_issues_count":20,"license":{"key":"apache-2.0","name":"Apache License 2.0","spdx_id":"Apache-2.0","url":"https://api.github.com/licenses/apache-2.0","node_id":"MDc6TGljZW5zZTI="},"forks":21,"open_issues":20,"watchers":118,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/spotify/styx/pulls/621"},"html":{"href":"https://github.com/spotify/styx/pull/621"},"issue":{"href":"https://api.github.com/repos/spotify/styx/issues/621"},"comments":{"href":"https://api.github.com/repos/spotify/styx/issues/621/comments"},"review_comments":{"href":"https://api.github.com/repos/spotify/styx/pulls/621/comments"},"review_comment":{"href":"https://api.github.com/repos/spotify/styx/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/spotify/styx/pulls/621/commits"},"statuses":{"href":"https://api.github.com/repos/spotify/styx/statuses/665dd91fdead7a468619299ad9b163636ac627be"}},"author_association":"CONTRIBUTOR"}}
{"actor":{"display_login":"danielnorberg"}}