{"action":"created","comment":{"url":"https://api.github.com/repos/eclipse/che/pulls/comments/64425409","id":64425409,"diff_hunk":"@@ -928,6 +939,50 @@ public void testDeleteProject() throws Exception {\n     }\n \n     @Test\n+    public void testDeleteProjectsConcurrently() throws Exception {\n+        int threadNumber = 100;\n+        ExecutorService executor = Executors.newFixedThreadPool(threadNumber);\n+        CountDownLatch countDownLatch = new CountDownLatch(threadNumber);\n+        List<Future<ContainerResponse>> futures = new LinkedList<>();\n+\n+\n+        for (int i = 0; i < threadNumber; i++) {\n+            addMockedProjectConfigDto(ptRegistry.getProjectType(\"my_project_type\"), \"my_project_name\" + i);\n+        }\n+\n+        IntStream.range(0, threadNumber).forEach(","path":"wsagent/che-core-api-project/src/test/java/org/eclipse/che/api/project/server/ProjectServiceTest.java","position":104,"original_position":104,"commit_id":"4088de7e58045e3a0d50d22d23ed601cc80a7f86","original_commit_id":"4088de7e58045e3a0d50d22d23ed601cc80a7f86","user":{"login":"benoitf","id":436777,"avatar_url":"https://avatars.githubusercontent.com/u/436777?v=3","gravatar_id":"","url":"https://api.github.com/users/benoitf","html_url":"https://github.com/benoitf","followers_url":"https://api.github.com/users/benoitf/followers","following_url":"https://api.github.com/users/benoitf/following{/other_user}","gists_url":"https://api.github.com/users/benoitf/gists{/gist_id}","starred_url":"https://api.github.com/users/benoitf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/benoitf/subscriptions","organizations_url":"https://api.github.com/users/benoitf/orgs","repos_url":"https://api.github.com/users/benoitf/repos","events_url":"https://api.github.com/users/benoitf/events{/privacy}","received_events_url":"https://api.github.com/users/benoitf/received_events","type":"User","site_admin":false},"body":"my concern was only to unify the two loops into a single loop. (because we iterate on the same numbers but in a different way)\r\nand yes to avoid hacks like int j = i; you've to use IntStream","created_at":"2016-05-24T16:23:20Z","updated_at":"2016-05-24T16:23:20Z","html_url":"https://github.com/eclipse/che/pull/1343#discussion_r64425409","pull_request_url":"https://api.github.com/repos/eclipse/che/pulls/1343","_links":{"self":{"href":"https://api.github.com/repos/eclipse/che/pulls/comments/64425409"},"html":{"href":"https://github.com/eclipse/che/pull/1343#discussion_r64425409"},"pull_request":{"href":"https://api.github.com/repos/eclipse/che/pulls/1343"}}},"pull_request":{"url":"https://api.github.com/repos/eclipse/che/pulls/1343","id":71206830,"html_url":"https://github.com/eclipse/che/pull/1343","diff_url":"https://github.com/eclipse/che/pull/1343.diff","patch_url":"https://github.com/eclipse/che/pull/1343.patch","issue_url":"https://api.github.com/repos/eclipse/che/issues/1343","number":1343,"state":"open","locked":false,"title":"CHE-1143: fixed synchronization issue during removal of several projects","user":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"body":"Hi,\r\n\r\nRecently I was working on some bugs in Eclipse Che project management infrastructure. The report itself looks like:\r\n\r\nWhen we delete several projects simultaneously (in the same running workspace with project api):\r\n\r\n1. We get `204` status on each request however projects are not deleted\r\n2. In some cases we get `500` status combined with `java.util.ConcurrentModificationException`\r\n\r\nThe first point is a separate question and there are several ideological problems that should be solved first. As far as I know, the discussion is not yet complete.\r\n\r\nAnd for the second item I created a dedicated test to cover the bug. Though the bug is not reproduced anymore most likely thanks to [this commit](https://github.com/eclipse/che/commit/658410556f8d0d5c91316890111ff7dc9d57cba1#diff-9414b806e2941cc99a2d071cd5b982fc), I've spotted another problem. We rarely get exception:  \r\n```\r\njava.lang.AssertionError: Error: Comparison method violates its general contract! expected [204] but found [500]\r\n        at org.testng.Assert.fail(Assert.java:94)\r\n        at org.testng.Assert.failNotEquals(Assert.java:494)\r\n        at org.testng.Assert.assertEquals(Assert.java:123)\r\n        at org.testng.Assert.assertEquals(Assert.java:370)\r\n        at org.eclipse.che.api.project.server.ProjectServiceTest.testDeleteProjectsConcurrently(ProjectServiceTest.java:981)\r\n```\r\nwhen we try to simultaneously remove 100 projects (in 100 threads), while it is always reproduced for 1k+ threads.\r\n\r\nI assume that the problem is the following. At some point of project deletion we are to get its filesystem parent's sorted children list, the process is a set of two operations: get and sort the list. These operations are not synchronized so sometimes it happens to change somehow state of children collections by another thread. This results in a following exception: \r\n```\r\njava.lang.IllegalArgumentException: Comparison method violates its general contract!\r\n    at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:835)\r\n    at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:453)\r\n    at java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:392)\r\n    at java.util.ComparableTimSort.sort(ComparableTimSort.java:191)\r\n    at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)\r\n    at java.util.Arrays.sort(Arrays.java:472)\r\n    at java.util.Collections.sort(Collections.java:155)\r\n```\r\nThat basically happens when arrays of children that are to be merged are indeed not sorted (because of impact of the other thread: there are changes of some elements' state or elements themselves after they are already sorted) or elements are sorted but children's compareTo method cannot fulfill its contract (e.g. does not provide transitivity).\r\n\r\nMy suggestion is to synchronize those two operations, which should not have great impact on performance (taking into account the kind of operations that we have here) and at the same time we will not impact general logic of such a low level component as file system. ","created_at":"2016-05-24T11:58:13Z","updated_at":"2016-05-24T16:23:20Z","closed_at":null,"merged_at":null,"merge_commit_sha":"f5bbbd5e346c350f579fa6dfc0c3ade740a3c1df","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/eclipse/che/pulls/1343/commits","review_comments_url":"https://api.github.com/repos/eclipse/che/pulls/1343/comments","review_comment_url":"https://api.github.com/repos/eclipse/che/pulls/comments{/number}","comments_url":"https://api.github.com/repos/eclipse/che/issues/1343/comments","statuses_url":"https://api.github.com/repos/eclipse/che/statuses/4088de7e58045e3a0d50d22d23ed601cc80a7f86","head":{"label":"dkuleshov:CHE-1143","ref":"CHE-1143","sha":"4088de7e58045e3a0d50d22d23ed601cc80a7f86","user":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"repo":{"id":59562244,"name":"che","full_name":"dkuleshov/che","owner":{"login":"dkuleshov","id":1632360,"avatar_url":"https://avatars.githubusercontent.com/u/1632360?v=3","gravatar_id":"","url":"https://api.github.com/users/dkuleshov","html_url":"https://github.com/dkuleshov","followers_url":"https://api.github.com/users/dkuleshov/followers","following_url":"https://api.github.com/users/dkuleshov/following{/other_user}","gists_url":"https://api.github.com/users/dkuleshov/gists{/gist_id}","starred_url":"https://api.github.com/users/dkuleshov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dkuleshov/subscriptions","organizations_url":"https://api.github.com/users/dkuleshov/orgs","repos_url":"https://api.github.com/users/dkuleshov/repos","events_url":"https://api.github.com/users/dkuleshov/events{/privacy}","received_events_url":"https://api.github.com/users/dkuleshov/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/dkuleshov/che","description":"Eclipse Che: Next-generation Eclipse IDE.  Open source workspace server and cloud IDE.","fork":true,"url":"https://api.github.com/repos/dkuleshov/che","forks_url":"https://api.github.com/repos/dkuleshov/che/forks","keys_url":"https://api.github.com/repos/dkuleshov/che/keys{/key_id}","collaborators_url":"https://api.github.com/repos/dkuleshov/che/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/dkuleshov/che/teams","hooks_url":"https://api.github.com/repos/dkuleshov/che/hooks","issue_events_url":"https://api.github.com/repos/dkuleshov/che/issues/events{/number}","events_url":"https://api.github.com/repos/dkuleshov/che/events","assignees_url":"https://api.github.com/repos/dkuleshov/che/assignees{/user}","branches_url":"https://api.github.com/repos/dkuleshov/che/branches{/branch}","tags_url":"https://api.github.com/repos/dkuleshov/che/tags","blobs_url":"https://api.github.com/repos/dkuleshov/che/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/dkuleshov/che/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/dkuleshov/che/git/refs{/sha}","trees_url":"https://api.github.com/repos/dkuleshov/che/git/trees{/sha}","statuses_url":"https://api.github.com/repos/dkuleshov/che/statuses/{sha}","languages_url":"https://api.github.com/repos/dkuleshov/che/languages","stargazers_url":"https://api.github.com/repos/dkuleshov/che/stargazers","contributors_url":"https://api.github.com/repos/dkuleshov/che/contributors","subscribers_url":"https://api.github.com/repos/dkuleshov/che/subscribers","subscription_url":"https://api.github.com/repos/dkuleshov/che/subscription","commits_url":"https://api.github.com/repos/dkuleshov/che/commits{/sha}","git_commits_url":"https://api.github.com/repos/dkuleshov/che/git/commits{/sha}","comments_url":"https://api.github.com/repos/dkuleshov/che/comments{/number}","issue_comment_url":"https://api.github.com/repos/dkuleshov/che/issues/comments{/number}","contents_url":"https://api.github.com/repos/dkuleshov/che/contents/{+path}","compare_url":"https://api.github.com/repos/dkuleshov/che/compare/{base}...{head}","merges_url":"https://api.github.com/repos/dkuleshov/che/merges","archive_url":"https://api.github.com/repos/dkuleshov/che/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/dkuleshov/che/downloads","issues_url":"https://api.github.com/repos/dkuleshov/che/issues{/number}","pulls_url":"https://api.github.com/repos/dkuleshov/che/pulls{/number}","milestones_url":"https://api.github.com/repos/dkuleshov/che/milestones{/number}","notifications_url":"https://api.github.com/repos/dkuleshov/che/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/dkuleshov/che/labels{/name}","releases_url":"https://api.github.com/repos/dkuleshov/che/releases{/id}","deployments_url":"https://api.github.com/repos/dkuleshov/che/deployments","created_at":"2016-05-24T10:05:28Z","updated_at":"2016-05-24T10:05:45Z","pushed_at":"2016-05-24T12:32:51Z","git_url":"git://github.com/dkuleshov/che.git","ssh_url":"git@github.com:dkuleshov/che.git","clone_url":"https://github.com/dkuleshov/che.git","svn_url":"https://github.com/dkuleshov/che","homepage":"http://eclipse.org/che","size":25106,"stargazers_count":0,"watchers_count":0,"language":"Java","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"eclipse:master","ref":"master","sha":"492f213b0d49412766fe8d5b670d8e523dc585b7","user":{"login":"eclipse","id":56974,"avatar_url":"https://avatars.githubusercontent.com/u/56974?v=3","gravatar_id":"","url":"https://api.github.com/users/eclipse","html_url":"https://github.com/eclipse","followers_url":"https://api.github.com/users/eclipse/followers","following_url":"https://api.github.com/users/eclipse/following{/other_user}","gists_url":"https://api.github.com/users/eclipse/gists{/gist_id}","starred_url":"https://api.github.com/users/eclipse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eclipse/subscriptions","organizations_url":"https://api.github.com/users/eclipse/orgs","repos_url":"https://api.github.com/users/eclipse/repos","events_url":"https://api.github.com/users/eclipse/events{/privacy}","received_events_url":"https://api.github.com/users/eclipse/received_events","type":"Organization","site_admin":false},"repo":{"id":32935745,"name":"che","full_name":"eclipse/che","owner":{"login":"eclipse","id":56974,"avatar_url":"https://avatars.githubusercontent.com/u/56974?v=3","gravatar_id":"","url":"https://api.github.com/users/eclipse","html_url":"https://github.com/eclipse","followers_url":"https://api.github.com/users/eclipse/followers","following_url":"https://api.github.com/users/eclipse/following{/other_user}","gists_url":"https://api.github.com/users/eclipse/gists{/gist_id}","starred_url":"https://api.github.com/users/eclipse/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/eclipse/subscriptions","organizations_url":"https://api.github.com/users/eclipse/orgs","repos_url":"https://api.github.com/users/eclipse/repos","events_url":"https://api.github.com/users/eclipse/events{/privacy}","received_events_url":"https://api.github.com/users/eclipse/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/eclipse/che","description":"Eclipse Che: Next-generation Eclipse IDE.  Open source workspace server and cloud IDE.","fork":false,"url":"https://api.github.com/repos/eclipse/che","forks_url":"https://api.github.com/repos/eclipse/che/forks","keys_url":"https://api.github.com/repos/eclipse/che/keys{/key_id}","collaborators_url":"https://api.github.com/repos/eclipse/che/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/eclipse/che/teams","hooks_url":"https://api.github.com/repos/eclipse/che/hooks","issue_events_url":"https://api.github.com/repos/eclipse/che/issues/events{/number}","events_url":"https://api.github.com/repos/eclipse/che/events","assignees_url":"https://api.github.com/repos/eclipse/che/assignees{/user}","branches_url":"https://api.github.com/repos/eclipse/che/branches{/branch}","tags_url":"https://api.github.com/repos/eclipse/che/tags","blobs_url":"https://api.github.com/repos/eclipse/che/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/eclipse/che/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/eclipse/che/git/refs{/sha}","trees_url":"https://api.github.com/repos/eclipse/che/git/trees{/sha}","statuses_url":"https://api.github.com/repos/eclipse/che/statuses/{sha}","languages_url":"https://api.github.com/repos/eclipse/che/languages","stargazers_url":"https://api.github.com/repos/eclipse/che/stargazers","contributors_url":"https://api.github.com/repos/eclipse/che/contributors","subscribers_url":"https://api.github.com/repos/eclipse/che/subscribers","subscription_url":"https://api.github.com/repos/eclipse/che/subscription","commits_url":"https://api.github.com/repos/eclipse/che/commits{/sha}","git_commits_url":"https://api.github.com/repos/eclipse/che/git/commits{/sha}","comments_url":"https://api.github.com/repos/eclipse/che/comments{/number}","issue_comment_url":"https://api.github.com/repos/eclipse/che/issues/comments{/number}","contents_url":"https://api.github.com/repos/eclipse/che/contents/{+path}","compare_url":"https://api.github.com/repos/eclipse/che/compare/{base}...{head}","merges_url":"https://api.github.com/repos/eclipse/che/merges","archive_url":"https://api.github.com/repos/eclipse/che/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/eclipse/che/downloads","issues_url":"https://api.github.com/repos/eclipse/che/issues{/number}","pulls_url":"https://api.github.com/repos/eclipse/che/pulls{/number}","milestones_url":"https://api.github.com/repos/eclipse/che/milestones{/number}","notifications_url":"https://api.github.com/repos/eclipse/che/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/eclipse/che/labels{/name}","releases_url":"https://api.github.com/repos/eclipse/che/releases{/id}","deployments_url":"https://api.github.com/repos/eclipse/che/deployments","created_at":"2015-03-26T15:31:01Z","updated_at":"2016-05-24T14:53:36Z","pushed_at":"2016-05-24T15:24:52Z","git_url":"git://github.com/eclipse/che.git","ssh_url":"git@github.com:eclipse/che.git","clone_url":"https://github.com/eclipse/che.git","svn_url":"https://github.com/eclipse/che","homepage":"http://eclipse.org/che","size":34040,"stargazers_count":2846,"watchers_count":2846,"language":"Java","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":404,"mirror_url":null,"open_issues_count":104,"forks":404,"open_issues":104,"watchers":2846,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/eclipse/che/pulls/1343"},"html":{"href":"https://github.com/eclipse/che/pull/1343"},"issue":{"href":"https://api.github.com/repos/eclipse/che/issues/1343"},"comments":{"href":"https://api.github.com/repos/eclipse/che/issues/1343/comments"},"review_comments":{"href":"https://api.github.com/repos/eclipse/che/pulls/1343/comments"},"review_comment":{"href":"https://api.github.com/repos/eclipse/che/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/eclipse/che/pulls/1343/commits"},"statuses":{"href":"https://api.github.com/repos/eclipse/che/statuses/4088de7e58045e3a0d50d22d23ed601cc80a7f86"}}}}
{"actor":{"display_login":"benoitf"}}