{"action":"created","issue":{"url":"https://api.github.com/repos/gradle/gradle/issues/2669","repository_url":"https://api.github.com/repos/gradle/gradle","labels_url":"https://api.github.com/repos/gradle/gradle/issues/2669/labels{/name}","comments_url":"https://api.github.com/repos/gradle/gradle/issues/2669/comments","events_url":"https://api.github.com/repos/gradle/gradle/issues/2669/events","html_url":"https://github.com/gradle/gradle/issues/2669","id":248426364,"node_id":"MDU6SXNzdWUyNDg0MjYzNjQ=","number":2669,"title":"Improve parallel tests concurrency","user":{"login":"tomwidmer","id":1503640,"node_id":"MDQ6VXNlcjE1MDM2NDA=","avatar_url":"https://avatars0.githubusercontent.com/u/1503640?v=4","gravatar_id":"","url":"https://api.github.com/users/tomwidmer","html_url":"https://github.com/tomwidmer","followers_url":"https://api.github.com/users/tomwidmer/followers","following_url":"https://api.github.com/users/tomwidmer/following{/other_user}","gists_url":"https://api.github.com/users/tomwidmer/gists{/gist_id}","starred_url":"https://api.github.com/users/tomwidmer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tomwidmer/subscriptions","organizations_url":"https://api.github.com/users/tomwidmer/orgs","repos_url":"https://api.github.com/users/tomwidmer/repos","events_url":"https://api.github.com/users/tomwidmer/events{/privacy}","received_events_url":"https://api.github.com/users/tomwidmer/received_events","type":"User","site_admin":false},"labels":[{"id":460356753,"node_id":"MDU6TGFiZWw0NjAzNTY3NTM=","url":"https://api.github.com/repos/gradle/gradle/labels/a:feature","name":"a:feature","color":"0e8a16","default":false},{"id":478384995,"node_id":"MDU6TGFiZWw0NzgzODQ5OTU=","url":"https://api.github.com/repos/gradle/gradle/labels/from:contributor","name":"from:contributor","color":"0052cc","default":false},{"id":281288685,"node_id":"MDU6TGFiZWwyODEyODg2ODU=","url":"https://api.github.com/repos/gradle/gradle/labels/in:testing","name":"in:testing","color":"d4c5f9","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2017-08-07T14:19:39Z","updated_at":"2018-06-28T03:59:51Z","closed_at":null,"author_association":"NONE","body":"When running unit tests in parallel using maxParallelForks, tests are allocated to forks statically, using a round-robin algorithm, on startup. This means that there is a chance that all your slow tests will be allocated to 1 fork, and as a result using more than 1 fork will have almost no time saving over running a single fork. \r\n\r\n### Expected Behavior\r\nWhile there are test classes that haven't yet started, no fork should be idle.\r\n\r\n### Current Behavior\r\nYou can have a whole series of tests run without any parallelism, if more slow tests happen to be allocated to a particular fork than to others, causing the run to take much longer than it should.\r\n\r\n### Context\r\nThis has been an issue with long running Selenium tests (> 1 minute each) running against cloud selenium providers who offer a fixed number of simultaneous tests. Test run time is often 50+% longer than it should be, due to it ending up running just 1 test at a time towards the end of the test suite.\r\n\r\n### Steps to Reproduce (for bugs)\r\nCreate 2 slow test classes (1 minute each) and 2 fast test classes (<1 second). Run them with maxParallelForks = 2. Depending on how the classes are allocated to forks, runtime will be either 1 minute or 2 minutes, a 100% difference.\r\n\r\n### Your Environment\r\nSelenium testing using cloud providers (with limited parallelism)\r\nMac and Linux"},"comment":{"url":"https://api.github.com/repos/gradle/gradle/issues/comments/400903615","html_url":"https://github.com/gradle/gradle/issues/2669#issuecomment-400903615","issue_url":"https://api.github.com/repos/gradle/gradle/issues/2669","id":400903615,"node_id":"MDEyOklzc3VlQ29tbWVudDQwMDkwMzYxNQ==","user":{"login":"4n00p","id":4590808,"node_id":"MDQ6VXNlcjQ1OTA4MDg=","avatar_url":"https://avatars0.githubusercontent.com/u/4590808?v=4","gravatar_id":"","url":"https://api.github.com/users/4n00p","html_url":"https://github.com/4n00p","followers_url":"https://api.github.com/users/4n00p/followers","following_url":"https://api.github.com/users/4n00p/following{/other_user}","gists_url":"https://api.github.com/users/4n00p/gists{/gist_id}","starred_url":"https://api.github.com/users/4n00p/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/4n00p/subscriptions","organizations_url":"https://api.github.com/users/4n00p/orgs","repos_url":"https://api.github.com/users/4n00p/repos","events_url":"https://api.github.com/users/4n00p/events{/privacy}","received_events_url":"https://api.github.com/users/4n00p/received_events","type":"User","site_admin":false},"created_at":"2018-06-28T03:59:51Z","updated_at":"2018-06-28T03:59:51Z","author_association":"NONE","body":"What do you think about this idea? Any guidance?  I think it would offer better speedup than moving the longest running tests to a single Test task:\r\n\r\nThe optimal case is one that shows the ideal speedup (adding a 2nd process reduces duration by 50%).  This is when each test is the same duration (as seen in [screenshots/Figure_3.png](https://raw.githubusercontent.com/4n00p/gradle_log_test_timeline/master/screenshots/Figure_3.png) )  \r\n\r\nI want to get my build as close to the optimal case above.  Using an older JUnit test report, start by constructing a list with test class name + duration and sort by the duration.  Then, dynamically create Test tasks that each only include up to N tests, where N == maxParallelForks and the test class exists in the sourceSet (the JUnit test report and list won't have subproject information).  I will have to make sure that every task has its N \"slots\" full with tests.  If many tests are similar in  duration, then I can have single Test task contain 2*N, etc.  \r\n\r\nThis would create many Test tasks within a subproject, but hopefully the overhead from that would be minimal.\r\nWhy so many Test tasks? Simply to prevent queuing of multiple, long-running tests (the original problem in this ticket :)  )  I'm sure you all know this, but here is my example:  \r\n\r\n```\r\nIf maxParallelForks = 2, and the sourceSet contains JUnit tests:\r\n Test1,Test2,Test3,Test4.\r\nAssume these tests happen to be in sorted order of duration.\r\nIf there is one task, Task1, with sourceSet:\r\nTest1 = duration of 100 seconds\r\nTest2 =                     90 seconds\r\nTest3 =                     80 seconds\r\nTest4 =                     70 seconds\r\n\r\nIt may result in a total task duration of 190 seconds, depending on \r\nhow the JUnit tests are assigned:\r\norder of execution  ---->\r\n0 seconds                  100 seconds               190 seconds\r\n         Task1:Test1 (100 sec) | Task1:Test2 (90 sec)\r\n         Task1:Test3 (80 sec)   | Task1:Test4 (70 sec)\r\n\r\nIf maxParallelForks = 2 and there are 2 Tasks:\r\n\r\nTask 1:\r\nTest1 = duration of 100 seconds\r\nTest2 =                    90 seconds\r\n\r\nTask 2:\r\nTest3 =             80 seconds\r\nTest4 =             70 seconds\r\n\r\norder of execution  ---->\r\n0 seconds                  100 seconds                      180 seconds\r\n         Task1:Test1 (100 sec) | Task2:Test2 (80 sec)\r\n         Task1:Test2 (90 sec)   | Task2:Test4 (70 sec)\r\n\r\nTask1 and Task2 complete these Tests in 180 seconds \r\n(assuming not much overhead switching from Task1 to Task2)\r\n```\r\nThe tests are assigned from the sourceSet in what seems to be a round-robin assignment to workers from the source tree...? Well, no matter, it is a set and no order is guaranteed.  \r\nIn summary:\r\n- create a list of JUnit test names, sorted by duration \r\n- per subproject, dynamically create Test tasks with \"include\" statements. The include should take the first N tests (that are in the sourceSet) then the next Test task takes the next N tests,etc.\r\n\r\nAnother approach I was thinking about was to use JavaExec to \"blast\" the Junit tests in parallel...then somehow collect the html output into a single report.\r\n"}}
{"actor":{"display_login":"4n00p"}}