{"action":"created","issue":{"url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/564","repository_url":"https://api.github.com/repos/web-platform-tests/results-collection","labels_url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/564/labels{/name}","comments_url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/564/comments","events_url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/564/events","html_url":"https://github.com/web-platform-tests/results-collection/issues/564","id":324476843,"number":564,"title":"WPT runner timeout","user":{"login":"jugglinmike","id":677252,"avatar_url":"https://avatars2.githubusercontent.com/u/677252?v=4","gravatar_id":"","url":"https://api.github.com/users/jugglinmike","html_url":"https://github.com/jugglinmike","followers_url":"https://api.github.com/users/jugglinmike/followers","following_url":"https://api.github.com/users/jugglinmike/following{/other_user}","gists_url":"https://api.github.com/users/jugglinmike/gists{/gist_id}","starred_url":"https://api.github.com/users/jugglinmike/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jugglinmike/subscriptions","organizations_url":"https://api.github.com/users/jugglinmike/orgs","repos_url":"https://api.github.com/users/jugglinmike/repos","events_url":"https://api.github.com/users/jugglinmike/events{/privacy}","received_events_url":"https://api.github.com/users/jugglinmike/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-05-18T16:33:57Z","updated_at":"2018-05-30T13:12:47Z","closed_at":null,"author_association":"COLLABORATOR","body":"A recent collection failed due to an as-yet unknown error originating in the WPT CLI.\r\n\r\nThe collection was for Safari as provided by Sauce Labs, specifically \"chunk\" number 84 of 100.\r\n\r\nWe have previously experienced unexpected closure of the Sauce Labs Connect tunnel (see gh-544). At first glance, the logs from that process seemed to indicate that the latest error was due to the same problem:\r\n\r\n    20180517 220229.293 [16324] Secure remote tunnel VM provisioned.\r\n    20180517 220229.293 [16324] Tunnel ID: 49a19b33c8a349d59dc353c8411ccfee\r\n    20180517 220229.293 [16324] Using no proxy for connecting to tunnel VM.\r\n    20180517 220229.294 [16324] Starting Selenium listener...\r\n    20180517 220229.294 [16324] Establishing secure TLS connection to tunnel...\r\n    20180517 220229.294 [16324] [EVDNS] Added nameserver 10.101.0.2:53 as 0x7fa79800dc50\r\n    20180517 220229.295 [16324] MAIN connecting to maki82227.miso.saucelabs.com:443\r\n    20180517 220229.392 [16324] MAIN connecting to KGP server maki82227.miso.saucelabs.com took 96 ms\r\n    20180517 220229.544 [16324] Selenium listener started on port 4445.\r\n    20180517 220230.548 [16324] Sauce Connect is up, you may start your tests.\r\n\r\n    (about 18,000 lines elidated)\r\n\r\n    20180517 222103.273 [16324] PROXY 127.0.0.1:36670 (172.20.47.221) <- 200 web-platform.test:8000 (537 bytes)\r\n    20180517 222104.613 [16324] PROXY resolved 'web-platform.test' to '127.0.0.1'\r\n    20180517 222104.613 [16324] PROXY 127.0.0.1:36684 (172.20.47.221) -> GET http://web-platform.test:8000/css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-4.html (618 bytes)\r\n    20180517 222104.615 [16324] PROXY 127.0.0.1:36684 (172.20.47.221) <- 200 web-platform.test:8000 (711 bytes)\r\n    20180517 222106.466 [16324] PROXY resolved 'web-platform.test' to '127.0.0.1'\r\n    20180517 222106.466 [16324] PROXY 127.0.0.1:36696 (172.20.47.221) -> GET http://web-platform.test:8000/css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-4-ref.html (622 bytes)\r\n    20180517 222106.469 [16324] PROXY 127.0.0.1:36696 (172.20.47.221) <- 200 web-platform.test:8000 (537 bytes)\r\n    20180517 225236.457 [16324] Got signal terminated\r\n    20180517 225236.460 [16324] Cleaning up.\r\n    20180517 225236.460 [16324] Removing tunnel 49a19b33c8a349d59dc353c8411ccfee.\r\n    20180517 225236.460 [16324] Waiting for any active jobs using this tunnel to finish.\r\n    20180517 225236.460 [16324] Press CTRL-C again to shut down immediately.\r\n    20180517 225236.460 [16324] Note: if you do this, tests that are still running will fail.\r\n\r\nHowever, in this case, there were 30 minutes of inactivity prior to the closure. The Sauce Labs documentation is pretty clear about this:\r\n\r\n> As a safety measure to prevent tests from running indefinitely, Sauce limits\r\n> the duration of tests to 30 minutes by default. You can adjust this limit on\r\n> per-job basis and the maximum value is 10800 seconds. \r\n\r\nhttps://wiki.saucelabs.com/display/DOCS/Test+Configuration+Options\r\n\r\nThis shifts the focus to the WPT CLI: why was the test hanging? Here are the WPT CLI logs at the time of the Sauce Labs Connect closure:\r\n\r\n    2018-05-17 22:20:41,996 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:20:56,539 INFO validate-wpt-results wpt-run:stdout TEST-START | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-1.html\r\n    2018-05-17 22:20:56,540 INFO validate-wpt-results wpt-run:stdout Test requires OS-level window focus\r\n    2018-05-17 22:20:59,191 INFO validate-wpt-results wpt-run:stdout TEST-PASS | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-1.html | took 2651ms\r\n    2018-05-17 22:20:59,191 INFO validate-wpt-results wpt-run:stdout TEST-START | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-2.html\r\n    2018-05-17 22:20:59,192 INFO validate-wpt-results wpt-run:stdout Test requires OS-level window focus\r\n    2018-05-17 22:21:01,710 INFO validate-wpt-results wpt-run:stdout TEST-PASS | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-2.html | took 2519ms\r\n    2018-05-17 22:21:01,711 INFO validate-wpt-results wpt-run:stdout TEST-START | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-3.html\r\n    2018-05-17 22:21:01,712 INFO validate-wpt-results wpt-run:stdout Test requires OS-level window focus\r\n    2018-05-17 22:21:04,114 INFO validate-wpt-results wpt-run:stdout TEST-PASS | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-3.html | took 2404ms\r\n    2018-05-17 22:21:04,115 INFO validate-wpt-results wpt-run:stdout TEST-START | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-4.html\r\n    2018-05-17 22:21:04,116 INFO validate-wpt-results wpt-run:stdout Test requires OS-level window focus\r\n    2018-05-17 22:21:26,352 INFO validate-wpt-results wpt-run:stdout TEST-UNEXPECTED-TIMEOUT | /css/vendor-imports/mozilla/mozilla-central-reftests/ib-split/emptyspan-4.html | expected PASS\r\n    2018-05-17 22:21:26,352 INFO validate-wpt-results wpt-run:stdout TEST-INFO took 22238ms\r\n    2018-05-17 22:21:36,353 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:21:36,362 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:26:46,364 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:26:46,371 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:26:46,376 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:31:56,378 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:31:56,385 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:31:56,388 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:37:06,389 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:37:06,396 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:37:06,400 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:42:16,401 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:42:16,408 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:42:16,411 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:47:26,413 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:47:26,422 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:47:26,423 INFO validate-wpt-results wpt-run:stdout Starting runner\r\n    2018-05-17 22:52:36,427 INFO validate-wpt-results wpt-run:stdout Forcibly terminating runner process\r\n    2018-05-17 22:52:36,434 INFO validate-wpt-results wpt-run:stdout u'stop': ()\r\n    2018-05-17 22:52:36,434 INFO validate-wpt-results wpt-run:stdout Max restarts exceeded\r\n    2018-05-17 22:52:36,436 INFO validate-wpt-results wpt-run:stdout Got 492 unexpected results\r\n    2018-05-17 22:52:36,440 INFO validate-wpt-results wpt-run:stdout SUITE-END | took 3005s\r\n    2018-05-17 22:52:36,457 INFO validate-wpt-results wpt-run:stdout 17 May 22:52:36 - Got signal terminated\r\n    2018-05-17 22:52:36,461 INFO validate-wpt-results wpt-run:stdout 17 May 22:52:36 - Cleaning up.\r\n    2018-05-17 22:52:36,461 INFO validate-wpt-results wpt-run:stdout 17 May 22:52:36 - Removing tunnel 49a19b33c8a349d59dc353c8411ccfee.\r\n\r\nWith 5 minutes of silence between each \"starting\" and \"terminating\" output, it appears as though [the \"TestRunnerManager\" instance is timing out](https://github.com/w3c/web-platform-tests/blob/cfd016a717248813af9d419833c376029ec70671/tools/wptrunner/wptrunner/testrunner.py#L467-L489).\r\n\r\nTo learn more, we will have to increase the level of debugging. I am reluctant to do that given the significant increase in storage this will require and the very low frequency of this event. I'm filing this issue so we have a place to report future occurrences and reevaluate the necessity of further investigation.\r\n\r\nFinally, while the system is capable of re-trying collection in response to failure, recovery failed in this case due to a previously-reported bug in the WPT CLI: https://github.com/w3c/web-platform-tests/issues/10724)."},"comment":{"url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/comments/393156288","html_url":"https://github.com/web-platform-tests/results-collection/issues/564#issuecomment-393156288","issue_url":"https://api.github.com/repos/web-platform-tests/results-collection/issues/564","id":393156288,"user":{"login":"gsnedders","id":176218,"avatar_url":"https://avatars2.githubusercontent.com/u/176218?v=4","gravatar_id":"","url":"https://api.github.com/users/gsnedders","html_url":"https://github.com/gsnedders","followers_url":"https://api.github.com/users/gsnedders/followers","following_url":"https://api.github.com/users/gsnedders/following{/other_user}","gists_url":"https://api.github.com/users/gsnedders/gists{/gist_id}","starred_url":"https://api.github.com/users/gsnedders/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gsnedders/subscriptions","organizations_url":"https://api.github.com/users/gsnedders/orgs","repos_url":"https://api.github.com/users/gsnedders/repos","events_url":"https://api.github.com/users/gsnedders/events{/privacy}","received_events_url":"https://api.github.com/users/gsnedders/received_events","type":"User","site_admin":false},"created_at":"2018-05-30T13:12:47Z","updated_at":"2018-05-30T13:12:47Z","author_association":"COLLABORATOR","body":"It's probably something related to \"restarting\" the browser and that not working?"}}
{"actor":{"display_login":"gsnedders"}}