{"action":"created","issue":{"url":"https://api.github.com/repos/w3c/web-platform-tests/issues/10853","repository_url":"https://api.github.com/repos/w3c/web-platform-tests","labels_url":"https://api.github.com/repos/w3c/web-platform-tests/issues/10853/labels{/name}","comments_url":"https://api.github.com/repos/w3c/web-platform-tests/issues/10853/comments","events_url":"https://api.github.com/repos/w3c/web-platform-tests/issues/10853/events","html_url":"https://github.com/w3c/web-platform-tests/issues/10853","id":320377967,"number":10853,"title":"Disambiguating uncaught errors","user":{"login":"jugglinmike","id":677252,"avatar_url":"https://avatars2.githubusercontent.com/u/677252?v=4","gravatar_id":"","url":"https://api.github.com/users/jugglinmike","html_url":"https://github.com/jugglinmike","followers_url":"https://api.github.com/users/jugglinmike/followers","following_url":"https://api.github.com/users/jugglinmike/following{/other_user}","gists_url":"https://api.github.com/users/jugglinmike/gists{/gist_id}","starred_url":"https://api.github.com/users/jugglinmike/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jugglinmike/subscriptions","organizations_url":"https://api.github.com/users/jugglinmike/orgs","repos_url":"https://api.github.com/users/jugglinmike/repos","events_url":"https://api.github.com/users/jugglinmike/events{/privacy}","received_events_url":"https://api.github.com/users/jugglinmike/received_events","type":"User","site_admin":false},"labels":[{"id":45230790,"url":"https://api.github.com/repos/w3c/web-platform-tests/labels/infra","name":"infra","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-05-04T18:15:34Z","updated_at":"2018-05-06T12:01:22Z","closed_at":null,"author_association":"COLLABORATOR","body":"Single page tests are convenient because they shorten test code by two lines (i.e. `test(function(){` and `});`). However, they also introduce an ambiguity which makes differentiating harness errors and harness errors difficult (or maybe impossible).\r\n\r\nThe ambiguity is apparent any time a test produces an uncaught error without defining any tests. This may represent one of two states:\r\n\r\n- a valid \"single page test\" that has failed\r\n- a test that has been invalidated by exceptional behavior outside of a sub-test\r\n\r\nAt this moment, it is not possible for testharness.js to differentiate between these two states. Today, it conservatively assumes the former. This is a problem because it means WPT cannot detect (or reject) tests that are invalid in this way. gh-10706 is the most recent example we have identified.\r\n\r\nAs far as I know, \"single page tests\" are only offered as a convenience to test authors. The ambiguity described above trumps the convenience, so if the motivation really is so simple, then we should remove the feature.\r\n\r\nThere may be cases where the semantics under test necessitate that functionality. In that case, we could preserve the feature and eliminate the ambiguity by requiring an explicit \"opt-in\" pattern. This might take the form of metadata in the test code (e.g. `// META: single=true`) or a filename-based feature pattern (e.g. `.single.html`).\r\n\r\nIn either case, all \"single page tests\" defined in WPT today would need to be updated. [My attempts to estimate the number of single-page tests have been woefully misguided.](https://w3.logbot.info/testing/20180504#c403285-c403319) I think I'll need to instrument WPT and do a complete run to get an accurate number. Before investing that time, I'd like to get feedback from the folks who would know better than me.\r\n\r\n@jgraham @gsnedders @zcorpan @foolip What do you think about eliminating single page tests? Or requiring explicit opt-in?"},"comment":{"url":"https://api.github.com/repos/w3c/web-platform-tests/issues/comments/386874315","html_url":"https://github.com/w3c/web-platform-tests/issues/10853#issuecomment-386874315","issue_url":"https://api.github.com/repos/w3c/web-platform-tests/issues/10853","id":386874315,"user":{"login":"zcorpan","id":244772,"avatar_url":"https://avatars2.githubusercontent.com/u/244772?v=4","gravatar_id":"","url":"https://api.github.com/users/zcorpan","html_url":"https://github.com/zcorpan","followers_url":"https://api.github.com/users/zcorpan/followers","following_url":"https://api.github.com/users/zcorpan/following{/other_user}","gists_url":"https://api.github.com/users/zcorpan/gists{/gist_id}","starred_url":"https://api.github.com/users/zcorpan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zcorpan/subscriptions","organizations_url":"https://api.github.com/users/zcorpan/orgs","repos_url":"https://api.github.com/users/zcorpan/repos","events_url":"https://api.github.com/users/zcorpan/events{/privacy}","received_events_url":"https://api.github.com/users/zcorpan/received_events","type":"User","site_admin":false},"created_at":"2018-05-06T12:01:22Z","updated_at":"2018-05-06T12:01:22Z","author_association":"OWNER","body":"We could have an easier to remember opt in like `single()`"}}
{"actor":{"display_login":"zcorpan"}}