{"action":"created","issue":{"url":"https://api.github.com/repos/ampproject/amphtml/issues/14360","repository_url":"https://api.github.com/repos/ampproject/amphtml","labels_url":"https://api.github.com/repos/ampproject/amphtml/issues/14360/labels{/name}","comments_url":"https://api.github.com/repos/ampproject/amphtml/issues/14360/comments","events_url":"https://api.github.com/repos/ampproject/amphtml/issues/14360/events","html_url":"https://github.com/ampproject/amphtml/issues/14360","id":310581350,"number":14360,"title":"Make AMP tests less flaky","user":{"login":"rsimha","id":26553114,"avatar_url":"https://avatars1.githubusercontent.com/u/26553114?v=4","gravatar_id":"","url":"https://api.github.com/users/rsimha","html_url":"https://github.com/rsimha","followers_url":"https://api.github.com/users/rsimha/followers","following_url":"https://api.github.com/users/rsimha/following{/other_user}","gists_url":"https://api.github.com/users/rsimha/gists{/gist_id}","starred_url":"https://api.github.com/users/rsimha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsimha/subscriptions","organizations_url":"https://api.github.com/users/rsimha/orgs","repos_url":"https://api.github.com/users/rsimha/repos","events_url":"https://api.github.com/users/rsimha/events{/privacy}","received_events_url":"https://api.github.com/users/rsimha/received_events","type":"User","site_admin":false},"labels":[{"id":530202281,"url":"https://api.github.com/repos/ampproject/amphtml/labels/P2:%20Soon","name":"P2: Soon","color":"e99695","default":false},{"id":284526766,"url":"https://api.github.com/repos/ampproject/amphtml/labels/Related%20to:%20Flaky%20Tests","name":"Related to: Flaky Tests","color":"c5def5","default":false}],"state":"open","locked":false,"assignee":{"login":"rsimha","id":26553114,"avatar_url":"https://avatars1.githubusercontent.com/u/26553114?v=4","gravatar_id":"","url":"https://api.github.com/users/rsimha","html_url":"https://github.com/rsimha","followers_url":"https://api.github.com/users/rsimha/followers","following_url":"https://api.github.com/users/rsimha/following{/other_user}","gists_url":"https://api.github.com/users/rsimha/gists{/gist_id}","starred_url":"https://api.github.com/users/rsimha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsimha/subscriptions","organizations_url":"https://api.github.com/users/rsimha/orgs","repos_url":"https://api.github.com/users/rsimha/repos","events_url":"https://api.github.com/users/rsimha/events{/privacy}","received_events_url":"https://api.github.com/users/rsimha/received_events","type":"User","site_admin":false},"assignees":[{"login":"rsimha","id":26553114,"avatar_url":"https://avatars1.githubusercontent.com/u/26553114?v=4","gravatar_id":"","url":"https://api.github.com/users/rsimha","html_url":"https://github.com/rsimha","followers_url":"https://api.github.com/users/rsimha/followers","following_url":"https://api.github.com/users/rsimha/following{/other_user}","gists_url":"https://api.github.com/users/rsimha/gists{/gist_id}","starred_url":"https://api.github.com/users/rsimha/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rsimha/subscriptions","organizations_url":"https://api.github.com/users/rsimha/orgs","repos_url":"https://api.github.com/users/rsimha/repos","events_url":"https://api.github.com/users/rsimha/events{/privacy}","received_events_url":"https://api.github.com/users/rsimha/received_events","type":"User","site_admin":false}],"milestone":{"url":"https://api.github.com/repos/ampproject/amphtml/milestones/58","html_url":"https://github.com/ampproject/amphtml/milestone/58","labels_url":"https://api.github.com/repos/ampproject/amphtml/milestones/58/labels","id":3231415,"number":58,"title":"Sprint H1 April 2018","description":"","creator":{"login":"adelinamart","id":24213793,"avatar_url":"https://avatars2.githubusercontent.com/u/24213793?v=4","gravatar_id":"","url":"https://api.github.com/users/adelinamart","html_url":"https://github.com/adelinamart","followers_url":"https://api.github.com/users/adelinamart/followers","following_url":"https://api.github.com/users/adelinamart/following{/other_user}","gists_url":"https://api.github.com/users/adelinamart/gists{/gist_id}","starred_url":"https://api.github.com/users/adelinamart/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/adelinamart/subscriptions","organizations_url":"https://api.github.com/users/adelinamart/orgs","repos_url":"https://api.github.com/users/adelinamart/repos","events_url":"https://api.github.com/users/adelinamart/events{/privacy}","received_events_url":"https://api.github.com/users/adelinamart/received_events","type":"User","site_admin":false},"open_issues":11,"closed_issues":0,"state":"open","created_at":"2018-03-30T16:04:32Z","updated_at":"2018-04-02T19:24:41Z","due_on":"2018-04-14T07:00:00Z","closed_at":null},"comments":0,"created_at":"2018-04-02T19:24:41Z","updated_at":"2018-04-02T19:30:02Z","closed_at":null,"author_association":"COLLABORATOR","body":"The number of test failures / flakes during PR and push builds on Travis due to dirty global state appears to have increased over the recent weeks. It's time to explore ideas that will reduce flakiness by isolating tests from one another and eliminating dirty global state.\r\n\r\nSome ideas:\r\n- Update `beforeEach` and `afterEach` in [test/_init_tests.js](https://github.com/ampproject/amphtml/blob/master/test/_init_tests.js#L265-L331) to be more thorough with how they clear state between tests\r\n- Prevent tests from performing operations that modify global state\r\n\r\n(More ideas welcome via comments below)\r\n\r\n![](https://user-images.githubusercontent.com/26553114/38212126-aae87880-368a-11e8-9dc8-d6d83742d4e3.jpg)\r\n"},"comment":{"url":"https://api.github.com/repos/ampproject/amphtml/issues/comments/378019260","html_url":"https://github.com/ampproject/amphtml/issues/14360#issuecomment-378019260","issue_url":"https://api.github.com/repos/ampproject/amphtml/issues/14360","id":378019260,"user":{"login":"alanorozco","id":254946,"avatar_url":"https://avatars2.githubusercontent.com/u/254946?v=4","gravatar_id":"","url":"https://api.github.com/users/alanorozco","html_url":"https://github.com/alanorozco","followers_url":"https://api.github.com/users/alanorozco/followers","following_url":"https://api.github.com/users/alanorozco/following{/other_user}","gists_url":"https://api.github.com/users/alanorozco/gists{/gist_id}","starred_url":"https://api.github.com/users/alanorozco/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alanorozco/subscriptions","organizations_url":"https://api.github.com/users/alanorozco/orgs","repos_url":"https://api.github.com/users/alanorozco/repos","events_url":"https://api.github.com/users/alanorozco/events{/privacy}","received_events_url":"https://api.github.com/users/alanorozco/received_events","type":"User","site_admin":false},"created_at":"2018-04-02T19:30:02Z","updated_at":"2018-04-02T19:30:02Z","author_association":"MEMBER","body":"# 1. Stub experiment toggles by default\r\n\r\nSome sort of test-time replacement of `isExperimentOn` and `toggleExperiment` so that they use stubs instead of affecting the window state.\r\n\r\n# 2. Forbid experiment toggles in test\r\n\r\nDisadvantage: tests need to be rewritten to stub a static method that checks for the experiment."}}
{"actor":{"display_login":"alanorozco"}}