{"action":"created","comment":{"url":"https://api.github.com/repos/servo/servo/pulls/comments/71901586","id":71901586,"diff_hunk":"@@ -0,0 +1,66 @@\n+Servo Page Load Time Test\n+==============\n+\n+# Prerequisites\n+\n+* Python3\n+\n+# Basic Usage\n+\n+## Prepare the test runner\n+\n+* Clone this repo\n+* Download [tp5n.zip](http://people.mozilla.org/~jmaher/taloszips/zips/tp5n.zip), extract it to `page_load_test/`\n+* Run `prepare_manifest.sh` to transform the tp5n manifest to our format\n+* `python3 -m virtualenv venv; source venv/bin/activate; pip install treeherder-client`\n+* Setup your Treeherder client ID and secret as environment variables `TREEHERDER_CLIENT_ID` and `TREEHERDER_CLIENT_SECRET`\n+\n+## Build Servo\n+* Clone the servo repo\n+* Compile release build\n+* Run `git_log_to_json.sh` in the servo repo, save the output as `revision.json`\n+* Put your `servo` binary, `revision.json` and `resources` folder in `etc/ci/performance/servo/`\n+\n+## Run\n+* Run `test_all.sh`\n+* Test results are submitted to https://treeherder.mozilla.org/#/jobs?repo=servo\n+\n+# How it works\n+\n+* The testcase is from tp5, every testcase will run 20 times, and we take the median.\n+* Some of the tests will make Servo run forever, it's disabled right now. See https://github.com/servo/servo/issues/11087\n+* Each testcase is a subtest on Perfherder, and their summary time is the geometric mean of all the subtests.\n+* Notice that the test is different from the Talos TP5 test we run for Gecko. So you can NOT conclude that Servo is \"faster\" or \"slower\" than Gecko from this test.\n+\n+# Unit tests\n+\n+You can run all unit tests (include 3rd-party libraries) with `python -m pytest`.\n+\n+Individual test can be run by `python -m pytest <filename>`:\n+\n+* `test_runner.py`\n+* `test_submit_to_perfherder.py`\n+\n+# Advanced Usage\n+\n+## Test Perfherder Locally\n+\n+If you want to test the data submission code in `submit_to_perfherder.py`, you can setup a local treeherder VM. If you don't need to test `submit_to_perfherder.py`, you can skip this step.","path":"etc/ci/performance/README.md","position":48,"original_position":48,"commit_id":"d3254723a2c3c73ef6c228660792b6c477c0e393","original_commit_id":"d3254723a2c3c73ef6c228660792b6c477c0e393","user":{"login":"aneeshusa","id":2085567,"avatar_url":"https://avatars.githubusercontent.com/u/2085567?v=3","gravatar_id":"","url":"https://api.github.com/users/aneeshusa","html_url":"https://github.com/aneeshusa","followers_url":"https://api.github.com/users/aneeshusa/followers","following_url":"https://api.github.com/users/aneeshusa/following{/other_user}","gists_url":"https://api.github.com/users/aneeshusa/gists{/gist_id}","starred_url":"https://api.github.com/users/aneeshusa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aneeshusa/subscriptions","organizations_url":"https://api.github.com/users/aneeshusa/orgs","repos_url":"https://api.github.com/users/aneeshusa/repos","events_url":"https://api.github.com/users/aneeshusa/events{/privacy}","received_events_url":"https://api.github.com/users/aneeshusa/received_events","type":"User","site_admin":false},"body":"Is this needed to run `test_submit_to_perfherder.py`? Please add a note either way.","created_at":"2016-07-22T15:53:57Z","updated_at":"2016-07-22T15:53:57Z","html_url":"https://github.com/servo/servo/pull/11107#discussion_r71901586","pull_request_url":"https://api.github.com/repos/servo/servo/pulls/11107","_links":{"self":{"href":"https://api.github.com/repos/servo/servo/pulls/comments/71901586"},"html":{"href":"https://github.com/servo/servo/pull/11107#discussion_r71901586"},"pull_request":{"href":"https://api.github.com/repos/servo/servo/pulls/11107"}}},"pull_request":{"url":"https://api.github.com/repos/servo/servo/pulls/11107","id":69478361,"html_url":"https://github.com/servo/servo/pull/11107","diff_url":"https://github.com/servo/servo/pull/11107.diff","patch_url":"https://github.com/servo/servo/pull/11107.patch","issue_url":"https://api.github.com/repos/servo/servo/issues/11107","number":11107,"state":"open","locked":false,"title":"Bug 10452 - Page Load Time Test Runner","user":{"login":"shinglyu","id":3250983,"avatar_url":"https://avatars.githubusercontent.com/u/3250983?v=3","gravatar_id":"","url":"https://api.github.com/users/shinglyu","html_url":"https://github.com/shinglyu","followers_url":"https://api.github.com/users/shinglyu/followers","following_url":"https://api.github.com/users/shinglyu/following{/other_user}","gists_url":"https://api.github.com/users/shinglyu/gists{/gist_id}","starred_url":"https://api.github.com/users/shinglyu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shinglyu/subscriptions","organizations_url":"https://api.github.com/users/shinglyu/orgs","repos_url":"https://api.github.com/users/shinglyu/repos","events_url":"https://api.github.com/users/shinglyu/events{/privacy}","received_events_url":"https://api.github.com/users/shinglyu/received_events","type":"User","site_admin":false},"body":"This is a minimal test runner for bug #10452 . \r\n\r\nThe main entry point is `test_all.sh`, which will trigger `runner.py` (run the performance test), then run `submit_to_perfherder.py` (push performance data to perfherder)\r\n\r\nThere are also some utility script scattered around.\r\n\r\nYou can also check `README.md` for detail.\n\n<!-- Reviewable:start -->\n---\nThis change is [<img src=\"https://reviewable.io/review_button.svg\" height=\"35\" align=\"absmiddle\" alt=\"Reviewable\"/>](https://reviewable.io/reviews/servo/servo/11107)\n<!-- Reviewable:end -->\n","created_at":"2016-05-10T09:56:27Z","updated_at":"2016-07-22T15:53:57Z","closed_at":null,"merged_at":null,"merge_commit_sha":"98e03d2d46a8677dffca46d143fc8c31f2e36ef5","assignee":{"login":"aneeshusa","id":2085567,"avatar_url":"https://avatars.githubusercontent.com/u/2085567?v=3","gravatar_id":"","url":"https://api.github.com/users/aneeshusa","html_url":"https://github.com/aneeshusa","followers_url":"https://api.github.com/users/aneeshusa/followers","following_url":"https://api.github.com/users/aneeshusa/following{/other_user}","gists_url":"https://api.github.com/users/aneeshusa/gists{/gist_id}","starred_url":"https://api.github.com/users/aneeshusa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aneeshusa/subscriptions","organizations_url":"https://api.github.com/users/aneeshusa/orgs","repos_url":"https://api.github.com/users/aneeshusa/repos","events_url":"https://api.github.com/users/aneeshusa/events{/privacy}","received_events_url":"https://api.github.com/users/aneeshusa/received_events","type":"User","site_admin":false},"assignees":[{"login":"aneeshusa","id":2085567,"avatar_url":"https://avatars.githubusercontent.com/u/2085567?v=3","gravatar_id":"","url":"https://api.github.com/users/aneeshusa","html_url":"https://github.com/aneeshusa","followers_url":"https://api.github.com/users/aneeshusa/followers","following_url":"https://api.github.com/users/aneeshusa/following{/other_user}","gists_url":"https://api.github.com/users/aneeshusa/gists{/gist_id}","starred_url":"https://api.github.com/users/aneeshusa/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aneeshusa/subscriptions","organizations_url":"https://api.github.com/users/aneeshusa/orgs","repos_url":"https://api.github.com/users/aneeshusa/repos","events_url":"https://api.github.com/users/aneeshusa/events{/privacy}","received_events_url":"https://api.github.com/users/aneeshusa/received_events","type":"User","site_admin":false}],"milestone":null,"commits_url":"https://api.github.com/repos/servo/servo/pulls/11107/commits","review_comments_url":"https://api.github.com/repos/servo/servo/pulls/11107/comments","review_comment_url":"https://api.github.com/repos/servo/servo/pulls/comments{/number}","comments_url":"https://api.github.com/repos/servo/servo/issues/11107/comments","statuses_url":"https://api.github.com/repos/servo/servo/statuses/d3254723a2c3c73ef6c228660792b6c477c0e393","head":{"label":"shinglyu:servo-perf","ref":"servo-perf","sha":"d3254723a2c3c73ef6c228660792b6c477c0e393","user":{"login":"shinglyu","id":3250983,"avatar_url":"https://avatars.githubusercontent.com/u/3250983?v=3","gravatar_id":"","url":"https://api.github.com/users/shinglyu","html_url":"https://github.com/shinglyu","followers_url":"https://api.github.com/users/shinglyu/followers","following_url":"https://api.github.com/users/shinglyu/following{/other_user}","gists_url":"https://api.github.com/users/shinglyu/gists{/gist_id}","starred_url":"https://api.github.com/users/shinglyu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shinglyu/subscriptions","organizations_url":"https://api.github.com/users/shinglyu/orgs","repos_url":"https://api.github.com/users/shinglyu/repos","events_url":"https://api.github.com/users/shinglyu/events{/privacy}","received_events_url":"https://api.github.com/users/shinglyu/received_events","type":"User","site_admin":false},"repo":{"id":26213633,"name":"servo","full_name":"shinglyu/servo","owner":{"login":"shinglyu","id":3250983,"avatar_url":"https://avatars.githubusercontent.com/u/3250983?v=3","gravatar_id":"","url":"https://api.github.com/users/shinglyu","html_url":"https://github.com/shinglyu","followers_url":"https://api.github.com/users/shinglyu/followers","following_url":"https://api.github.com/users/shinglyu/following{/other_user}","gists_url":"https://api.github.com/users/shinglyu/gists{/gist_id}","starred_url":"https://api.github.com/users/shinglyu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/shinglyu/subscriptions","organizations_url":"https://api.github.com/users/shinglyu/orgs","repos_url":"https://api.github.com/users/shinglyu/repos","events_url":"https://api.github.com/users/shinglyu/events{/privacy}","received_events_url":"https://api.github.com/users/shinglyu/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/shinglyu/servo","description":"The Servo Browser Engine","fork":true,"url":"https://api.github.com/repos/shinglyu/servo","forks_url":"https://api.github.com/repos/shinglyu/servo/forks","keys_url":"https://api.github.com/repos/shinglyu/servo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/shinglyu/servo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/shinglyu/servo/teams","hooks_url":"https://api.github.com/repos/shinglyu/servo/hooks","issue_events_url":"https://api.github.com/repos/shinglyu/servo/issues/events{/number}","events_url":"https://api.github.com/repos/shinglyu/servo/events","assignees_url":"https://api.github.com/repos/shinglyu/servo/assignees{/user}","branches_url":"https://api.github.com/repos/shinglyu/servo/branches{/branch}","tags_url":"https://api.github.com/repos/shinglyu/servo/tags","blobs_url":"https://api.github.com/repos/shinglyu/servo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/shinglyu/servo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/shinglyu/servo/git/refs{/sha}","trees_url":"https://api.github.com/repos/shinglyu/servo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/shinglyu/servo/statuses/{sha}","languages_url":"https://api.github.com/repos/shinglyu/servo/languages","stargazers_url":"https://api.github.com/repos/shinglyu/servo/stargazers","contributors_url":"https://api.github.com/repos/shinglyu/servo/contributors","subscribers_url":"https://api.github.com/repos/shinglyu/servo/subscribers","subscription_url":"https://api.github.com/repos/shinglyu/servo/subscription","commits_url":"https://api.github.com/repos/shinglyu/servo/commits{/sha}","git_commits_url":"https://api.github.com/repos/shinglyu/servo/git/commits{/sha}","comments_url":"https://api.github.com/repos/shinglyu/servo/comments{/number}","issue_comment_url":"https://api.github.com/repos/shinglyu/servo/issues/comments{/number}","contents_url":"https://api.github.com/repos/shinglyu/servo/contents/{+path}","compare_url":"https://api.github.com/repos/shinglyu/servo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/shinglyu/servo/merges","archive_url":"https://api.github.com/repos/shinglyu/servo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/shinglyu/servo/downloads","issues_url":"https://api.github.com/repos/shinglyu/servo/issues{/number}","pulls_url":"https://api.github.com/repos/shinglyu/servo/pulls{/number}","milestones_url":"https://api.github.com/repos/shinglyu/servo/milestones{/number}","notifications_url":"https://api.github.com/repos/shinglyu/servo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/shinglyu/servo/labels{/name}","releases_url":"https://api.github.com/repos/shinglyu/servo/releases{/id}","deployments_url":"https://api.github.com/repos/shinglyu/servo/deployments","created_at":"2014-11-05T10:00:00Z","updated_at":"2016-03-21T04:17:16Z","pushed_at":"2016-07-22T02:20:38Z","git_url":"git://github.com/shinglyu/servo.git","ssh_url":"git@github.com:shinglyu/servo.git","clone_url":"https://github.com/shinglyu/servo.git","svn_url":"https://github.com/shinglyu/servo","homepage":"","size":200245,"stargazers_count":1,"watchers_count":1,"language":null,"has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":1,"default_branch":"master"}},"base":{"label":"servo:master","ref":"master","sha":"638eb80110cf07ec2f07e43fd64a48042fe2925b","user":{"login":"servo","id":2566135,"avatar_url":"https://avatars.githubusercontent.com/u/2566135?v=3","gravatar_id":"","url":"https://api.github.com/users/servo","html_url":"https://github.com/servo","followers_url":"https://api.github.com/users/servo/followers","following_url":"https://api.github.com/users/servo/following{/other_user}","gists_url":"https://api.github.com/users/servo/gists{/gist_id}","starred_url":"https://api.github.com/users/servo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/servo/subscriptions","organizations_url":"https://api.github.com/users/servo/orgs","repos_url":"https://api.github.com/users/servo/repos","events_url":"https://api.github.com/users/servo/events{/privacy}","received_events_url":"https://api.github.com/users/servo/received_events","type":"Organization","site_admin":false},"repo":{"id":3390243,"name":"servo","full_name":"servo/servo","owner":{"login":"servo","id":2566135,"avatar_url":"https://avatars.githubusercontent.com/u/2566135?v=3","gravatar_id":"","url":"https://api.github.com/users/servo","html_url":"https://github.com/servo","followers_url":"https://api.github.com/users/servo/followers","following_url":"https://api.github.com/users/servo/following{/other_user}","gists_url":"https://api.github.com/users/servo/gists{/gist_id}","starred_url":"https://api.github.com/users/servo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/servo/subscriptions","organizations_url":"https://api.github.com/users/servo/orgs","repos_url":"https://api.github.com/users/servo/repos","events_url":"https://api.github.com/users/servo/events{/privacy}","received_events_url":"https://api.github.com/users/servo/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/servo/servo","description":"The Servo Browser Engine","fork":false,"url":"https://api.github.com/repos/servo/servo","forks_url":"https://api.github.com/repos/servo/servo/forks","keys_url":"https://api.github.com/repos/servo/servo/keys{/key_id}","collaborators_url":"https://api.github.com/repos/servo/servo/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/servo/servo/teams","hooks_url":"https://api.github.com/repos/servo/servo/hooks","issue_events_url":"https://api.github.com/repos/servo/servo/issues/events{/number}","events_url":"https://api.github.com/repos/servo/servo/events","assignees_url":"https://api.github.com/repos/servo/servo/assignees{/user}","branches_url":"https://api.github.com/repos/servo/servo/branches{/branch}","tags_url":"https://api.github.com/repos/servo/servo/tags","blobs_url":"https://api.github.com/repos/servo/servo/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/servo/servo/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/servo/servo/git/refs{/sha}","trees_url":"https://api.github.com/repos/servo/servo/git/trees{/sha}","statuses_url":"https://api.github.com/repos/servo/servo/statuses/{sha}","languages_url":"https://api.github.com/repos/servo/servo/languages","stargazers_url":"https://api.github.com/repos/servo/servo/stargazers","contributors_url":"https://api.github.com/repos/servo/servo/contributors","subscribers_url":"https://api.github.com/repos/servo/servo/subscribers","subscription_url":"https://api.github.com/repos/servo/servo/subscription","commits_url":"https://api.github.com/repos/servo/servo/commits{/sha}","git_commits_url":"https://api.github.com/repos/servo/servo/git/commits{/sha}","comments_url":"https://api.github.com/repos/servo/servo/comments{/number}","issue_comment_url":"https://api.github.com/repos/servo/servo/issues/comments{/number}","contents_url":"https://api.github.com/repos/servo/servo/contents/{+path}","compare_url":"https://api.github.com/repos/servo/servo/compare/{base}...{head}","merges_url":"https://api.github.com/repos/servo/servo/merges","archive_url":"https://api.github.com/repos/servo/servo/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/servo/servo/downloads","issues_url":"https://api.github.com/repos/servo/servo/issues{/number}","pulls_url":"https://api.github.com/repos/servo/servo/pulls{/number}","milestones_url":"https://api.github.com/repos/servo/servo/milestones{/number}","notifications_url":"https://api.github.com/repos/servo/servo/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/servo/servo/labels{/name}","releases_url":"https://api.github.com/repos/servo/servo/releases{/id}","deployments_url":"https://api.github.com/repos/servo/servo/deployments","created_at":"2012-02-08T19:07:25Z","updated_at":"2016-07-22T09:25:49Z","pushed_at":"2016-07-22T15:46:54Z","git_url":"git://github.com/servo/servo.git","ssh_url":"git@github.com:servo/servo.git","clone_url":"https://github.com/servo/servo.git","svn_url":"https://github.com/servo/servo","homepage":"https://servo.org/","size":271815,"stargazers_count":7596,"watchers_count":7596,"language":null,"has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":1217,"mirror_url":null,"open_issues_count":1628,"forks":1217,"open_issues":1628,"watchers":7596,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/servo/servo/pulls/11107"},"html":{"href":"https://github.com/servo/servo/pull/11107"},"issue":{"href":"https://api.github.com/repos/servo/servo/issues/11107"},"comments":{"href":"https://api.github.com/repos/servo/servo/issues/11107/comments"},"review_comments":{"href":"https://api.github.com/repos/servo/servo/pulls/11107/comments"},"review_comment":{"href":"https://api.github.com/repos/servo/servo/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/servo/servo/pulls/11107/commits"},"statuses":{"href":"https://api.github.com/repos/servo/servo/statuses/d3254723a2c3c73ef6c228660792b6c477c0e393"}}}}
{"actor":{"display_login":"aneeshusa"}}