{"action":"opened","number":392,"pull_request":{"url":"https://api.github.com/repos/schematics/schematics/pulls/392","id":59715763,"html_url":"https://github.com/schematics/schematics/pull/392","diff_url":"https://github.com/schematics/schematics/pull/392.diff","patch_url":"https://github.com/schematics/schematics/pull/392.patch","issue_url":"https://api.github.com/repos/schematics/schematics/issues/392","number":392,"state":"open","locked":false,"title":"Benchmark tests","user":{"login":"lkraider","id":52256,"avatar_url":"https://avatars.githubusercontent.com/u/52256?v=3","gravatar_id":"","url":"https://api.github.com/users/lkraider","html_url":"https://github.com/lkraider","followers_url":"https://api.github.com/users/lkraider/followers","following_url":"https://api.github.com/users/lkraider/following{/other_user}","gists_url":"https://api.github.com/users/lkraider/gists{/gist_id}","starred_url":"https://api.github.com/users/lkraider/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/lkraider/subscriptions","organizations_url":"https://api.github.com/users/lkraider/orgs","repos_url":"https://api.github.com/users/lkraider/repos","events_url":"https://api.github.com/users/lkraider/events{/privacy}","received_events_url":"https://api.github.com/users/lkraider/received_events","type":"User","site_admin":false},"body":"This branch adds multiple benchmarks tools that can be used to test the performance of Schematics.\r\n\r\nThe following tools are used:\r\n - [pytest-benchmark](https://pytest-benchmark.readthedocs.org)\r\n - [vmprof](https://vmprof.readthedocs.org)\r\n - [cProfile](https://docs.python.org/2/library/profile.html#module-cProfile)\r\n - [airspeed velocity](https://asv.readthedocs.org)\r\n\r\nTo run the tests using `tox`:\r\n```\r\ntox -e benchmark\r\n```\r\nThis will run pytest-benchmark and compare it to a baseline benchmark (saved from my machine, so results will be inconsistent), it will generate some SVG histograms on the project folder. You can run the tests multiple times while checking out revisions in git, and the histograms will be updated with all runs, allowing to compare outputs.\r\n![screen shot 2016-02-17 at 19 48 08](https://cloud.githubusercontent.com/assets/52256/13126020/fff13c88-d5af-11e5-86d0-acdb7dcb4e0d.png)\r\n\r\n\r\nVmprof will also run and upload data to their online viewer. Examples:\r\nhttp://vmprof.com/#/0a0e3965f9eb8c0f89efcf67095e6960\r\nhttp://vmprof.com/#/ed6458cf55dcc3b0bfc69566f53db5e6\r\n\r\nFinally, the output of cProfile is also printed.\r\n\r\nTox will not run airspeed velocity, to do that install the tool in your virtualenv and run the script:\r\n```\r\npip install asv\r\nsh benchmarks/run.sh\r\n```\r\nThis tool allows to compare performance both in time and memory across multiple commits. Example:\r\n![screen shot 2016-02-17 at 19 46 22](https://cloud.githubusercontent.com/assets/52256/13125928/33ed2ee4-d5af-11e5-8383-b368452b44ee.png)\r\n\r\nI added just some basic test functions, would be great to review and expand them. Also, maybe see if it's worth to add some regression tests for Travis to run on PRs.","created_at":"2016-02-17T22:00:29Z","updated_at":"2016-02-17T22:00:29Z","closed_at":null,"merged_at":null,"merge_commit_sha":null,"assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/schematics/schematics/pulls/392/commits","review_comments_url":"https://api.github.com/repos/schematics/schematics/pulls/392/comments","review_comment_url":"https://api.github.com/repos/schematics/schematics/pulls/comments{/number}","comments_url":"https://api.github.com/repos/schematics/schematics/issues/392/comments","statuses_url":"https://api.github.com/repos/schematics/schematics/statuses/8b46a4b2ac0e00662718b68ecd0e5a5bdea79c30","head":{"label":"nKey:benchmark","ref":"benchmark","sha":"8b46a4b2ac0e00662718b68ecd0e5a5bdea79c30","user":{"login":"nKey","id":855190,"avatar_url":"https://avatars.githubusercontent.com/u/855190?v=3","gravatar_id":"","url":"https://api.github.com/users/nKey","html_url":"https://github.com/nKey","followers_url":"https://api.github.com/users/nKey/followers","following_url":"https://api.github.com/users/nKey/following{/other_user}","gists_url":"https://api.github.com/users/nKey/gists{/gist_id}","starred_url":"https://api.github.com/users/nKey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nKey/subscriptions","organizations_url":"https://api.github.com/users/nKey/orgs","repos_url":"https://api.github.com/users/nKey/repos","events_url":"https://api.github.com/users/nKey/events{/privacy}","received_events_url":"https://api.github.com/users/nKey/received_events","type":"Organization","site_admin":false},"repo":{"id":6658396,"name":"schematics","full_name":"nKey/schematics","owner":{"login":"nKey","id":855190,"avatar_url":"https://avatars.githubusercontent.com/u/855190?v=3","gravatar_id":"","url":"https://api.github.com/users/nKey","html_url":"https://github.com/nKey","followers_url":"https://api.github.com/users/nKey/followers","following_url":"https://api.github.com/users/nKey/following{/other_user}","gists_url":"https://api.github.com/users/nKey/gists{/gist_id}","starred_url":"https://api.github.com/users/nKey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nKey/subscriptions","organizations_url":"https://api.github.com/users/nKey/orgs","repos_url":"https://api.github.com/users/nKey/repos","events_url":"https://api.github.com/users/nKey/events{/privacy}","received_events_url":"https://api.github.com/users/nKey/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/nKey/schematics","description":"Python Data Structures for Humans™.","fork":true,"url":"https://api.github.com/repos/nKey/schematics","forks_url":"https://api.github.com/repos/nKey/schematics/forks","keys_url":"https://api.github.com/repos/nKey/schematics/keys{/key_id}","collaborators_url":"https://api.github.com/repos/nKey/schematics/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/nKey/schematics/teams","hooks_url":"https://api.github.com/repos/nKey/schematics/hooks","issue_events_url":"https://api.github.com/repos/nKey/schematics/issues/events{/number}","events_url":"https://api.github.com/repos/nKey/schematics/events","assignees_url":"https://api.github.com/repos/nKey/schematics/assignees{/user}","branches_url":"https://api.github.com/repos/nKey/schematics/branches{/branch}","tags_url":"https://api.github.com/repos/nKey/schematics/tags","blobs_url":"https://api.github.com/repos/nKey/schematics/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/nKey/schematics/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/nKey/schematics/git/refs{/sha}","trees_url":"https://api.github.com/repos/nKey/schematics/git/trees{/sha}","statuses_url":"https://api.github.com/repos/nKey/schematics/statuses/{sha}","languages_url":"https://api.github.com/repos/nKey/schematics/languages","stargazers_url":"https://api.github.com/repos/nKey/schematics/stargazers","contributors_url":"https://api.github.com/repos/nKey/schematics/contributors","subscribers_url":"https://api.github.com/repos/nKey/schematics/subscribers","subscription_url":"https://api.github.com/repos/nKey/schematics/subscription","commits_url":"https://api.github.com/repos/nKey/schematics/commits{/sha}","git_commits_url":"https://api.github.com/repos/nKey/schematics/git/commits{/sha}","comments_url":"https://api.github.com/repos/nKey/schematics/comments{/number}","issue_comment_url":"https://api.github.com/repos/nKey/schematics/issues/comments{/number}","contents_url":"https://api.github.com/repos/nKey/schematics/contents/{+path}","compare_url":"https://api.github.com/repos/nKey/schematics/compare/{base}...{head}","merges_url":"https://api.github.com/repos/nKey/schematics/merges","archive_url":"https://api.github.com/repos/nKey/schematics/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/nKey/schematics/downloads","issues_url":"https://api.github.com/repos/nKey/schematics/issues{/number}","pulls_url":"https://api.github.com/repos/nKey/schematics/pulls{/number}","milestones_url":"https://api.github.com/repos/nKey/schematics/milestones{/number}","notifications_url":"https://api.github.com/repos/nKey/schematics/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/nKey/schematics/labels{/name}","releases_url":"https://api.github.com/repos/nKey/schematics/releases{/id}","deployments_url":"https://api.github.com/repos/nKey/schematics/deployments","created_at":"2012-11-12T18:36:17Z","updated_at":"2014-07-31T22:36:26Z","pushed_at":"2016-02-17T21:59:11Z","git_url":"git://github.com/nKey/schematics.git","ssh_url":"git@github.com:nKey/schematics.git","clone_url":"https://github.com/nKey/schematics.git","svn_url":"https://github.com/nKey/schematics","homepage":"http://groups.google.com/group/schematics-dev","size":2026,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"develop"}},"base":{"label":"schematics:development","ref":"development","sha":"0ead9c19a9e48de13d7d3967c3e9781fac2f45c0","user":{"login":"schematics","id":7866441,"avatar_url":"https://avatars.githubusercontent.com/u/7866441?v=3","gravatar_id":"","url":"https://api.github.com/users/schematics","html_url":"https://github.com/schematics","followers_url":"https://api.github.com/users/schematics/followers","following_url":"https://api.github.com/users/schematics/following{/other_user}","gists_url":"https://api.github.com/users/schematics/gists{/gist_id}","starred_url":"https://api.github.com/users/schematics/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schematics/subscriptions","organizations_url":"https://api.github.com/users/schematics/orgs","repos_url":"https://api.github.com/users/schematics/repos","events_url":"https://api.github.com/users/schematics/events{/privacy}","received_events_url":"https://api.github.com/users/schematics/received_events","type":"Organization","site_admin":false},"repo":{"id":1199632,"name":"schematics","full_name":"schematics/schematics","owner":{"login":"schematics","id":7866441,"avatar_url":"https://avatars.githubusercontent.com/u/7866441?v=3","gravatar_id":"","url":"https://api.github.com/users/schematics","html_url":"https://github.com/schematics","followers_url":"https://api.github.com/users/schematics/followers","following_url":"https://api.github.com/users/schematics/following{/other_user}","gists_url":"https://api.github.com/users/schematics/gists{/gist_id}","starred_url":"https://api.github.com/users/schematics/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/schematics/subscriptions","organizations_url":"https://api.github.com/users/schematics/orgs","repos_url":"https://api.github.com/users/schematics/repos","events_url":"https://api.github.com/users/schematics/events{/privacy}","received_events_url":"https://api.github.com/users/schematics/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/schematics/schematics","description":"Python Data Structures for Humans™.","fork":false,"url":"https://api.github.com/repos/schematics/schematics","forks_url":"https://api.github.com/repos/schematics/schematics/forks","keys_url":"https://api.github.com/repos/schematics/schematics/keys{/key_id}","collaborators_url":"https://api.github.com/repos/schematics/schematics/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/schematics/schematics/teams","hooks_url":"https://api.github.com/repos/schematics/schematics/hooks","issue_events_url":"https://api.github.com/repos/schematics/schematics/issues/events{/number}","events_url":"https://api.github.com/repos/schematics/schematics/events","assignees_url":"https://api.github.com/repos/schematics/schematics/assignees{/user}","branches_url":"https://api.github.com/repos/schematics/schematics/branches{/branch}","tags_url":"https://api.github.com/repos/schematics/schematics/tags","blobs_url":"https://api.github.com/repos/schematics/schematics/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/schematics/schematics/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/schematics/schematics/git/refs{/sha}","trees_url":"https://api.github.com/repos/schematics/schematics/git/trees{/sha}","statuses_url":"https://api.github.com/repos/schematics/schematics/statuses/{sha}","languages_url":"https://api.github.com/repos/schematics/schematics/languages","stargazers_url":"https://api.github.com/repos/schematics/schematics/stargazers","contributors_url":"https://api.github.com/repos/schematics/schematics/contributors","subscribers_url":"https://api.github.com/repos/schematics/schematics/subscribers","subscription_url":"https://api.github.com/repos/schematics/schematics/subscription","commits_url":"https://api.github.com/repos/schematics/schematics/commits{/sha}","git_commits_url":"https://api.github.com/repos/schematics/schematics/git/commits{/sha}","comments_url":"https://api.github.com/repos/schematics/schematics/comments{/number}","issue_comment_url":"https://api.github.com/repos/schematics/schematics/issues/comments{/number}","contents_url":"https://api.github.com/repos/schematics/schematics/contents/{+path}","compare_url":"https://api.github.com/repos/schematics/schematics/compare/{base}...{head}","merges_url":"https://api.github.com/repos/schematics/schematics/merges","archive_url":"https://api.github.com/repos/schematics/schematics/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/schematics/schematics/downloads","issues_url":"https://api.github.com/repos/schematics/schematics/issues{/number}","pulls_url":"https://api.github.com/repos/schematics/schematics/pulls{/number}","milestones_url":"https://api.github.com/repos/schematics/schematics/milestones{/number}","notifications_url":"https://api.github.com/repos/schematics/schematics/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/schematics/schematics/labels{/name}","releases_url":"https://api.github.com/repos/schematics/schematics/releases{/id}","deployments_url":"https://api.github.com/repos/schematics/schematics/deployments","created_at":"2010-12-27T02:25:29Z","updated_at":"2016-02-17T07:32:32Z","pushed_at":"2016-02-16T16:40:41Z","git_url":"git://github.com/schematics/schematics.git","ssh_url":"git@github.com:schematics/schematics.git","clone_url":"https://github.com/schematics/schematics.git","svn_url":"https://github.com/schematics/schematics","homepage":"http://schematics.readthedocs.org/","size":2070,"stargazers_count":1517,"watchers_count":1517,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":195,"mirror_url":null,"open_issues_count":33,"forks":195,"open_issues":33,"watchers":1517,"default_branch":"development"}},"_links":{"self":{"href":"https://api.github.com/repos/schematics/schematics/pulls/392"},"html":{"href":"https://github.com/schematics/schematics/pull/392"},"issue":{"href":"https://api.github.com/repos/schematics/schematics/issues/392"},"comments":{"href":"https://api.github.com/repos/schematics/schematics/issues/392/comments"},"review_comments":{"href":"https://api.github.com/repos/schematics/schematics/pulls/392/comments"},"review_comment":{"href":"https://api.github.com/repos/schematics/schematics/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/schematics/schematics/pulls/392/commits"},"statuses":{"href":"https://api.github.com/repos/schematics/schematics/statuses/8b46a4b2ac0e00662718b68ecd0e5a5bdea79c30"}},"merged":false,"mergeable":null,"mergeable_state":"unknown","merged_by":null,"comments":0,"review_comments":0,"commits":7,"additions":6293,"deletions":1,"changed_files":9}}