{"action":"closed","issue":{"url":"https://api.github.com/repos/mre/hyperjson/issues/18","repository_url":"https://api.github.com/repos/mre/hyperjson","labels_url":"https://api.github.com/repos/mre/hyperjson/issues/18/labels{/name}","comments_url":"https://api.github.com/repos/mre/hyperjson/issues/18/comments","events_url":"https://api.github.com/repos/mre/hyperjson/issues/18/events","html_url":"https://github.com/mre/hyperjson/issues/18","id":338302380,"node_id":"MDU6SXNzdWUzMzgzMDIzODA=","number":18,"title":"Port Python benchmark to pytest-benchmark","user":{"login":"mre","id":175809,"node_id":"MDQ6VXNlcjE3NTgwOQ==","avatar_url":"https://avatars2.githubusercontent.com/u/175809?v=4","gravatar_id":"","url":"https://api.github.com/users/mre","html_url":"https://github.com/mre","followers_url":"https://api.github.com/users/mre/followers","following_url":"https://api.github.com/users/mre/following{/other_user}","gists_url":"https://api.github.com/users/mre/gists{/gist_id}","starred_url":"https://api.github.com/users/mre/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mre/subscriptions","organizations_url":"https://api.github.com/users/mre/orgs","repos_url":"https://api.github.com/users/mre/repos","events_url":"https://api.github.com/users/mre/events{/privacy}","received_events_url":"https://api.github.com/users/mre/received_events","type":"User","site_admin":false},"labels":[{"id":905789459,"node_id":"MDU6TGFiZWw5MDU3ODk0NTk=","url":"https://api.github.com/repos/mre/hyperjson/labels/enhancement","name":"enhancement","color":"a2eeef","default":true},{"id":905789461,"node_id":"MDU6TGFiZWw5MDU3ODk0NjE=","url":"https://api.github.com/repos/mre/hyperjson/labels/good%20first%20issue","name":"good first issue","color":"7057ff","default":true}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-07-04T14:43:23Z","updated_at":"2018-07-07T12:41:50Z","closed_at":"2018-07-07T12:41:50Z","author_association":"OWNER","body":"Our current Python benchmarks are based on `ujson's` script [here](https://github.com/esnme/ultrajson/blob/master/tests/benchmark.py).\r\nIt's a bit clunky to use and not really idiomatic (redundant code, custom implementation for printing the results as a table).\r\nIn addition to that, it's hard to make code changes and see the performance impact, because the measurements fluctuate a lot from run to run.\r\n\r\nA modern alternative is [pytest-benchmark](https://pytest-benchmark.readthedocs.io/en/latest/). Since we already use pytest, it's would fit in well with the rest of our testing pipeline.\r\nIt provides powerful output methods for different use-cases:\r\n\r\n![grafik](https://user-images.githubusercontent.com/175809/42283153-e46248b8-7fa8-11e8-8159-34512cd8b57e.png)\r\n\r\n![grafik](https://user-images.githubusercontent.com/175809/42283159-e78e0b62-7fa8-11e8-8a86-b0d5f948ada6.png)\r\n\r\nIt can also plot histograms to visualize the error bars as a box-plot.\r\n\r\n![grafik](https://user-images.githubusercontent.com/175809/42283274-44445c62-7fa9-11e8-8430-238716c7017d.png)\r\n\r\nIt would be nice to move to pytest-benchmark as we could compare the measurements after each change using Travis CI.\r\n\r\n"}}
{"actor":{"display_login":"mre"}}