{"action":"created","issue":{"url":"https://api.github.com/repos/vmprof/vmprof-python/issues/71","repository_url":"https://api.github.com/repos/vmprof/vmprof-python","labels_url":"https://api.github.com/repos/vmprof/vmprof-python/issues/71/labels{/name}","comments_url":"https://api.github.com/repos/vmprof/vmprof-python/issues/71/comments","events_url":"https://api.github.com/repos/vmprof/vmprof-python/issues/71/events","html_url":"https://github.com/vmprof/vmprof-python/pull/71","id":154669936,"number":71,"title":"Gzip profile files","user":{"login":"jonashaag","id":175722,"avatar_url":"https://avatars.githubusercontent.com/u/175722?v=3","gravatar_id":"","url":"https://api.github.com/users/jonashaag","html_url":"https://github.com/jonashaag","followers_url":"https://api.github.com/users/jonashaag/followers","following_url":"https://api.github.com/users/jonashaag/following{/other_user}","gists_url":"https://api.github.com/users/jonashaag/gists{/gist_id}","starred_url":"https://api.github.com/users/jonashaag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonashaag/subscriptions","organizations_url":"https://api.github.com/users/jonashaag/orgs","repos_url":"https://api.github.com/users/jonashaag/repos","events_url":"https://api.github.com/users/jonashaag/events{/privacy}","received_events_url":"https://api.github.com/users/jonashaag/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-05-13T09:26:37Z","updated_at":"2016-06-03T09:31:32Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/vmprof/vmprof-python/pulls/71","html_url":"https://github.com/vmprof/vmprof-python/pull/71","diff_url":"https://github.com/vmprof/vmprof-python/pull/71.diff","patch_url":"https://github.com/vmprof/vmprof-python/pull/71.patch"},"body":"Depends on #70 \r\n\r\nGzip profile files, reduces file size up to 97%. Backwards compatible. Increases parse duration by about 5-10%. (Note that my other PR reduces parse duration by 150-250%.)\r\n\r\nI also tried other file size reduction techniques:\r\n\r\n- Use 16-bit integer for instruction pointers instead of 32-bit ones (`short` instead of `long`). Map 32-bit IPs to 16-bit IPs using an associative array when writing the profile. This works well and isn't too difficult to implement. It cuts profile file size up to 50%.\r\n- Count repeated IPs within a trace (recursion etc.) and replace by a `(IP, count)` tuple. Easy to implement, saves 10% in my benchmarks (depends on the program that is being profiled of course)\r\n\r\nAn idea I did not look into is de-duplicating whole traces, i.e. for each trace look if an equivalent trace has already been written and if so only use a pointer to that other trace.\r\n\r\nGzipping everything is probably the easiest and most robust solution to reducing file sizes."},"comment":{"url":"https://api.github.com/repos/vmprof/vmprof-python/issues/comments/223534087","html_url":"https://github.com/vmprof/vmprof-python/pull/71#issuecomment-223534087","issue_url":"https://api.github.com/repos/vmprof/vmprof-python/issues/71","id":223534087,"user":{"login":"methane","id":199592,"avatar_url":"https://avatars.githubusercontent.com/u/199592?v=3","gravatar_id":"","url":"https://api.github.com/users/methane","html_url":"https://github.com/methane","followers_url":"https://api.github.com/users/methane/followers","following_url":"https://api.github.com/users/methane/following{/other_user}","gists_url":"https://api.github.com/users/methane/gists{/gist_id}","starred_url":"https://api.github.com/users/methane/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/methane/subscriptions","organizations_url":"https://api.github.com/users/methane/orgs","repos_url":"https://api.github.com/users/methane/repos","events_url":"https://api.github.com/users/methane/events{/privacy}","received_events_url":"https://api.github.com/users/methane/received_events","type":"User","site_admin":false},"created_at":"2016-06-03T09:31:32Z","updated_at":"2016-06-03T09:31:32Z","body":"gzip output may affects sigprof.\r\nI think piping to outer gzip (or lzop, lz4c, etc...) is better for accurate profiling."}}
{"actor":{"display_login":"methane"}}