{"action":"created","issue":{"url":"https://api.github.com/repos/coreos/etcd/issues/5186","repository_url":"https://api.github.com/repos/coreos/etcd","labels_url":"https://api.github.com/repos/coreos/etcd/issues/5186/labels{/name}","comments_url":"https://api.github.com/repos/coreos/etcd/issues/5186/comments","events_url":"https://api.github.com/repos/coreos/etcd/issues/5186/events","html_url":"https://github.com/coreos/etcd/issues/5186","id":150961915,"number":5186,"title":"Evaluate compression layer","user":{"login":"gyuho","id":6799218,"avatar_url":"https://avatars.githubusercontent.com/u/6799218?v=3","gravatar_id":"","url":"https://api.github.com/users/gyuho","html_url":"https://github.com/gyuho","followers_url":"https://api.github.com/users/gyuho/followers","following_url":"https://api.github.com/users/gyuho/following{/other_user}","gists_url":"https://api.github.com/users/gyuho/gists{/gist_id}","starred_url":"https://api.github.com/users/gyuho/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gyuho/subscriptions","organizations_url":"https://api.github.com/users/gyuho/orgs","repos_url":"https://api.github.com/users/gyuho/repos","events_url":"https://api.github.com/users/gyuho/events{/privacy}","received_events_url":"https://api.github.com/users/gyuho/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/coreos/etcd/labels/kind/enhancement","name":"kind/enhancement","color":"c7def8"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-04-25T19:45:06Z","updated_at":"2016-05-02T03:08:06Z","closed_at":null,"body":"#### 2016.04.26 ~\r\n\r\n- [X] try other different compression algorithms\r\n  - [X] no compression\r\n  - [X] [`github.com/bkaradzic/go-lz4`](https://godoc.org/github.com/bkaradzic/go-lz4)\r\n  - [X] [`github.com/cloudflare/golz4`](https://godoc.org/github.com/cloudflare/golz4)\r\n  - [X] [`github.com/golang/snappy`](https://godoc.org/github.com/golang/snappy)\r\n  - [X] [`github.com/klauspost/compress/snappy`](https://godoc.org/github.com/klauspost/compress/snappy)\r\n  - [X] [`compress/gzip`](https://golang.org/pkg/compress/gzip) with different compression levels\r\n  - [X] [`github.com/klauspost/compress/gzip`](https://godoc.org/github.com/klauspost/compress/gzip) with different compression levels\r\n  - [X] [`compress/zlib`](https://golang.org/pkg/compress/zlib/)\r\n  - [X] [`github.com/klauspost/compress/zlib`](https://godoc.org/github.com/klauspost/compress/zlib)\r\n  - [X] [`compress/bzip2`](https://golang.org/pkg/compress/bzip2/)\r\n  - [X] evaluate `sync.Pool` performance effect\r\n  - [X] measure compression rate\r\n- [X] maximize CPU usage\r\n![03-avg-cpu](https://cloud.githubusercontent.com/assets/6799218/14796278/5f9fbc16-0ae1-11e6-8b4e-0819623705a4.png)\r\n- try different compression path\r\n  - [X] gRPC layer\r\n  - [X] rafthttp layer\r\n  - [ ] storage layer\r\n\r\n<br><br><hr>\r\n#### 2016.04.25\r\n\r\nhttps://github.com/coreos/etcd/pull/5183\r\n\r\n![01-avg-cpu](https://cloud.githubusercontent.com/assets/6799218/14796246/33001d4a-0ae1-11e6-9690-ffa2e09bd168.png)\r\n![01-avg-latency-ms](https://cloud.githubusercontent.com/assets/6799218/14796249/330190d0-0ae1-11e6-8c59-1ef9ebb28d79.png)\r\n![01-avg-memory](https://cloud.githubusercontent.com/assets/6799218/14796247/33007c04-0ae1-11e6-9be7-52923f8f2c5e.png)\r\n![01-throughput](https://cloud.githubusercontent.com/assets/6799218/14796248/3300b87c-0ae1-11e6-9054-6517fee51b0e.png)\r\n\r\n```\r\n# fastest to the right\r\ngzip < cgzip < cgzip level 2 < snappy C++ < snappy (fastest) == no compression\r\n# cgzip, snappy C++ need CGO, so we might skip them\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/coreos/etcd/issues/comments/216097209","html_url":"https://github.com/coreos/etcd/issues/5186#issuecomment-216097209","issue_url":"https://api.github.com/repos/coreos/etcd/issues/5186","id":216097209,"user":{"login":"xiang90","id":4479947,"avatar_url":"https://avatars.githubusercontent.com/u/4479947?v=3","gravatar_id":"","url":"https://api.github.com/users/xiang90","html_url":"https://github.com/xiang90","followers_url":"https://api.github.com/users/xiang90/followers","following_url":"https://api.github.com/users/xiang90/following{/other_user}","gists_url":"https://api.github.com/users/xiang90/gists{/gist_id}","starred_url":"https://api.github.com/users/xiang90/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/xiang90/subscriptions","organizations_url":"https://api.github.com/users/xiang90/orgs","repos_url":"https://api.github.com/users/xiang90/repos","events_url":"https://api.github.com/users/xiang90/events{/privacy}","received_events_url":"https://api.github.com/users/xiang90/received_events","type":"User","site_admin":false},"created_at":"2016-05-02T03:08:06Z","updated_at":"2016-05-02T03:08:06Z","body":"@gyuho Then we should not make etcd overloaded. Or the benchmark does not make really sense to me. In another word, we need to understand why adding compression overloads etcd server. It seems the CPU load actually also drops."}}