{"action":"created","comment":{"url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments/186269956","pull_request_review_id":117822739,"id":186269956,"diff_hunk":"@@ -115,11 +115,28 @@ zio_compress_data(enum zio_compress c, abd_t *src, void *dst, size_t s_len)\n \tif (c == ZIO_COMPRESS_EMPTY)\n \t\treturn (s_len);\n \n+\t/* No compression algorithms can read from ABDs directly */\n+\tvoid *tmp = abd_borrow_buf_copy(src, s_len);\n+\n+\t/* for gzip 6+, use lz4 as a heuristic for compressibility */\n+\tif (ci->ci_compress == gzip_compress && ci->ci_level >= 6) {\n+\t    /* lz4 must find some compression */\n+            if (s_len) {\n+                d_len = s_len - 1;\n+            } else {\n+                d_len = s_len;\n+            }","path":"module/zfs/zio_compress.c","position":14,"original_position":14,"commit_id":"de4a0c31b5d269e39114181465ddbba448a4db3f","original_commit_id":"de4a0c31b5d269e39114181465ddbba448a4db3f","user":{"login":"RubenKelevra","id":614929,"avatar_url":"https://avatars0.githubusercontent.com/u/614929?v=4","gravatar_id":"","url":"https://api.github.com/users/RubenKelevra","html_url":"https://github.com/RubenKelevra","followers_url":"https://api.github.com/users/RubenKelevra/followers","following_url":"https://api.github.com/users/RubenKelevra/following{/other_user}","gists_url":"https://api.github.com/users/RubenKelevra/gists{/gist_id}","starred_url":"https://api.github.com/users/RubenKelevra/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/RubenKelevra/subscriptions","organizations_url":"https://api.github.com/users/RubenKelevra/orgs","repos_url":"https://api.github.com/users/RubenKelevra/repos","events_url":"https://api.github.com/users/RubenKelevra/events{/privacy}","received_events_url":"https://api.github.com/users/RubenKelevra/received_events","type":"User","site_admin":false},"body":"@behlendorf this need also an additional check, if the data-size before compression is actually smaller than one sector size, since it might compress with gzip that well, that it can be stored in the block pointer by the embedded_data, right?","created_at":"2018-05-05T19:19:21Z","updated_at":"2018-05-05T19:19:21Z","html_url":"https://github.com/zfsonlinux/zfs/pull/7481#discussion_r186269956","pull_request_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481","author_association":"NONE","_links":{"self":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments/186269956"},"html":{"href":"https://github.com/zfsonlinux/zfs/pull/7481#discussion_r186269956"},"pull_request":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481"}},"in_reply_to_id":185165706},"pull_request":{"url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481","id":184511467,"html_url":"https://github.com/zfsonlinux/zfs/pull/7481","diff_url":"https://github.com/zfsonlinux/zfs/pull/7481.diff","patch_url":"https://github.com/zfsonlinux/zfs/pull/7481.patch","issue_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/7481","number":7481,"state":"open","locked":false,"title":"[WIP] gzip compression heuristic","user":{"login":"rbrewer123","id":743058,"avatar_url":"https://avatars3.githubusercontent.com/u/743058?v=4","gravatar_id":"","url":"https://api.github.com/users/rbrewer123","html_url":"https://github.com/rbrewer123","followers_url":"https://api.github.com/users/rbrewer123/followers","following_url":"https://api.github.com/users/rbrewer123/following{/other_user}","gists_url":"https://api.github.com/users/rbrewer123/gists{/gist_id}","starred_url":"https://api.github.com/users/rbrewer123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbrewer123/subscriptions","organizations_url":"https://api.github.com/users/rbrewer123/orgs","repos_url":"https://api.github.com/users/rbrewer123/repos","events_url":"https://api.github.com/users/rbrewer123/events{/privacy}","received_events_url":"https://api.github.com/users/rbrewer123/received_events","type":"User","site_admin":false},"body":"<!--- Provide a general summary of your changes in the Title above -->\r\n\r\n<!---\r\nDocumentation on ZFS Buildbot options can be found at\r\nhttps://github.com/zfsonlinux/zfs/wiki/Buildbot-Options\r\n-->\r\n\r\n### Description\r\n<!--- Describe your changes in detail -->\r\nThis change uses lz4 as a heuristic to decide if a block is worth compressing with gzip levels 6-9.  If lz4 provides any compression at all, gzip is attempted. Otherwise, the block is assumed incompressible and gzip is skipped.  \r\n\r\n### Motivation and Context\r\n<!--- Why is this change required? What problem does it solve? -->\r\n<!--- If it fixes an open issue, please link to the issue here. -->\r\nThe goal of this change is to improve gzip compression speed for incompressible data, without significantly affecting performance on compressible data. #7373 provides additional discussion and rationale.\r\n\r\n### How Has This Been Tested?\r\n<!--- Please describe in detail how you tested your changes. -->\r\n<!--- Include details of your testing environment, and the tests you ran to -->\r\n<!--- see how your change affects other areas of the code, etc. -->\r\n<!--- If your change is a performance enhancement, please provide benchmarks here. -->\r\nI haven't tested this yet. I will add benchmarks here as I test this. I encourage others with particular workloads to post their own benchmarks.\r\n\r\nThis change does not properly deal with the QAT hardware compression yet.  It might be that if this heuristic check is viable, it should go in gzip_compress(}.  The issue with that is `d_len` has already been reduced by 12.5%. I could use `s_len` instead, but it feels error prone.  For now I'm posting this for benchmarking purposes.\r\n\r\n### Types of changes\r\n<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->\r\n- [ ] Bug fix (non-breaking change which fixes an issue)\r\n- [ ] New feature (non-breaking change which adds functionality)\r\n- [x] Performance enhancement (non-breaking change which improves efficiency)\r\n- [ ] Code cleanup (non-breaking change which makes code smaller or more readable)\r\n- [ ] Breaking change (fix or feature that would cause existing functionality to change)\r\n- [ ] Documentation (a change to man pages or other documentation)\r\n\r\n### Checklist:\r\n<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->\r\n<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->\r\n- [ ] My code follows the ZFS on Linux code style requirements.\r\n- [ ] I have updated the documentation accordingly.\r\n- [ ] I have read the **CONTRIBUTING** document.\r\n- [ ] I have added tests to cover my changes.\r\n- [ ] All new and existing tests passed.\r\n- [ ] All commit messages are properly formatted and contain `Signed-off-by`.\r\n- [ ] Change has been approved by a ZFS on Linux member.\r\n","created_at":"2018-04-27T02:49:49Z","updated_at":"2018-05-05T19:19:21Z","closed_at":null,"merged_at":null,"merge_commit_sha":"49000187805b5d573bbd9b97266d9b13b8166eea","assignee":null,"assignees":[],"requested_reviewers":[],"requested_teams":[],"labels":[],"milestone":null,"commits_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481/commits","review_comments_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481/comments","review_comment_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments{/number}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/7481/comments","statuses_url":"https://api.github.com/repos/zfsonlinux/zfs/statuses/de4a0c31b5d269e39114181465ddbba448a4db3f","head":{"label":"rbrewer123:feature-compress-heuristic","ref":"feature-compress-heuristic","sha":"de4a0c31b5d269e39114181465ddbba448a4db3f","user":{"login":"rbrewer123","id":743058,"avatar_url":"https://avatars3.githubusercontent.com/u/743058?v=4","gravatar_id":"","url":"https://api.github.com/users/rbrewer123","html_url":"https://github.com/rbrewer123","followers_url":"https://api.github.com/users/rbrewer123/followers","following_url":"https://api.github.com/users/rbrewer123/following{/other_user}","gists_url":"https://api.github.com/users/rbrewer123/gists{/gist_id}","starred_url":"https://api.github.com/users/rbrewer123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbrewer123/subscriptions","organizations_url":"https://api.github.com/users/rbrewer123/orgs","repos_url":"https://api.github.com/users/rbrewer123/repos","events_url":"https://api.github.com/users/rbrewer123/events{/privacy}","received_events_url":"https://api.github.com/users/rbrewer123/received_events","type":"User","site_admin":false},"repo":{"id":130348384,"name":"zfs","full_name":"rbrewer123/zfs","owner":{"login":"rbrewer123","id":743058,"avatar_url":"https://avatars3.githubusercontent.com/u/743058?v=4","gravatar_id":"","url":"https://api.github.com/users/rbrewer123","html_url":"https://github.com/rbrewer123","followers_url":"https://api.github.com/users/rbrewer123/followers","following_url":"https://api.github.com/users/rbrewer123/following{/other_user}","gists_url":"https://api.github.com/users/rbrewer123/gists{/gist_id}","starred_url":"https://api.github.com/users/rbrewer123/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rbrewer123/subscriptions","organizations_url":"https://api.github.com/users/rbrewer123/orgs","repos_url":"https://api.github.com/users/rbrewer123/repos","events_url":"https://api.github.com/users/rbrewer123/events{/privacy}","received_events_url":"https://api.github.com/users/rbrewer123/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/rbrewer123/zfs","description":"ZFS on Linux - the official OpenZFS implementation for Linux.","fork":true,"url":"https://api.github.com/repos/rbrewer123/zfs","forks_url":"https://api.github.com/repos/rbrewer123/zfs/forks","keys_url":"https://api.github.com/repos/rbrewer123/zfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/rbrewer123/zfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/rbrewer123/zfs/teams","hooks_url":"https://api.github.com/repos/rbrewer123/zfs/hooks","issue_events_url":"https://api.github.com/repos/rbrewer123/zfs/issues/events{/number}","events_url":"https://api.github.com/repos/rbrewer123/zfs/events","assignees_url":"https://api.github.com/repos/rbrewer123/zfs/assignees{/user}","branches_url":"https://api.github.com/repos/rbrewer123/zfs/branches{/branch}","tags_url":"https://api.github.com/repos/rbrewer123/zfs/tags","blobs_url":"https://api.github.com/repos/rbrewer123/zfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/rbrewer123/zfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/rbrewer123/zfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/rbrewer123/zfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/rbrewer123/zfs/statuses/{sha}","languages_url":"https://api.github.com/repos/rbrewer123/zfs/languages","stargazers_url":"https://api.github.com/repos/rbrewer123/zfs/stargazers","contributors_url":"https://api.github.com/repos/rbrewer123/zfs/contributors","subscribers_url":"https://api.github.com/repos/rbrewer123/zfs/subscribers","subscription_url":"https://api.github.com/repos/rbrewer123/zfs/subscription","commits_url":"https://api.github.com/repos/rbrewer123/zfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/rbrewer123/zfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/rbrewer123/zfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/rbrewer123/zfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/rbrewer123/zfs/contents/{+path}","compare_url":"https://api.github.com/repos/rbrewer123/zfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/rbrewer123/zfs/merges","archive_url":"https://api.github.com/repos/rbrewer123/zfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/rbrewer123/zfs/downloads","issues_url":"https://api.github.com/repos/rbrewer123/zfs/issues{/number}","pulls_url":"https://api.github.com/repos/rbrewer123/zfs/pulls{/number}","milestones_url":"https://api.github.com/repos/rbrewer123/zfs/milestones{/number}","notifications_url":"https://api.github.com/repos/rbrewer123/zfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/rbrewer123/zfs/labels{/name}","releases_url":"https://api.github.com/repos/rbrewer123/zfs/releases{/id}","deployments_url":"https://api.github.com/repos/rbrewer123/zfs/deployments","created_at":"2018-04-20T10:37:28Z","updated_at":"2018-04-20T10:37:37Z","pushed_at":"2018-04-25T02:20:22Z","git_url":"git://github.com/rbrewer123/zfs.git","ssh_url":"git@github.com:rbrewer123/zfs.git","clone_url":"https://github.com/rbrewer123/zfs.git","svn_url":"https://github.com/rbrewer123/zfs","homepage":"http://zfsonlinux.org/","size":46242,"stargazers_count":0,"watchers_count":0,"language":"C","has_issues":false,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"archived":false,"open_issues_count":0,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"zfsonlinux:master","ref":"master","sha":"089500e79234c9cb741090ade96baec266f0ace4","user":{"login":"zfsonlinux","id":575083,"avatar_url":"https://avatars3.githubusercontent.com/u/575083?v=4","gravatar_id":"","url":"https://api.github.com/users/zfsonlinux","html_url":"https://github.com/zfsonlinux","followers_url":"https://api.github.com/users/zfsonlinux/followers","following_url":"https://api.github.com/users/zfsonlinux/following{/other_user}","gists_url":"https://api.github.com/users/zfsonlinux/gists{/gist_id}","starred_url":"https://api.github.com/users/zfsonlinux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zfsonlinux/subscriptions","organizations_url":"https://api.github.com/users/zfsonlinux/orgs","repos_url":"https://api.github.com/users/zfsonlinux/repos","events_url":"https://api.github.com/users/zfsonlinux/events{/privacy}","received_events_url":"https://api.github.com/users/zfsonlinux/received_events","type":"Organization","site_admin":false},"repo":{"id":437011,"name":"zfs","full_name":"zfsonlinux/zfs","owner":{"login":"zfsonlinux","id":575083,"avatar_url":"https://avatars3.githubusercontent.com/u/575083?v=4","gravatar_id":"","url":"https://api.github.com/users/zfsonlinux","html_url":"https://github.com/zfsonlinux","followers_url":"https://api.github.com/users/zfsonlinux/followers","following_url":"https://api.github.com/users/zfsonlinux/following{/other_user}","gists_url":"https://api.github.com/users/zfsonlinux/gists{/gist_id}","starred_url":"https://api.github.com/users/zfsonlinux/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zfsonlinux/subscriptions","organizations_url":"https://api.github.com/users/zfsonlinux/orgs","repos_url":"https://api.github.com/users/zfsonlinux/repos","events_url":"https://api.github.com/users/zfsonlinux/events{/privacy}","received_events_url":"https://api.github.com/users/zfsonlinux/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/zfsonlinux/zfs","description":"ZFS on Linux - the official OpenZFS implementation for Linux.","fork":false,"url":"https://api.github.com/repos/zfsonlinux/zfs","forks_url":"https://api.github.com/repos/zfsonlinux/zfs/forks","keys_url":"https://api.github.com/repos/zfsonlinux/zfs/keys{/key_id}","collaborators_url":"https://api.github.com/repos/zfsonlinux/zfs/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/zfsonlinux/zfs/teams","hooks_url":"https://api.github.com/repos/zfsonlinux/zfs/hooks","issue_events_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/events{/number}","events_url":"https://api.github.com/repos/zfsonlinux/zfs/events","assignees_url":"https://api.github.com/repos/zfsonlinux/zfs/assignees{/user}","branches_url":"https://api.github.com/repos/zfsonlinux/zfs/branches{/branch}","tags_url":"https://api.github.com/repos/zfsonlinux/zfs/tags","blobs_url":"https://api.github.com/repos/zfsonlinux/zfs/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/zfsonlinux/zfs/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/zfsonlinux/zfs/git/refs{/sha}","trees_url":"https://api.github.com/repos/zfsonlinux/zfs/git/trees{/sha}","statuses_url":"https://api.github.com/repos/zfsonlinux/zfs/statuses/{sha}","languages_url":"https://api.github.com/repos/zfsonlinux/zfs/languages","stargazers_url":"https://api.github.com/repos/zfsonlinux/zfs/stargazers","contributors_url":"https://api.github.com/repos/zfsonlinux/zfs/contributors","subscribers_url":"https://api.github.com/repos/zfsonlinux/zfs/subscribers","subscription_url":"https://api.github.com/repos/zfsonlinux/zfs/subscription","commits_url":"https://api.github.com/repos/zfsonlinux/zfs/commits{/sha}","git_commits_url":"https://api.github.com/repos/zfsonlinux/zfs/git/commits{/sha}","comments_url":"https://api.github.com/repos/zfsonlinux/zfs/comments{/number}","issue_comment_url":"https://api.github.com/repos/zfsonlinux/zfs/issues/comments{/number}","contents_url":"https://api.github.com/repos/zfsonlinux/zfs/contents/{+path}","compare_url":"https://api.github.com/repos/zfsonlinux/zfs/compare/{base}...{head}","merges_url":"https://api.github.com/repos/zfsonlinux/zfs/merges","archive_url":"https://api.github.com/repos/zfsonlinux/zfs/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/zfsonlinux/zfs/downloads","issues_url":"https://api.github.com/repos/zfsonlinux/zfs/issues{/number}","pulls_url":"https://api.github.com/repos/zfsonlinux/zfs/pulls{/number}","milestones_url":"https://api.github.com/repos/zfsonlinux/zfs/milestones{/number}","notifications_url":"https://api.github.com/repos/zfsonlinux/zfs/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/zfsonlinux/zfs/labels{/name}","releases_url":"https://api.github.com/repos/zfsonlinux/zfs/releases{/id}","deployments_url":"https://api.github.com/repos/zfsonlinux/zfs/deployments","created_at":"2009-12-14T20:20:34Z","updated_at":"2018-05-05T11:57:01Z","pushed_at":"2018-05-05T18:53:36Z","git_url":"git://github.com/zfsonlinux/zfs.git","ssh_url":"git@github.com:zfsonlinux/zfs.git","clone_url":"https://github.com/zfsonlinux/zfs.git","svn_url":"https://github.com/zfsonlinux/zfs","homepage":"http://zfsonlinux.org/","size":46387,"stargazers_count":3257,"watchers_count":3257,"language":"C","has_issues":true,"has_projects":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":757,"mirror_url":null,"archived":false,"open_issues_count":946,"license":{"key":"other","name":"Other","spdx_id":null,"url":null},"forks":757,"open_issues":946,"watchers":3257,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481"},"html":{"href":"https://github.com/zfsonlinux/zfs/pull/7481"},"issue":{"href":"https://api.github.com/repos/zfsonlinux/zfs/issues/7481"},"comments":{"href":"https://api.github.com/repos/zfsonlinux/zfs/issues/7481/comments"},"review_comments":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481/comments"},"review_comment":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/zfsonlinux/zfs/pulls/7481/commits"},"statuses":{"href":"https://api.github.com/repos/zfsonlinux/zfs/statuses/de4a0c31b5d269e39114181465ddbba448a4db3f"}},"author_association":"NONE"}}
{"actor":{"display_login":"RubenKelevra"}}