{"action":"created","issue":{"url":"https://api.github.com/repos/MariaDB/server/issues/261","repository_url":"https://api.github.com/repos/MariaDB/server","labels_url":"https://api.github.com/repos/MariaDB/server/issues/261/labels{/name}","comments_url":"https://api.github.com/repos/MariaDB/server/issues/261/comments","events_url":"https://api.github.com/repos/MariaDB/server/issues/261/events","html_url":"https://github.com/MariaDB/server/pull/261","id":192177551,"number":261,"title":"MDEV-11371 - Big column compressed(innodb)","user":{"login":"GCSAdmin","id":1744964,"avatar_url":"https://avatars.githubusercontent.com/u/1744964?v=3","gravatar_id":"","url":"https://api.github.com/users/GCSAdmin","html_url":"https://github.com/GCSAdmin","followers_url":"https://api.github.com/users/GCSAdmin/followers","following_url":"https://api.github.com/users/GCSAdmin/following{/other_user}","gists_url":"https://api.github.com/users/GCSAdmin/gists{/gist_id}","starred_url":"https://api.github.com/users/GCSAdmin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/GCSAdmin/subscriptions","organizations_url":"https://api.github.com/users/GCSAdmin/orgs","repos_url":"https://api.github.com/users/GCSAdmin/repos","events_url":"https://api.github.com/users/GCSAdmin/events{/privacy}","received_events_url":"https://api.github.com/users/GCSAdmin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"svoj","id":7700472,"avatar_url":"https://avatars.githubusercontent.com/u/7700472?v=3","gravatar_id":"","url":"https://api.github.com/users/svoj","html_url":"https://github.com/svoj","followers_url":"https://api.github.com/users/svoj/followers","following_url":"https://api.github.com/users/svoj/following{/other_user}","gists_url":"https://api.github.com/users/svoj/gists{/gist_id}","starred_url":"https://api.github.com/users/svoj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/svoj/subscriptions","organizations_url":"https://api.github.com/users/svoj/orgs","repos_url":"https://api.github.com/users/svoj/repos","events_url":"https://api.github.com/users/svoj/events{/privacy}","received_events_url":"https://api.github.com/users/svoj/received_events","type":"User","site_admin":false},"assignees":[{"login":"svoj","id":7700472,"avatar_url":"https://avatars.githubusercontent.com/u/7700472?v=3","gravatar_id":"","url":"https://api.github.com/users/svoj","html_url":"https://github.com/svoj","followers_url":"https://api.github.com/users/svoj/followers","following_url":"https://api.github.com/users/svoj/following{/other_user}","gists_url":"https://api.github.com/users/svoj/gists{/gist_id}","starred_url":"https://api.github.com/users/svoj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/svoj/subscriptions","organizations_url":"https://api.github.com/users/svoj/orgs","repos_url":"https://api.github.com/users/svoj/repos","events_url":"https://api.github.com/users/svoj/events{/privacy}","received_events_url":"https://api.github.com/users/svoj/received_events","type":"User","site_admin":false}],"milestone":null,"comments":6,"created_at":"2016-11-29T03:16:06Z","updated_at":"2016-11-29T09:00:42Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/MariaDB/server/pulls/261","html_url":"https://github.com/MariaDB/server/pull/261","diff_url":"https://github.com/MariaDB/server/pull/261.diff","patch_url":"https://github.com/MariaDB/server/pull/261.patch"},"body":"  Some big columns(blob/text/varchar/varbinary) waste a lot of space, we introduce “compressed” into column definition when create or alter a table. \r\n  When a column was defined as a compressed column, the column data will be compressed using zlib (other compress algorithm not support yet). \r\n  We could get a better compression ratio and performance, more flexibility (vs compressed row format)\r\n  For example:\r\n    Create table tcompress (\r\n            C1 int,\r\n            C2 blob compressed,\r\n            C3 text  compressed,\r\n\t\t            C4 text) engine = innodb\r\n\r\n  We achieve this 'big columns compress' function by following step:\r\n  1. Support 'compressed' syntax, and save this attribute in .frm file \r\n  2. Store the 'compressed' attribute in innodb layer, we add DATA_IS_COMPRESSED flag in prtype\r\n  3. If needed , we do compress in row_mysql_store_col_in_innobase_format and do decompress in row_sel_store_mysql_field\r\n  4. Use a compress header to control how to compress/decompress.\r\n    Compress Header is 1 Byte, \r\n    7 Bit: Always 1, mean compressed; \r\n    5-6 Bit: Compressed algorithm - Always 0, means zlib.It maybe support other compression algorithm in the future.\r\n    0-3 Bit: Bytes of \"Record Original Length\"\r\n    Record Original Length: 1-4 Bytes*/\r\n \r\n  We add system global variable 'field_compress_min_len' was used to control that only compress the column if the data length exceeds 'field_compress_min_len'. Default 128.\r\n  Also we add 3 error num: \r\n  ER_FIELD_TYPE_NOT_ALLOWED_AS_COMPRESSED_FIELD: support text/blob/varchar/varbinary column has compress attribute only.\r\n  ER_FIELD_CAN_NOT_COMPRESSED_AND_INDEX: column has compress attribute can not be an index\r\n  ER_FIELD_CAN_NOT_COMPRESSED_IN_CURRENT_ENGINESS: column compress be supported in innodb only\r\n"},"comment":{"url":"https://api.github.com/repos/MariaDB/server/issues/comments/263513560","html_url":"https://github.com/MariaDB/server/pull/261#issuecomment-263513560","issue_url":"https://api.github.com/repos/MariaDB/server/issues/261","id":263513560,"user":{"login":"janlindstrom","id":6338705,"avatar_url":"https://avatars.githubusercontent.com/u/6338705?v=3","gravatar_id":"","url":"https://api.github.com/users/janlindstrom","html_url":"https://github.com/janlindstrom","followers_url":"https://api.github.com/users/janlindstrom/followers","following_url":"https://api.github.com/users/janlindstrom/following{/other_user}","gists_url":"https://api.github.com/users/janlindstrom/gists{/gist_id}","starred_url":"https://api.github.com/users/janlindstrom/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/janlindstrom/subscriptions","organizations_url":"https://api.github.com/users/janlindstrom/orgs","repos_url":"https://api.github.com/users/janlindstrom/repos","events_url":"https://api.github.com/users/janlindstrom/events{/privacy}","received_events_url":"https://api.github.com/users/janlindstrom/received_events","type":"User","site_admin":false},"created_at":"2016-11-29T09:00:42Z","updated_at":"2016-11-29T09:00:42Z","body":"Thank you for your contribution, however I would implement compressed columns in upper layer not inside InnoDB. Benefits would include \r\n\r\n- Reduced payload provided to InnoDB (and possible other storage engines supporting this)\r\n- Reduced payload provided out from InnoDB \r\n- This would also work for replication i.e. reduced payload on binary log ad column would be compressed on binary log\r\n- Do we really need to uncompress these columns always to InnoDB buffer pool ?\r\n"}}
{"actor":{"display_login":"janlindstrom"}}