{"action":"closed","issue":{"url":"https://api.github.com/repos/facebook/zstd/issues/1385","repository_url":"https://api.github.com/repos/facebook/zstd","labels_url":"https://api.github.com/repos/facebook/zstd/issues/1385/labels{/name}","comments_url":"https://api.github.com/repos/facebook/zstd/issues/1385/comments","events_url":"https://api.github.com/repos/facebook/zstd/issues/1385/events","html_url":"https://github.com/facebook/zstd/issues/1385","id":372753599,"node_id":"MDU6SXNzdWUzNzI3NTM1OTk=","number":1385,"title":"ZSTD_decompress(NULL, 0, ...) returns -ZSTD_error_dstSize_tooSmall","user":{"login":"veprbl","id":245573,"node_id":"MDQ6VXNlcjI0NTU3Mw==","avatar_url":"https://avatars1.githubusercontent.com/u/245573?v=4","gravatar_id":"","url":"https://api.github.com/users/veprbl","html_url":"https://github.com/veprbl","followers_url":"https://api.github.com/users/veprbl/followers","following_url":"https://api.github.com/users/veprbl/following{/other_user}","gists_url":"https://api.github.com/users/veprbl/gists{/gist_id}","starred_url":"https://api.github.com/users/veprbl/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/veprbl/subscriptions","organizations_url":"https://api.github.com/users/veprbl/orgs","repos_url":"https://api.github.com/users/veprbl/repos","events_url":"https://api.github.com/users/veprbl/events{/privacy}","received_events_url":"https://api.github.com/users/veprbl/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":{"login":"Cyan4973","id":750081,"node_id":"MDQ6VXNlcjc1MDA4MQ==","avatar_url":"https://avatars0.githubusercontent.com/u/750081?v=4","gravatar_id":"","url":"https://api.github.com/users/Cyan4973","html_url":"https://github.com/Cyan4973","followers_url":"https://api.github.com/users/Cyan4973/followers","following_url":"https://api.github.com/users/Cyan4973/following{/other_user}","gists_url":"https://api.github.com/users/Cyan4973/gists{/gist_id}","starred_url":"https://api.github.com/users/Cyan4973/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Cyan4973/subscriptions","organizations_url":"https://api.github.com/users/Cyan4973/orgs","repos_url":"https://api.github.com/users/Cyan4973/repos","events_url":"https://api.github.com/users/Cyan4973/events{/privacy}","received_events_url":"https://api.github.com/users/Cyan4973/received_events","type":"User","site_admin":false},"assignees":[{"login":"Cyan4973","id":750081,"node_id":"MDQ6VXNlcjc1MDA4MQ==","avatar_url":"https://avatars0.githubusercontent.com/u/750081?v=4","gravatar_id":"","url":"https://api.github.com/users/Cyan4973","html_url":"https://github.com/Cyan4973","followers_url":"https://api.github.com/users/Cyan4973/followers","following_url":"https://api.github.com/users/Cyan4973/following{/other_user}","gists_url":"https://api.github.com/users/Cyan4973/gists{/gist_id}","starred_url":"https://api.github.com/users/Cyan4973/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Cyan4973/subscriptions","organizations_url":"https://api.github.com/users/Cyan4973/orgs","repos_url":"https://api.github.com/users/Cyan4973/repos","events_url":"https://api.github.com/users/Cyan4973/events{/privacy}","received_events_url":"https://api.github.com/users/Cyan4973/received_events","type":"User","site_admin":false}],"milestone":null,"comments":3,"created_at":"2018-10-23T00:30:58Z","updated_at":"2018-10-26T21:43:16Z","closed_at":"2018-10-26T21:43:16Z","author_association":"NONE","body":"I'm maintaining an arrow-cpp (https://github.com/apache/arrow) package in nixpkgs. arrow-cpp can use zstd as one of the compression backends. Since we made an upgrade for zstd from 1.3.5 to 1.3.6 one of the tests in arrow-cpp started to fail. After some debugging I found that the source of failure is the call to `ZSTD_decompress` with `dstCapacity=0` and `dst=NULL` that is not working in zstd 1.3.6+. I've come up with a minimal reproducing example: \r\n```c++\r\n#include <stdio.h>\r\n#include <stdlib.h>\r\n#include <string.h>\r\n\r\n#include <zstd.h>\r\n\r\n#define MAX_UNCOMPRESSED_SIZE 4096\r\n#define MAX_COMPRESSED_SIZE 4096\r\n\r\n#define CHECK_ERROR(code) \\\r\n  if (ZSTD_isError(code)) { \\\r\n    printf(\"Error: %s\\n\", ZSTD_getErrorName(code)); \\\r\n    exit(EXIT_FAILURE); \\\r\n  }\r\n\r\nint main(void) {\r\n  char uncompressed[MAX_UNCOMPRESSED_SIZE];\r\n  char compressed[MAX_COMPRESSED_SIZE];\r\n  int compression_level = 1;\r\n  size_t input_len = 0;\r\n  memset(uncompressed, 0, sizeof(uncompressed));\r\n  size_t compressed_size = ZSTD_compress(compressed, MAX_COMPRESSED_SIZE, uncompressed, input_len, compression_level);\r\n  CHECK_ERROR(compressed_size);\r\n  printf(\"compressed_size = %zu\\n\", compressed_size);\r\n\r\n  {\r\n    printf(\"test1\\n\");\r\n    memset(uncompressed, 0, sizeof(uncompressed));\r\n    size_t uncompressed_size = ZSTD_decompress(uncompressed, MAX_UNCOMPRESSED_SIZE, compressed, compressed_size);\r\n    CHECK_ERROR(uncompressed_size);\r\n    printf(\"uncompressed_size = %zu\\n\", uncompressed_size);\r\n  }\r\n  {\r\n    printf(\"test2\\n\");\r\n    memset(uncompressed, 0, sizeof(uncompressed));\r\n    size_t uncompressed_size = ZSTD_decompress(uncompressed, 0, compressed, compressed_size);\r\n    CHECK_ERROR(uncompressed_size);\r\n    printf(\"uncompressed_size = %zu\\n\", uncompressed_size);\r\n  }\r\n  {\r\n    printf(\"test3\\n\");\r\n    size_t uncompressed_size = ZSTD_decompress(NULL, 0, compressed, compressed_size);\r\n    CHECK_ERROR(uncompressed_size);\r\n    printf(\"uncompressed_size = %zu\\n\", uncompressed_size);\r\n  }\r\n  return EXIT_SUCCESS;\r\n}\r\n```\r\nOn zstd 1.3.5:\r\n```\r\ncompressed_size = 9\r\ntest1\r\nuncompressed_size = 0\r\ntest2\r\nuncompressed_size = 0\r\ntest3\r\nuncompressed_size = 0\r\n```\r\nOn zstd 1.3.7:\r\n```\r\ncompressed_size = 9\r\ntest1\r\nuncompressed_size = 0\r\ntest2\r\nuncompressed_size = 0\r\ntest3\r\nError: Destination buffer is too small\r\n```\r\nI'm not very familiar with arrow-cpp's codebase, but, from what I understand, the situation when `dstCapacity=0` and `dst=NULL` is possible at runtime in arrow-cpp, for example, when reading \"parquet\" files with empty columns. It also seems that all other decompressors (GZIP, ZLIB, LZ4, SNAPPY, BROTLI) can handle these zero-length output buffers starting at NULL, as they pass the same test. I was wondering if it is possible to address this issue in zstd."}}
{"actor":{"display_login":"Cyan4973"}}