{"action":"created","issue":{"url":"https://api.github.com/repos/carlomedas/4mc/issues/12","repository_url":"https://api.github.com/repos/carlomedas/4mc","labels_url":"https://api.github.com/repos/carlomedas/4mc/issues/12/labels{/name}","comments_url":"https://api.github.com/repos/carlomedas/4mc/issues/12/comments","events_url":"https://api.github.com/repos/carlomedas/4mc/issues/12/events","html_url":"https://github.com/carlomedas/4mc/issues/12","id":161656261,"number":12,"title":"FourMcOutputStream.compress doesn't handle incompressible data correctly","user":{"login":"advancedxy","id":807537,"avatar_url":"https://avatars.githubusercontent.com/u/807537?v=3","gravatar_id":"","url":"https://api.github.com/users/advancedxy","html_url":"https://github.com/advancedxy","followers_url":"https://api.github.com/users/advancedxy/followers","following_url":"https://api.github.com/users/advancedxy/following{/other_user}","gists_url":"https://api.github.com/users/advancedxy/gists{/gist_id}","starred_url":"https://api.github.com/users/advancedxy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/advancedxy/subscriptions","organizations_url":"https://api.github.com/users/advancedxy/orgs","repos_url":"https://api.github.com/users/advancedxy/repos","events_url":"https://api.github.com/users/advancedxy/events{/privacy}","received_events_url":"https://api.github.com/users/advancedxy/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-06-22T11:22:44Z","updated_at":"2016-07-25T16:10:06Z","closed_at":null,"body":"Suppose input is a 4MB buffer and is incompressible. The corresponding compressed size would be lz4_compressBound(4MB) > 4MB.\r\n\r\nFourMcOutputStream.compress() will trigger the Lz4Compressor to compress input. And returns 4MB in\r\n\r\n```\r\n        int len = compressor.compress(buffer, 0, buffer.length);\r\n```\r\n\r\nFourMcOutputStream detects that it should use uncompressed data directly. After getting the raw uncompressed data, there is still compressed data  left in compressor.compressedDirectBuf, thus the `compressor.finished()` returns  false. Which triggers another call of `FourMcOutputStream.compress()` in `FourMcOutputStream.finish()`. It will try to get uncompressed again, which results an `BufferUnderflowException`.  \r\n```\r\n    @Override\r\n    public void finish() throws IOException {\r\n        if (!compressor.finished()) {\r\n            compressor.finish();\r\n            while (!compressor.finished()) {\r\n                compress();\r\n            }\r\n        }\r\n    }\r\n```\r\n\r\nA quick fix to this problem would be allocating a big enough buffer when creating FourMcOutputStream. When the buffer is big enough to hold all the compressed data, line 196 \r\n\r\n```\r\n        int len = compressor.compress(buffer, 0, buffer.length);\r\n```\r\n\r\nin FourMcOutputStream will transfer all the data in compressedDirectBuf in to buffer, thus set the compressor.finish() to true.\r\n\r\nFix would be as followed in FourCodec.java:\r\n```\r\n    @Override\r\n    public CompressionOutputStream createOutputStream(OutputStream out, Compressor compressor) throws IOException {\r\n        int bufferPlusOverhead = Lz4Compressor.compressBound(FOURMC_MAX_BLOCK_SIZE);\r\n        return new FourMcOutputStream(out, compressor, bufferPlusOverhead);\r\n    }\r\n\r\n``` \r\n\r\nIf you believe this fix is ok to accept, I would send a pr later. However, I do think there should be a better way to solve this problem as we really don't need a buffer that's larger than 4MB."},"comment":{"url":"https://api.github.com/repos/carlomedas/4mc/issues/comments/235000622","html_url":"https://github.com/carlomedas/4mc/issues/12#issuecomment-235000622","issue_url":"https://api.github.com/repos/carlomedas/4mc/issues/12","id":235000622,"user":{"login":"advancedxy","id":807537,"avatar_url":"https://avatars.githubusercontent.com/u/807537?v=3","gravatar_id":"","url":"https://api.github.com/users/advancedxy","html_url":"https://github.com/advancedxy","followers_url":"https://api.github.com/users/advancedxy/followers","following_url":"https://api.github.com/users/advancedxy/following{/other_user}","gists_url":"https://api.github.com/users/advancedxy/gists{/gist_id}","starred_url":"https://api.github.com/users/advancedxy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/advancedxy/subscriptions","organizations_url":"https://api.github.com/users/advancedxy/orgs","repos_url":"https://api.github.com/users/advancedxy/repos","events_url":"https://api.github.com/users/advancedxy/events{/privacy}","received_events_url":"https://api.github.com/users/advancedxy/received_events","type":"User","site_admin":false},"created_at":"2016-07-25T16:10:06Z","updated_at":"2016-07-25T16:10:06Z","body":"@carlomedas ping"}}
{"actor":{"display_login":"advancedxy"}}