{"action":"created","issue":{"url":"https://api.github.com/repos/aws4s/aws4s/issues/27","repository_url":"https://api.github.com/repos/aws4s/aws4s","labels_url":"https://api.github.com/repos/aws4s/aws4s/issues/27/labels{/name}","comments_url":"https://api.github.com/repos/aws4s/aws4s/issues/27/comments","events_url":"https://api.github.com/repos/aws4s/aws4s/issues/27/events","html_url":"https://github.com/aws4s/aws4s/issues/27","id":309397232,"number":27,"title":"S3 upload fails when body at least 10kb in size","user":{"login":"Malax","id":177877,"avatar_url":"https://avatars2.githubusercontent.com/u/177877?v=4","gravatar_id":"","url":"https://api.github.com/users/Malax","html_url":"https://github.com/Malax","followers_url":"https://api.github.com/users/Malax/followers","following_url":"https://api.github.com/users/Malax/following{/other_user}","gists_url":"https://api.github.com/users/Malax/gists{/gist_id}","starred_url":"https://api.github.com/users/Malax/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Malax/subscriptions","organizations_url":"https://api.github.com/users/Malax/orgs","repos_url":"https://api.github.com/users/Malax/repos","events_url":"https://api.github.com/users/Malax/events{/privacy}","received_events_url":"https://api.github.com/users/Malax/received_events","type":"User","site_admin":false},"labels":[{"id":768164868,"url":"https://api.github.com/repos/aws4s/aws4s/labels/bug","name":"bug","color":"ee0701","default":true}],"state":"open","locked":false,"assignee":{"login":"amrhassan","id":130985,"avatar_url":"https://avatars2.githubusercontent.com/u/130985?v=4","gravatar_id":"","url":"https://api.github.com/users/amrhassan","html_url":"https://github.com/amrhassan","followers_url":"https://api.github.com/users/amrhassan/followers","following_url":"https://api.github.com/users/amrhassan/following{/other_user}","gists_url":"https://api.github.com/users/amrhassan/gists{/gist_id}","starred_url":"https://api.github.com/users/amrhassan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amrhassan/subscriptions","organizations_url":"https://api.github.com/users/amrhassan/orgs","repos_url":"https://api.github.com/users/amrhassan/repos","events_url":"https://api.github.com/users/amrhassan/events{/privacy}","received_events_url":"https://api.github.com/users/amrhassan/received_events","type":"User","site_admin":false},"assignees":[{"login":"amrhassan","id":130985,"avatar_url":"https://avatars2.githubusercontent.com/u/130985?v=4","gravatar_id":"","url":"https://api.github.com/users/amrhassan","html_url":"https://github.com/amrhassan","followers_url":"https://api.github.com/users/amrhassan/followers","following_url":"https://api.github.com/users/amrhassan/following{/other_user}","gists_url":"https://api.github.com/users/amrhassan/gists{/gist_id}","starred_url":"https://api.github.com/users/amrhassan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amrhassan/subscriptions","organizations_url":"https://api.github.com/users/amrhassan/orgs","repos_url":"https://api.github.com/users/amrhassan/repos","events_url":"https://api.github.com/users/amrhassan/events{/privacy}","received_events_url":"https://api.github.com/users/amrhassan/received_events","type":"User","site_admin":false}],"milestone":null,"comments":0,"created_at":"2018-03-28T14:41:39Z","updated_at":"2018-03-29T14:47:04Z","closed_at":null,"author_association":"NONE","body":"When the body of a `putObject` is 10kb or larger in size, the process will fail with:\r\n\r\n```\r\norg.aws4s.ErrorResponse: Error response:\r\nstatus:   501 Not Implemented,\r\ntype:\r\ncode:     NotImplemented\r\nmessage:  A header you provided implies functionality that is not implemented\r\n```\r\n\r\nI tracked the issue down and found out that the `Transfer-Encoding` header caues the problem. Blaze internally creates chunks of 10kb, that's why this is the magic limit. AWS seems to support chunked uploads as long as the request bodies are signed and the `Content-Length` header is provided, even with chunked encoding.\r\n\r\nThis issue can be easily reproduced by increasing the `data` field size to 10kb in `org.aws4s.s3.S3SmokeTest`.\r\n\r\nUnfortunately I don't know how to work around this issue, otherwise I would have created a PR with a fix.\r\n"},"comment":{"url":"https://api.github.com/repos/aws4s/aws4s/issues/comments/377259169","html_url":"https://github.com/aws4s/aws4s/issues/27#issuecomment-377259169","issue_url":"https://api.github.com/repos/aws4s/aws4s/issues/27","id":377259169,"user":{"login":"amrhassan","id":130985,"avatar_url":"https://avatars2.githubusercontent.com/u/130985?v=4","gravatar_id":"","url":"https://api.github.com/users/amrhassan","html_url":"https://github.com/amrhassan","followers_url":"https://api.github.com/users/amrhassan/followers","following_url":"https://api.github.com/users/amrhassan/following{/other_user}","gists_url":"https://api.github.com/users/amrhassan/gists{/gist_id}","starred_url":"https://api.github.com/users/amrhassan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amrhassan/subscriptions","organizations_url":"https://api.github.com/users/amrhassan/orgs","repos_url":"https://api.github.com/users/amrhassan/repos","events_url":"https://api.github.com/users/amrhassan/events{/privacy}","received_events_url":"https://api.github.com/users/amrhassan/received_events","type":"User","site_admin":false},"created_at":"2018-03-29T14:47:04Z","updated_at":"2018-03-29T14:47:04Z","author_association":"CONTRIBUTOR","body":"Thank you for investigating and reporting this.\r\n\r\nIt seems that my implementation is rather naive and only works with the single chunk as you said. I can make it work by running the payload stream entirely first then pass the resulting bytes to http4s which treats it then as a single unchunked request, which works.\r\n\r\nThis of course assumes that the payload is small enough to fit all in memory. I will change it to be like that while I investigate further into how I can make the chunked uploading work."}}
{"actor":{"display_login":"amrhassan"}}