{"action":"created","issue":{"url":"https://api.github.com/repos/janko-m/shrine/issues/252","repository_url":"https://api.github.com/repos/janko-m/shrine","labels_url":"https://api.github.com/repos/janko-m/shrine/issues/252/labels{/name}","comments_url":"https://api.github.com/repos/janko-m/shrine/issues/252/comments","events_url":"https://api.github.com/repos/janko-m/shrine/issues/252/events","html_url":"https://github.com/janko-m/shrine/issues/252","id":298129934,"number":252,"title":"Support for IO objects of unknown size","user":{"login":"willcosgrove","id":77887,"avatar_url":"https://avatars0.githubusercontent.com/u/77887?v=4","gravatar_id":"","url":"https://api.github.com/users/willcosgrove","html_url":"https://github.com/willcosgrove","followers_url":"https://api.github.com/users/willcosgrove/followers","following_url":"https://api.github.com/users/willcosgrove/following{/other_user}","gists_url":"https://api.github.com/users/willcosgrove/gists{/gist_id}","starred_url":"https://api.github.com/users/willcosgrove/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/willcosgrove/subscriptions","organizations_url":"https://api.github.com/users/willcosgrove/orgs","repos_url":"https://api.github.com/users/willcosgrove/repos","events_url":"https://api.github.com/users/willcosgrove/events{/privacy}","received_events_url":"https://api.github.com/users/willcosgrove/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-02-18T23:35:08Z","updated_at":"2018-02-19T12:45:52Z","closed_at":null,"author_association":"NONE","body":"I'm trying to implement a streaming processor.  Something that can take the given IO, and stream it to a program that will convert it, and stream the output to be uploaded back to Shrine.  But Shrine expects all valid IO to respond to `size` which is something that can't be known ahead of time in this case.\r\n\r\nWrote an extension of the `S3` storage that uses multipart uploads when the size of the IO is unknown, but I found out the size restraint is placed further up the chain than that.  First in `_enforce_io` then in `extract_size` called indirectly by `get_metadata`.  Size could potentially be required in more places, but I stopped my search here.\r\n\r\nI have two questions about this:\r\n\r\n1. Is this something that sounds like it should be possible to build within Shrine's plugin framework, or would this require a restructuring of Shrine's core to be able to support IOs with no known size?\r\n2. If it would require a restructuring of Shrine's core, would you see supporting this use case as a reasonable goal of Shrine, or as outside of the scope of what you want Shrine to be?"},"comment":{"url":"https://api.github.com/repos/janko-m/shrine/issues/comments/366683217","html_url":"https://github.com/janko-m/shrine/issues/252#issuecomment-366683217","issue_url":"https://api.github.com/repos/janko-m/shrine/issues/252","id":366683217,"user":{"login":"janko-m","id":795488,"avatar_url":"https://avatars2.githubusercontent.com/u/795488?v=4","gravatar_id":"","url":"https://api.github.com/users/janko-m","html_url":"https://github.com/janko-m","followers_url":"https://api.github.com/users/janko-m/followers","following_url":"https://api.github.com/users/janko-m/following{/other_user}","gists_url":"https://api.github.com/users/janko-m/gists{/gist_id}","starred_url":"https://api.github.com/users/janko-m/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/janko-m/subscriptions","organizations_url":"https://api.github.com/users/janko-m/orgs","repos_url":"https://api.github.com/users/janko-m/repos","events_url":"https://api.github.com/users/janko-m/events{/privacy}","received_events_url":"https://api.github.com/users/janko-m/received_events","type":"User","site_admin":false},"created_at":"2018-02-19T12:45:52Z","updated_at":"2018-02-19T12:45:52Z","author_association":"OWNER","body":"Thank you for bringing this to my attention!\r\n\r\nFirst of all, streaming support is something that's very important to me, because it's crucial when dealing with large files, and I designed Shrine with this in mind. For example, `Shrine::UploadedFile` objects allow you to stream uploaded files directly from the storage.\r\n\r\nUntil now I haven't thought of the use case of uploading a stream of unknown size to a storage. This definitely makes sense, and I believe that no parts of Shrine rely on size to be known other than the ones you've found.\r\n\r\nDo you think you could send me a pull request which addresses the things you've mentioned (making `#size` method optional and the S3 support)?"}}
{"actor":{"display_login":"janko-m"}}