{"action":"created","issue":{"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/2871","repository_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python","labels_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/2871/labels{/name}","comments_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/2871/comments","events_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/2871/events","html_url":"https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2871","id":195793234,"number":2871,"title":"Streaming upload/download for blobs","user":{"login":"cristicbz","id":801328,"avatar_url":"https://avatars2.githubusercontent.com/u/801328?v=4","gravatar_id":"","url":"https://api.github.com/users/cristicbz","html_url":"https://github.com/cristicbz","followers_url":"https://api.github.com/users/cristicbz/followers","following_url":"https://api.github.com/users/cristicbz/following{/other_user}","gists_url":"https://api.github.com/users/cristicbz/gists{/gist_id}","starred_url":"https://api.github.com/users/cristicbz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cristicbz/subscriptions","organizations_url":"https://api.github.com/users/cristicbz/orgs","repos_url":"https://api.github.com/users/cristicbz/repos","events_url":"https://api.github.com/users/cristicbz/events{/privacy}","received_events_url":"https://api.github.com/users/cristicbz/received_events","type":"User","site_admin":false},"labels":[{"id":77678602,"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/labels/api:%20storage","name":"api: storage","color":"f8f8f8","default":false},{"id":77438582,"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/labels/type:%20feature%20request","name":"type: feature request","color":"c5def5","default":false}],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2016-12-15T12:26:51Z","updated_at":"2018-03-21T20:12:50Z","closed_at":"2017-08-11T17:05:02Z","author_association":"NONE","body":"We're in the process of moving from S3 to GCS and we've hit a snag: with `boto` S3 one can stream a download using the `read` method on a key, and stream an upload using multiple 'multipart' calls: `initiate_multipart_upload`, `upload_part_from*` and `complete_upload`.\r\n\r\nIt's imperfect, but this allows simple streaming uploads and downloads of very large files. Furthermore, stick some buffering in there and you can easily wrap S3 reads and writes into file-like objects (without seeking), like [`smart_open`](https://github.com/RaRe-Technologies/smart_open/blob/master/smart_open/smart_open_lib.py#L562) does.\r\n\r\nI think that the API supports all the pieces necessary to implement this in the client (which would be greatly appreciated!), but in the meantime can anyone suggest a workaround which does not require a temporary file?\r\n\r\nThanks!\r\n"},"comment":{"url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/comments/375080141","html_url":"https://github.com/GoogleCloudPlatform/google-cloud-python/issues/2871#issuecomment-375080141","issue_url":"https://api.github.com/repos/GoogleCloudPlatform/google-cloud-python/issues/2871","id":375080141,"user":{"login":"gastlich","id":1146430,"avatar_url":"https://avatars3.githubusercontent.com/u/1146430?v=4","gravatar_id":"","url":"https://api.github.com/users/gastlich","html_url":"https://github.com/gastlich","followers_url":"https://api.github.com/users/gastlich/followers","following_url":"https://api.github.com/users/gastlich/following{/other_user}","gists_url":"https://api.github.com/users/gastlich/gists{/gist_id}","starred_url":"https://api.github.com/users/gastlich/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gastlich/subscriptions","organizations_url":"https://api.github.com/users/gastlich/orgs","repos_url":"https://api.github.com/users/gastlich/repos","events_url":"https://api.github.com/users/gastlich/events{/privacy}","received_events_url":"https://api.github.com/users/gastlich/received_events","type":"User","site_admin":false},"created_at":"2018-03-21T20:12:50Z","updated_at":"2018-03-21T20:12:50Z","author_association":"NONE","body":"@linar-jether do you know if there is any other way to achieve what you have done in your code snippet? It's not working anymore, because of the changes in `reusable-media`, which expects to invoke `seek` on stream object"}}
{"actor":{"display_login":"gastlich"}}