{"action":"created","issue":{"url":"https://api.github.com/repos/piskvorky/smart_open/issues/56","labels_url":"https://api.github.com/repos/piskvorky/smart_open/issues/56/labels{/name}","comments_url":"https://api.github.com/repos/piskvorky/smart_open/issues/56/comments","events_url":"https://api.github.com/repos/piskvorky/smart_open/issues/56/events","html_url":"https://github.com/piskvorky/smart_open/issues/56","id":126461485,"number":56,"title":"Cached reads","user":{"login":"AndreaCrotti","id":53640,"avatar_url":"https://avatars.githubusercontent.com/u/53640?v=3","gravatar_id":"","url":"https://api.github.com/users/AndreaCrotti","html_url":"https://github.com/AndreaCrotti","followers_url":"https://api.github.com/users/AndreaCrotti/followers","following_url":"https://api.github.com/users/AndreaCrotti/following{/other_user}","gists_url":"https://api.github.com/users/AndreaCrotti/gists{/gist_id}","starred_url":"https://api.github.com/users/AndreaCrotti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndreaCrotti/subscriptions","organizations_url":"https://api.github.com/users/AndreaCrotti/orgs","repos_url":"https://api.github.com/users/AndreaCrotti/repos","events_url":"https://api.github.com/users/AndreaCrotti/events{/privacy}","received_events_url":"https://api.github.com/users/AndreaCrotti/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-01-13T16:43:43Z","updated_at":"2016-01-13T17:20:18Z","closed_at":null,"body":"The fact that smart_open is streaming is great and nice to use.\r\n\r\nHowever sometimes we still need to get the whole file.\r\n\r\nWould there be a way to actually cache the file content, to make sure that we don't actually re-read the whole file every time?\r\n\r\nS3 files for example in our case never change, passing around the smart_open read object they would be re-read completely every time right?\r\n\r\nSomething like\r\n\r\n```python\r\nsmart_open.smart_open('s3://bucket/key', cached=True)\r\n```\r\n\r\nfor example could be a possible API for this?\r\nDo you think it's feasible/desirable?"},"comment":{"url":"https://api.github.com/repos/piskvorky/smart_open/issues/comments/171369567","html_url":"https://github.com/piskvorky/smart_open/issues/56#issuecomment-171369567","issue_url":"https://api.github.com/repos/piskvorky/smart_open/issues/56","id":171369567,"user":{"login":"AndreaCrotti","id":53640,"avatar_url":"https://avatars.githubusercontent.com/u/53640?v=3","gravatar_id":"","url":"https://api.github.com/users/AndreaCrotti","html_url":"https://github.com/AndreaCrotti","followers_url":"https://api.github.com/users/AndreaCrotti/followers","following_url":"https://api.github.com/users/AndreaCrotti/following{/other_user}","gists_url":"https://api.github.com/users/AndreaCrotti/gists{/gist_id}","starred_url":"https://api.github.com/users/AndreaCrotti/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AndreaCrotti/subscriptions","organizations_url":"https://api.github.com/users/AndreaCrotti/orgs","repos_url":"https://api.github.com/users/AndreaCrotti/repos","events_url":"https://api.github.com/users/AndreaCrotti/events{/privacy}","received_events_url":"https://api.github.com/users/AndreaCrotti/received_events","type":"User","site_admin":false},"created_at":"2016-01-13T17:20:18Z","updated_at":"2016-01-13T17:20:18Z","body":"Actually this can be solved quite simply like this\r\n```python\r\n\r\n@cachetools.cached(cache={})\r\ndef cached_streamer(file_path):\r\n    \"\"\"Given a path read the whole thing upfront and cache the result\r\n    so that it does not need to be re-read again\r\n    \"\"\"\r\n    return [line for line in smart_open.smart_open(file_path)]\r\n\r\n```\r\n\r\nEven if this is really make sense only for quite small files that can be actually loaded in memory, so I think using the smart_open internals something much smarter could be done.."}}