{"action":"created","issue":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/21478","repository_url":"https://api.github.com/repos/pandas-dev/pandas","labels_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21478/labels{/name}","comments_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21478/comments","events_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21478/events","html_url":"https://github.com/pandas-dev/pandas/pull/21478","id":332487533,"node_id":"MDExOlB1bGxSZXF1ZXN0MTk0OTU5MDQ1","number":21478,"title":"BUG/REG: file-handle object handled incorrectly in to_csv","user":{"login":"minggli","id":14131823,"node_id":"MDQ6VXNlcjE0MTMxODIz","avatar_url":"https://avatars0.githubusercontent.com/u/14131823?v=4","gravatar_id":"","url":"https://api.github.com/users/minggli","html_url":"https://github.com/minggli","followers_url":"https://api.github.com/users/minggli/followers","following_url":"https://api.github.com/users/minggli/following{/other_user}","gists_url":"https://api.github.com/users/minggli/gists{/gist_id}","starred_url":"https://api.github.com/users/minggli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/minggli/subscriptions","organizations_url":"https://api.github.com/users/minggli/orgs","repos_url":"https://api.github.com/users/minggli/repos","events_url":"https://api.github.com/users/minggli/events{/privacy}","received_events_url":"https://api.github.com/users/minggli/received_events","type":"User","site_admin":false},"labels":[{"id":47229171,"node_id":"MDU6TGFiZWw0NzIyOTE3MQ==","url":"https://api.github.com/repos/pandas-dev/pandas/labels/IO%20CSV","name":"IO CSV","color":"5319e7","default":false},{"id":32815646,"node_id":"MDU6TGFiZWwzMjgxNTY0Ng==","url":"https://api.github.com/repos/pandas-dev/pandas/labels/Regression","name":"Regression","color":"e11d21","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/pandas-dev/pandas/milestones/57","html_url":"https://github.com/pandas-dev/pandas/milestone/57","labels_url":"https://api.github.com/repos/pandas-dev/pandas/milestones/57/labels","id":3407592,"node_id":"MDk6TWlsZXN0b25lMzQwNzU5Mg==","number":57,"title":"0.23.2","description":"","creator":{"login":"jreback","id":953992,"node_id":"MDQ6VXNlcjk1Mzk5Mg==","avatar_url":"https://avatars2.githubusercontent.com/u/953992?v=4","gravatar_id":"","url":"https://api.github.com/users/jreback","html_url":"https://github.com/jreback","followers_url":"https://api.github.com/users/jreback/followers","following_url":"https://api.github.com/users/jreback/following{/other_user}","gists_url":"https://api.github.com/users/jreback/gists{/gist_id}","starred_url":"https://api.github.com/users/jreback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreback/subscriptions","organizations_url":"https://api.github.com/users/jreback/orgs","repos_url":"https://api.github.com/users/jreback/repos","events_url":"https://api.github.com/users/jreback/events{/privacy}","received_events_url":"https://api.github.com/users/jreback/received_events","type":"User","site_admin":false},"open_issues":32,"closed_issues":14,"state":"open","created_at":"2018-06-07T10:56:05Z","updated_at":"2018-06-14T19:56:15Z","due_on":"2018-06-29T07:00:00Z","closed_at":null},"comments":2,"created_at":"2018-06-14T17:03:53Z","updated_at":"2018-06-14T23:28:30Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/pandas-dev/pandas/pulls/21478","html_url":"https://github.com/pandas-dev/pandas/pull/21478","diff_url":"https://github.com/pandas-dev/pandas/pull/21478.diff","patch_url":"https://github.com/pandas-dev/pandas/pull/21478.patch"},"body":"- [x] closes #https://github.com/pandas-dev/pandas/issues/21471\r\n- [x] tests added / passed\r\n- [x] passes `git diff upstream/master -u -- \"*.py\" | flake8 --diff`\r\n- [ ] whatsnew entry\r\n\r\nThis error related to PR #21249 and https://github.com/pandas-dev/pandas/issues/21227. This is never supported use case and to use file-handle in to_csv with compression, the file-object itself should be a compression archive such as:\r\n```\r\nwith gzip.open('test.txt.gz', 'wt') as f:\r\n    pd.DataFrame([0,1],index=['a','b'], columns=['c']).to_csv(f, sep='\\t')\r\n```\r\n\r\nRegressed to 0.22 to_csv with support for zipfile. zipfile doesn't support writing csv strings to a zip archive using a file-handle. So buffer is used to catch the writing and dump into zip archive in one go. The other scenarios remain unchanged."},"comment":{"url":"https://api.github.com/repos/pandas-dev/pandas/issues/comments/397468307","html_url":"https://github.com/pandas-dev/pandas/pull/21478#issuecomment-397468307","issue_url":"https://api.github.com/repos/pandas-dev/pandas/issues/21478","id":397468307,"node_id":"MDEyOklzc3VlQ29tbWVudDM5NzQ2ODMwNw==","user":{"login":"minggli","id":14131823,"node_id":"MDQ6VXNlcjE0MTMxODIz","avatar_url":"https://avatars0.githubusercontent.com/u/14131823?v=4","gravatar_id":"","url":"https://api.github.com/users/minggli","html_url":"https://github.com/minggli","followers_url":"https://api.github.com/users/minggli/followers","following_url":"https://api.github.com/users/minggli/following{/other_user}","gists_url":"https://api.github.com/users/minggli/gists{/gist_id}","starred_url":"https://api.github.com/users/minggli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/minggli/subscriptions","organizations_url":"https://api.github.com/users/minggli/orgs","repos_url":"https://api.github.com/users/minggli/repos","events_url":"https://api.github.com/users/minggli/events{/privacy}","received_events_url":"https://api.github.com/users/minggli/received_events","type":"User","site_admin":false},"created_at":"2018-06-14T23:28:30Z","updated_at":"2018-06-14T23:28:30Z","author_association":"CONTRIBUTOR","body":"agree that for extremely large zip output, memory usage will increase when writing csv to StringIO first and then call `getvalue()`. With 8G RAM MacOS, enough to write a dataframe [100m, 3]. the memory increase is only when zip compression, for other use cases, there should be no difference as 0.22. the custom class just provides a file-like that accepts string into zip archive when ZipFile class doesn't really offer that. \r\n\r\nThe ability to write zip archive is added https://github.com/pandas-dev/pandas/issues/17778 where only read zip was supported, whereas we have round-trip ability for others. the motivation is consistency but also zip compression format is quite common and strange not to be supported.\r\n\r\nthe zip class is not only for to_csv but provides utility for to_json and to_pickle case too."}}
{"actor":{"display_login":"minggli"}}