{"action":"created","issue":{"url":"https://api.github.com/repos/wireservice/agate/issues/624","repository_url":"https://api.github.com/repos/wireservice/agate","labels_url":"https://api.github.com/repos/wireservice/agate/issues/624/labels{/name}","comments_url":"https://api.github.com/repos/wireservice/agate/issues/624/comments","events_url":"https://api.github.com/repos/wireservice/agate/issues/624/events","html_url":"https://github.com/wireservice/agate/issues/624","id":159723823,"node_id":"MDU6SXNzdWUxNTk3MjM4MjM=","number":624,"title":"to_csv encoding problem?","user":{"login":"tf13","id":16142258,"node_id":"MDQ6VXNlcjE2MTQyMjU4","avatar_url":"https://avatars0.githubusercontent.com/u/16142258?v=4","gravatar_id":"","url":"https://api.github.com/users/tf13","html_url":"https://github.com/tf13","followers_url":"https://api.github.com/users/tf13/followers","following_url":"https://api.github.com/users/tf13/following{/other_user}","gists_url":"https://api.github.com/users/tf13/gists{/gist_id}","starred_url":"https://api.github.com/users/tf13/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tf13/subscriptions","organizations_url":"https://api.github.com/users/tf13/orgs","repos_url":"https://api.github.com/users/tf13/repos","events_url":"https://api.github.com/users/tf13/events{/privacy}","received_events_url":"https://api.github.com/users/tf13/received_events","type":"User","site_admin":false},"labels":[{"id":95150051,"node_id":"MDU6TGFiZWw5NTE1MDA1MQ==","url":"https://api.github.com/repos/wireservice/agate/labels/bug","name":"bug","color":"f7c6c7","default":true},{"id":95150314,"node_id":"MDU6TGFiZWw5NTE1MDMxNA==","url":"https://api.github.com/repos/wireservice/agate/labels/priority-normal","name":"priority-normal","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-06-10T21:07:50Z","updated_at":"2018-09-05T15:04:49Z","closed_at":null,"author_association":"NONE","body":"I'm getting a UnicodeEncodeError when I try to send a table to_csv. The table was made with a join of two other tables, both loaded from csv using utf-8 encoding. The character that seems to have caused the error is apparently just an en-dash -- nothing too exotic. Not sure if I'm doing something wrong or if something worse is going on.\n\nThe traceback:\n\n```\nTraceback (most recent call last):\n  File \"path/goes/here/lobbyists.py\", line 57, in <module>\n    combo.to_csv('combo.csv')\n  File \"/Users/theofrancis/miniconda3/lib/python3.5/site-packages/agate/table/__init__.py\", line 408, in to_csv\n    writer.writerow(tuple(csv_funcs[i](d) for i, d in enumerate(row)))\n  File \"/Users/theofrancis/miniconda3/lib/python3.5/site-packages/agate/csv_py3.py\", line 91, in writerow\n    self.writer.writerow(row)\nUnicodeEncodeError: 'ascii' codec can't encode character '\\u2013' in position 282: ordinal not in range(128)\n\n```\n"},"comment":{"url":"https://api.github.com/repos/wireservice/agate/issues/comments/418765116","html_url":"https://github.com/wireservice/agate/issues/624#issuecomment-418765116","issue_url":"https://api.github.com/repos/wireservice/agate/issues/624","id":418765116,"node_id":"MDEyOklzc3VlQ29tbWVudDQxODc2NTExNg==","user":{"login":"bitlather","id":148455,"node_id":"MDQ6VXNlcjE0ODQ1NQ==","avatar_url":"https://avatars0.githubusercontent.com/u/148455?v=4","gravatar_id":"","url":"https://api.github.com/users/bitlather","html_url":"https://github.com/bitlather","followers_url":"https://api.github.com/users/bitlather/followers","following_url":"https://api.github.com/users/bitlather/following{/other_user}","gists_url":"https://api.github.com/users/bitlather/gists{/gist_id}","starred_url":"https://api.github.com/users/bitlather/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bitlather/subscriptions","organizations_url":"https://api.github.com/users/bitlather/orgs","repos_url":"https://api.github.com/users/bitlather/repos","events_url":"https://api.github.com/users/bitlather/events{/privacy}","received_events_url":"https://api.github.com/users/bitlather/received_events","type":"User","site_admin":false},"created_at":"2018-09-05T15:04:49Z","updated_at":"2018-09-05T15:04:49Z","author_association":"NONE","body":"This was among the top results when I was looking up the error:\r\n\r\n> UnicodeEncodeError: 'ascii' codec can't encode character u'\\xe9' in position 0: ordinal not in range(128)\r\n\r\nI thought I'd show how I fixed it for anyone else who ends up here.\r\n\r\n### Replicating The Problem\r\n\r\nConsider this script:\r\n\r\n```\r\nimport csv\r\n\r\ndata = [[\"a\", \"b\", u'\\xe9']]\r\n\r\nwith open(\"output.csv\", \"w\") as csv_file:\r\n    writer = csv.writer(csv_file, quoting=csv.QUOTE_ALL)\r\n    writer.writerows(data)\r\n```\r\n\r\nWhen I run it, I get:\r\n\r\n```\r\n$ python tmp_csv_script.py \r\nTraceback (most recent call last):\r\n  File \"tmp_csv_script.py\", line 7, in <module>\r\n    writer.writerows(data)\r\nUnicodeEncodeError: 'ascii' codec can't encode character u'\\xe9' in position 0: ordinal not in range(128)\r\n```\r\n\r\n### Solution\r\n\r\nSet default encoding to utf8.\r\n\r\n```\r\nimport csv\r\n\r\nimport sys\r\nreload(sys)\r\nsys.setdefaultencoding('utf8')\r\n\r\ndata = [[\"a\", \"b\", u'\\xe9']]\r\n\r\nwith open(\"output.csv\", \"w\") as csv_file:\r\n    writer = csv.writer(csv_file, quoting=csv.QUOTE_ALL)\r\n    writer.writerows(data)\r\n```\r\n\r\nRunning this was successful for me, and the output looked like this:\r\n\r\n```\r\n\"a\",\"b\",\"é\"\r\n```\r\n\r\n### Further Reading\r\n\r\nThere is an excellent blog article that might be able to help you out more, if you don't want to set the default encoding:\r\n\r\n[Python: UnicodeEncodeError: 'ascii' codec can't encode character u'\\xfc' in position 11: ordinal not in range(128)](https://markhneedham.com/blog/2015/05/21/python-unicodeencodeerror-ascii-codec-cant-encode-character-uxfc-in-position-11-ordinal-not-in-range128/)"}}
{"actor":{"display_login":"bitlather"}}