{"action":"created","comment":{"url":"https://api.github.com/repos/astropy/astropy/pulls/comments/72755800","id":72755800,"diff_hunk":"@@ -1429,6 +1429,12 @@ def dtype(self):\n                 else:\n                     dt = np.dtype((dt.base, dim))\n \n+            # On Python 2, force the `name` to `str`\n+            # because Numpy structured arrays can't handle unicode `name`\n+            # See https://github.com/astropy/astropy/issues/5204\n+            if PY2:  # pragma: py2\n+                name = str(name)","path":"astropy/io/fits/column.py","position":17,"original_position":17,"commit_id":"89a92d2721ae56db7fb65e4738e047621f301ec4","original_commit_id":"9f138609cacdf9b13134e82cc427ceb06531ebf4","user":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"body":"@saimn - I'm getting the same nice `ValueError` on both Python 2 and 3.\r\nCan you post a code snippet to trigger the potential `UnicodeEncodeError` you mention?","created_at":"2016-07-29T08:11:59Z","updated_at":"2016-07-29T08:11:59Z","html_url":"https://github.com/astropy/astropy/pull/5204#discussion_r72755800","pull_request_url":"https://api.github.com/repos/astropy/astropy/pulls/5204","_links":{"self":{"href":"https://api.github.com/repos/astropy/astropy/pulls/comments/72755800"},"html":{"href":"https://github.com/astropy/astropy/pull/5204#discussion_r72755800"},"pull_request":{"href":"https://api.github.com/repos/astropy/astropy/pulls/5204"}}},"pull_request":{"url":"https://api.github.com/repos/astropy/astropy/pulls/5204","id":79218580,"html_url":"https://github.com/astropy/astropy/pull/5204","diff_url":"https://github.com/astropy/astropy/pull/5204.diff","patch_url":"https://github.com/astropy/astropy/pull/5204.patch","issue_url":"https://api.github.com/repos/astropy/astropy/issues/5204","number":5204,"state":"open","locked":false,"title":"Handle unicode FITS BinTable column names on Python 2","user":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"body":"The following example looks good, no?\r\n\r\n```python\r\n# Put this in a file: test.py\r\nfrom __future__ import unicode_literals\r\nfrom astropy.io import fits\r\ncol = fits.Column(name='spam', format='E', array=[42.])\r\nhdu = fits.BinTableHDU.from_columns([col])\r\n```\r\n\r\nIt follows the [Astropy coding guidelines](http://docs.astropy.org/en/latest/development/codeguide.html) that recommend using `from __future__ import unicode_literals` and it's a simplified version of the example from the docs how to create a `BinTable` (see [here](http://docs.astropy.org/en/latest/io/fits/index.html#creating-a-new-table-file)).\r\n\r\nIt works on Python 3, but on Python 2 it fails with a cryptic error message:\r\n```\r\n$ python2 test.py\r\nTraceback (most recent call last):\r\n  File \"test.py\", line 4, in <module>\r\n    hdu = fits.BinTableHDU.from_columns([col])\r\n  File \"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/io/fits/hdu/table.py\", line 126, in from_columns\r\n    data = FITS_rec.from_columns(coldefs, nrows=nrows, fill=fill)\r\n  File \"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/io/fits/fitsrec.py\", line 342, in from_columns\r\n    raw_data = np.empty(columns.dtype.itemsize * nrows, dtype=np.uint8)\r\n  File \"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/utils/decorators.py\", line 498, in __get__\r\n    val = self.fget(obj)\r\n  File \"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/io/fits/column.py\", line 1435, in dtype\r\n    return nh.realign_dtype(np.dtype(fields), offsets)\r\nTypeError: data type not understood\r\n```\r\n(This is with latest Astropy and Numpy, I think the error will be the same with all older versions of Astropy and Numpy.)\r\n\r\nIn Gammapy and Fermipy development, we encountered this issue this week (see https://github.com/gammapy/gammapy/pull/648#issuecomment-235850515 and https://github.com/fermiPy/fermipy/pull/63), cc @jjlk @woodmd\r\n\r\nAfter a lot of head-scratching, I found the explanation of the issue and suggested workaround to force the column name to bytes on Python 2 only and use `name=str('spam')` here:\r\nhttp://docs.astropy.org/en/latest/development/codeguide.html#not-so-fast-on-that-unicode-thing\r\n\r\nI also found #4805 by @pllim where she got bitten by the same issue and proposed to do automatic conversion from unicode to bytes for dtype field names on Python 2 in `astropy.io.fits`:\r\nhttps://github.com/numpy/numpy/issues/2407#issuecomment-195500248\r\nThat PR was closed without being merged after discussion with @saimn .\r\n\r\nIn https://github.com/numpy/numpy/issues/2407#issuecomment-195500248, @pv said that this change would probably even be welcome in Numpy itself.\r\n\r\nSince we'll need to support Numpy versions without that fix for a long time, I'm +1 to implement that change in `astropy.io.fits`.\r\n\r\nDoes that sound like a good idea to everyone or is there someone against this change?\r\n\r\n@saimn - I think your argument against #4805 was just about an issue with how the fix was implemented, and not against automatic conversion from unicode to bytes for dtype field names on Python 2 by encoding to ascii in principle?","created_at":"2016-07-28T15:37:28Z","updated_at":"2016-07-29T08:11:59Z","closed_at":null,"merged_at":null,"merge_commit_sha":"fb6e5643e600b010f01ae88a363e7ba33aaa033e","assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/astropy/astropy/milestones/41","html_url":"https://github.com/astropy/astropy/milestones/v1.2.2","labels_url":"https://api.github.com/repos/astropy/astropy/milestones/41/labels","id":1847447,"number":41,"title":"v1.2.2","description":"","creator":{"login":"astrofrog","id":314716,"avatar_url":"https://avatars.githubusercontent.com/u/314716?v=3","gravatar_id":"","url":"https://api.github.com/users/astrofrog","html_url":"https://github.com/astrofrog","followers_url":"https://api.github.com/users/astrofrog/followers","following_url":"https://api.github.com/users/astrofrog/following{/other_user}","gists_url":"https://api.github.com/users/astrofrog/gists{/gist_id}","starred_url":"https://api.github.com/users/astrofrog/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astrofrog/subscriptions","organizations_url":"https://api.github.com/users/astrofrog/orgs","repos_url":"https://api.github.com/users/astrofrog/repos","events_url":"https://api.github.com/users/astrofrog/events{/privacy}","received_events_url":"https://api.github.com/users/astrofrog/received_events","type":"User","site_admin":false},"open_issues":7,"closed_issues":11,"state":"open","created_at":"2016-06-24T16:24:28Z","updated_at":"2016-07-28T16:54:44Z","due_on":null,"closed_at":null},"commits_url":"https://api.github.com/repos/astropy/astropy/pulls/5204/commits","review_comments_url":"https://api.github.com/repos/astropy/astropy/pulls/5204/comments","review_comment_url":"https://api.github.com/repos/astropy/astropy/pulls/comments{/number}","comments_url":"https://api.github.com/repos/astropy/astropy/issues/5204/comments","statuses_url":"https://api.github.com/repos/astropy/astropy/statuses/89a92d2721ae56db7fb65e4738e047621f301ec4","head":{"label":"cdeil:issue-5204","ref":"issue-5204","sha":"89a92d2721ae56db7fb65e4738e047621f301ec4","user":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"repo":{"id":2844165,"name":"astropy","full_name":"cdeil/astropy","owner":{"login":"cdeil","id":852409,"avatar_url":"https://avatars.githubusercontent.com/u/852409?v=3","gravatar_id":"","url":"https://api.github.com/users/cdeil","html_url":"https://github.com/cdeil","followers_url":"https://api.github.com/users/cdeil/followers","following_url":"https://api.github.com/users/cdeil/following{/other_user}","gists_url":"https://api.github.com/users/cdeil/gists{/gist_id}","starred_url":"https://api.github.com/users/cdeil/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cdeil/subscriptions","organizations_url":"https://api.github.com/users/cdeil/orgs","repos_url":"https://api.github.com/users/cdeil/repos","events_url":"https://api.github.com/users/cdeil/events{/privacy}","received_events_url":"https://api.github.com/users/cdeil/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/cdeil/astropy","description":"Main AstroPy repository","fork":true,"url":"https://api.github.com/repos/cdeil/astropy","forks_url":"https://api.github.com/repos/cdeil/astropy/forks","keys_url":"https://api.github.com/repos/cdeil/astropy/keys{/key_id}","collaborators_url":"https://api.github.com/repos/cdeil/astropy/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/cdeil/astropy/teams","hooks_url":"https://api.github.com/repos/cdeil/astropy/hooks","issue_events_url":"https://api.github.com/repos/cdeil/astropy/issues/events{/number}","events_url":"https://api.github.com/repos/cdeil/astropy/events","assignees_url":"https://api.github.com/repos/cdeil/astropy/assignees{/user}","branches_url":"https://api.github.com/repos/cdeil/astropy/branches{/branch}","tags_url":"https://api.github.com/repos/cdeil/astropy/tags","blobs_url":"https://api.github.com/repos/cdeil/astropy/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/cdeil/astropy/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/cdeil/astropy/git/refs{/sha}","trees_url":"https://api.github.com/repos/cdeil/astropy/git/trees{/sha}","statuses_url":"https://api.github.com/repos/cdeil/astropy/statuses/{sha}","languages_url":"https://api.github.com/repos/cdeil/astropy/languages","stargazers_url":"https://api.github.com/repos/cdeil/astropy/stargazers","contributors_url":"https://api.github.com/repos/cdeil/astropy/contributors","subscribers_url":"https://api.github.com/repos/cdeil/astropy/subscribers","subscription_url":"https://api.github.com/repos/cdeil/astropy/subscription","commits_url":"https://api.github.com/repos/cdeil/astropy/commits{/sha}","git_commits_url":"https://api.github.com/repos/cdeil/astropy/git/commits{/sha}","comments_url":"https://api.github.com/repos/cdeil/astropy/comments{/number}","issue_comment_url":"https://api.github.com/repos/cdeil/astropy/issues/comments{/number}","contents_url":"https://api.github.com/repos/cdeil/astropy/contents/{+path}","compare_url":"https://api.github.com/repos/cdeil/astropy/compare/{base}...{head}","merges_url":"https://api.github.com/repos/cdeil/astropy/merges","archive_url":"https://api.github.com/repos/cdeil/astropy/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/cdeil/astropy/downloads","issues_url":"https://api.github.com/repos/cdeil/astropy/issues{/number}","pulls_url":"https://api.github.com/repos/cdeil/astropy/pulls{/number}","milestones_url":"https://api.github.com/repos/cdeil/astropy/milestones{/number}","notifications_url":"https://api.github.com/repos/cdeil/astropy/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/cdeil/astropy/labels{/name}","releases_url":"https://api.github.com/repos/cdeil/astropy/releases{/id}","deployments_url":"https://api.github.com/repos/cdeil/astropy/deployments","created_at":"2011-11-24T15:11:07Z","updated_at":"2014-07-21T19:57:03Z","pushed_at":"2016-07-29T06:47:38Z","git_url":"git://github.com/cdeil/astropy.git","ssh_url":"git@github.com:cdeil/astropy.git","clone_url":"https://github.com/cdeil/astropy.git","svn_url":"https://github.com/cdeil/astropy","homepage":"astropy.org","size":46177,"stargazers_count":0,"watchers_count":0,"language":"Python","has_issues":false,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":0,"forks":0,"open_issues":0,"watchers":0,"default_branch":"master"}},"base":{"label":"astropy:master","ref":"master","sha":"e8f6c7d2787577c8b01e4fbf765c7600701044ea","user":{"login":"astropy","id":847984,"avatar_url":"https://avatars.githubusercontent.com/u/847984?v=3","gravatar_id":"","url":"https://api.github.com/users/astropy","html_url":"https://github.com/astropy","followers_url":"https://api.github.com/users/astropy/followers","following_url":"https://api.github.com/users/astropy/following{/other_user}","gists_url":"https://api.github.com/users/astropy/gists{/gist_id}","starred_url":"https://api.github.com/users/astropy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astropy/subscriptions","organizations_url":"https://api.github.com/users/astropy/orgs","repos_url":"https://api.github.com/users/astropy/repos","events_url":"https://api.github.com/users/astropy/events{/privacy}","received_events_url":"https://api.github.com/users/astropy/received_events","type":"Organization","site_admin":false},"repo":{"id":2081289,"name":"astropy","full_name":"astropy/astropy","owner":{"login":"astropy","id":847984,"avatar_url":"https://avatars.githubusercontent.com/u/847984?v=3","gravatar_id":"","url":"https://api.github.com/users/astropy","html_url":"https://github.com/astropy","followers_url":"https://api.github.com/users/astropy/followers","following_url":"https://api.github.com/users/astropy/following{/other_user}","gists_url":"https://api.github.com/users/astropy/gists{/gist_id}","starred_url":"https://api.github.com/users/astropy/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/astropy/subscriptions","organizations_url":"https://api.github.com/users/astropy/orgs","repos_url":"https://api.github.com/users/astropy/repos","events_url":"https://api.github.com/users/astropy/events{/privacy}","received_events_url":"https://api.github.com/users/astropy/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/astropy/astropy","description":"Repository for the Astropy core package","fork":false,"url":"https://api.github.com/repos/astropy/astropy","forks_url":"https://api.github.com/repos/astropy/astropy/forks","keys_url":"https://api.github.com/repos/astropy/astropy/keys{/key_id}","collaborators_url":"https://api.github.com/repos/astropy/astropy/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/astropy/astropy/teams","hooks_url":"https://api.github.com/repos/astropy/astropy/hooks","issue_events_url":"https://api.github.com/repos/astropy/astropy/issues/events{/number}","events_url":"https://api.github.com/repos/astropy/astropy/events","assignees_url":"https://api.github.com/repos/astropy/astropy/assignees{/user}","branches_url":"https://api.github.com/repos/astropy/astropy/branches{/branch}","tags_url":"https://api.github.com/repos/astropy/astropy/tags","blobs_url":"https://api.github.com/repos/astropy/astropy/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/astropy/astropy/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/astropy/astropy/git/refs{/sha}","trees_url":"https://api.github.com/repos/astropy/astropy/git/trees{/sha}","statuses_url":"https://api.github.com/repos/astropy/astropy/statuses/{sha}","languages_url":"https://api.github.com/repos/astropy/astropy/languages","stargazers_url":"https://api.github.com/repos/astropy/astropy/stargazers","contributors_url":"https://api.github.com/repos/astropy/astropy/contributors","subscribers_url":"https://api.github.com/repos/astropy/astropy/subscribers","subscription_url":"https://api.github.com/repos/astropy/astropy/subscription","commits_url":"https://api.github.com/repos/astropy/astropy/commits{/sha}","git_commits_url":"https://api.github.com/repos/astropy/astropy/git/commits{/sha}","comments_url":"https://api.github.com/repos/astropy/astropy/comments{/number}","issue_comment_url":"https://api.github.com/repos/astropy/astropy/issues/comments{/number}","contents_url":"https://api.github.com/repos/astropy/astropy/contents/{+path}","compare_url":"https://api.github.com/repos/astropy/astropy/compare/{base}...{head}","merges_url":"https://api.github.com/repos/astropy/astropy/merges","archive_url":"https://api.github.com/repos/astropy/astropy/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/astropy/astropy/downloads","issues_url":"https://api.github.com/repos/astropy/astropy/issues{/number}","pulls_url":"https://api.github.com/repos/astropy/astropy/pulls{/number}","milestones_url":"https://api.github.com/repos/astropy/astropy/milestones{/number}","notifications_url":"https://api.github.com/repos/astropy/astropy/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/astropy/astropy/labels{/name}","releases_url":"https://api.github.com/repos/astropy/astropy/releases{/id}","deployments_url":"https://api.github.com/repos/astropy/astropy/deployments","created_at":"2011-07-21T01:33:49Z","updated_at":"2016-07-27T19:52:26Z","pushed_at":"2016-07-29T06:47:39Z","git_url":"git://github.com/astropy/astropy.git","ssh_url":"git@github.com:astropy/astropy.git","clone_url":"https://github.com/astropy/astropy.git","svn_url":"https://github.com/astropy/astropy","homepage":"www.astropy.org","size":68187,"stargazers_count":907,"watchers_count":907,"language":"Python","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":576,"mirror_url":null,"open_issues_count":757,"forks":576,"open_issues":757,"watchers":907,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/astropy/astropy/pulls/5204"},"html":{"href":"https://github.com/astropy/astropy/pull/5204"},"issue":{"href":"https://api.github.com/repos/astropy/astropy/issues/5204"},"comments":{"href":"https://api.github.com/repos/astropy/astropy/issues/5204/comments"},"review_comments":{"href":"https://api.github.com/repos/astropy/astropy/pulls/5204/comments"},"review_comment":{"href":"https://api.github.com/repos/astropy/astropy/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/astropy/astropy/pulls/5204/commits"},"statuses":{"href":"https://api.github.com/repos/astropy/astropy/statuses/89a92d2721ae56db7fb65e4738e047621f301ec4"}}}}
{"actor":{"display_login":"cdeil"}}