{"action":"created","issue":{"url":"https://api.github.com/repos/probcomp/bayeslite/issues/361","labels_url":"https://api.github.com/repos/probcomp/bayeslite/issues/361/labels{/name}","comments_url":"https://api.github.com/repos/probcomp/bayeslite/issues/361/comments","events_url":"https://api.github.com/repos/probcomp/bayeslite/issues/361/events","html_url":"https://github.com/probcomp/bayeslite/issues/361","id":126979762,"number":361,"title":"`unicode` and `json.loads/json.dumps` disagree on floats","user":{"login":"riastradh-probcomp","id":8646066,"avatar_url":"https://avatars.githubusercontent.com/u/8646066?v=3","gravatar_id":"","url":"https://api.github.com/users/riastradh-probcomp","html_url":"https://github.com/riastradh-probcomp","followers_url":"https://api.github.com/users/riastradh-probcomp/followers","following_url":"https://api.github.com/users/riastradh-probcomp/following{/other_user}","gists_url":"https://api.github.com/users/riastradh-probcomp/gists{/gist_id}","starred_url":"https://api.github.com/users/riastradh-probcomp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/riastradh-probcomp/subscriptions","organizations_url":"https://api.github.com/users/riastradh-probcomp/orgs","repos_url":"https://api.github.com/users/riastradh-probcomp/repos","events_url":"https://api.github.com/users/riastradh-probcomp/events{/privacy}","received_events_url":"https://api.github.com/users/riastradh-probcomp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-01-15T23:30:05Z","updated_at":"2016-01-18T17:46:12Z","closed_at":null,"body":"    >>> import json\r\n    >>> json.loads(json.dumps({2/3.: 0}))\r\n    {u'0.6666666666666666': 0}\r\n    >>> unicode(2/3.)\r\n    u'0.666666666667'\r\n\r\nWhen creating crosscat categorical column metadata in `crosscat_categorical_metadata`, we may inadvertently create a code_to_value dict keyed by floats -- because sqlite3 returns them for some reason -- instead of strings, and then we get whatever encoding json uses.  Then `_crosscat_data` tries to look up a value in the code_to_value data obtained by passing the float through `unicode` rather than whatever the json library uses.\r\n\r\nThe easy fix for this is to just call `unicode` when constructing the table.  But we probably *also* want to modify the queries in `_crosscat_data` and `_crosscat_metadata_categorical` to explicitly request that we get the *string* values in the table, not parsed as floats as sqlite3 will helpfully do for us.\r\n\r\n(Adam tripped over this issue in a table where the column should have been numerical, not categorical -- but that's a separate issue.  Even if the column is made categorical, this bug remains.)"},"comment":{"url":"https://api.github.com/repos/probcomp/bayeslite/issues/comments/172603446","html_url":"https://github.com/probcomp/bayeslite/issues/361#issuecomment-172603446","issue_url":"https://api.github.com/repos/probcomp/bayeslite/issues/361","id":172603446,"user":{"login":"madeleineth","id":3121668,"avatar_url":"https://avatars.githubusercontent.com/u/3121668?v=3","gravatar_id":"","url":"https://api.github.com/users/madeleineth","html_url":"https://github.com/madeleineth","followers_url":"https://api.github.com/users/madeleineth/followers","following_url":"https://api.github.com/users/madeleineth/following{/other_user}","gists_url":"https://api.github.com/users/madeleineth/gists{/gist_id}","starred_url":"https://api.github.com/users/madeleineth/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/madeleineth/subscriptions","organizations_url":"https://api.github.com/users/madeleineth/orgs","repos_url":"https://api.github.com/users/madeleineth/repos","events_url":"https://api.github.com/users/madeleineth/events{/privacy}","received_events_url":"https://api.github.com/users/madeleineth/received_events","type":"User","site_admin":false},"created_at":"2016-01-18T17:46:12Z","updated_at":"2016-01-18T17:46:12Z","body":"For what it's worth: `unicode` (and `str`) should not be used to encode Python objects in an invertible way. If you want to encode them invertibly, use `repr`. `json.dumps` and `repr` give an exact (though not unique) string representation. `unicode` and `str` do not.\r\n\r\n    >>> json.dumps(2.0/3.0)\r\n    '0.6666666666666666'\r\n    >>> repr(2.0/3.0)\r\n    '0.6666666666666666'\r\n    >>> str(2.0/3.0)\r\n    '0.666666666667'\r\n    >>> float(str(2.0/3.0)) - 2.0/3.0\r\n    3.333999742949345e-13\r\n    >>> float(repr(2.0/3.0)) - 2.0/3.0\r\n    0.0\r\n    >>> import sys\r\n    >>> sys.version\r\n    '2.7.6 (default, Jun 22 2015, 17:58:13) \\n[GCC 4.8.2]'\r\n"}}