{"action":"created","issue":{"url":"https://api.github.com/repos/SAP/PyHDB/issues/102","repository_url":"https://api.github.com/repos/SAP/PyHDB","labels_url":"https://api.github.com/repos/SAP/PyHDB/issues/102/labels{/name}","comments_url":"https://api.github.com/repos/SAP/PyHDB/issues/102/comments","events_url":"https://api.github.com/repos/SAP/PyHDB/issues/102/events","html_url":"https://github.com/SAP/PyHDB/issues/102","id":321752650,"number":102,"title":"cesu8 decode error","user":{"login":"jstorryefx","id":39140237,"avatar_url":"https://avatars0.githubusercontent.com/u/39140237?v=4","gravatar_id":"","url":"https://api.github.com/users/jstorryefx","html_url":"https://github.com/jstorryefx","followers_url":"https://api.github.com/users/jstorryefx/followers","following_url":"https://api.github.com/users/jstorryefx/following{/other_user}","gists_url":"https://api.github.com/users/jstorryefx/gists{/gist_id}","starred_url":"https://api.github.com/users/jstorryefx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jstorryefx/subscriptions","organizations_url":"https://api.github.com/users/jstorryefx/orgs","repos_url":"https://api.github.com/users/jstorryefx/repos","events_url":"https://api.github.com/users/jstorryefx/events{/privacy}","received_events_url":"https://api.github.com/users/jstorryefx/received_events","type":"User","site_admin":false},"labels":[{"id":156736808,"url":"https://api.github.com/repos/SAP/PyHDB/labels/bug","name":"bug","color":"fc2929","default":true},{"id":942374875,"url":"https://api.github.com/repos/SAP/PyHDB/labels/need-info","name":"need-info","color":"0d387c","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2018-05-09T22:51:39Z","updated_at":"2018-05-23T16:26:04Z","closed_at":null,"author_association":"NONE","body":"I'm receiving an UnicodeDecodeError error when I try to pull data from HANA using the PyHDB library while iterating through the results in the following code.\r\n```\r\nfrom sqlalchemy import create_engine\r\nengine = create_engine('hana+pyhdb://%s:%s@%s:%s' % (username, password, host, port))\r\nconnection = engine.connect()\r\nresult = connection.execute(\"select * from TASKS\")\r\nfor row in result.fetchall():\r\n   test = row\r\nconnection.close()\r\n```\r\n\r\nI've narrowed it down to the following snippet.\r\n\r\n```\r\nimport pyhdb.cesu8\r\ninput = b'\\xed\\xa0\\xbd'\r\npyhdb.cesu8.decode(input)\r\n```\r\n\r\nWhich generates the error:\r\n\r\n`UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 0: invalid continuation byte`\r\n\r\nIs this the expected behavior? My assumption is that if the data can be stored in HANA we should be able to read it with the library but my knowledge of utf-8/cesu8 is limited.\r\n\r\nThanks,\r\n\r\nJon"},"comment":{"url":"https://api.github.com/repos/SAP/PyHDB/issues/comments/391411200","html_url":"https://github.com/SAP/PyHDB/issues/102#issuecomment-391411200","issue_url":"https://api.github.com/repos/SAP/PyHDB/issues/102","id":391411200,"user":{"login":"wadeej","id":31515450,"avatar_url":"https://avatars3.githubusercontent.com/u/31515450?v=4","gravatar_id":"","url":"https://api.github.com/users/wadeej","html_url":"https://github.com/wadeej","followers_url":"https://api.github.com/users/wadeej/followers","following_url":"https://api.github.com/users/wadeej/following{/other_user}","gists_url":"https://api.github.com/users/wadeej/gists{/gist_id}","starred_url":"https://api.github.com/users/wadeej/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wadeej/subscriptions","organizations_url":"https://api.github.com/users/wadeej/orgs","repos_url":"https://api.github.com/users/wadeej/repos","events_url":"https://api.github.com/users/wadeej/events{/privacy}","received_events_url":"https://api.github.com/users/wadeej/received_events","type":"User","site_admin":false},"created_at":"2018-05-23T16:26:04Z","updated_at":"2018-05-23T16:26:04Z","author_association":"NONE","body":"It seems like the trouble is that CESU-8 _SHOULD_ pass in 6 byte long sequence, but what we are running into is at some point SAP Hana is passing just a 3 byte sequence (`b'\\xed\\xa0\\xbd'` in @jstorryefx 's case) and the so the conversion fails.  A 6 byte CESU-8 sequence should correspond to a 4 byte UTF-8 sequence, but I do not know if the same holds try for a 3 byte sequence (i.e. does 3 byte CESU-8 correspond to a 2 byte UTF-8?)"}}
{"actor":{"display_login":"wadeej"}}