{"action":"created","issue":{"url":"https://api.github.com/repos/msgpack/msgpack-python/issues/126","repository_url":"https://api.github.com/repos/msgpack/msgpack-python","labels_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/126/labels{/name}","comments_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/126/comments","events_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/126/events","html_url":"https://github.com/msgpack/msgpack-python/issues/126","id":58342085,"number":126,"title":"memoryview objects are not fully supported","user":{"login":"himikof","id":43698,"avatar_url":"https://avatars.githubusercontent.com/u/43698?v=3","gravatar_id":"","url":"https://api.github.com/users/himikof","html_url":"https://github.com/himikof","followers_url":"https://api.github.com/users/himikof/followers","following_url":"https://api.github.com/users/himikof/following{/other_user}","gists_url":"https://api.github.com/users/himikof/gists{/gist_id}","starred_url":"https://api.github.com/users/himikof/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/himikof/subscriptions","organizations_url":"https://api.github.com/users/himikof/orgs","repos_url":"https://api.github.com/users/himikof/repos","events_url":"https://api.github.com/users/himikof/events{/privacy}","received_events_url":"https://api.github.com/users/himikof/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2015-02-20T11:50:20Z","updated_at":"2016-05-12T12:12:31Z","closed_at":null,"body":"`memoryview` is the python3 type for non-owning memory buffer objects, also backported to python 2.7. `unpackb` and `Unpacker.feed` should unpack them without copying, and packing functions should handle them the same as `bytes`.\r\nThe current support is quite limited due to multiple issues:\r\n### Python 3.3\r\n```\r\n>>> msgpack._unpacker.unpackb(memoryview(b'\\x91\\xc3'))\r\n[True]\r\n>>> msgpack.fallback.unpackb(memoryview(b'\\x91\\xc3'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/nofitserov/.local/lib64/python3.3/site-packages/msgpack/fallback.py\", line 93, in unpackb\r\n    ret = unpacker._fb_unpack()\r\n  File \"/home/nofitserov/.local/lib64/python3.3/site-packages/msgpack/fallback.py\", line 383, in _fb_unpack\r\n    typ, n, obj = self._read_header(execute, write_bytes)\r\n  File \"/home/nofitserov/.local/lib64/python3.3/site-packages/msgpack/fallback.py\", line 274, in _read_header\r\n    b = ord(c)\r\nTypeError: ord() expected string of length 1, but memoryview found\r\n>>> msgpack._packer.Packer().pack(memoryview(b'abc'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"_packer.pyx\", line 224, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:224)\r\n  File \"_packer.pyx\", line 226, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:226)\r\n  File \"_packer.pyx\", line 221, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:221)\r\nTypeError: can't serialize <memory at 0x7f37c3e41460>\r\n>>> msgpack.fallback.Packer().pack(memoryview(b'abc'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/nofitserov/.local/lib64/python3.3/site-packages/msgpack/fallback.py\", line 618, in pack\r\n    self._pack(obj)\r\n  File \"/home/nofitserov/.local/lib64/python3.3/site-packages/msgpack/fallback.py\", line 615, in _pack\r\n    raise TypeError(\"Cannot serialize %r\" % obj)\r\nTypeError: Cannot serialize <memory at 0x7f37c3e41390>\r\n```\r\n### Python 2.7\r\n```\r\n>>> msgpack._unpacker.unpackb(memoryview(b'\\x91\\xc3'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"_unpacker.pyx\", line 105, in msgpack._unpacker.unpackb (msgpack/_unpacker.cpp:105)\r\nTypeError: expected a readable buffer object\r\n>>> msgpack.fallback.unpackb(memoryview(b'\\x91\\xc3'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/nofitserov/.local/lib64/python2.7/site-packages/msgpack/fallback.py\", line 93, in unpackb\r\n    ret = unpacker._fb_unpack()\r\n  File \"/home/nofitserov/.local/lib64/python2.7/site-packages/msgpack/fallback.py\", line 381, in _fb_unpack\r\n    typ, n, obj = self._read_header(execute, write_bytes)\r\n  File \"/home/nofitserov/.local/lib64/python2.7/site-packages/msgpack/fallback.py\", line 272, in _read_header\r\n    b = ord(c)\r\nTypeError: ord() expected string of length 1, but memoryview found\r\n>>> msgpack._packer.Packer().pack(memoryview(b'abc'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"_packer.pyx\", line 206, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:206)\r\n  File \"_packer.pyx\", line 208, in msgpack._packer.Packer.pack (msgpack/_packer.cpp:208)\r\n  File \"_packer.pyx\", line 203, in msgpack._packer.Packer._pack (msgpack/_packer.cpp:203)\r\nTypeError: can't serialize <memory at 0x7f506e40ddf8>\r\n>>> msgpack.fallback.Packer().pack(memoryview(b'abc'))\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/nofitserov/.local/lib64/python2.7/site-packages/msgpack/fallback.py\", line 616, in pack\r\n    self._pack(obj)\r\n  File \"/home/nofitserov/.local/lib64/python2.7/site-packages/msgpack/fallback.py\", line 613, in _pack\r\n    raise TypeError(\"Cannot serialize %r\" % obj)\r\nTypeError: Cannot serialize <memory at 0x7f506e40ddf8>\r\n```\r\nThe only available workaround right now is to explicitly convert `memoryview` objects to `bytes`, needlessly copying the contents, which degrades performance, especially for unpacking large objects."},"comment":{"url":"https://api.github.com/repos/msgpack/msgpack-python/issues/comments/218738830","html_url":"https://github.com/msgpack/msgpack-python/issues/126#issuecomment-218738830","issue_url":"https://api.github.com/repos/msgpack/msgpack-python/issues/126","id":218738830,"user":{"login":"nxsofsys","id":1502388,"avatar_url":"https://avatars.githubusercontent.com/u/1502388?v=3","gravatar_id":"","url":"https://api.github.com/users/nxsofsys","html_url":"https://github.com/nxsofsys","followers_url":"https://api.github.com/users/nxsofsys/followers","following_url":"https://api.github.com/users/nxsofsys/following{/other_user}","gists_url":"https://api.github.com/users/nxsofsys/gists{/gist_id}","starred_url":"https://api.github.com/users/nxsofsys/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/nxsofsys/subscriptions","organizations_url":"https://api.github.com/users/nxsofsys/orgs","repos_url":"https://api.github.com/users/nxsofsys/repos","events_url":"https://api.github.com/users/nxsofsys/events{/privacy}","received_events_url":"https://api.github.com/users/nxsofsys/received_events","type":"User","site_admin":false},"created_at":"2016-05-12T12:12:31Z","updated_at":"2016-05-12T12:12:31Z","body":"> msgpack.unpackb(memoryview(b'\\x91\\xc3'))\r\n\r\nI think memoryview should be packed before unpack. Can you try following code:\r\n\r\n`msgpack.unpackb(msgpack.packb(memoryview(b'\\x91\\xc3')))`\r\n"}}