{"action":"created","issue":{"url":"https://api.github.com/repos/h5py/h5py/issues/581","repository_url":"https://api.github.com/repos/h5py/h5py","labels_url":"https://api.github.com/repos/h5py/h5py/issues/581/labels{/name}","comments_url":"https://api.github.com/repos/h5py/h5py/issues/581/comments","events_url":"https://api.github.com/repos/h5py/h5py/issues/581/events","html_url":"https://github.com/h5py/h5py/issues/581","id":84011201,"number":581,"title":"Compound with vlen field","user":{"login":"aothms","id":1096535,"avatar_url":"https://avatars.githubusercontent.com/u/1096535?v=3","gravatar_id":"","url":"https://api.github.com/users/aothms","html_url":"https://github.com/aothms","followers_url":"https://api.github.com/users/aothms/followers","following_url":"https://api.github.com/users/aothms/following{/other_user}","gists_url":"https://api.github.com/users/aothms/gists{/gist_id}","starred_url":"https://api.github.com/users/aothms/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/aothms/subscriptions","organizations_url":"https://api.github.com/users/aothms/orgs","repos_url":"https://api.github.com/users/aothms/repos","events_url":"https://api.github.com/users/aothms/events{/privacy}","received_events_url":"https://api.github.com/users/aothms/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":6,"created_at":"2015-06-02T12:57:38Z","updated_at":"2016-02-11T02:37:20Z","closed_at":null,"body":"Hi,\r\n\r\nPlease see the following code snippet that illustrates an issue (or misunderstanding from my side) on how to create a compound type with a vlen member. Note that the dataset is created successfully, but assignment into the dataset fails.\r\n\r\nIt's difficult for me to see what's going on exactly, but it seems at some point the vlen field becomes an opaque type, and as such it seems that `h5py.h5t.py_create(x).dtype != x` when wrapping the vlen field into the compound structure. Not sure if this last observation is relevant, but this equality does seem to uphold in the case of simple integers e.g `h5py.h5t.py_create(np.dtype('int32')).dtype == np.dtype('int32')`.\r\n\r\nAny advise? Thanks in advance.\r\n\r\n~~~\r\nimport h5py\r\nimport numpy as np\r\n\r\nf = h5py.File(\"test.hdf\", \"w\")\r\ng = f.create_group(\"test\")\r\n\r\nvlen_int32 = h5py.special_dtype(vlen=np.dtype('int32'))\r\nvlen_int32_h5 = h5py.h5t.py_create(vlen_int32)\r\ncompound = h5py.h5t.create(h5py.h5t.COMPOUND, h5py.h5t.NATIVE_INT64.get_size() + vlen_int32_h5.get_size())\r\ncompound.insert(\"int\", 0, h5py.h5t.NATIVE_INT64)\r\ncompound.insert(\"ints\", h5py.h5t.NATIVE_INT64.get_size(), vlen_int32_h5)\r\ncompound.commit(g.id, \"int_ints\")\r\n\r\ndset = g.create_dataset(\"int_ints_dset\", (1,), dtype=compound)\r\n# both approaches below fail with TypeError: expected a readable buffer object\r\ndset[0] = np.array([1,(2,3,4)], dtype=compound.dtype)\r\ndset[0] = [1,(2,3,4)]\r\n~~~"},"comment":{"url":"https://api.github.com/repos/h5py/h5py/issues/comments/182677352","html_url":"https://github.com/h5py/h5py/issues/581#issuecomment-182677352","issue_url":"https://api.github.com/repos/h5py/h5py/issues/581","id":182677352,"user":{"login":"jreadey","id":7785492,"avatar_url":"https://avatars.githubusercontent.com/u/7785492?v=3","gravatar_id":"","url":"https://api.github.com/users/jreadey","html_url":"https://github.com/jreadey","followers_url":"https://api.github.com/users/jreadey/followers","following_url":"https://api.github.com/users/jreadey/following{/other_user}","gists_url":"https://api.github.com/users/jreadey/gists{/gist_id}","starred_url":"https://api.github.com/users/jreadey/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jreadey/subscriptions","organizations_url":"https://api.github.com/users/jreadey/orgs","repos_url":"https://api.github.com/users/jreadey/repos","events_url":"https://api.github.com/users/jreadey/events{/privacy}","received_events_url":"https://api.github.com/users/jreadey/received_events","type":"User","site_admin":false},"created_at":"2016-02-11T02:37:20Z","updated_at":"2016-02-11T02:37:20Z","body":"@aothms - my example was working fine on py3.4, but when I tried it on py27 I got the same crash.  It looks like updating my numpy from 1.10.1 to 1.10.4 fixed it.  \r\n\r\nFor reference here's the complete source:\r\n```\r\nimport h5py\r\nimport numpy as np\r\n# create an dataset with compound type and array element\r\nprint(h5py.version.info)\r\nf = h5py.File(\"compound_array_vlen_str_dset.h5\", \"w\") \r\ndt_vlen = h5py.special_dtype(vlen=str)\r\ndt_arr = np.dtype((dt_vlen, (4,)))\r\ndt_compound = np.dtype([('VALUE1', np.float64), ('VALUE2', np.int64), ('VALUE3', dt_arr) ])\r\ndset = f.create_dataset('DS1', (), dtype=dt_compound)\r\nval = dset[()]\r\nprint(val)\r\nval[0] = 3.12\r\nval[1] = 42\r\nval[2][0] = b'abcd'\r\nval[2][1] = b'bcda'\r\nval[2][2] = b'abdc'\r\nval[2][3] = b'dcab'\r\nprint(val)\r\nf.close()\r\n```\r\n\r\nAnd the output:\r\n```\r\nh5py    2.5.0\r\nHDF5    1.8.15\r\nPython  2.7.11 |Anaconda 2.3.0 (x86_64)| (default, Dec  6 2015, 18:57:58)\r\n[GCC 4.2.1 (Apple Inc. build 5577)]\r\nsys.platform    darwin\r\nsys.maxsize     9223372036854775807\r\nnumpy   1.10.4\r\n\r\n(0.0, 0, ['', '', '', ''])\r\n(3.12, 42, ['abcd', 'bcda', 'abdc', 'dcab'])\r\n```\r\nThis is on Mac OS X, let me know if it works for you on 32-bit Windows.\r\n\r\nVLEN array of ints works too.  As above, I'm not using the low-level api, just passing the dtype to create_dataset:\r\n```\r\nimport h5py\r\nimport numpy as np\r\n\r\n\r\n# create an dataset with compound type and array element\r\nprint(h5py.version.info)\r\nf = h5py.File(\"compound_array_vlen_str_dset.h5\", \"w\") \r\ndt_vlen = h5py.special_dtype(vlen=np.int32)\r\ndt_arr = np.dtype((dt_vlen, (4,)))\r\ndt_compound = np.dtype([('VALUE1', np.float64), ('VALUE2', np.int64), ('VALUE3', dt_arr) ])\r\ndset = f.create_dataset('DS1', (), dtype=dt_compound)\r\nval = dset[()]\r\nprint(val)\r\nval[0] = 3.12\r\nval[1] = 42\r\nval[2][0] = (1,)\r\nval[2][1] = (1,2)\r\nval[2][2] = (1,2,3)\r\nval[2][3] = (1,2,3,4)\r\nprint(val)\r\nf.close()\r\n```\r\n\r\n"}}