{"action":"created","issue":{"url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/400","repository_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb","labels_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/400/labels{/name}","comments_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/400/comments","events_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/400/events","html_url":"https://github.com/NeurodataWithoutBorders/pynwb/pull/400","id":307014625,"number":400,"title":"Enh/hdf5comp","user":{"login":"oruebel","id":10999845,"avatar_url":"https://avatars0.githubusercontent.com/u/10999845?v=4","gravatar_id":"","url":"https://api.github.com/users/oruebel","html_url":"https://github.com/oruebel","followers_url":"https://api.github.com/users/oruebel/followers","following_url":"https://api.github.com/users/oruebel/following{/other_user}","gists_url":"https://api.github.com/users/oruebel/gists{/gist_id}","starred_url":"https://api.github.com/users/oruebel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oruebel/subscriptions","organizations_url":"https://api.github.com/users/oruebel/orgs","repos_url":"https://api.github.com/users/oruebel/repos","events_url":"https://api.github.com/users/oruebel/events{/privacy}","received_events_url":"https://api.github.com/users/oruebel/received_events","type":"User","site_admin":false},"labels":[{"id":661411412,"url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/labels/enhancement","name":"enhancement","color":"84b6eb","default":true}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":{"url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/milestones/4","html_url":"https://github.com/NeurodataWithoutBorders/pynwb/milestone/4","labels_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/milestones/4/labels","id":2860080,"number":4,"title":"NWB 2.0 Full","description":"Full stable release of the NWB 2.0","creator":{"login":"oruebel","id":10999845,"avatar_url":"https://avatars0.githubusercontent.com/u/10999845?v=4","gravatar_id":"","url":"https://api.github.com/users/oruebel","html_url":"https://github.com/oruebel","followers_url":"https://api.github.com/users/oruebel/followers","following_url":"https://api.github.com/users/oruebel/following{/other_user}","gists_url":"https://api.github.com/users/oruebel/gists{/gist_id}","starred_url":"https://api.github.com/users/oruebel/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/oruebel/subscriptions","organizations_url":"https://api.github.com/users/oruebel/orgs","repos_url":"https://api.github.com/users/oruebel/repos","events_url":"https://api.github.com/users/oruebel/events{/privacy}","received_events_url":"https://api.github.com/users/oruebel/received_events","type":"User","site_admin":false},"open_issues":26,"closed_issues":14,"state":"open","created_at":"2017-10-23T17:41:16Z","updated_at":"2018-04-04T16:36:16Z","due_on":"2018-06-30T07:00:00Z","closed_at":null},"comments":3,"created_at":"2018-03-20T19:52:51Z","updated_at":"2018-04-07T11:56:17Z","closed_at":null,"author_association":"CONTRIBUTOR","pull_request":{"url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/pulls/400","html_url":"https://github.com/NeurodataWithoutBorders/pynwb/pull/400","diff_url":"https://github.com/NeurodataWithoutBorders/pynwb/pull/400.diff","patch_url":"https://github.com/NeurodataWithoutBorders/pynwb/pull/400.patch"},"body":"## Motivation\r\n\r\nEnable users to  directly specify advanced I/O options for the writing of datasets, specifically chunking and compression. \r\n\r\n# Summary of changes\r\n* Expanded H5DataIO to enable users to specify more advanced I/O settings\r\n* Updated the HDF5IO backend to support the use of the expanded I/O settings of H5DataIO\r\n* Removed direct support Iterable from H5DataIO.write_dataset and instead require users to explicitly wrap iterables using DataChunkIterator. This is to avoid implicit behavior. For convenience I added a factory function for DataChunkIterator to generate it from an Iterable. \r\n* Added max_shape and dtype functions to AbstractDataChunkIterator to ensure we only assume functionality that is on the base class, rather than DataChunkIterator.\r\n* Added/expanded tests to test the new options\r\n\r\n## How to test the behavior?\r\n\r\nThe following is a direct test derived from the test suit that uses the FORM API only. \r\n\r\n```\r\nimport tempfile\r\nimport numpy as np\r\nfrom pynwb.form.backends.hdf5.h5tools import HDF5IO\r\nfrom pynwb.form.backends.hdf5 import H5DataIO\r\nfrom pynwb.form.build import DatasetBuilder\r\ntest_temp_file = tempfile.NamedTemporaryFile()\r\ntest_temp_file.close()\r\nio = HDF5IO(test_temp_file.name)\r\nf = io._file\r\na = H5DataIO(np.arange(30).reshape(5, 2, 3), compression='gzip', compression_opts=5, shuffle=True, fletcher32=True)\r\nio.write_dataset(f, DatasetBuilder('test_dataset', a, attributes={}))\r\ndset = f['test_dataset']\r\nprint(\"Compression: %s\" % str(dset.compression))\r\nprint(\"Compression Options: %s\" % str(dset.compression_opts))\r\nprint(\"Shuffle: %s\" % str(dset.shuffle))\r\nprint(\"Fletcher32: %s\" %  str(dset.fletcher32))\r\nio.close()\r\n```\r\n\r\nWhen using PyNWB containers one woulld hand the H5DataIO object as an input for a specific dataset, e.g., the data of a TimeSeries.\r\n\r\n## Checklist\r\n\r\n- [X] Have you checked our [Contributing](https://github.com/NeurodataWithoutBorders/pynwb/blob/dev/docs/CONTRIBUTING.rst) document?\r\n- [X] Have you ensured the PR description clearly describes problem and the solution?\r\n- [X] Is your contribution compliant with our coding style ? This can be checked running `flake8` from the source directory.\r\n- [X] Have you checked to ensure that there aren't other open [Pull Requests](https://github.com/NeurodataWithoutBorders/pynwb/pulls) for the same change?\r\n- [X] Have you included the relevant issue number using `#XXX` notation where `XXX` is the issue number ?\r\n"},"comment":{"url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/comments/379464145","html_url":"https://github.com/NeurodataWithoutBorders/pynwb/pull/400#issuecomment-379464145","issue_url":"https://api.github.com/repos/NeurodataWithoutBorders/pynwb/issues/400","id":379464145,"user":{"login":"bendichter","id":844306,"avatar_url":"https://avatars0.githubusercontent.com/u/844306?v=4","gravatar_id":"","url":"https://api.github.com/users/bendichter","html_url":"https://github.com/bendichter","followers_url":"https://api.github.com/users/bendichter/followers","following_url":"https://api.github.com/users/bendichter/following{/other_user}","gists_url":"https://api.github.com/users/bendichter/gists{/gist_id}","starred_url":"https://api.github.com/users/bendichter/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bendichter/subscriptions","organizations_url":"https://api.github.com/users/bendichter/orgs","repos_url":"https://api.github.com/users/bendichter/repos","events_url":"https://api.github.com/users/bendichter/events{/privacy}","received_events_url":"https://api.github.com/users/bendichter/received_events","type":"User","site_admin":false},"created_at":"2018-04-07T11:56:17Z","updated_at":"2018-04-07T11:56:17Z","author_association":"CONTRIBUTOR","body":"@jcfr had some good ideas about using sha tags to ensure that linked files are correct."}}
{"actor":{"display_login":"bendichter"}}