{"action":"created","issue":{"url":"https://api.github.com/repos/nipy/nibabel/issues/391","labels_url":"https://api.github.com/repos/nipy/nibabel/issues/391/labels{/name}","comments_url":"https://api.github.com/repos/nipy/nibabel/issues/391/comments","events_url":"https://api.github.com/repos/nipy/nibabel/issues/391/events","html_url":"https://github.com/nipy/nibabel/pull/391","id":120226158,"number":391,"title":"Streamlines API","user":{"login":"MarcCote","id":660004,"avatar_url":"https://avatars.githubusercontent.com/u/660004?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcCote","html_url":"https://github.com/MarcCote","followers_url":"https://api.github.com/users/MarcCote/followers","following_url":"https://api.github.com/users/MarcCote/following{/other_user}","gists_url":"https://api.github.com/users/MarcCote/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcCote/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcCote/subscriptions","organizations_url":"https://api.github.com/users/MarcCote/orgs","repos_url":"https://api.github.com/users/MarcCote/repos","events_url":"https://api.github.com/users/MarcCote/events{/privacy}","received_events_url":"https://api.github.com/users/MarcCote/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":50,"created_at":"2015-12-03T17:25:04Z","updated_at":"2016-01-29T12:44:46Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/nipy/nibabel/pulls/391","html_url":"https://github.com/nipy/nibabel/pull/391","diff_url":"https://github.com/nipy/nibabel/pull/391.diff","patch_url":"https://github.com/nipy/nibabel/pull/391.patch"},"body":"This is take two at providing a general API to manage different streamlines file format (the code can all be found in `nibabel.streamlines.`). This is related to this somewhat outdated [BIAP5](https://github.com/nipy/nibabel/wiki/BIAP5). I initially wanted to make two PRs but as @Garyfallidis [pointed out](https://github.com/nipy/nibabel/pull/243#issuecomment-160810062), without the entire PR \"it will make it difficult to try this with real data\".\r\n\r\n\r\nThis new streamline API is divided in two main parts:\r\n\r\n1. Representation of streamlines in memory (`Tractogram` and `CompactList` classes)\r\n2. Loading and saving streamlines from/to different file format (`TractogramFile` class)\r\n\r\n*We use the term 'Tractogram' to refer to a set of streamlines, usually the output of a tractography algorithm.*\r\n\r\n![streamlines_api](https://cloud.githubusercontent.com/assets/660004/11518705/c1f11fb0-9861-11e5-8bda-13cbabc86056.png)\r\n\r\n\r\n#### CompactList\r\nThe `CompactList` class needs to be explained first as it is at the heart of the streamlines representation. I guess it could be seen as a sort of jagged/ragged array to store a list of 2D arrays that have different length but a same number of dimensions (e.g. a list of list of points). For efficiency, we store a list of streamlines contiguously in memory using what we call a `CompactList`. This was needed as @Garyfallidis and I realized that a list of numpy arrays has too much overhead in term of memory when handling >1M streamlines (loading a tractogram of 2.5Gb could take up to 5Gb in RAM!).\r\n\r\nA `CompactList` behaves exactly like a `list` of `2D ndarray`. That is, it supports appending, extending, indexing (supports slicing too), iterating, etc. as a `list` object. For example, indexing a `CompactList` object will return a 2D `ndarray`, as one would expect, representing the 3D points of the selected streamline. Note that, while `CompactList` does not support deletion (i.e. no `del` nor `remove`), one can efficiently use slicing to achieve the same goal.\r\n\r\nA `CompactList` object can be instantiated using any iterable yielding `2D ndarray`:\r\n```\r\nimport numpy as np\r\nimport nibabel as nib\r\ndata = [np.random.rand(10, 3), np.random.rand(3, 3)]\r\nclist = nib.streamlines.CompactList(data)\r\n```\r\n\r\nSlicing a `CompactList` object returns a new `CompactList` object acting as a view (*no memory duplication*). Of course, one can use the `copy` method to \"force\" the memory duplication.\r\n\r\n\r\n#### Tractogram\r\nThis part handles streamlines in memory using the Python class `Tractogram`. This class provides a mechanism to keep data associated to points of every streamline (`data_per_point` property). It can also keep data associated to each streamline (`data_per_streamline` property). \r\n\r\nRegarding the reference space we used for the streamlines of a tractogram live in, we decided to follow the NiBabel convention: the streamlines are assumed to be in **RAS+**, points are expressed in **millimetre** and the coordinate of a voxel (eg. (0,0,0) or (1,2,3)) refers to the **centre of that voxel**.\r\n\r\nA `Tractogram` objects supports indexing, slicing and iterating. For example, indexing a `Tractogram` object will return a `TractogramItem` object, representing a single streamline with its associated data. Note that, while `Tractogram` does not support deletion (i.e. no `del` nor `remove`), one can efficiently use slicing to achieve the same goal. Again, slicing a `Tractogram` object returns a new `Tractogram` object acting as a view (*no memory duplication*).\r\n\r\nHere a small example of instantiating a `Tractogram` object:\r\n```\r\nimport numpy as np\r\nimport nibabel as nib\r\ndata = [np.random.rand(10, 3), np.random.rand(3, 3)]\r\ntractogram = nib.streamlines.Tractogram(streamlines=data)\r\ncolors = [(1, 0, 0), (0, 1, 0)]\r\ntractogram.data_per_streamlines['colors'] = colors\r\n```\r\n\r\n\r\n#### Tractogram File\r\nThis part handles the saving and loading of `Tractogram` objects from/to the disk. Right now, as a proof of concept, only the TRK file format is supported (support for TCK is already in progress and will be added in a following PR).\r\n\r\nThe `TractogramFile` class is abstract. Each file format should have its own class inheriting `TractogramFile` (e.g., `TrkFile`).\r\n\r\n\r\n\r\n#### Realistic example\r\n\r\nHere a small example of loading a TRK file, computing the length of each streamlines, adding that information to the tractogram and saving it back.\r\n```\r\nimport nibabel as nib\r\ntrk = nib.streamlines.load('my_streamlines.trk')\r\n\r\nfrom dipy.tracking.streamline import length\r\nlengths = length(trk.tractogram.streamlines)\r\n\r\ntrk.tractogram.data_per_streamline['lengths'] = lengths\r\n\r\nnew_trk = nib.streamlines.TrkFile(trk.tractogram, trk.header)\r\nnib.streamlines.save(new_trk, \"my_streamlines.trk\")\r\n```"},"comment":{"url":"https://api.github.com/repos/nipy/nibabel/issues/comments/176733161","html_url":"https://github.com/nipy/nibabel/pull/391#issuecomment-176733161","issue_url":"https://api.github.com/repos/nipy/nibabel/issues/391","id":176733161,"user":{"login":"MarcCote","id":660004,"avatar_url":"https://avatars.githubusercontent.com/u/660004?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcCote","html_url":"https://github.com/MarcCote","followers_url":"https://api.github.com/users/MarcCote/followers","following_url":"https://api.github.com/users/MarcCote/following{/other_user}","gists_url":"https://api.github.com/users/MarcCote/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcCote/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcCote/subscriptions","organizations_url":"https://api.github.com/users/MarcCote/orgs","repos_url":"https://api.github.com/users/MarcCote/repos","events_url":"https://api.github.com/users/MarcCote/events{/privacy}","received_events_url":"https://api.github.com/users/MarcCote/received_events","type":"User","site_admin":false},"created_at":"2016-01-29T12:44:46Z","updated_at":"2016-01-29T12:44:46Z","body":"Ok, I don't mind be coherent with NiBabel. I'll add a affine property."}}