{"action":"created","issue":{"url":"https://api.github.com/repos/nipy/nibabel/issues/391","repository_url":"https://api.github.com/repos/nipy/nibabel","labels_url":"https://api.github.com/repos/nipy/nibabel/issues/391/labels{/name}","comments_url":"https://api.github.com/repos/nipy/nibabel/issues/391/comments","events_url":"https://api.github.com/repos/nipy/nibabel/issues/391/events","html_url":"https://github.com/nipy/nibabel/pull/391","id":120226158,"number":391,"title":"Streamlines API","user":{"login":"MarcCote","id":660004,"avatar_url":"https://avatars.githubusercontent.com/u/660004?v=3","gravatar_id":"","url":"https://api.github.com/users/MarcCote","html_url":"https://github.com/MarcCote","followers_url":"https://api.github.com/users/MarcCote/followers","following_url":"https://api.github.com/users/MarcCote/following{/other_user}","gists_url":"https://api.github.com/users/MarcCote/gists{/gist_id}","starred_url":"https://api.github.com/users/MarcCote/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/MarcCote/subscriptions","organizations_url":"https://api.github.com/users/MarcCote/orgs","repos_url":"https://api.github.com/users/MarcCote/repos","events_url":"https://api.github.com/users/MarcCote/events{/privacy}","received_events_url":"https://api.github.com/users/MarcCote/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":103,"created_at":"2015-12-03T17:25:04Z","updated_at":"2016-06-01T16:01:50Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/nipy/nibabel/pulls/391","html_url":"https://github.com/nipy/nibabel/pull/391","diff_url":"https://github.com/nipy/nibabel/pull/391.diff","patch_url":"https://github.com/nipy/nibabel/pull/391.patch"},"body":"This is take two at providing a general API to manage different streamlines file format (the code can all be found in `nibabel.streamlines.`). This is related to this somewhat outdated [BIAP5](https://github.com/nipy/nibabel/wiki/BIAP5). I initially wanted to make two PRs but as @Garyfallidis [pointed out](https://github.com/nipy/nibabel/pull/243#issuecomment-160810062), without the entire PR \"it will make it difficult to try this with real data\".\r\n\r\n\r\nThis new streamline API is divided in two main parts:\r\n\r\n1. Representation of streamlines in memory (`Tractogram` and `ArraySequence` classes)\r\n2. Loading and saving streamlines from/to different file format (`TractogramFile` class)\r\n\r\n*We use the term 'Tractogram' to refer to a set of streamlines, usually the output of a tractography algorithm.*\r\n\r\n![streamlines_api](https://cloud.githubusercontent.com/assets/660004/12959160/4606db04-d003-11e5-9161-778f488c8bfd.png)\r\n\r\n#### ArraySequence\r\nThe `ArraySequence` class needs to be explained first as it is at the heart of the streamlines representation. I guess it could be seen as a sort of jagged/ragged array to store a list of 2D arrays that have different length but a same number of dimensions (e.g. a list of list of points). For efficiency, we store a list of streamlines contiguously in memory using what we call a `ArraySequence`. This was needed as @Garyfallidis and I realized that a list of numpy arrays has too much overhead in term of memory when handling >1M streamlines (loading a tractogram of 2.5Gb could take up to 5Gb in RAM!).\r\n\r\nA `ArraySequence` behaves exactly like a `list` of `2D ndarray`. That is, it supports appending, extending, indexing (supports slicing too), iterating, etc. as a `list` object. For example, indexing a `ArraySequence` object will return a 2D `ndarray`, as one would expect, representing the 3D points of the selected streamline. Note that, while `ArraySequence` does not support deletion (i.e. no `del` nor `remove`), one can efficiently use slicing to achieve the same goal.\r\n\r\nA `ArraySequence` object can be instantiated using any iterable yielding `2D array`:\r\n```\r\nimport numpy as np\r\nimport nibabel as nib\r\ndata = [np.random.rand(10, 3), np.random.rand(3, 3)]\r\nseq = nib.streamlines.ArraySequence(data)\r\n```\r\n\r\nSlicing a `ArraySequence` object returns a new `ArraySequence` object acting as a view (*no memory duplication*). Of course, one can use the `copy` method to \"force\" the memory duplication.\r\n\r\n\r\n#### Tractogram\r\nThis part handles streamlines in memory using the Python class `Tractogram`. This class provides a mechanism to keep data associated to points of every streamline (`data_per_point` property). It can also keep data associated to each streamline (`data_per_streamline` property). \r\n\r\nRegarding the reference space we used for the streamlines of a tractogram live in, we decided to follow the NiBabel convention: the streamlines are assumed to be in **RAS+**, points are expressed in **millimetre** and the coordinate of a voxel (eg. (0,0,0) or (1,2,3)) refers to the **centre of that voxel**.\r\n\r\nA `Tractogram` objects supports indexing, slicing and iterating. For example, indexing a `Tractogram` object will return a `TractogramItem` object, representing a single streamline with its associated data. Note that, while `Tractogram` does not support deletion (i.e. no `del` nor `remove`), one can efficiently use slicing to achieve the same goal. Again, slicing a `Tractogram` object returns a new `Tractogram` object acting as a view (*no memory duplication*).\r\n\r\nHere a small example of instantiating a `Tractogram` object:\r\n```\r\nimport numpy as np\r\nimport nibabel as nib\r\ndata = [np.random.rand(10, 3), np.random.rand(3, 3)]\r\ntractogram = nib.streamlines.Tractogram(streamlines=data)\r\ncolors = [(1, 0, 0), (0, 1, 0)]\r\ntractogram.data_per_streamlines['colors'] = colors\r\n```\r\n\r\n\r\n#### Tractogram File\r\nThis part handles the saving and loading of `Tractogram` objects from/to the disk. Right now, as a proof of concept, only the TRK file format is supported (support for TCK is already in progress and will be added in a following PR).\r\n\r\nThe `TractogramFile` class is abstract. Each file format should have its own class inheriting `TractogramFile` (e.g., `TrkFile`).\r\n\r\n\r\n\r\n#### Realistic example\r\n\r\nHere a small example of loading a TRK file, computing the length of each streamlines, adding that information to the tractogram and saving it back.\r\n```\r\nimport nibabel as nib\r\ntrk = nib.streamlines.load('my_streamlines.trk')\r\n\r\nfrom dipy.tracking.streamline import length\r\nlengths = length(trk.tractogram.streamlines)\r\n\r\ntrk.tractogram.data_per_streamline['lengths'] = lengths\r\n\r\nnew_trk = nib.streamlines.TrkFile(trk.tractogram, trk.header)\r\nnib.streamlines.save(new_trk, \"my_streamlines.trk\")\r\n```\r\nEDIT: Renamed `CompactList` to `ArraySequence`"},"comment":{"url":"https://api.github.com/repos/nipy/nibabel/issues/comments/223040631","html_url":"https://github.com/nipy/nibabel/pull/391#issuecomment-223040631","issue_url":"https://api.github.com/repos/nipy/nibabel/issues/391","id":223040631,"user":{"login":"Garyfallidis","id":134276,"avatar_url":"https://avatars.githubusercontent.com/u/134276?v=3","gravatar_id":"","url":"https://api.github.com/users/Garyfallidis","html_url":"https://github.com/Garyfallidis","followers_url":"https://api.github.com/users/Garyfallidis/followers","following_url":"https://api.github.com/users/Garyfallidis/following{/other_user}","gists_url":"https://api.github.com/users/Garyfallidis/gists{/gist_id}","starred_url":"https://api.github.com/users/Garyfallidis/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Garyfallidis/subscriptions","organizations_url":"https://api.github.com/users/Garyfallidis/orgs","repos_url":"https://api.github.com/users/Garyfallidis/repos","events_url":"https://api.github.com/users/Garyfallidis/events{/privacy}","received_events_url":"https://api.github.com/users/Garyfallidis/received_events","type":"User","site_admin":false},"created_at":"2016-06-01T16:01:50Z","updated_at":"2016-06-01T16:01:50Z","body":"Hi @matthew-brett and @MarcCote today is 1st of June. Only 20 days left for releasing and getting ready for the exhibition, hackathon and workshop at OHBM. This PR seems ready to be merged and any other minor issues e.g. the TRK header extension, docs etc. can be updated in additional smaller PRs. Let me know if you agree to merge this PR today. "}}
{"actor":{"display_login":"Garyfallidis"}}