{"action":"created","issue":{"url":"https://api.github.com/repos/datproject/discussions/issues/46","repository_url":"https://api.github.com/repos/datproject/discussions","labels_url":"https://api.github.com/repos/datproject/discussions/issues/46/labels{/name}","comments_url":"https://api.github.com/repos/datproject/discussions/issues/46/comments","events_url":"https://api.github.com/repos/datproject/discussions/issues/46/events","html_url":"https://github.com/datproject/discussions/issues/46","id":160950729,"number":46,"title":"Bioinformatics use case (RNA-Seq analysis)","user":{"login":"joehand","id":684965,"avatar_url":"https://avatars.githubusercontent.com/u/684965?v=3","gravatar_id":"","url":"https://api.github.com/users/joehand","html_url":"https://github.com/joehand","followers_url":"https://api.github.com/users/joehand/followers","following_url":"https://api.github.com/users/joehand/following{/other_user}","gists_url":"https://api.github.com/users/joehand/gists{/gist_id}","starred_url":"https://api.github.com/users/joehand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joehand/subscriptions","organizations_url":"https://api.github.com/users/joehand/orgs","repos_url":"https://api.github.com/users/joehand/repos","events_url":"https://api.github.com/users/joehand/events{/privacy}","received_events_url":"https://api.github.com/users/joehand/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/datproject/discussions/labels/discussion","name":"discussion","color":"ededed"},{"url":"https://api.github.com/repos/datproject/discussions/labels/use%20case","name":"use case","color":"ededed"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2016-06-17T18:41:59Z","updated_at":"2016-06-17T18:42:07Z","closed_at":null,"body":"_From @olgabot on July 6, 2014 22:55_\n\nHi @jbenet and @maxogden! Thank you so much for the time you took to meet with @mlovci and me this weekend. Here's an overview of our current data management situation and what our ideal case would be.\r\n\r\n## What we have now\r\n\r\nCurrently, we host a [`datapackages.json`](http://sauron.ucsd.edu/flotilla_projects/neural_diff_chr22/datapackage.json) file which contains `resources` with the names `\"experiment_design\"` (metadata on the samples, e.g. celltype and colors to plot them with), `\"expression\"` (gene expression data), `\"splicing\"` (scores of which version of a gene was used). Then, at the end of the file, we have an attribute called `\"species\"` (e.g. `\"hg19\"` for the human genome build 19) that only works with `hg19` and `mm10` (*Mus musculus* aka house mouse genome build 10) because it points to the URL \"http://sauron.ucsd.edu/flotilla_projects/<SPECIES>/datapackage.json\", which we hand-curated. So if the data we use is from one of these two species, we can grab the data.\r\n\r\nTry this:\r\n\r\nOn a command line:\r\n```\r\ngit clone git@github.com:YeoLab/flotilla\r\ncd flotilla\r\npip install -e .\r\n```\r\n\r\nIn Python:\r\n```python\r\nimport flotilla\r\nstudy = flotilla.embark(\"http://sauron.ucsd.edu/flotilla_projects/neural_diff_chr22/datapackage.json\")\r\n```\r\n\r\nThis will load the data from our server from sauron.ucsd.edu, and since you haven't downloaded anything with that filename yet, it will download it. Additionally, this is a test dataset with only information from human chromosome 22, so it is loadable on a regular laptop. Feel free to look through the code and JSON files. `flotilla.data_model.Study.from_data_package` does most of the heavy lifting in loading the data. Keep in mind that the project is also in a pre-alpha stage, and has a long way to go :)\r\n\r\n## What we would like\r\n\r\nTwo major issues are:\r\n* Get the data in the `neural_diff_chr22` datapackage into a `pandas.DataFrame` object which can then be imported into `flotilla`. \r\n    * Currently this is managed by the URL in the `datapackage.json` file for that file, but it should first check locally for the data and be able to be loaded offline, if you already have the data downloaded.\r\n* Grab related data, e.g. descriptions of genes and their functions given an ID like `ENSG00000100320` and get the \"gene symbol\" (i.e. the familiar name that we know it by) of RBFOX2 and that this gene is an RNA-binding protein involved in alternative splicing and neural development. \r\n    * Currently this is is managed by the `\"species\"` attribute, but ideally it would be something like `ENSEMBL_v75_homo_sapiens` which would link to the human data here: http://uswest.ensembl.org/info/data/ftp/index.html and then grab gene annotation (`gtf` files)/sequence information (`fasta` files) as necessary by the analysis.\r\n    * Relatedly, there is apparently an \"[eHive](http://uswest.ensembl.org/info/docs/eHive/index.html)\" system on ENSEMBL for data processing. I haven't explored it yet, but it may be good to be aware of.\r\n    * Another major issue is how to *merge analyses of different species' data*. For example, the ENSEMBL website has mappings of human and mouse versions of genes that we could use to compare gene expression. Plus there's the [HAVANA](http://www.sanger.ac.uk/research/projects/vertebrategenome/havana/) project which categorizes orthologous (evolutionarily related) genes between different vertebrates. But what if I want to compare across non-traditional species? And many of them, not just between two? I would like to be able to easily grab these data, submit a job (either to our local supercomputer or to Amazon AWS) which runs a script that outputs a mapping with some unique keys that you could merge all your different data on.\r\n\r\nIdeally, we could do something like this:\r\n\r\n```\r\nstudy = flotilla.embark('neurons')\r\n```\r\n\r\nWhich would fetch our mouse and human neuron data, which has some kind of link to ENSEMBL and attach all the relevant metadata about mouse and human genes, and give common keys where possible.\r\n\r\n@mlovci - please add on with anything I missed.\n\n_Copied from original issue: maxogden/dat#135_"},"comment":{"url":"https://api.github.com/repos/datproject/discussions/issues/comments/226849313","html_url":"https://github.com/datproject/discussions/issues/46#issuecomment-226849313","issue_url":"https://api.github.com/repos/datproject/discussions/issues/46","id":226849313,"user":{"login":"joehand","id":684965,"avatar_url":"https://avatars.githubusercontent.com/u/684965?v=3","gravatar_id":"","url":"https://api.github.com/users/joehand","html_url":"https://github.com/joehand","followers_url":"https://api.github.com/users/joehand/followers","following_url":"https://api.github.com/users/joehand/following{/other_user}","gists_url":"https://api.github.com/users/joehand/gists{/gist_id}","starred_url":"https://api.github.com/users/joehand/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joehand/subscriptions","organizations_url":"https://api.github.com/users/joehand/orgs","repos_url":"https://api.github.com/users/joehand/repos","events_url":"https://api.github.com/users/joehand/events{/privacy}","received_events_url":"https://api.github.com/users/joehand/received_events","type":"User","site_admin":false},"created_at":"2016-06-17T18:42:06Z","updated_at":"2016-06-17T18:42:06Z","body":"_From @olgabot on August 19, 2014 3:28_\n\nYes, definitely! Out of curiosity, what's your current hosting solution? If\r\nwe have some data but no public server, how do we proceed?\r\n\r\nSent from my mobile device.\r\nOn Aug 18, 2014 8:01 PM, \"Max Ogden\" <notifications@github.com> wrote:\r\n\r\n> @mlovci <https://github.com/mlovci> @olgabot <https://github.com/olgabot>\r\n> Heya! We are finally done with the first stable version of dat. Our new\r\n> website is up at http://dat-data.com/\r\n>\r\n> It would be good to do a google hangout with you guys and @bmpvieira\r\n> <https://github.com/bmpvieira> and @mafintosh\r\n> <https://github.com/mafintosh> to discuss how we could start modeling\r\n> your data pipeline with dat!\r\n>\r\n> —\r\n> Reply to this email directly or view it on GitHub\r\n> <https://github.com/maxogden/dat/issues/135#issuecomment-52584914>.\r\n>"}}
{"actor":{"display_login":"joehand"}}