{"action":"created","issue":{"url":"https://api.github.com/repos/cBioPortal/GSoC/issues/10","repository_url":"https://api.github.com/repos/cBioPortal/GSoC","labels_url":"https://api.github.com/repos/cBioPortal/GSoC/issues/10/labels{/name}","comments_url":"https://api.github.com/repos/cBioPortal/GSoC/issues/10/comments","events_url":"https://api.github.com/repos/cBioPortal/GSoC/issues/10/events","html_url":"https://github.com/cBioPortal/GSoC/issues/10","id":130387365,"number":10,"title":"ETL pipeline for ICGC","user":{"login":"jjgao","id":840895,"avatar_url":"https://avatars.githubusercontent.com/u/840895?v=3","gravatar_id":"","url":"https://api.github.com/users/jjgao","html_url":"https://github.com/jjgao","followers_url":"https://api.github.com/users/jjgao/followers","following_url":"https://api.github.com/users/jjgao/following{/other_user}","gists_url":"https://api.github.com/users/jjgao/gists{/gist_id}","starred_url":"https://api.github.com/users/jjgao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jjgao/subscriptions","organizations_url":"https://api.github.com/users/jjgao/orgs","repos_url":"https://api.github.com/users/jjgao/repos","events_url":"https://api.github.com/users/jjgao/events{/privacy}","received_events_url":"https://api.github.com/users/jjgao/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cBioPortal/GSoC/labels/cBioPortal","name":"cBioPortal","color":"207de5"},{"url":"https://api.github.com/repos/cBioPortal/GSoC/labels/Java","name":"Java","color":"5319e7"},{"url":"https://api.github.com/repos/cBioPortal/GSoC/labels/Pipeline","name":"Pipeline","color":"207de5"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":5,"created_at":"2016-02-01T15:16:38Z","updated_at":"2016-03-22T15:56:56Z","closed_at":null,"body":"**Background**: The [International Cancer Genome Consortium (ICGC)](https://icgc.org/) is an scientific consortium for collaboration among the world's leading cancer genomics researchers. The [ICGC Data Portal](https://dcc.icgc.org/) aggregates cancer genomics data from its consortium member projects, including [The Cancer Genome Atlas (TCGA)](http://cancergenome.nih.gov/) projects. While the cBioPortal has all the TCGA projects, we haven't integrated many of the ICGC projects from other sources.\r\n\r\n**Goal**: Building a public ETL (Extract, Transform, and Load) pipeline for ICGC data.\r\n\r\n**Approach**: \r\n\r\n- Extract data from the [ICGC Data Portal](https://dcc.icgc.org/).\r\n- Transform the extracted data to the [cBioProtal file formats](https://github.com/cBioPortal/cbioportal/wiki/File-Formats)\r\n- (Optional) Improve our load pipeline.\r\n\r\n**Need skills**: Bioinformatics, Java, Spring Batch\r\n\r\n**Possible mentors**: [Benjamin Gross](https://github.com/n1zea144), [Zack Heins](https://github.com/zheins)\r\n"},"comment":{"url":"https://api.github.com/repos/cBioPortal/GSoC/issues/comments/199879322","html_url":"https://github.com/cBioPortal/GSoC/issues/10#issuecomment-199879322","issue_url":"https://api.github.com/repos/cBioPortal/GSoC/issues/10","id":199879322,"user":{"login":"zulrich91","id":8017054,"avatar_url":"https://avatars.githubusercontent.com/u/8017054?v=3","gravatar_id":"","url":"https://api.github.com/users/zulrich91","html_url":"https://github.com/zulrich91","followers_url":"https://api.github.com/users/zulrich91/followers","following_url":"https://api.github.com/users/zulrich91/following{/other_user}","gists_url":"https://api.github.com/users/zulrich91/gists{/gist_id}","starred_url":"https://api.github.com/users/zulrich91/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/zulrich91/subscriptions","organizations_url":"https://api.github.com/users/zulrich91/orgs","repos_url":"https://api.github.com/users/zulrich91/repos","events_url":"https://api.github.com/users/zulrich91/events{/privacy}","received_events_url":"https://api.github.com/users/zulrich91/received_events","type":"User","site_admin":false},"created_at":"2016-03-22T15:56:56Z","updated_at":"2016-03-22T15:56:56Z","body":"@n1zea144 \r\nThanks for the reply, My questions are as follows\r\n\r\nHow is data from the TCGA extracted ? \r\n\r\nI ask this question because the only way I could figure out for extracting data from the ICGC Data portal is a manual process, where by one has to search the available repositories and download a manifest that suits his choice. I thought knowing how TCGA data is extracted could be of help. \r\n\r\nShould in case the data will be downloaded manually how will this be uploaded ? Will it require a GUI or the scope of the project does not require this ?\r\n\r\nI wish to know the scope of the transformation ? Will a transformation be done for each of the  12 available data formats ? \r\n\r\nWill a transformation consist of generating 4 files(a datafile, its meta data file, cancer_type.txt and meta_study.txt ) ? from each file extracted from the ICGC Data portal ?\r\n\r\nI submitted a draft proposal lately yesterday, i will appreciate you review so that we come out with a feasible final proposal. \r\n\r\nThanks.\r\n\r\n\r\n"}}