{"action":"created","issue":{"url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/37","repository_url":"https://api.github.com/repos/Clinical-Genomics/moneypenny","labels_url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/37/labels{/name}","comments_url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/37/comments","events_url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/37/events","html_url":"https://github.com/Clinical-Genomics/moneypenny/issues/37","id":174039552,"number":37,"title":"New proposed workflow","user":{"login":"robinandeer","id":1220232,"avatar_url":"https://avatars.githubusercontent.com/u/1220232?v=3","gravatar_id":"","url":"https://api.github.com/users/robinandeer","html_url":"https://github.com/robinandeer","followers_url":"https://api.github.com/users/robinandeer/followers","following_url":"https://api.github.com/users/robinandeer/following{/other_user}","gists_url":"https://api.github.com/users/robinandeer/gists{/gist_id}","starred_url":"https://api.github.com/users/robinandeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robinandeer/subscriptions","organizations_url":"https://api.github.com/users/robinandeer/orgs","repos_url":"https://api.github.com/users/robinandeer/repos","events_url":"https://api.github.com/users/robinandeer/events{/privacy}","received_events_url":"https://api.github.com/users/robinandeer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":7,"created_at":"2016-08-30T14:46:14Z","updated_at":"2016-08-31T08:10:42Z","closed_at":null,"body":"# Analyze\r\n\r\nThe main package handling analysis (for now only MIP) is called **Trailblazer** (TB). It has a web interface to track status of running/complete/failed runs and a CLI to interact with them in the terminal.\r\n\r\nWe have another tool called **Moneypenny** (MP) which is the only tool that takes into account information from multiple \"apps\" like e.g. Trailblazer and LIMS. So **MP is tool that provides shortcuts to run the separate apps.**\r\n\r\n## Start analysis\r\n\r\nserver: rastapopoulos\r\n\r\n```bash\r\n$ moneypenny start [--email robin.andeer@scilifelab.se] cust003 16104\r\n```\r\n\r\n### Limitations\r\nOnly works for samples run with gene panels in the clinical master gene list. You can then start the analysis \"manually\":\r\n\r\n```bash\r\n$ trailblazer analyze start [--email robin.andeer@scilifelab.se] --ccp /mnt/hds/proj/bioinfo/MIP_ANALYSIS/cust003/16104 --analysis-type genomes --customer cust003 --family 16104 **--gene-list cust002-NMD.txt**\r\n```\r\n\r\nYou can also just generate the pedigree in case you want to place it in a custom location:\r\n\r\n```bash\r\n$ cglims pedigree cust003 16104 > /tmp/16104_pedigree.txt\r\n```\r\n\r\n## Restart analysis after \"GATKVariantRecalibration\" fail\r\n\r\n```bash\r\n$ trailblazer analyze restart max-gaussian --case cust003-16104\r\n```\r\n\r\n## Implementation\r\n\r\nStarting MIP from within a Python script is currently not possible. This is why the above commands will write a generic Bash script that it will place in a predetermined directory.\r\n\r\nA crontab runs a script that looks in the same directory for scripts to execute and remove. This is how we achieve automation.\r\n\r\n# Post-analysis\r\n\r\nThere's a lot of data from the analysis that needs to be taken care of; coverage, VCFs, QC metrics, genotypes etc. Hence there's also a number of tools dedicated to taking care of processing this data. However, there's one tool that takes over right after an analysis is finished: **Housekeeper**.\r\n\r\n*Housekeeper* takes care of extracting interesting files from the analysis output and stores it in a way that makes it easy to retrieve the files later on.\r\n\r\n## Store analysis\r\n\r\nserver: rastapopoulos\r\n\r\n```bash\r\n$ housekeeper add mip /mnt/hds/proj/bioinfo/MIP_ANALYSIS/cust003/16104/genomes/16104/16104_config.yaml\r\n```\r\n\r\n## Automation\r\n\r\nThere's a solution to automatically fetch recently completed analyses and store them in *Housekeeper*. This command could be run in a crontab.\r\n\r\n```bash\r\n$ moneypenny analysis store [--since 16 08 15]\r\n```\r\n\r\n*Moneypenny* will ask *Trailblazer* for completed analyses after the given date (defaults to 2 weeks) and ask *Housekeeper* to store them unless they have already been taken care of.\r\n\r\n> Some analyses might run without fails but you still want to stop the dataflow and rerun (e.g. wrong gene list). How do you handle this?\r\n\r\n# Downstream\r\n\r\nWe've now run our pipeline and store our raw result files and are ready to start processing it for real. Data uploads from analysis output is generally collected under `moneypenny analysis <subcommand>`.\r\n\r\nserver: rastapopoulos\r\n\r\n```bash\r\n$ moneypenny analysis coverage cust003-16104\r\n$ moneypenny analysis genotype cust003-16104\r\n$ moneypenny analysis qc cust003-16104\r\n```\r\n\r\nThese commands will work as so: *MP* will as *HK* for the files that are required to load data for the particular tool/app and pass those along to Chanjo/Approved/Taboo.\r\n\r\n> I'll likely bundle these commands to do it all at once but for now I think this is fine. Of course there's also the commands to upload variants to Scout and Loqusdb under development.\r\n\r\n## Additional data: Taboo\r\n\r\nTaboo requires some more data from LIMS and the MAF Excel books to do it's thing. Excel books are uploaded manually via the web interface. Information about expected sample sex is added from LIMS via *MP*. It's added *per sample* which means we need to figure out some LIMS sample ids first:\r\n\r\n```bash\r\n$ cglims get --field name cust003-16104\r\nADM1678A1: 16104-I-1A\r\nADM1678A2: 16104-II-1U\r\nADM1678A3: 16104-II-2U\r\n```\r\n\r\n...and now we can populate the Taboo database with the additional information:\r\n\r\n```bash\r\n$ moneypenny add genotype ADM1678A1\r\n...\r\n```"},"comment":{"url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/comments/243690670","html_url":"https://github.com/Clinical-Genomics/moneypenny/issues/37#issuecomment-243690670","issue_url":"https://api.github.com/repos/Clinical-Genomics/moneypenny/issues/37","id":243690670,"user":{"login":"robinandeer","id":1220232,"avatar_url":"https://avatars.githubusercontent.com/u/1220232?v=3","gravatar_id":"","url":"https://api.github.com/users/robinandeer","html_url":"https://github.com/robinandeer","followers_url":"https://api.github.com/users/robinandeer/followers","following_url":"https://api.github.com/users/robinandeer/following{/other_user}","gists_url":"https://api.github.com/users/robinandeer/gists{/gist_id}","starred_url":"https://api.github.com/users/robinandeer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/robinandeer/subscriptions","organizations_url":"https://api.github.com/users/robinandeer/orgs","repos_url":"https://api.github.com/users/robinandeer/repos","events_url":"https://api.github.com/users/robinandeer/events{/privacy}","received_events_url":"https://api.github.com/users/robinandeer/received_events","type":"User","site_admin":false},"created_at":"2016-08-31T08:10:42Z","updated_at":"2016-08-31T08:10:42Z","body":"Okey, I'll just clarify my thoughts but I'm seeing your point and consistency is important!\r\n\r\n1. MP is a first pass option - it's what users interact with 99% of the time\r\n2. If MP is too strict or can't handle the additional 1% you drop down to the individual tool\r\n\r\nIndividual tools should map as closely to the thing they are providing an abstraction on top of. In case of TB it maps functionality on top of MIP which is why it tries to not stray too far from MIP conventions. I want using e.g. TB to be as intuitive as possible to use for people that **already know how to use MIP**, not necessarily people how are familiar with MP.\r\n\r\nDoes it make sense as well?"}}
{"actor":{"display_login":"robinandeer"}}