{"action":"created","issue":{"url":"https://api.github.com/repos/secastel/phaser/issues/41","repository_url":"https://api.github.com/repos/secastel/phaser","labels_url":"https://api.github.com/repos/secastel/phaser/issues/41/labels{/name}","comments_url":"https://api.github.com/repos/secastel/phaser/issues/41/comments","events_url":"https://api.github.com/repos/secastel/phaser/issues/41/events","html_url":"https://github.com/secastel/phaser/issues/41","id":286156950,"number":41,"title":"Issues with VCF merge after running phaser","user":{"login":"everestial","id":11068812,"avatar_url":"https://avatars2.githubusercontent.com/u/11068812?v=4","gravatar_id":"","url":"https://api.github.com/users/everestial","html_url":"https://github.com/everestial","followers_url":"https://api.github.com/users/everestial/followers","following_url":"https://api.github.com/users/everestial/following{/other_user}","gists_url":"https://api.github.com/users/everestial/gists{/gist_id}","starred_url":"https://api.github.com/users/everestial/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/everestial/subscriptions","organizations_url":"https://api.github.com/users/everestial/orgs","repos_url":"https://api.github.com/users/everestial/repos","events_url":"https://api.github.com/users/everestial/events{/privacy}","received_events_url":"https://api.github.com/users/everestial/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-01-05T00:32:43Z","updated_at":"2018-03-02T14:37:23Z","closed_at":null,"author_association":"NONE","body":"Hi @secastel \r\n\r\nI may have raised this issue before, but the problem with VCF merge still exists. I am not sure if VCF from other callers are running into this issue, but with GATK (HaplotypeCaller) generated VCF's, I can run run phaser. But, the single sample VCF's that are not generated cannot be merged back.\r\n\r\nThis is the error message if it helps:\r\n\r\n`$ java -jar -Xmx16g /home/everestial007/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar -T CombineVariants -R lyrata_genome.fa -V ms01e_phased.vcf -V ms02g_phased.vcf -V ms03g_phased.vcf -V ms04h_phased.vcf -o F1.phased_variants.Final.vcf`\r\n\r\n    INFO  19:25:42,458 HelpFormatter - -------------------------------------------------------------------------------- \r\n    INFO  19:25:42,460 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.7-0-gcfedb67, Compiled 2016/12/12 11:21:18 \r\n    INFO  19:25:42,461 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute \r\n    INFO  19:25:42,461 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk \r\n    INFO  19:25:42,461 HelpFormatter - [Thu Jan 04 19:25:42 EST 2018] Executing on Linux 4.10.0-42-generic amd64 \r\n    INFO  19:25:42,461 HelpFormatter - OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 \r\n    INFO  19:25:42,465 HelpFormatter - Program Args: -T CombineVariants -R lyrata_genome.fa -V ms01e_phased.vcf -V ms02g_phased.vcf -V ms03g_phased.vcf -V ms04h_phased.vcf -o F1.phased_variants.Final.vcf \r\n    INFO  19:25:42,469 HelpFormatter - Executing as everestial007@everestial007-Inspiron-3647 on Linux 4.10.0-42-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12. \r\n    INFO  19:25:42,469 HelpFormatter - Date/Time: 2018/01/04 19:25:42 \r\n    INFO  19:25:42,469 HelpFormatter - -------------------------------------------------------------------------------- \r\n    INFO  19:25:42,469 HelpFormatter - -------------------------------------------------------------------------------- \r\n    INFO  19:25:42,492 GenomeAnalysisEngine - Strictness is SILENT \r\n    INFO  19:25:42,741 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000 \r\n    INFO  19:25:43,091 GenomeAnalysisEngine - Preparing for traversal \r\n    INFO  19:25:43,094 GenomeAnalysisEngine - Done preparing for traversal \r\n    INFO  19:25:43,094 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] \r\n    INFO  19:25:43,095 ProgressMeter -                 | processed |    time |    per 1M |           |   total | remaining \r\n    INFO  19:25:43,095 ProgressMeter -        Location |     sites | elapsed |     sites | completed | runtime |   runtime \r\n    ##### ERROR --\r\n    ##### ERROR stack trace \r\n    java.lang.NumberFormatException: For input string: \"\"\r\n\t    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)\r\n\t    at java.lang.Integer.parseInt(Integer.java:592)\r\n\t    at java.lang.Integer.valueOf(Integer.java:766)\r\n\t    at htsjdk.variant.vcf.AbstractVCFCodec.createGenotypeMap(AbstractVCFCodec.java:717)\r\n\t    at htsjdk.variant.vcf.AbstractVCFCodec$LazyVCFGenotypesParser.parse(AbstractVCFCodec.java:129)\r\n\t    at htsjdk.variant.variantcontext.LazyGenotypesContext.decode(LazyGenotypesContext.java:158)\r\n\t    at htsjdk.variant.variantcontext.LazyGenotypesContext.getGenotypes(LazyGenotypesContext.java:148)\r\n\t    at htsjdk.variant.variantcontext.GenotypesContext.iterator(GenotypesContext.java:465)\r\n\t    at org.broadinstitute.gatk.utils.variant.GATKVariantContextUtils.mergeGenotypes(GATKVariantContextUtils.java:1556)\r\n\t    at org.broadinstitute.gatk.utils.variant.GATKVariantContextUtils.simpleMerge(GATKVariantContextUtils.java:1224)\r\n\t    at org.broadinstitute.gatk.tools.walkers.variantutils.CombineVariants.map(CombineVariants.java:361)\r\n\t    at org.broadinstitute.gatk.tools.walkers.variantutils.CombineVariants.map(CombineVariants.java:143)\r\n\t    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:267)\r\n\t    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano$TraverseLociMap.apply(TraverseLociNano.java:255)\r\n\t    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:274)\r\n\t    at org.broadinstitute.gatk.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)\r\n\t    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)\r\n\t    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)\r\n\t    at org.broadinstitute.gatk.engine.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)\r\n\t    at org.broadinstitute.gatk.engine.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:98)\r\n\t    at org.broadinstitute.gatk.engine.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:316)\r\n\t    at org.broadinstitute.gatk.engine.CommandLineExecutable.execute(CommandLineExecutable.java:123)\r\n\t    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:256)\r\n\t    at org.broadinstitute.gatk.utils.commandline.CommandLineProgram.start(CommandLineProgram.java:158)\r\n\t    at org.broadinstitute.gatk.engine.CommandLineGATK.main(CommandLineGATK.java:108)\r\n    ##### ERROR ------------------------------------------------------------------------------------------\r\n    ##### ERROR A GATK RUNTIME ERROR has occurred (version 3.7-0-gcfedb67):\r\n    ##### ERROR\r\n    ##### ERROR This might be a bug. Please check the documentation guide to see if this is a known problem.\r\n    ##### ERROR If not, please post the error message, with stack trace, to the GATK forum.\r\n    ##### ERROR Visit our website and forum for extensive documentation and answers to \r\n    ##### ERROR commonly asked questions https://software.broadinstitute.org/gatk\r\n    ##### ERROR\r\n    ##### ERROR MESSAGE: For input string: \"\"\r\n    ##### ERROR ------------------------------------------------------------------------------------------\r\n\r\n\r\nThe issue at hand is that even if I can merge the data using other tools, I will still have problems because I further have to use GATK to do my downstream analyses. Can you please check the issue if time permits?\r\n\r\n\r\nHere are my files. \r\n\r\n[ms01e_phased.vcf.gz](https://github.com/secastel/phaser/files/1605182/ms01e_phased.vcf.gz)\r\n[ms02g_phased.vcf.gz](https://github.com/secastel/phaser/files/1605183/ms02g_phased.vcf.gz)\r\n[ms03g_phased.vcf.gz](https://github.com/secastel/phaser/files/1605184/ms03g_phased.vcf.gz)\r\n[ms04h_phased.vcf.gz](https://github.com/secastel/phaser/files/1605185/ms04h_phased.vcf.gz)\r\n\r\n\r\n\r\n  "},"comment":{"url":"https://api.github.com/repos/secastel/phaser/issues/comments/369937598","html_url":"https://github.com/secastel/phaser/issues/41#issuecomment-369937598","issue_url":"https://api.github.com/repos/secastel/phaser/issues/41","id":369937598,"user":{"login":"everestial","id":11068812,"avatar_url":"https://avatars2.githubusercontent.com/u/11068812?v=4","gravatar_id":"","url":"https://api.github.com/users/everestial","html_url":"https://github.com/everestial","followers_url":"https://api.github.com/users/everestial/followers","following_url":"https://api.github.com/users/everestial/following{/other_user}","gists_url":"https://api.github.com/users/everestial/gists{/gist_id}","starred_url":"https://api.github.com/users/everestial/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/everestial/subscriptions","organizations_url":"https://api.github.com/users/everestial/orgs","repos_url":"https://api.github.com/users/everestial/repos","events_url":"https://api.github.com/users/everestial/events{/privacy}","received_events_url":"https://api.github.com/users/everestial/received_events","type":"User","site_admin":false},"created_at":"2018-03-02T14:37:23Z","updated_at":"2018-03-02T14:37:23Z","author_association":"NONE","body":"@secastel : I again had this issue. I think I may have found the reason why vcf's arenot merging using `GATK CombineVariants`. \r\n\r\n**Here is what I did for troubleshooting:**\r\n  - I ran phaser on multi sample vcf's, which then gives me multiple single sample VCF's. \r\n  - I tried to merge them using `GATK combine variants`, then received the the same error, that I posted earlier on this issue.\r\n\r\n  - Unable to merge I ran the GATK `validate variants`, which gives me the error message:\r\n    \r\n    everestial007@.../outputs_RBphased_VCF$ jav\r\na -jar -Xmx6g /home/everestial007/GenomeAnalysisTK-3.8/GenomeAnalysisTK.jar -T ValidateVariants -R ${ref_genome} -V ms01e_phased.vcf\r\nINFO  09:27:37,686 HelpFormatter - ---------------------------------------------------------------------------------- \r\nINFO  09:27:37,690 HelpFormatter - The Genome Analysis Toolkit (GATK) v3.8-0-ge9d806836, Compiled 2017/07/28 21:26:50 \r\nINFO  09:27:37,690 HelpFormatter - Copyright (c) 2010-2016 The Broad Institute \r\nINFO  09:27:37,691 HelpFormatter - For support and documentation go to https://software.broadinstitute.org/gatk \r\nINFO  09:27:37,691 HelpFormatter - [Fri Mar 02 09:27:37 EST 2018] Executing on Linux 4.13.0-32-generic amd64 \r\nINFO  09:27:37,692 HelpFormatter - OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12 \r\nINFO  09:27:37,698 HelpFormatter - Program Args: -T ValidateVariants -R /media/everestial007/SeagateBackup4.0TB2/New_Alignment_Set/RefNindex_lyrata/lyrata_genome.fa -V ms01e_phased.vcf \r\nINFO  09:27:37,701 HelpFormatter - Executing as everestial007@everestial007-Inspiron-3647 on Linux 4.13.0-32-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_151-8u151-b12-0ubuntu0.16.04.2-b12. \r\nINFO  09:27:37,702 HelpFormatter - Date/Time: 2018/03/02 09:27:37 \r\nINFO  09:27:37,702 HelpFormatter - ---------------------------------------------------------------------------------- \r\nINFO  09:27:37,702 HelpFormatter - ---------------------------------------------------------------------------------- \r\nERROR StatusLogger Unable to create class org.apache.logging.log4j.core.impl.Log4jContextFactory specified in jar:file:/home/everestial007/GenomeAnalysisTK-3.8/GenomeAnalysisTK.jar!/META-INF/log4j-provider.properties\r\nERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...\r\nINFO  09:27:37,986 GenomeAnalysisEngine - Deflater: IntelDeflater \r\nINFO  09:27:37,987 GenomeAnalysisEngine - Inflater: IntelInflater \r\nINFO  09:27:37,988 GenomeAnalysisEngine - Strictness is SILENT \r\nINFO  09:27:38,603 GenomeAnalysisEngine - Downsampling Settings: Method: BY_SAMPLE, Target Coverage: 1000 \r\nINFO  09:27:39,014 GenomeAnalysisEngine - Preparing for traversal \r\nINFO  09:27:39,017 GenomeAnalysisEngine - Done preparing for traversal \r\nINFO  09:27:39,018 ProgressMeter - [INITIALIZATION COMPLETE; STARTING PROCESSING] \r\nINFO  09:27:39,018 ProgressMeter -                 | processed |    time |    per 1M |           |   total | remaining \r\nINFO  09:27:39,018 ProgressMeter -        Location |     sites | elapsed |     sites | completed | runtime |   runtime \r\n##### ERROR ------------------------------------------------------------------------------------------\r\n##### ERROR A USER ERROR has occurred (version 3.8-0-ge9d806836): \r\n##### ERROR\r\n##### ERROR This means that one or more arguments or inputs in your command are incorrect.\r\n##### ERROR The error message below tells you what is the problem.\r\n##### ERROR\r\n##### ERROR If the problem is an invalid argument, please check the online documentation guide\r\n##### ERROR (or rerun your command with --help) to view allowable command-line arguments for this tool.\r\n##### ERROR\r\n##### ERROR Visit our website and forum for extensive documentation and answers to \r\n##### ERROR commonly asked questions https://software.broadinstitute.org/gatk\r\n##### ERROR\r\n##### ERROR Please do NOT post this error to the GATK forum unless you have really tried to fix it yourself.\r\n##### ERROR\r\n##### ERROR MESSAGE: File /media/everestial007/SeagateBackup4.0TB2/RNAseq_Data_Analyses/phaser_to_ASE_on_DiploidGenome/outputs_RBphased_VCF/ms01e_phased.vcf fails strict validation: the Allele Number (AN) tag is incorrect for the record at position 2:181028, 6 vs. 2\r\n##### ERROR ------------------------------------------------------------------------------------------\r\n\r\nYou can see that there is problem with `AN` tag and the reason I think this is happening is because `phaser` reads in multi sample VCF but outputs only one sample VCF, where the \"AN\" values are not recalculated. So, GATK is throwing some error because VCF is having validation problems. Either there needs to be a method to ignore this flag in GATK or there needs to be method to update `AN` if phaser is outputting single sample VCF's. \r\n\r\nPossible non-optimal solution. I could use a `awk` one liner to bring mutiple sample together but `PG` and `PI` tags are not added at same consistent positions in all the samples.\r\n\r\nFor now, I am able to merge the VCFs using \"bcftools\" and GATK doesnot throw validation error on this \r\nmerge VCF because the number of samples before running \"phaser\" and after running \"bcftools merge\" has matched \"AN\" values.\r\n\r\nMy issues is solved for now, but I am leaving this message so potential data structure problem can be sorted out either with GATK or phaser in the future.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"everestial"}}