{"action":"created","issue":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1452","repository_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen","labels_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1452/labels{/name}","comments_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1452/comments","events_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1452/events","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1452","id":163278555,"number":1452,"title":"Error during gemini database creation","user":{"login":"pfujita","id":3877668,"avatar_url":"https://avatars.githubusercontent.com/u/3877668?v=3","gravatar_id":"","url":"https://api.github.com/users/pfujita","html_url":"https://github.com/pfujita","followers_url":"https://api.github.com/users/pfujita/followers","following_url":"https://api.github.com/users/pfujita/following{/other_user}","gists_url":"https://api.github.com/users/pfujita/gists{/gist_id}","starred_url":"https://api.github.com/users/pfujita/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pfujita/subscriptions","organizations_url":"https://api.github.com/users/pfujita/orgs","repos_url":"https://api.github.com/users/pfujita/repos","events_url":"https://api.github.com/users/pfujita/events{/privacy}","received_events_url":"https://api.github.com/users/pfujita/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-06-30T22:13:48Z","updated_at":"2016-09-13T18:03:02Z","closed_at":null,"body":"Hello,\r\n\r\nWe just updated to the latest dev version yesterday and I've been trying to test gvcf generation with various callers. I'm running into an error when our process gets to gemini database creation. I don't know if this is at all related to this other issue with gemini support #1284 or maybe this check-in (69bb98ac8abb3ab68ab320410ef584df696124d3) - grasping at straws. Here is the sample.yaml file we're using (and below it is the error output we're seeing):\r\n<pre>\r\nfc_date: 2016-05-07 00:00:15.708284\r\nfc_name: 160208_SN404_0473_AHHFLHBCXX\r\nupload:\r\n  dir: /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106\r\ndetails:\r\n  - files: [/mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/Sample_R14-18106_R1.fastq, /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/Sam\r\n    description: R14-18106\r\n    analysis: 'variant2'\r\n    genome_build: GRCh37\r\n    lane: R14-18106\r\n    algorithm:\r\n      aligner: bwa\r\n      trim_reads: false\r\n      mark_duplicates: true\r\n      recalibrate: true\r\n      realign: true\r\n      variantcaller: [gatk, gatk-haplotype, freebayes, samtools, vardict]\r\n      ensemble:\r\n       numpass: 1\r\n       use_filtered: false\r\n      tools_on: [qualimap_full, gvcf]\r\n      coverage: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      coverage_interval: regional\r\n      coverage_depth: high\r\n      variant_regions: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      clinical_reporting: true\r\n</pre>\r\n\r\n\r\n\r\n[2016-06-30T21:31Z] Timing: structural variation validation\r\n[2016-06-30T21:31Z] multiprocessing: validate_sv\r\n[2016-06-30T21:31Z] Timing: heterogeneity\r\n[2016-06-30T21:31Z] Timing: population database\r\n[2016-06-30T21:31Z] multiprocessing: prep_gemini_db\r\n[2016-06-30T21:31Z] Create gemini database for /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/R14-18106-ensemble-decompose-effects.vcf.gz : R14-18106\r\n[2016-06-30T21:31Z] CADD scores are being loaded (to skip use:--skip-cadd).\r\n[2016-06-30T21:31Z] Traceback (most recent call last):\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/bin/gemini\", line 6, in <module>\r\n[2016-06-30T21:31Z]     gemini.gemini_main.main()\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 1185, in main\r\n[2016-06-30T21:31Z]     args.func(parser, args)\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 198, in load_fn\r\n[2016-06-30T21:31Z]     gemini_load.load(parser, args)\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 51, in load\r\n[2016-06-30T21:31Z]     load_singlecore(args)\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 60, in load_singlecore\r\n[2016-06-30T21:31Z]     gemini_loader.populate_from_vcf()\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 203, in populate_from_vcf\r\n[2016-06-30T21:31Z]     (variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)\r\n[2016-06-30T21:31Z]   File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 358, in _prepare_variation\r\n[2016-06-30T21:31Z]     hom_ref = var.num_hom_ref\r\n[2016-06-30T21:31Z]   File \"cyvcf2/cyvcf2.pyx\", line 642, in cyvcf2.cyvcf2.Variant.num_hom_ref.__get__ (cyvcf2/cyvcf2.c:16261)\r\n[2016-06-30T21:31Z]   File \"cyvcf2/cyvcf2.pyx\", line 697, in cyvcf2.cyvcf2.Variant.gt_types.__get__ (cyvcf2/cyvcf2.c:17047)\r\n[2016-06-30T21:31Z] ValueError: range() step argument must not be zero\r\n[2016-06-30T21:31Z] Uncaught exception occurred\r\nTraceback (most recent call last):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 21, in run\r\n    _do_run(cmd, checks, log_stdout)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 95, in _do_run\r\n    raise subprocess.CalledProcessError(exitcode, error_msg)\r\nCalledProcessError: Command 'set -o pipefail; /mnt/speed/qc/sequencing/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/R14-18106-ensemble-decompose-effects.vcf.gz -t snpEff --cores 1 --tempdir /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/tx/tmp3IC13v /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/tx/tmp3IC13v/R14-18106-ensemble.db\r\nCADD scores are being loaded (to skip use:--skip-cadd).\r\nTraceback (most recent call last):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/bin/gemini\", line 6, in <module>\r\n    gemini.gemini_main.main()\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 1185, in main\r\n    args.func(parser, args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 198, in load_fn\r\n    gemini_load.load(parser, args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 51, in load\r\n    load_singlecore(args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 60, in load_singlecore\r\n    gemini_loader.populate_from_vcf()\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 203, in populate_from_vcf\r\n    (variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 358, in _prepare_variation\r\n    hom_ref = var.num_hom_ref\r\n  File \"cyvcf2/cyvcf2.pyx\", line 642, in cyvcf2.cyvcf2.Variant.num_hom_ref.__get__ (cyvcf2/cyvcf2.c:16261)\r\n  File \"cyvcf2/cyvcf2.pyx\", line 697, in cyvcf2.cyvcf2.Variant.gt_types.__get__ (cyvcf2/cyvcf2.c:17047)\r\nValueError: range() step argument must not be zero\r\n' returned non-zero exit status 1\r\nTraceback (most recent call last):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/bin/bcbio_nextgen.py\", line 226, in <module>\r\n    main(**kwargs)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/bin/bcbio_nextgen.py\", line 43, in main\r\n    run_main(**kwargs)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 43, in run_main\r\n    fc_dir, run_info_yaml)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 87, in _run_toplevel\r\n    for xs in pipeline(config, run_info_yaml, parallel, dirs, samples):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/pipeline/main.py\", line 185, in variant2pipeline\r\n    samples = population.prep_db_parallel(samples, run_parallel)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py\", line 281, in prep_db_parallel\r\n    output = parallel_fn(\"prep_gemini_db\", to_process)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py\", line 28, in run_parallel\r\n    return run_multicore(fn, items, config, parallel=parallel)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multi.py\", line 86, in run_multicore\r\n    for data in joblib.Parallel(parallel[\"num_jobs\"])(joblib.delayed(fn)(x) for x in items):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 804, in __call__\r\n    while self.dispatch_one_batch(iterator):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 662, in dispatch_one_batch\r\n    self._dispatch(tasks)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 570, in _dispatch\r\n    job = ImmediateComputeBatch(batch)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 183, in __init__\r\n    self.results = batch()\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/joblib/parallel.py\", line 72, in __call__\r\n    return [func(*args, **kwargs) for func, args, kwargs in self.items]\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/utils.py\", line 51, in wrapper\r\n    return apply(f, *args, **kwargs)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/distributed/multitasks.py\", line 232, in prep_gemini_db\r\n    return population.prep_gemini_db(*args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py\", line 36, in prep_gemini_db\r\n    gemini_db = create_gemini_db(gemini_vcf, data, gemini_db, ped_file)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/variation/population.py\", line 79, in create_gemini_db\r\n    do.run(cmd, \"Create gemini database for %s\" % gemini_vcf, data)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 21, in run\r\n    _do_run(cmd, checks, log_stdout)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py\", line 95, in _do_run\r\n    raise subprocess.CalledProcessError(exitcode, error_msg)\r\nsubprocess.CalledProcessError: Command 'set -o pipefail; /mnt/speed/qc/sequencing/bcbio/anaconda/bin/gemini  load  --passonly --skip-gerp-bp  -v /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/R14-18106-ensemble-decompose-effects.vcf.gz -t snpEff --cores 1 --tempdir /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/tx/tmp3IC13v /mnt/speed/qc/sequencing/ihg-pipeline/EV-13-5-8/Sample_R14-18106/gemini/tx/tmp3IC13v/R14-18106-ensemble.db\r\nCADD scores are being loaded (to skip use:--skip-cadd).\r\nTraceback (most recent call last):\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/bin/gemini\", line 6, in <module>\r\n    gemini.gemini_main.main()\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 1185, in main\r\n    args.func(parser, args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py\", line 198, in load_fn\r\n    gemini_load.load(parser, args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 51, in load\r\n    load_singlecore(args)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py\", line 60, in load_singlecore\r\n    gemini_loader.populate_from_vcf()\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 203, in populate_from_vcf\r\n    (variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)\r\n  File \"/mnt/speed/qc/sequencing/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py\", line 358, in _prepare_variation\r\n    hom_ref = var.num_hom_ref\r\n  File \"cyvcf2/cyvcf2.pyx\", line 642, in cyvcf2.cyvcf2.Variant.num_hom_ref.__get__ (cyvcf2/cyvcf2.c:16261)\r\n  File \"cyvcf2/cyvcf2.pyx\", line 697, in cyvcf2.cyvcf2.Variant.gt_types.__get__ (cyvcf2/cyvcf2.c:17047)\r\nValueError: range() step argument must not be zero\r\n' returned non-zero exit status 1"},"comment":{"url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/comments/246769145","html_url":"https://github.com/chapmanb/bcbio-nextgen/issues/1452#issuecomment-246769145","issue_url":"https://api.github.com/repos/chapmanb/bcbio-nextgen/issues/1452","id":246769145,"user":{"login":"pfujita","id":3877668,"avatar_url":"https://avatars.githubusercontent.com/u/3877668?v=3","gravatar_id":"","url":"https://api.github.com/users/pfujita","html_url":"https://github.com/pfujita","followers_url":"https://api.github.com/users/pfujita/followers","following_url":"https://api.github.com/users/pfujita/following{/other_user}","gists_url":"https://api.github.com/users/pfujita/gists{/gist_id}","starred_url":"https://api.github.com/users/pfujita/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pfujita/subscriptions","organizations_url":"https://api.github.com/users/pfujita/orgs","repos_url":"https://api.github.com/users/pfujita/repos","events_url":"https://api.github.com/users/pfujita/events{/privacy}","received_events_url":"https://api.github.com/users/pfujita/received_events","type":"User","site_admin":false},"created_at":"2016-09-13T18:03:02Z","updated_at":"2016-09-13T18:03:02Z","body":"Hi again,\r\n\r\nWe're trying to get joint sample calling with gvcf output. We did find this other thread about gvcfs and FreeBayes #1134, but we're not sure if it's the same issue. We have \r\n\r\ntools_on: [gvcf] \r\n\r\nin our sample.yaml but we're getting this GATK error:\r\n\r\n```\r\n##### ERROR MESSAGE: Invalid command line: \r\nArgument emitRefConfidence has a bad value: Can only be used in single sample mode currently. Use the sample_name argument to run on a single sample out of a multi-sample BAM file.\r\n```\r\n\r\n\r\nThis is a snippet of the sample.yaml file we're using:\r\n\r\n```\r\nfc_date: 20160726\r\nfc_name: HHFLHBCXX\r\nupload:\r\n  dir: /mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/\r\ndetails:\r\n  - files: [/mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_Water-EX-17/Sample_Water_R1.fastq, /mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_Water-EX-17/Sample_Water_R2.fastq]\r\n    description: Sample_Water_EX-17\r\n    analysis: 'variant2'\r\n    genome_build: GRCh37\r\n    lane: Sample_Water_EX-17\r\n    algorithm:\r\n      aligner: bwa\r\n      trim_reads: false\r\n      recalibrate: gatk\r\n      realign: gatk\r\n      variantcaller: [gatk-haplotype,freebayes,samtools,gatk]\r\n      ensemble:\r\n       numpass: 1\r\n       use_filtered: true\r\n      tools_off: [gemini]\r\n      tools_on: [qualimap_full,gvcf]\r\n      coverage: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      coverage_interval: regional\r\n      coverage_depth: high\r\n      variant_regions: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      clinical_reporting: true\r\n    metadata: \r\n      batch: Batch1\r\n  - files: [/mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_NA19678/Sample_NA19678_R1.fastq, /mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_NA19678/Sample_NA19678_R2.fastq]\r\n    description: Sample_NA19678\r\n    analysis: 'variant2'\r\n    genome_build: GRCh37\r\n    lane: Sample_NA19678\r\n    algorithm:\r\n      aligner: bwa\r\n      trim_reads: false\r\n      recalibrate: gatk\r\n      realign: gatk\r\n      variantcaller: [gatk-haplotype,freebayes,samtools,gatk]\r\n      ensemble:\r\n       numpass: 1\r\n       use_filtered: true\r\n      tools_off: [gemini]\r\n      tools_on: [qualimap_full,gvcf]\r\n      coverage: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      coverage_interval: regional\r\n      coverage_depth: high\r\n      variant_regions: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      clinical_reporting: true\r\n    metadata:\r\n      batch: Batch1\r\n  - files: [/mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_HG03727/Sample_HG03727_R1.fastq, /mnt/speed/qc/sequencing/ihg-pipeline/joint-calling/Sample_HG03727/Sample_HG03727_R2.fastq]\r\n    description: Sample_HG03727\r\n    analysis: 'variant2'\r\n    genome_build: GRCh37\r\n    lane: Sample_HG03727\r\n    algorithm:\r\n      aligner: bwa\r\n      trim_reads: false\r\n      recalibrate: gatk\r\n      realign: gatk\r\n      variantcaller: [gatk-haplotype,freebayes,samtools,gatk]\r\n      ensemble:\r\n       numpass: 1\r\n       use_filtered: true\r\n      tools_off: [gemini]\r\n      tools_on: [qualimap_full,gvcf]\r\n      coverage: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      coverage_interval: regional\r\n      coverage_depth: high\r\n      variant_regions: /mnt/speed/qc/sequencing/biodata/capture_regions/xgen-exome-research-panel-targets-cols1-2-noChr.bed\r\n      clinical_reporting: true\r\n\r\n```\r\n"}}
{"actor":{"display_login":"pfujita"}}