{"action":"created","issue":{"url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/89","repository_url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines","labels_url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/89/labels{/name}","comments_url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/89/comments","events_url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/89/events","html_url":"https://github.com/kundajelab/atac_dnase_pipelines/issues/89","id":286286573,"number":89,"title":"ataqqc failed: 'filename'.trim_gc.txt does not exist","user":{"login":"jonasungerback","id":15617009,"avatar_url":"https://avatars1.githubusercontent.com/u/15617009?v=4","gravatar_id":"","url":"https://api.github.com/users/jonasungerback","html_url":"https://github.com/jonasungerback","followers_url":"https://api.github.com/users/jonasungerback/followers","following_url":"https://api.github.com/users/jonasungerback/following{/other_user}","gists_url":"https://api.github.com/users/jonasungerback/gists{/gist_id}","starred_url":"https://api.github.com/users/jonasungerback/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jonasungerback/subscriptions","organizations_url":"https://api.github.com/users/jonasungerback/orgs","repos_url":"https://api.github.com/users/jonasungerback/repos","events_url":"https://api.github.com/users/jonasungerback/events{/privacy}","received_events_url":"https://api.github.com/users/jonasungerback/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":31,"created_at":"2018-01-05T13:18:57Z","updated_at":"2018-03-14T18:05:55Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\nfirst and foremost, thank you so much for this pipeline. Standardization is much needed in the ATAC-seq world. I am however encountering an issue in the qc-step that I am too much a beginner for to locate. I did install the pipeline with the --recursive option and as long as I am giving the -no_xcor option it starts and runs (it fails with it and I am not sure why) into the ataqc steps but here I am encountering the error in the title. The part of the error message looks like this:\r\n\r\nI am also attaching the zipped report.\r\n\r\nThank you so much in advance!\r\nBest,\r\nJonas\r\n> Num   52     ID   task.ataqc.ataqc_rep1.line_109.id_113     Name   ataqc rep1    Thread   thread_226     PID   47197 | Num | 52 | ID | task.ataqc.ataqc_rep1.line_109.id_113 | Name | ataqc rep1 | Thread | thread_226 | PID | 47197 | OK   false      Exit Code   1      Retries       State   ERROR     Dep.   ERROR     Cpus        Mem | OK | false | Exit Code | 1 | Retries |   | State | ERROR | Dep. | ERROR | Cpus |   | Mem |   | Start   2018-01-05 09:36:39     End   2018-01-05 09:36:39     Elapsed   00:00:00     Timeout  00:00:-1    Wall Timeout   100 days | Start | 2018-01-05 09:36:39 | End | 2018-01-05 09:36:39 | Elapsed | 00:00:00 | Timeout | 00:00:-1 | Wall Timeout | 100 days | Input files    /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.fastq.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.align.log /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.nodup.pbc.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.dup.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/signal/macs2/rep1/NALM6_index_1.trim.nodup.tn5.pf.pval.signal.bigwig /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/pseudo_reps/rep1/NALM6_ATAC_ENCODE_pipeline_rep1-pr.IDR0.1.filt.narrowPeak.gz       Output files   /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.html /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.txt      Dependencies | Input files | /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.fastq.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.align.log /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.nodup.pbc.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.dup.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/signal/macs2/rep1/NALM6_index_1.trim.nodup.tn5.pf.pval.signal.bigwig /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/pseudo_reps/rep1/NALM6_ATAC_ENCODE_pipeline_rep1-pr.IDR0.1.filt.narrowPeak.gz | Output files | /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.html /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.txt | Dependencies |   | # SYS command. line 111   if [[ -f $(which conda) && $(conda env list \\| grep bds_atac \\| wc -l) != \"0\" ]]; then source activate bds_atac; sleep 5; fi;  export PATH=/mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/atac_dnase_pipelines/.:/mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/atac_dnase_pipelines/modules:/mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/atac_dnase_pipelines/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)  # SYS command. line 114   export _JAVA_OPTIONS=\"-Xms256M -Xmx20G -XX:ParallelGCThreads=1\"  # SYS command. line 116   if [ \"${TMPDIR}\" != \"\" ] && [ -d \"${TMPDIR}\" ]; then \\ \t\t\texport _JAVA_OPTIONS=\"${_JAVA_OPTIONS} -Djava.io.tmpdir=${TMPDIR}\"; \\ \t\tfi  # SYS command. line 119   cd /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1  # SYS command. line 122   if [ -f \"$(which picard)\" ]; then export PICARDROOT=\"$(dirname $(which picard))/../share/picard\"*; fi  # SYS command. line 124   /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/atac_dnase_pipelines/ataqc/run_ataqc.py \\ \t\t    --workdir /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1 \\ \t\t    --outdir /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1 \\ \t\t    --outprefix NALM6_index_1.trim \\ \t\t    --genome hg19 \\ \t\t    --chromsizes /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/hg19.chrom.sizes \\ \t\t    --ref /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/male.hg19.fa \\ \t\t    --tss /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/hg19_gencode_tss_unique.bed.gz \\ \t\t    --dnase /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz \\ \t\t    --blacklist /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/wgEncodeDacMapabilityConsensusExcludable.bed.gz \\ \t\t    --prom /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_prom_p2.bed.gz \\ \t\t    --enh /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_enh_p2.bed.gz \\ \t\t    --pbc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.nodup.pbc.qc\\ \t\t     --fastq1 /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.fastq.gz \\ \t\t    --alignedbam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.bam \\ \t\t    --alignmentlog /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.align.log \\ \t\t    --coordsortbam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.bam \\ \t\t    --duplog /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.dup.qc \\ \t\t    --finalbam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.bam \\ \t\t    --finalbed /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\ \t\t    --bigwig /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/signal/macs2/rep1/NALM6_index_1.trim.nodup.tn5.pf.pval.signal.bigwig \\ \t\t    --peaks /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/pseudo_reps/rep1/NALM6_ATAC_ENCODE_pipeline_rep1-pr.IDR0.1.filt.narrowPeak.gz  --naive_overlap_peaks /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/overlap/optimal_set/NALM6_ATAC_ENCODE_pipeline_ppr.naive_overlap.filt.narrowPeak.gz  --idr_peaks /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/optimal_set/NALM6_ATAC_ENCODE_pipeline_ppr.IDR0.1.filt.narrowPeak.gz  \\ \t\t     --reg2map /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/dnase_avgs_reg2map_p10_merged_named.pvals.gz  --reg2map_bed /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz  --meta /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/eid_to_mnemonic.txt  # SYS command. line 147   rm -f test.log test.png  # SYS command. line 149   TASKTIME=$[$(date +%s)-${STARTTIME}]; echo \"Task has finished (${TASKTIME} seconds).\"; sleep 0       --------------------Stdout-------------------- bedtools sort -i /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz  \\| bedtools merge -i stdin \\| bedtools intersect -u -nonamecheck -a /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz -b stdin \\| wc -l zcat -f /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\| wc -l bedtools sort -i /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/wgEncodeDacMapabilityConsensusExcludable.bed.gz  \\| bedtools merge -i stdin \\| bedtools intersect -u -nonamecheck -a /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz -b stdin \\| wc -l zcat -f /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\| wc -l bedtools sort -i /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_prom_p2.bed.gz  \\| bedtools merge -i stdin \\| bedtools intersect -u -nonamecheck -a /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz -b stdin \\| wc -l zcat -f /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\| wc -l bedtools sort -i /mnt/data/bioinfo_tools_and_refs/bioinfo_tools/kundajelab_ENCODE_atac_dnase_pipeline/bds_pipeline_genome_data/hg19/ataqc/reg2map_honeybadger2_dnase_enh_p2.bed.gz  \\| bedtools merge -i stdin \\| bedtools intersect -u -nonamecheck -a /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz -b stdin \\| wc -l zcat -f /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\| wc -l bedtools sort -i /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/pseudo_reps/rep1/NALM6_ATAC_ENCODE_pipeline_rep1-pr.IDR0.1.filt.narrowPeak.gz  \\| bedtools merge -i stdin \\| bedtools intersect -u -nonamecheck -a /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz -b stdin \\| wc -l zcat -f /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz \\| wc -l    --------------------Stderr--------------------     parser = TextFileReader(filepath_or_buffer, **kwds)   File \"/opt/anaconda3/envs/bds_atac/lib/python2.7/site-packages/pandas/io/parsers.py\", line 818, in __init__     self._make_engine(self.engine)   File \"/opt/anaconda3/envs/bds_atac/lib/python2.7/site-packages/pandas/io/parsers.py\", line 1049, in _make_engine     self._engine = CParserWrapper(self.f, **self.options)   File \"/opt/anaconda3/envs/bds_atac/lib/python2.7/site-packages/pandas/io/parsers.py\", line 1695, in __init__     self._reader = parsers.TextReader(src, **kwds)   File \"pandas/_libs/parsers.pyx\", line 402, in pandas._libs.parsers.TextReader.__cinit__   File \"pandas/_libs/parsers.pyx\", line 718, in pandas._libs.parsers.TextReader._setup_parser_source IOError: File /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_gc.txt does not exist\r\n-- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | --\r\n52\r\ntask.ataqc.ataqc_rep1.line_109.id_113\r\nataqc rep1\r\nthread_226\r\n47197\r\nfalse\r\n1\r\n \r\nERROR\r\nERROR\r\n \r\n \r\n2018-01-05 09:36:39\r\n2018-01-05 09:36:39\r\n00:00:00\r\n00:00:-1\r\n100 days\r\n/mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.fastq.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.align.log /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.nodup.pbc.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim.dup.qc /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.bam /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/align/rep1/NALM6_index_1.trim.nodup.tn5.tagAlign.gz /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/signal/macs2/rep1/NALM6_index_1.trim.nodup.tn5.pf.pval.signal.bigwig /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/peak/macs2/idr/pseudo_reps/rep1/NALM6_ATAC_ENCODE_pipeline_rep1-pr.IDR0.1.filt.narrowPeak.gz\r\n/mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.html /mnt/data/common/jonas/atacseq/NALM6_ATAC_ENCODE_pipeline/out/qc/rep1/NALM6_index_1.trim_qc.txt\r\n[atac.bds.20180104_113531_124.report.html.zip](https://github.com/kundajelab/atac_dnase_pipelines/files/1606673/atac.bds.20180104_113531_124.report.html.zip)\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/comments/373120348","html_url":"https://github.com/kundajelab/atac_dnase_pipelines/issues/89#issuecomment-373120348","issue_url":"https://api.github.com/repos/kundajelab/atac_dnase_pipelines/issues/89","id":373120348,"user":{"login":"leepc12","id":8625660,"avatar_url":"https://avatars0.githubusercontent.com/u/8625660?v=4","gravatar_id":"","url":"https://api.github.com/users/leepc12","html_url":"https://github.com/leepc12","followers_url":"https://api.github.com/users/leepc12/followers","following_url":"https://api.github.com/users/leepc12/following{/other_user}","gists_url":"https://api.github.com/users/leepc12/gists{/gist_id}","starred_url":"https://api.github.com/users/leepc12/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/leepc12/subscriptions","organizations_url":"https://api.github.com/users/leepc12/orgs","repos_url":"https://api.github.com/users/leepc12/repos","events_url":"https://api.github.com/users/leepc12/events{/privacy}","received_events_url":"https://api.github.com/users/leepc12/received_events","type":"User","site_admin":false},"created_at":"2018-03-14T18:05:55Z","updated_at":"2018-03-14T18:05:55Z","author_association":"COLLABORATOR","body":"@jonasungerback did you `git pull` the latest fix commit? did you run with `-no_random_source`?"}}
{"actor":{"display_login":"leepc12"}}