text: user3839: I was trying to develop a pipeline using nextflow and plink. My code runs without any error, but got only two output files instead of 22 (one for each chromosome). Here is part my code block, any help?
  process step5 {
  publishDir params.publish, mode: 'copy'
  input:
  val(hrc) from HRC_1000G_check_bim
  val(hrc_site) from hrc_sites_file
  tuple val(prefix), val(chr_name), path(bfiles) from step4_frq
  output:
  tuple val("${prefix}"), val(chr_name), path("LOG-${prefix}${chr_name}.step4-HRC.txt") into step4_log
  tuple val("${prefix}"), val(chr_name), path("Exclude-${prefix}${chr_name}.step4-HRC.txt") into step4_exclude
  tuple val("${prefix}"), val(chr_name), path("Chromosome-${prefix}${chr_name}.step4-HRC.txt") into step4_chromosome
  tuple val("${prefix}"), val(chr_name), path("Position-${prefix}${chr_name}.step4-HRC.txt") into step4_position
  tuple val("${prefix}"), val(chr_name), path("Strand-Flip-${prefix}${chr_name}.step4-HRC.txt") into step4_strand
  tuple val("${prefix}"), val(chr_name), path("Force-Allele1-${prefix}${chr_name}.step4-HRC.txt") into step4_allele
  tuple val("${prefix}"), val(chr_name), path("ID-${prefix}${chr_name}.step4-HRC.txt") into step4_id
  script:
  """
  perl "${hrc}" -b "${prefix}${chr_name}.step4.bim" -f "${prefix}${chr_name}.step4.frq" -r "${hrc_site}" -h
  """
  }
  
  The output files are:
  gsa-out/Chromosome-chr22.step4-HRC.txt     gsa-out/Force-Allele1-chr3.step4-HRC.txt  gsa-out/Position-chr22.step4-HRC.txt
  gsa-out/Chromosome-chr3.step4-HRC.txt      gsa-out/ID-chr22.step4-HRC.txt            gsa-out/Position-chr3.step4-HRC.txt
  gsa-out/Exclude-chr22.step4-HRC.txt        gsa-out/ID-chr3.step4-HRC.txt             gsa-out/Strand-Flip-chr22.step4-HRC.txt
  gsa-out/Exclude-chr3.step4-HRC.txt         gsa-out/LOG-chr22.step4-HRC.txt           gsa-out/Strand-Flip-chr3.step4-HRC.txt
  gsa-out/Force-Allele1-chr22.step4-HRC.txt  gsa-out/LOG-chr3.step4-HRC.txt
  
  I am not sure why I am getting only 2, instead of 22 output files. With similar output channel previously generated 22 output files in the same script. For eaxmple:
  process step1b {
  publishDir params.publish, mode: 'copy'
  input:
  tuple val(prefix), val(chr_name), path(bfiles) from step1a_out
  output:
  tuple val("${prefix}"), val(chr_name), path("${prefix}${chr_name}.step1c.{bed,bim,fam}") into step1b_results
  path("${prefix}${chr_name}.step1c.log") into step1b_logs
  path("${prefix}${chr_name}.step1b.snplist.txt") into step1b_snplist
  script:
  
  """
  awk '{ if ((\\\$5=="T" && \\\$6=="A")||(\\\$5=="A" && \\\$6=="T")||(\\\$5=="C" && \\\$6=="G")||(\\\$5=="G" && \\\$6=="C")) print \\\$2, "ambig" ; else print \\\$2 ;}' "${bfiles[1]
  }" | grep ambig > "${prefix}${chr_name}.step1b.snplist.txt"
  plink1.9 --bfile "${prefix}${chr_name}.step1b" --exclude "${prefix}${chr_name}.step1b.snplist.txt" --make-bed --out "${prefix}${chr_name}.step1c"
  """
  }
  
  
  user10775: I suspect your HRC_1000G_check_bim and hrc_sites_file channels are queue channels when they should be value channels. Please again see how multiple channels work. Also, third-party scripts, like your Perl script, would be better placed in a directory called bin/ at the root of your project repository. Nextflow adds this directory to your $PATH, so scripts in this directory will be automatically accessible in your pipeline without needing to localize them using a separate channel. This way also avoids having to specify an absolute path to invoke them. Just make sure your script is executable: chmod +x your_script.pl
  hrc_sites = file( params.hrc_sites_file )
  
  process step5 {
  
      input:
      tuple val(prefix), val(chr_name), path(bfiles) from step4_frq
      path hrc_sites
  
      """
      your_script.pl \\
          -b "${prefix}${chr_name}.step4.bim" \\
          -f "${prefix}${chr_name}.step4.frq" \\
          -r "${hrc_sites}" \\
          -h
      """
  }
  
  
original_text: 3839: <p>I was trying to develop a pipeline using nextflow and plink. My code runs without any error, but got only two output files instead of 22 (one for each chromosome). Here is part my code block, any help?</p>
  <pre><code>process step5 {
  publishDir params.publish, mode: 'copy'
  input:
  val(hrc) from HRC_1000G_check_bim
  val(hrc_site) from hrc_sites_file
  tuple val(prefix), val(chr_name), path(bfiles) from step4_frq
  output:
  tuple val(&quot;<span class="math-container">${prefix}"), val(chr_name), path("LOG-$</span>{prefix}<span class="math-container">${chr_name}.step4-HRC.txt") into step4_log
  tuple val("$</span>{prefix}&quot;), val(chr_name), path(&quot;Exclude-<span class="math-container">${prefix}$</span>{chr_name}.step4-HRC.txt&quot;) into step4_exclude
  tuple val(&quot;<span class="math-container">${prefix}"), val(chr_name), path("Chromosome-$</span>{prefix}<span class="math-container">${chr_name}.step4-HRC.txt") into step4_chromosome
  tuple val("$</span>{prefix}&quot;), val(chr_name), path(&quot;Position-<span class="math-container">${prefix}$</span>{chr_name}.step4-HRC.txt&quot;) into step4_position
  tuple val(&quot;<span class="math-container">${prefix}"), val(chr_name), path("Strand-Flip-$</span>{prefix}<span class="math-container">${chr_name}.step4-HRC.txt") into step4_strand
  tuple val("$</span>{prefix}&quot;), val(chr_name), path(&quot;Force-Allele1-<span class="math-container">${prefix}$</span>{chr_name}.step4-HRC.txt&quot;) into step4_allele
  tuple val(&quot;<span class="math-container">${prefix}"), val(chr_name), path("ID-$</span>{prefix}<span class="math-container">${chr_name}.step4-HRC.txt") into step4_id
  script:
  """
  perl "$</span>{hrc}&quot; -b &quot;<span class="math-container">${prefix}$</span>{chr_name}.step4.bim&quot; -f &quot;<span class="math-container">${prefix}$</span>{chr_name}.step4.frq&quot; -r &quot;${hrc_site}&quot; -h
  &quot;&quot;&quot;
  }
  </code></pre>
  <p>The output files are:</p>
  <pre><code>gsa-out/Chromosome-chr22.step4-HRC.txt     gsa-out/Force-Allele1-chr3.step4-HRC.txt  gsa-out/Position-chr22.step4-HRC.txt
  gsa-out/Chromosome-chr3.step4-HRC.txt      gsa-out/ID-chr22.step4-HRC.txt            gsa-out/Position-chr3.step4-HRC.txt
  gsa-out/Exclude-chr22.step4-HRC.txt        gsa-out/ID-chr3.step4-HRC.txt             gsa-out/Strand-Flip-chr22.step4-HRC.txt
  gsa-out/Exclude-chr3.step4-HRC.txt         gsa-out/LOG-chr22.step4-HRC.txt           gsa-out/Strand-Flip-chr3.step4-HRC.txt
  gsa-out/Force-Allele1-chr22.step4-HRC.txt  gsa-out/LOG-chr3.step4-HRC.txt
  </code></pre>
  <p>I am not sure why I am getting only 2, instead of 22 output files. With similar output channel previously generated 22 output files in the same script. For eaxmple:</p>
  <pre><code>process step1b {
  publishDir params.publish, mode: 'copy'
  input:
  tuple val(prefix), val(chr_name), path(bfiles) from step1a_out
  output:
  tuple val(&quot;<span class="math-container">${prefix}"), val(chr_name), path("$</span>{prefix}<span class="math-container">${chr_name}.step1c.{bed,bim,fam}") into step1b_results
  path("$</span>{prefix}<span class="math-container">${chr_name}.step1c.log") into step1b_logs
  path("$</span>{prefix}${chr_name}.step1b.snplist.txt&quot;) into step1b_snplist
  script:
  
  &quot;&quot;&quot;
  awk '{ if ((\\\$5==&quot;T&quot; &amp;&amp; \\\$6==&quot;A&quot;)||(\\\$5==&quot;A&quot; &amp;&amp; \\\$6==&quot;T&quot;)||(\\\$5==&quot;C&quot; &amp;&amp; \\\$6==&quot;G&quot;)||(\\\$5==&quot;G&quot; &amp;&amp; \\\$6==&quot;C&quot;)) print \\\$2, &quot;ambig&quot; ; else print \\\$2 ;}' &quot;<span class="math-container">${bfiles[1]
  }" | grep ambig &gt; "$</span>{prefix}<span class="math-container">${chr_name}.step1b.snplist.txt"
  plink1.9 --bfile "$</span>{prefix}<span class="math-container">${chr_name}.step1b" --exclude "$</span>{prefix}<span class="math-container">${chr_name}.step1b.snplist.txt" --make-bed --out "$</span>{prefix}${chr_name}.step1c&quot;
  &quot;&quot;&quot;
  }
  </code></pre>
   10775: <p>I suspect your <code>HRC_1000G_check_bim</code> and <code>hrc_sites_file</code> channels are <a href="https://www.nextflow.io/docs/latest/channel.html#queue-channel" rel="nofollow noreferrer">queue channels</a> when they should be <a href="https://www.nextflow.io/docs/latest/channel.html#value-channel" rel="nofollow noreferrer">value channels</a>. Please again see <a href="https://www.nextflow.io/docs/latest/process.html#understand-how-multiple-input-channels-work" rel="nofollow noreferrer">how multiple channels work</a>. Also, <a href="https://www.nextflow.io/docs/latest/sharing.html#third-party-scripts" rel="nofollow noreferrer">third-party scripts</a>, like your Perl script, would be better placed in a directory called <code>bin/</code> at the root of your project repository. Nextflow adds this directory to your $PATH, so scripts in this directory will be automatically accessible in your pipeline without needing to localize them using a separate channel. This way also avoids having to specify an absolute path to invoke them. Just make sure your script is executable: <code>chmod +x your_script.pl</code></p>
  <pre><code>hrc_sites = file( params.hrc_sites_file )
  
  process step5 {
  
      input:
      tuple val(prefix), val(chr_name), path(bfiles) from step4_frq
      path hrc_sites
  
      &quot;&quot;&quot;
      your_script.pl \\
          -b &quot;<span class="math-container">${prefix}$</span>{chr_name}.step4.bim&quot; \\
          -f &quot;<span class="math-container">${prefix}$</span>{chr_name}.step4.frq&quot; \\
          -r &quot;${hrc_sites}&quot; \\
          -h
      &quot;&quot;&quot;
  }
  </code></pre>
  