text: user3505: Probably a silly oversight on my part, but I'm trying to filter all the vcfs in a directory with bcftools using a simple loop. My basic command is working fine:
  bcftools filter -i 'QUAL > 1000' -o filter/file1out.vcf file1in.vcf 
  
  but when I try to loop it, it echoes through each filename, but generates no output files or error messages.
  for f in *.vcf; do echo "filtering $f"; bcftools filter -i 'QUAL > 1000' -o filter/$f_out.vcf $f; done
  
  Am I not handling the output direction properly? Any help much appreciated!
  
  user37: For one-liners, xargs is often more convenient than the for loop:
  ls *.vcf | sed s,.vcf$,, \
    | xargs -i echo 'bcftools filter -i "QUAL > 1000" -o "filter/{}_out.vcf" "{}.vcf"' \
    | parallel
  
  Note that your command line (with @WouterDeCoster's fix) writes output to foo.vcf_out.vcf. My command line writes to foo_out.vcf.
  
  user298: Using a for loop for this is by far the safer option. However, you need to quote your variables and also make sure you're passing the right variable name. While you're at it, you can also remove the extension so that your filtered file name can be foo_out.vcf instead of foo.vcf_out.vcf (which is what your loop would create. Putting all that together:
  for f in *.vcf; do 
      echo "filtering $f"; 
  bcftools filter -i 'QUAL > 1000' -o filter/"${f//.vcf}"_out.vcf "$f"; done
  fi
  
  
  user681: I assume your shell is looking for the variable $f_out, but can't find it because you did not define it. It probably generated a hidden file with the name .vcf in your output directory.
  Instead, use the following: -o ${f}_out.vcf. This will "protect/limit" the bash variable to just $f rather than $f_out.
  When encountering issues like this it's worth putting echo in your loop right before your command, in this case bcftools (adding quotes whenever necessary). That would show you the command as how it is going to be executed and would show you, in this case, that the output name is incorrect.
  
original_text: 3505: <p>Probably a silly oversight on my part, but I'm trying to filter all the vcfs in a directory with bcftools using a simple loop. My basic command is working fine:</p>
  
  <pre><code>bcftools filter -i 'QUAL &gt; 1000' -o filter/file1out.vcf file1in.vcf 
  </code></pre>
  
  <p>but when I try to loop it, it echoes through each filename, but generates no output files or error messages.</p>
  
  <pre><code>for f in *.vcf; do echo "filtering <span class="math-container">$f"; bcftools filter -i 'QUAL &gt; 1000' -o filter/$</span>f_out.vcf $f; done
  </code></pre>
  
  <p>Am I not handling the output direction properly? Any help much appreciated!</p>
   37: <p>For one-liners, <code>xargs</code> is often more convenient than the <code>for</code> loop:</p>
  
  <pre class="lang-shell prettyprint-override"><code>ls *.vcf | sed s,.vcf$,, \
    | xargs -i echo 'bcftools filter -i "QUAL &gt; 1000" -o "filter/{}_out.vcf" "{}.vcf"' \
    | parallel
  </code></pre>
  
  <p>Note that your command line (with @WouterDeCoster's fix) writes output to <code>foo.vcf_out.vcf</code>. My command line writes to <code>foo_out.vcf</code>.</p>
   298: <p>Using a <code>for</code> loop for this is <a href="https://mywiki.wooledge.org/ParsingLs" rel="nofollow noreferrer">by far the safer option</a>. However, you need to <a href="https://unix.stackexchange.com/a/131767/22222">quote your variables</a> and also make sure you're passing the right variable name. While you're at it, you can also remove the extension so that your filtered file name can be <code>foo_out.vcf</code> instead of <code>foo.vcf_out.vcf</code> (which is what your loop would create. Putting all that together:</p>
  
  <pre><code>for f in *.vcf; do 
      echo "filtering <span class="math-container">$f"; 
  bcftools filter -i 'QUAL &gt; 1000' -o filter/"$</span>{f//.vcf}"_out.vcf "$f"; done
  fi
  </code></pre>
   681: <p>I assume your shell is looking for the variable <code>$f_out</code>, but can't find it because you did not define it. It probably generated a hidden file with the name <code>.vcf</code> in your output directory.</p>
  
  <p>Instead, use the following: <code>-o <span class="math-container">${f}_out.vcf</code>. This will "protect/limit" the bash variable to just <code>$</span>f</code> rather than <code>$f_out</code>.</p>
  
  <p>When encountering issues like this it's worth putting <code>echo</code> in your loop right before your command, in this case bcftools (adding quotes whenever necessary). That would show you the command as how it is going to be executed and would show you, in this case, that the output name is incorrect.</p>
  