text: user1377: Original tar.gz files look like below:
  UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz
  UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz
  UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz
  UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz
  
  Before extracting the tar.gz files, I changed their names based on names.text file as below and using the command:
  xargs -n2 mv -i <<<"$(sed '1d; s/$/.tar.gz/' names.txt)"
  
  Tar.gz_filenames       Sample_Names
  UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz                Sample2
  UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz                Sample1
  UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz                Sample3
  UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz                       Sample4
  
  After changing the names, looks like below:
  Sample1.tar.gz
  Sample2.tar.gz
  Sample3.tar.gz
  Sample4.tar.gz
  
  Using the tar.gz files after changing names I used the below code:
  for i in /doc/*.tar.gz
  do
      tar xvzf $i -C $TMPDIR
      for sample in $TMPDIR/*1.fastq
      do
          dir2="/destination/directory"
          base=$(basename $sample ".1.fastq")
          module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq | samtools view -Sb - > ${dir2}/${base}.bam
      done
  done
  
  When I run the above code, the output is like below:
  150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_2.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_2.fastq
  140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_2.fastq
  140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq
  110608_UNC11-SN627_0103_BD0CVVABXX.7_1.fastq
  110608_UNC11-SN627_0103_BD0CVVABXX.7_2.fastq
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  
  I dont mins whatever the name of fastq's in $TMPDIR but from $TMPDIR hisat2 should use those fastq's and give output bam files in /destination like below:
  Sample1.bam
  Sample2.bam
  Sample3.bam
  Sample4.bam
  
  I would like to run a script doing tar.gz files -> fastq -> bam (everything in loop)
  
  user77: Your samples end with _1.fastq and _2.fastq, so you need to change this:
  -1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq
  
  to this:
  -1 $TMPDIR/${base}_1.fastq -2 $TMPDIR/${base}_2.fastq
  
  
  user581: This should work for you:
  for i in *.tar.gz
  do
      dir2="/destination/directory"
      tar xvzf $i -C $TMPDIR
      files=($TMPDIR/*)
      module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - > ${dir2}/$i.bam
  done
  
  (If this is the sort of thing you'll be doing often, it would be worth investing the time to learn a more robust approach such as Snakemake or Nextflow).
  Edit:
  If you want to run on all files at the same time, the easiest way is find / xargs
  Change you script to this:
  for i in $@
  do
      dir2="/destination/directory"
      tar xvzf $i -C $TMPDIR
      files=($TMPDIR/*)
      module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - > ${dir2}/$i.bam
  done
  
  Then run it like this:
  find . -d 1 -name '*.tar.gz' | xargs -n 1 bash NAMEOFSCRIPT
  
  This works for SGE. I'm not sure about the details of SLURM. You might have to figure out how to run each job in the background or use gnu parallel or something. As @Ian Sudbury says, pipeline software can take care of all these details for you.
  
original_text: 1377: <p>Original tar.gz files look like below:</p>
  
  <pre><code>UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz
  UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz
  UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz
  UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz
  </code></pre>
  
  <p>Before extracting the tar.gz files, I changed their names based on names.text file as below and using the command:</p>
  
  <pre><code>xargs -n2 mv -i &lt;&lt;&lt;"$(sed '1d; s/$/.tar.gz/' names.txt)"
  
  Tar.gz_filenames       Sample_Names
  UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz                Sample2
  UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz                Sample1
  UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz                Sample3
  UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz                       Sample4
  </code></pre>
  
  <p>After changing the names, looks like below:</p>
  
  <pre><code>Sample1.tar.gz
  Sample2.tar.gz
  Sample3.tar.gz
  Sample4.tar.gz
  </code></pre>
  
  <p>Using the tar.gz files after changing names I used the below code:</p>
  
  <pre><code>for i in /doc/*.tar.gz
  do
      tar xvzf $i -C $TMPDIR
      for sample in $TMPDIR/*1.fastq
      do
          dir2="/destination/directory"
          base=$(basename $sample ".1.fastq")
          module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq | samtools view -Sb - &gt; ${dir2}/${base}.bam
      done
  done
  </code></pre>
  
  <p>When I run the above code, the output is like below:</p>
  
  <pre><code>150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_2.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq
  150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_2.fastq
  140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_2.fastq
  140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq
  110608_UNC11-SN627_0103_BD0CVVABXX.7_1.fastq
  110608_UNC11-SN627_0103_BD0CVVABXX.7_2.fastq
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  Warning: Could not open read file "/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq.1.fastq" for reading; skipping...
  Error: No input read files were valid
  (ERR): hisat2-align exited with value 1
  </code></pre>
  
  <p>I dont mins whatever the name of fastq's in <code>$TMPDIR</code> but from <code>$TMPDIR</code> hisat2 should use those fastq's and give output bam files in /destination like below:</p>
  
  <pre><code>Sample1.bam
  Sample2.bam
  Sample3.bam
  Sample4.bam
  </code></pre>
  
  <p>I would like to run a script doing tar.gz files -> fastq -> bam (everything in loop)</p>
   77: <p>Your samples end with <code>_1.fastq</code> and <code>_2.fastq</code>, so you need to change this:</p>
  
  <pre><code>-1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq
  </code></pre>
  
  <p>to this:</p>
  
  <pre><code>-1 $TMPDIR/${base}_1.fastq -2 $TMPDIR/${base}_2.fastq
  </code></pre>
   581: <p>This should work for you:</p>
  
  <pre><code>for i in *.tar.gz
  do
      dir2="/destination/directory"
      tar xvzf $i -C $TMPDIR
      files=($TMPDIR/*)
      module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - &gt; ${dir2}/$i.bam
  done
  </code></pre>
  
  <p>(If this is the sort of thing you'll be doing often, it would be worth investing the time to learn a more robust approach such as Snakemake or Nextflow).</p>
  
  <p>Edit:</p>
  
  <p>If you want to run on all files at the same time, the easiest way is <code>find</code> / <code>xargs</code></p>
  
  <p>Change you script to this:</p>
  
  <pre><code>for i in $@
  do
      dir2="/destination/directory"
      tar xvzf $i -C $TMPDIR
      files=($TMPDIR/*)
      module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - &gt; ${dir2}/$i.bam
  done
  </code></pre>
  
  <p>Then run it like this:</p>
  
  <pre><code>find . -d 1 -name '*.tar.gz' | xargs -n 1 bash NAMEOFSCRIPT
  </code></pre>
  
  <p>This works for SGE. I'm not sure about the details of SLURM. You might have to figure out how to run each job in the background or use <code>gnu parallel</code> or something. As <code>@Ian Sudbury</code> says, pipeline software can take care of all these details for you.</p>
  