text: user16196: I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:
  Adapter file:
  >IL_Multiplex_five_prime_fwd
  AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
  >IL_Multiplex_three_prime_rev
  AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC
  
  Program file:
  ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
  MINLEN:36
  SLIDINGWINDOW:4:15
  LEADING:3
  TRAILING:3
  
  In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:
  Sample workflow:
  version 1.0
   
  Workflow trim {
      input {
          File fastq1 = “R1.fastq.gz”
          File fastq2 = “R2.fastq.gz”
          File program_file = “trim_program.txt”
          File adapter_file = “adapters.fa”
      }
   
      Array[String] trim_program_array = read_lines(program_file)
   
      call trimmomatic {
          input:
              fastq1 = fastq1,
              fastq2 = fastq2,
              trim_program_array = trim_program_array,
              adapter_file = adaptor_file
      }
  }
   
  Task trimmomatic {
      input {
          File fastq1
          File fastq2
          File outfile1 = “trimmed_R1.fastq.gz”
          File outfile2 = “trimmed_R2.fastq.gz”
          Array[String] trim_program_array
          File adapter_file
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{sep = “ “ trim_program_array}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  
  My goal is for the first element of ~{sep = " " trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?
  
  user6955: Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with read_lines and won't get treated like WDL code. Since the lines from the program_file are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (~{}), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec here). So you can use optional inputs to make the different lines non-mandatory:
  task trimmomatic {
      input {
          File fastq1
          File fastq2
          String outfile1 = “trimmed_R1.fastq.gz”
          String outfile2 = “trimmed_R2.fastq.gz”
          File? adapter_file
          Int? minlen
          String? slidingwindow
          Int? leading
          Int? trailing
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{"ILLUMINACLIP:" + adapter_file + ":2:30:10:7:keepBothReads"} \
          ~{"MINLEN:" + minlen} \
          ~{"SLIDINGWINDOW:" + slidingwindow} \
          ~{"LEADING:" + leading} \
          ~{"TRAILING:" + trailing}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  
  If, say, adapter_file were to be omitted in the inputs, then the produced command should look something like this:
  TrimmomaticPE \
  R1.fq R2.fq \
  trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
   \
  MINLEN:36 \
  SLIDINGWINDOW:4:15 \
  LEADING:3 \
  TRAILING:3
  
  Also note that the output file paths in the input section should be Strings. Since they don't exist yet, the execution engine will complain that it can't find them if they are Files
  Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the :2:30:10:7:keepBothReads bit could be split up into additional inputs as well.
  
original_text: 16196: <p>I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:</p>
  <p>Adapter file:</p>
  <pre><code>&gt;IL_Multiplex_five_prime_fwd
  AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
  &gt;IL_Multiplex_three_prime_rev
  AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC
  </code></pre>
  <p>Program file:</p>
  <pre><code>ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads
  MINLEN:36
  SLIDINGWINDOW:4:15
  LEADING:3
  TRAILING:3
  </code></pre>
  <p>In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:</p>
  <p>Sample workflow:</p>
  <pre><code>version 1.0
   
  Workflow trim {
      input {
          File fastq1 = “R1.fastq.gz”
          File fastq2 = “R2.fastq.gz”
          File program_file = “trim_program.txt”
          File adapter_file = “adapters.fa”
      }
   
      Array[String] trim_program_array = read_lines(program_file)
   
      call trimmomatic {
          input:
              fastq1 = fastq1,
              fastq2 = fastq2,
              trim_program_array = trim_program_array,
              adapter_file = adaptor_file
      }
  }
   
  Task trimmomatic {
      input {
          File fastq1
          File fastq2
          File outfile1 = “trimmed_R1.fastq.gz”
          File outfile2 = “trimmed_R2.fastq.gz”
          Array[String] trim_program_array
          File adapter_file
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{sep = “ “ trim_program_array}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  </code></pre>
  <p>My goal is for the first element of ~{sep = &quot; &quot; trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?</p>
   6955: <p>Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with <code>read_lines</code> and won't get treated like WDL code. Since the lines from the <code>program_file</code> are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (<code>~{}</code>), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec <a href="https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter" rel="nofollow noreferrer">here</a>). So you can use optional inputs to make the different lines non-mandatory:</p>
  <pre><code>task trimmomatic {
      input {
          File fastq1
          File fastq2
          String outfile1 = “trimmed_R1.fastq.gz”
          String outfile2 = “trimmed_R2.fastq.gz”
          File? adapter_file
          Int? minlen
          String? slidingwindow
          Int? leading
          Int? trailing
       }
   
       command {
          TrimmomaticPE \
          ~{fastq1} ~{fastq2} \
          ~{outfile1} ~{outfile2} \
          ~{&quot;ILLUMINACLIP:&quot; + adapter_file + &quot;:2:30:10:7:keepBothReads&quot;} \
          ~{&quot;MINLEN:&quot; + minlen} \
          ~{&quot;SLIDINGWINDOW:&quot; + slidingwindow} \
          ~{&quot;LEADING:&quot; + leading} \
          ~{&quot;TRAILING:&quot; + trailing}
       }
   
       output {
          File out1 = outfile1
          File out2 = outfile2
       }
  }
  </code></pre>
  <p>If, say, <code>adapter_file</code> were to be omitted in the inputs, then the produced command should look something like this:</p>
  <pre><code>TrimmomaticPE \
  R1.fq R2.fq \
  trimmed_R1.fastq.gz trimmed_R2.fastq.gz \
   \
  MINLEN:36 \
  SLIDINGWINDOW:4:15 \
  LEADING:3 \
  TRAILING:3
  </code></pre>
  <p>Also note that the output file paths in the input section should be <code>String</code>s. Since they don't exist yet, the execution engine will complain that it can't find them if they are <code>File</code>s</p>
  <p>Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the <code>:2:30:10:7:keepBothReads</code> bit could be split up into additional inputs as well.</p>
  