title: Parallel execution of a snakemake rule with same input and a range of values for a single parameter
body: <p>I am transitioning a bash script to snakemake and I would like to parallelize a step I was previously handling with a for loop. The issue I am running into is that instead of running parallel processes, snakemake ends up trying to run one process with all parameters and fails.</p>
  
  <p>My original bash script runs a program multiple times for a range of values of the parameter <code>K</code>.</p>
  
  <pre><code>for num in {1..3}
  do
    structure.py -K $num --input=fileprefix --output=fileprefix
  done
  </code></pre>
  
  <p>There are multiple input files that start with <code>fileprefix</code>. And there are two main outputs per run, e.g. for K=1 they are <code>fileprefix.1.meanP</code>, <code>fileprefix.1.meanQ</code>. My config and snakemake files are as follows.</p>
  
  <p>Config:</p>
  
  <pre><code>cat config.yaml
  
  infile: fileprefix
  K:
    - 1
    - 2
    - 3
  </code></pre>
  
  <p>Snakemake:</p>
  
  <pre><code>configfile: 'config.yaml'
  
  rule all:
      input:
          expand("output/{sample}.{K}.{ext}",
              sample = config['infile'],
              K = config['K'],
              ext = ['meanQ', 'meanP'])
  
  rule structure:
      output:
          "output/{sample}.{K}.meanQ",
          "output/{sample}.{K}.meanP"
      params:
          prefix = config['infile'],
          K = config['K']
      threads: 3
      shell:
          """
          structure.py -K {params.K} \
          --input=output/{params.prefix} \
          --output=output/{params.prefix}
          """
  </code></pre>
  
  <p>This was executed with <code>snakemake --cores 3</code>. The problem persists when I only use one thread.</p>
  
  <p>I expected the outputs described above for each value of K, but the run fails with this error:</p>
  
  <pre><code>RuleException:
  CalledProcessError in line 84 of Snakefile:
  Command ' set -euo pipefail;  structure.py -K 1 2 3 --input=output/fileprefix \
  --output=output/fileprefix ' returned non-zero exit status 2.
    File "Snakefile", line 84, in __rule_Structure
    File "snake/lib/python3.6/concurrent/futures/thread.py", line 56, in run
  </code></pre>
  
  <p>When I set K to a single value such as <code>K = ['1']</code>, everything works. So the problem seems to be that <code>{params.K}</code> is being expanded to all values of K when the shell command is executed. I started teaching myself snakemake today, and it works really well, but I'm hitting a brick wall with this.</p>