text: user2005: I do not understand how to specify correctly the parameters on a SLURM cluster for snakemake to use them. I tried submitting the following SLURM file, but it does not work this way and the number of cores used is only 1, not 20:
  #!/bin/bash
  
  #SBATCH -p standard
  #SBATCH -A overall 
  #SBATCH --time=12:00:00
  #SBATCH --output=snakemake%A.out
  #SBATCH --error=snakemake%A.err
  #SBATCH --nodes=1
  #SBATCH --ntasks=1
  #SBATCH --cpus-per-task=20
  #SBATCH --mem=120000
  
  snakemake
  
  Then, I tried following snakemake tutorial. And created cluster.json based on the SLURM parameters that I need:
  {
      "__default__" :
      {
          "A" : "overall",
          "time" : "24:00:00",
          "nodes": 1,
          "ntasks": 1,
          "cpus" : 20,
          "p" : "standard",
          "mem": 120000,
          "output": "snakemake%A.out",
          "error": "snakemake%A.err"
      }
  }
  
  And ran snakemake inside a newly created snakemake.sh script:
  #!/bin/bash
  
  snakemake -j 999 --cluster-config cluster.json --cluster "sbatch -A 
  {cluster.A} -p {cluster.p} \ 
  -t {cluster.time} --output {cluster.output} --error {cluster.error} -- 
  nodes {cluster.nodes} \
  --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem 
  {cluster.mem}"
  
  And it is giving me an error now:
  
  sbatch: error: Unable to open file
    /bin/sh: line 1: -t: command not found
    Error submitting jobscript (exit code 127):
  
  I am now completely lost at what I shoould actually do. I would prefer plain regular .slurm file submission, but how to make snakemake use them? Any suggestions would be greatly appreciated.
  I removed \ - line separators in the snakemake.sh script:
  #!/bin/bash
  
  snakemake -j 10 --cluster-config cluster.json --cluster "sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --nodes {cluster.nodes} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}"
  
  And it started to run. It is not convenient though for me. I would rather prefer submitting just one job using .slurm file passing all of the parameters from #SBATCH. Is it possible?
  
  user77: You can omit --nodes, you need the following:
  #!/bin/bash
  #SBATCH --ntasks-per-node=1
  #SBATCH -c threads_from_snakemake
  #SBATCH -p some_partition
  
  Commands go here
  
  For slurm, you may want to modify my SlurmEasy script. We use that with snakemake all the time, usually of the form:
  snakemake -c "SlurmEasy -t {threads} -n {rule} --mem-per-cpu {cluster.memory}" ...
  
  You'll also want units with your memory requests, like 12G. Also, in general it's best not to submit snakemake itself to the cluster, rather run it on an interactive node and have it submit jobs on your behalf (if the admins complain point out that snakemake is barely using any resources on the head node).
  
original_text: 2005: <p>I do not understand how to specify correctly the parameters on a <code>SLURM</code> cluster for <code>snakemake</code> to use them. I tried submitting the following <code>SLURM</code> file, but it does not work this way and the number of cores used is only <code>1</code>, not <code>20</code>:</p>
  
  <pre><code>#!/bin/bash
  
  #SBATCH -p standard
  #SBATCH -A overall 
  #SBATCH --time=12:00:00
  #SBATCH --output=snakemake%A.out
  #SBATCH --error=snakemake%A.err
  #SBATCH --nodes=1
  #SBATCH --ntasks=1
  #SBATCH --cpus-per-task=20
  #SBATCH --mem=120000
  
  snakemake
  </code></pre>
  
  <p>Then, I tried following <code>snakemake</code> <a href="https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html" rel="noreferrer">tutorial</a>. And created <code>cluster.json</code> based on the <code>SLURM</code> parameters that I need:</p>
  
  <pre><code>{
      "__default__" :
      {
          "A" : "overall",
          "time" : "24:00:00",
          "nodes": 1,
          "ntasks": 1,
          "cpus" : 20,
          "p" : "standard",
          "mem": 120000,
          "output": "snakemake%A.out",
          "error": "snakemake%A.err"
      }
  }
  </code></pre>
  
  <p>And ran <code>snakemake</code> inside a newly created <code>snakemake.sh</code> script:</p>
  
  <pre><code>#!/bin/bash
  
  snakemake -j 999 --cluster-config cluster.json --cluster "sbatch -A 
  {cluster.A} -p {cluster.p} \ 
  -t {cluster.time} --output {cluster.output} --error {cluster.error} -- 
  nodes {cluster.nodes} \
  --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem 
  {cluster.mem}"
  </code></pre>
  
  <p>And it is giving me an error now:</p>
  
  <blockquote>
    <p>sbatch: error: Unable to open file<br>
    /bin/sh: line 1: -t: command not found
    Error submitting jobscript (exit code 127):</p>
  </blockquote>
  
  <p>I am now completely lost at what I shoould actually do. I would prefer plain regular <code>.slurm</code> file submission, but how to make <code>snakemake</code> use them? Any suggestions would be greatly appreciated.</p>
  
  <p>I removed <code>\</code> - line separators in the <code>snakemake.sh</code> script:</p>
  
  <pre><code>#!/bin/bash
  
  snakemake -j 10 --cluster-config cluster.json --cluster "sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --nodes {cluster.nodes} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}"
  </code></pre>
  
  <p>And it started to run. It is not convenient though for me. I would rather prefer submitting just one job using <code>.slurm</code> file passing all of the parameters from <code>#SBATCH</code>. Is it possible?</p>
   77: <p>You can omit <code>--nodes</code>, you need the following:</p>
  
  <pre><code>#!/bin/bash
  #SBATCH --ntasks-per-node=1
  #SBATCH -c threads_from_snakemake
  #SBATCH -p some_partition
  
  Commands go here
  </code></pre>
  
  <p>For slurm, you may want to modify my <a href="https://github.com/dpryan79/Misc/blob/master/MPIIE_internal/SlurmEasy" rel="noreferrer">SlurmEasy script</a>. We use that with snakemake all the time, usually of the form:</p>
  
  <pre><code>snakemake -c "SlurmEasy -t {threads} -n {rule} --mem-per-cpu {cluster.memory}" ...
  </code></pre>
  
  <p>You'll also want units with your memory requests, like <code>12G</code>. Also, in general it's best not to submit snakemake itself to the cluster, rather run it on an interactive node and have it submit jobs on your behalf (if the admins complain point out that snakemake is barely using any resources on the head node).</p>
  