title: Using Python multiprocessing in Slurm, and which combination of ntasks or ncpus I need
body: <p>I'm trying to run a python script on a slurm cluster, and I'm using python's built-in <code>multiprocessing</code> module. </p>
  
  <p>I'm using quite a simple set up, where for testing purpose, the example is:</p>
  
  <pre><code>len(arg_list)
  Out[2]: 5
  
  threads = multiprocessing.Pool(5)
  output = threads.map(func, arg_list)
  </code></pre>
  
  <p>So <code>func</code> is applied 5 times in parallel on 5 arguments in <code>arg_list</code>. What I want to know is how to allocate the correct amount of cpu's/tasks in slurm for this to work as expected. This is what the relevant part of my slurm batch script looks like:</p>
  
  <pre><code>#!/bin/bash
  
  # Runtime and memory
  #SBATCH --time=90:00:00
  #SBATCH --mem-per-cpu=2G
  
  # For parallel jobs
  #SBATCH --cpus-per-task=10
  ##SBATCH --nodes=2
  #SBATCH --ntasks=1
  ##SBATCH --ntasks-per-node=4  
  
  #### Your shell commands below this line ####
  
  srun ./script_wrapper.py 'test'
  </code></pre>
  
  <p>As you can see, at the moment I have <code>ntasks=1</code> and <code>cpus-per-task=10</code>. Note that the main bulk of func contains a scipy routine which tends to run on two cores (i.e uses 200% cpu usage, which is why I want 10 cpus and not 5). </p>
  
  <p>Is this the correct way to allocate resources for my purposes, because at the moment the job takes a lot longer than expected (more like it's running in a single thread). </p>
  
  <p>Do I need to set <code>ntasks=5</code> instead? Because my impression from online documentation was that <code>ntasks=5</code> would instead call <code>srun ./script_wrapper.py 'test'</code> five times instead, which is not what I want. Am I right in that assumption?</p>
  
  <p>Also, is there a way to easily check stuff like CPU usage and all the process id's of the python tasks called by multiprocessing.Pool? At the moment I'm trying with <code>sacct -u &lt;user&gt; --format=JobID,JobName,MaxRSS,Elapsed,AveCPU</code>, but the <code>AveCPU</code> and <code>MaxRSS</code> fields always come up empty for some reason (?) and while I see the first script as a process, I don't see the 5 others that should be called by multiprocessing. Example:</p>
  
  <pre><code>       JobID    JobName     MaxRSS    Elapsed     AveCPU 
  ------------ ---------- ---------- ---------- ---------- 
  16260892             GP              00:13:07            
  16260892.0   script_wr+              00:13:07            
  </code></pre>