title: SLURM: flag to auto-requeue jobs cancelled due to preemption?
body: <p>I'm running the following job array on SLURM:</p>
  <pre><code>#!/bin/bash
  
  #SBATCH --array=1-1000
  #SBATCH --partition=scavenge
  #SBATCH --mem=2g
  #SBATCH --time=1:00:00
  
  module load Python/3.6.4-iomkl-2018a
  python run.py ${SLURM_ARRAY_TASK_ID}
  </code></pre>
  <p>Many of my jobs error out with:</p>
  <pre><code>slurmstepd: error: *** JOB 63830645 ON p08r06n17 CANCELLED AT 2020-08-18T21:40:52 DUE TO PREEMPTION ***
  </code></pre>
  <p>I'd like to requeue those jobs automatically if they're preempted. Is it possible to do so? Any pointers on this query would be very appreciated!</p>