title: Submitting Open MPI jobs to SGE
body: <p>I've installed openmpi , not in <code>/usr/...</code> but in a <code>/commun/data/packages/openmpi/</code> , it was compiled with <code>--with-sge</code>.</p>
  
  <p>I've added a new PE in SGE as descibed in <a href="http://docs.oracle.com/cd/E19080-01/n1.grid.eng6/817-5677/6ml49n2c0/index.html" rel="nofollow">http://docs.oracle.com/cd/E19080-01/n1.grid.eng6/817-5677/6ml49n2c0/index.html</a></p>
  
  <pre><code># /commun/data/packages/openmpi/bin/ompi_info | grep gridengine
  MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.6.3)
  
  # qconf -sq all.q | grep pe_
  pe_list               make orte
  </code></pre>
  
  <p>Without SGE, the program runs without any problem, using several processors.</p>
  
  <pre><code>/commun/data/packages/openmpi/bin/orterun -np 20 ./a.out args
  </code></pre>
  
  <p>Now I want to submit my program to SGE</p>
  
  <p>In the Open MPI <a href="http://www.open-mpi.org/faq/?category=running#run-n1ge-or-sge" rel="nofollow">FAQ</a>, I read:</p>
  
  <pre><code># Allocate a SGE interactive job with 4 slots
  # from a parallel environment (PE) named 'orte'
  shell$ qsh -pe orte 4
  </code></pre>
  
  <p>but my output is:</p>
  
  <pre><code>qsh -pe orte 4
  Your job 84550 ("INTERACTIVE") has been submitted
  waiting for interactive job to be scheduled ...
  Could not start interactive job.
  </code></pre>
  
  <p>I've also tried the <code>mpirun</code> command embedded in a script:</p>
  
  <pre><code>$ cat ompi.sh
  #!/bin/sh
  /commun/data/packages/openmpi/bin/mpirun  \
      /path/to/a.out args
  </code></pre>
  
  <p>but it fails</p>
  
  <pre><code>$ cat ompi.sh.e84552
  error: executing task of job 84552 failed: execution daemon on host "node02" didn't accept task
  --------------------------------------------------------------------------
  A daemon (pid 18327) died unexpectedly with status 1 while attempting
  to launch so we are aborting.
  
  There may be more information reported by the environment (see above).
  
  This may be because the daemon was unable to find all the needed shared
  libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
  location of the shared libraries on the remote nodes and this will
  automatically be forwarded to the remote nodes.
  --------------------------------------------------------------------------
  error: executing task of job 84552 failed: execution daemon on host "node01" didn't accept task
  --------------------------------------------------------------------------
  mpirun noticed that the job aborted, but has no info as to the process
  that caused that situation.
  </code></pre>
  
  <p>How can I fix this?</p>
  
  <hr>
  
  <p>answer in the openmpi mailing list: <a href="http://www.open-mpi.org/community/lists/users/2013/02/21360.php" rel="nofollow">http://www.open-mpi.org/community/lists/users/2013/02/21360.php</a></p>