title: python luigi died unexpectedly with exit code -11
body: <p>I have a data pipeline with <code>luigi</code> that works perfectly fine if I put 1 worker to the task. However, if I put > 1 workers, then it dies (unexpectedly with exit code -11) in a stage with 2 dependencies. The code is rather complex, so a minimum example would be difficult to give. The gist of the matter is that I am doing the following things with <code>gensim</code>:</p>
  
  <ol>
  <li>Building a dictionary from some texts.</li>
  <li>Building a corpus from said texts and the dictionary (requires (1)).</li>
  <li>Training an LDA model from the corpus and dictionary (requires (1) and (2)).</li>
  </ol>
  
  <p>For some reason, step (3) crashes every time I put more than one worker, even if (1) and (2) are already completed...</p>
  
  <p>Any help would be greatly appreciated!</p>
  
  <p><strong>EDIT:</strong> Here is an example of the logging info. TrainLDA is task (3). There are still two tasks after that that require TrainLDA. All earlier tasks finished correctly. I substituted TrainLDA's arguments for <code>...</code> so that the output would be more readable. The additional info are just <code>print</code> statements we put to help us know what is happening.</p>
  
  <p>DEB</p>
  
  <pre><code>UG: Pending tasks: 3
  DEBUG: Asking scheduler for work...
  INFO: [pid 28851] Worker Worker(salt=514562349, workers=4, host=felipe.local, username=Felipe, pid=28825) running   TrainLDA(...)
  INFO: Done
  INFO: There are no more tasks to run at this time
  INFO: TrainLDA(...) is currently run by worker Worker(salt=514562349, workers=4, host=felipe.local, username=Felipe, pid=28825)
  ==============================
  Corriendo LDA de spanish con nivel de limpieza stopwords
  ==============================
  Número de tópicos: 40
  DEBUG: Asking scheduler for work...
  INFO: Done
  INFO: There are no more tasks to run at this time
  INFO: TrainLDA(...) is currently run by worker Worker(salt=514562349, workers=4, host=felipe.local, username=Felipe, pid=28825)
  DEBUG: Asking scheduler for work...
  INFO: Done
  INFO: There are no more tasks to run at this time
  INFO: TrainLDA(...) is currently run by worker Worker(salt=514562349, workers=4, host=felipe.local, username=Felipe, pid=28825)
  INFO: Worker task TrainLDA(...) died unexpectedly with exit code -11
  DEBUG: Asking scheduler for work...
  INFO: Done
  INFO: There are no more tasks to run at this time
  INFO: There are 2 pending tasks possibly being run by other workers
  INFO: There are 2 pending tasks unique to this worker
  INFO: Worker Worker(salt=514562349, workers=4, host=felipe.local, username=Felipe, pid=28825) was stopped. Shutting down Keep-Alive thread
  </code></pre>