title: Twisted: use of multiple threads and processes together
body: <p>The Twisted documentation led me to believe that it was OK to combine techniques such as <code>reactor.spawnProcess()</code> and <code>threads.deferToThread()</code> in the same application, that the reactor would handle this elegantly under the covers.  Upon actually trying it, I found that my application deadlocks.  Using multiple threads by themselves, or child processes by themselves, everything is fine.</p>
  
  <p>Looking into the reactor source, I find that the <code>SelectReactor.spawnProcess()</code> method simply calls <code>os.fork()</code> without any consideration for multiple threads that might be running.  This explains the deadlocks, because starting with the call to <code>os.fork()</code> you will have two processes with multiple concurrent threads running and doing who knows what with the same file descriptors.</p>
  
  <p>My question for SO is, what is the best strategy for solving this problem?</p>
  
  <p>What I have in mind is to subclass <code>SelectReactor</code>, so that it is a singleton and calls <code>os.fork()</code> only once, immediately when instantiated.  The child process will run in the background and act as a server for the parent (using object serialization over pipes to communicate back and forth).  The parent continues to run the application and may use threads as desired.  Calls to <code>spawnProcess()</code> in the parent will be delegated to the child process, which will be guaranteed to have only one thread running and can therefore call <code>os.fork()</code> safely.</p>
  
  <p>Has anyone done this before?  Is there a faster way?</p>