title: Pathos, Dask, futures, which one to use for parallel cluster application?
body: <p>I am confused here. I have an application that is CPU bounded so I went to implementing a parallelisation using multiprocess to overcome GIL issues.</p>
  
  <p>I first tried to use <code>multiprocessing</code> and <code>futures</code> but I faced a pickling issue so I went to <code>pathos</code> which uses <code>dill</code> as a <code>pickle</code> replacement.</p>
  
  <p>Everything is working but I am wondering if I am using the most "future proof" solution. I have seen also <code>dask</code>, but I don't know if it will work in case of pickling classes issues (see <a href="https://stackoverflow.com/q/29121979/6522112">Python: (Pathos) Multiprocessing vs. class methods</a>). From the doc, it uses <code>futures</code> so I am assuming that it won't do the job.</p>
  
  <p>Secondly, I would like to be able to use two servers at a time and I have seen that this is possible with <code>pathos</code> (also <code>dask</code>), but I don't understand how exactly this works. This answer <a href="https://stackoverflow.com/a/26948258/6522112">https://stackoverflow.com/a/26948258/6522112</a> only shows how to use one server. How about using 2 or more? I can't find any example about that although it seems possible as described by the package info.</p>
  
  <p>Thanks for your help!</p>