title: Distribute many independent, expensive operations over multiple cores in python
body: <p>Given a large <em>list</em> (1,000+) of completely independent <em>objects</em> that each need to be manipulated through some expensive function (~5 minutes each), what is the best way to distribute the work over other cores? Theoretically, I could just cut up the list into equal parts and serialize the data with cPickle (takes a few seconds) and launch a new python processes for each chunk--and it may just come to that if I intend to use multiple computers--but this feels like more of a hack than anything. Surely there is a more integrated way to do this using a multiprocessing library? Am I over-thinking this?</p>
  
  <p>Thanks.</p>