title: What can cause IOError: bad message length
body: <p>I am using python scripts to manipulate and extract information from 4D images (functional MRI scans). Part of the analysis is setup to run in parallel (for every subject) using the multiprocessing package:</p>
  
  <pre><code>pool = Pool(processes=numberCores)
  resultList = pool.map(SubjectProcesser, argList) # where arglist is the list of arguments passed to the process
  </code></pre>
  
  <p>These are applied to different kinds of files and different types of analysis. For one specific type of analysis, I get the following error:</p>
  
  <pre><code>Process PoolWorker-1:
  Traceback (most recent call last):
  File "/home2/user/epd/epd-7.2-2-rh5-x86_64/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home2/user/epd/epd-7.2-2-rh5-x86_64/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "/home2/surchs/epd/epd-7.2-2-rh5-x86_64/lib/python2.7/multiprocessing/pool.py", line 99, in worker
    put((job, i, result))
  File "/home2/user/epd/epd-7.2-2-rh5-x86_64/lib/python2.7/multiprocessing/queues.py", line 392, in put
    return send(obj)
  IOError: bad message length
  </code></pre>
  
  <p>I have narrowed it down to the point where it fails. The parallel processes apparently execute OK (determined by looking at my various debug printouts during different stages of my scripts) but then the failure happens during the remapping of the results.</p>
  
  <p>I have searched for this error message but haven't found any solutions yet. Since my scripts do work for all other types of analysis I am wondering what might be going on.</p>
  
  <h2>A bit about the analysis since I guess this plays into the problem</h2>
  
  <p>The different analysis are more or less timeseries extractions of voxels in the brain (imagine a brain as a 3D matrix with time as the fourth dimension and matrix elements are called voxels). Any point in the brain has an activation value for every point in time. Then a timeseries is the vector of all activation values of a given voxel over time.</p>
  
  <p>I then calculate the correlation coefficient between all the voxels (gives me a square correlation matrix with dimensions voxels by voxels) and return a vector of all the correlation coefficients (lower triangle of the matrix) as the output of the parallel processing.</p>
  
  <p>Now for all the analyses that don't throw the error, I am averaging multiple voxels (based on regional nodes) and then using the average timeseries for this region - effectively doing two things:</p>
  
  <ol>
  <li>drastically reducing the number of voxels (to the number of regions)</li>
  <li>getting rid of voxels which are always zero (as a result of the averaging. no region will contain only zero voxels)</li>
  </ol>
  
  <p>In contrast, the analysis that gives the above mentioned error uses all voxels timeseries in the brain, resulting in a much larger correlation matrix.</p>
  
  <p>I tried to get rid of zero-voxels by masking every subjects file and also, I am not getting any 'division by zero' errors but these are the only two things I can think of.</p>
  
  <p>Also, as said above, the parallel part of the processing runs through without problems. The error get's thrown after it ran, possibly during the remapping of the results.</p>
  
  <p>Any help would be greatly appreciated. Also, if I should provide additional details, please let me know.</p>