title: multiprocessing speed difference
body: <p>I am doing some sort of pattern matching on a dataset (English sentences) of 40 MB. To improve the speed I used <strong>multiprocessing</strong>. I created a <strong>Pool</strong> of <em>4</em> processes. Initially I kept the content of the file in a <em>variable</em>, then split it to 4 equal parts and sent it to the function.
  This is how I am doing:</p>
  
  <pre><code>nl_split =  content.split('\n') #The whole data kept in content
  lenNl=int(len(nl_split)/4)
  part1,part2,part3,part4 = '\n'.join(nl_split[0:lenNl]),'\n'.join(nl_split[lenNl+1:2*lenNl]),'\n'.join(nl_split[2*lenNl+1:3*lenNl]),'\n'.join(nl_split[3*lenNl+1:4*lenNl])
  pr = Pool(4)
  rValue=pr.map(match_some_pattern,[part1,part2,part3,part4])
  </code></pre>
  
  <p>Taking <code>90.148968935</code> Sec.</p>
  
  <p>As a <strong>second approach</strong>, I divided the file into 4 and passed to the function as seperate files.</p>
  
  <pre><code>pr = Pool(4)
  rValue=pr.map(match_some_pattern,['part1.txt','part2.txt','part3.txt','part4.txt'])
  </code></pre>
  
  <p>Taking <code>48.5109400749</code> Sec.</p>
  
  <p>When I compare their speed of execution, I found that the <strong>second approach is far better than the first one</strong>. I was thinking that the first approach should be better than the second one, since less file operations are there. But the result was opposite! </p>
  
  <p>Why second approach is consuming less time than the first one ?</p>