title: Multithreading for reading csv files python
body: <p>How do i parse a large (about 5GB) in python?</p>
  
  <p>My current datafile parser takes LOTS of time</p>
  
  <pre><code>def parse_data(file, op_file_test):
          with open(file, 'rb') as inf:
                  incsv = csv.reader(x.replace('\0', '') for x in inf)
                  col1, col2 = defaultdict(Counter), defaultdict(Counter)
                  for c1,c2,label in incsv:
                          try:
                                  col1[label][c1] += 1
                                  col2[label][c2] += 1
                          except ValueError:
                                  pass
  
          map_file =  open(op_file_test,'a')
          labels = sorted(col1)
          for lbl in labels:
                  print &gt;&gt;map_file, "%s, %s, %s, %s, %s" % (lbl, col1[lbl][str(0)], col1[lbl][str(1)], col2[lbl][str(0)],col2[lbl][str(1)])
          map_file.close()
  </code></pre>
  
  <p>What would be the most efficient way to do it?</p>